Bug report
Bug description:
The current getpath.py code tries determining base_prefix/base_exec_prefix by searching the location of the libpython library loaded in the current process, falling back to the location of the Python interpreter executable.
|
# First try to detect prefix by looking alongside our runtime library, if known |
|
if library and not prefix: |
|
library_dir = dirname(library) |
|
if ZIP_LANDMARK: |
|
if os_name == 'nt': |
|
# QUIRK: Windows does not search up for ZIP file |
|
if isfile(joinpath(library_dir, ZIP_LANDMARK)): |
|
prefix = library_dir |
|
else: |
|
prefix = search_up(library_dir, ZIP_LANDMARK) |
|
if STDLIB_SUBDIR and STDLIB_LANDMARKS and not prefix: |
|
if any(isfile(joinpath(library_dir, f)) for f in STDLIB_LANDMARKS): |
|
prefix = library_dir |
|
if not stdlib_dir_was_set_in_config: |
|
stdlib_dir = joinpath(prefix, STDLIB_SUBDIR) |
|
|
|
|
|
# Detect prefix by looking for zip file |
|
if ZIP_LANDMARK and executable_dir and not prefix: |
|
if os_name == 'nt': |
|
# QUIRK: Windows does not search up for ZIP file |
|
if isfile(joinpath(executable_dir, ZIP_LANDMARK)): |
|
prefix = executable_dir |
|
else: |
|
prefix = search_up(executable_dir, ZIP_LANDMARK) |
|
if prefix and not stdlib_dir_was_set_in_config: |
|
stdlib_dir = joinpath(prefix, STDLIB_SUBDIR) |
|
if not isdir(stdlib_dir): |
|
stdlib_dir = None |
|
|
|
|
|
# Detect prefix by searching from our executable location for the stdlib_dir |
|
if STDLIB_SUBDIR and STDLIB_LANDMARKS and executable_dir and not prefix: |
|
prefix = search_up(executable_dir, *STDLIB_LANDMARKS) |
|
if prefix and not stdlib_dir: |
|
stdlib_dir = joinpath(prefix, STDLIB_SUBDIR) |
Looking at the location of the libpython library in use first makes sense, as that is more reliable than looking at interpreter location — it works when embedding, where there isn't any interpreter executable, it works when the executable is not on base_prefix, etc. However, this is only currently supported on Windows and macOS framework builds.
|
/* Add the runtime library's path to the dict */ |
|
static int |
|
library_to_dict(PyObject *dict, const char *key) |
|
{ |
|
#ifdef MS_WINDOWS |
|
#ifdef Py_ENABLE_SHARED |
|
extern HMODULE PyWin_DLLhModule; |
|
if (PyWin_DLLhModule) { |
|
return winmodule_to_dict(dict, key, PyWin_DLLhModule); |
|
} |
|
#endif |
|
#elif defined(WITH_NEXT_FRAMEWORK) |
|
static char modPath[MAXPATHLEN + 1]; |
|
static int modPathInitialized = -1; |
|
if (modPathInitialized < 0) { |
|
modPathInitialized = 0; |
|
|
|
/* On Mac OS X we have a special case if we're running from a framework. |
|
This is because the python home should be set relative to the library, |
|
which is in the framework, not relative to the executable, which may |
|
be outside of the framework. Except when we're in the build |
|
directory... */ |
|
Dl_info pythonInfo; |
|
if (dladdr(&Py_Initialize, &pythonInfo)) { |
|
if (pythonInfo.dli_fname) { |
|
strncpy(modPath, pythonInfo.dli_fname, MAXPATHLEN); |
|
modPathInitialized = 1; |
|
} |
|
} |
|
} |
|
if (modPathInitialized > 0) { |
|
return decode_to_dict(dict, key, modPath); |
|
} |
|
#endif |
|
return PyDict_SetItemString(dict, key, Py_None) == 0; |
|
} |
The spotty platform support stroke me as odd, especially on macOS, as I see no apparent reason for only supporting framework builds, so I looked traced back the origin of this code.
The macOS logic goes back to Python 2.0, having been introduced in 54ecc3d. At this time, we were determining base_prefix/base_exec_prefix based on the Python interpreter location, which was problematic on OS X Frameworks, as the Python interpreter is provided via a launcher. The comment traces back to 55070f5 and is unrelated to the change made by that commit, it just highlights the special case for macOS framework builds.
In GH-29041, which introduced getpath.py, rewriting the old path initialization C code in Python, the logic changed to purposefully search the libpython location before the executable location, also adding Windows support. I imagine the existing macOS code was kept as-is as a mistake, leaving it behind the WITH_NEXT_FRAMEWORK flag, maybe under the assumption it was needed for some reason.
Considering the clear intent in the code, I am treating this a bug.
cc @zooba
CPython versions tested on:
CPython main branch
Operating systems tested on:
No response
Linked PRs
