[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Free-threaded libraries should be in lib/python3.14t (configure) #121103

Open
colesbury opened this issue Jun 27, 2024 · 2 comments
Open

Free-threaded libraries should be in lib/python3.14t (configure) #121103

colesbury opened this issue Jun 27, 2024 · 2 comments
Labels
3.13 bugs and security fixes 3.14 new features, bugs and security fixes release-blocker topic-free-threading type-bug An unexpected behavior, bug, or error

Comments

@colesbury
Copy link
Contributor
colesbury commented Jun 27, 2024 •
edited by bedevere-app bot
Loading

Bug report

Background

When using configure based Python installations, if two the free-threaded and default builds are installed to the same prefix, they will share the same lib directory. For example, when installing Python 3.14 the structure looks like:

include/python3.14/...
include/python3.14t/...
lib/libpython3.14.a
lib/libpython3.14t.a
lib/python3.14/...  # shared!

The include directories are not shared, which is good because they have different pyconfig.h files.

However, the lib/python3.14 is shared, which means that packages installed in the default build may be partially available in the free-threaded build and vice versa. This was unintended and can cause problems, such as confusing error messages and crashes.

For example, if I run:

  • python3.14 -m pip install numpy (install in default build)
  • python3.14t -c "import numpy"

I get a confusing error message:

Error importing numpy: you should not try to import numpy from its source directory... 
Traceback (most recent call last):
  File "/tmp/python-nogil/lib/python3.14/site-packages/numpy/_core/__init__.py", line 23, in <module>
    from . import multiarray
  File "/tmp/python-nogil/lib/python3.14/site-packages/numpy/_core/multiarray.py", line 10, in <module>
    from . import overrides
  File "/tmp/python-nogil/lib/python3.14/site-packages/numpy/_core/overrides.py", line 8, in <module>
    from numpy._core._multiarray_umath import (
        add_docstring,  _get_implementing_args, _ArrayFunctionDispatcher)
ModuleNotFoundError: No module named 'numpy._core._multiarray_umath'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/python-nogil/lib/python3.14/site-packages/numpy/__init__.py", line 114, in <module>
    from numpy.__config__ import show as show_config
  File "/tmp/python-nogil/lib/python3.14/site-packages/numpy/__config__.py", line 4, in <module>
    from numpy._core._multiarray_umath import (
    ...<3 lines>...
    )
  File "/tmp/python-nogil/lib/python3.14/site-packages/numpy/_core/__init__.py", line 49, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.14 from "/tmp/python-nogil/bin/python3.14t"
  * The NumPy version is: "2.0.0"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy._core._multiarray_umath'


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import numpy
  File "/tmp/python-nogil/lib/python3.14/site-packages/numpy/__init__.py", line 119, in <module>
    raise ImportError(msg) from e
ImportError: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

It would be better if installing NumPy in the default build did not make it available in the free-threaded build and vice versa.

Proposal

We should add the ABI suffix to the lib directory, like we do for the include directory. Specifically, we should use python$(LDVERSION) instead of python$(VERSION).For example, the free-threaded build would use lib/python3.14t in Python 3.14.

Debug builds have d as part of their ABI suffix (and LDVERSION), so this would incidentally affect installations of the debug configuration of the non-free-threaded build.

Linked PRs
@colesbury colesbury added type-bug An unexpected behavior, bug, or error 3.13 bugs and security fixes topic-free-threading 3.14 new features, bugs and security fixes labels Jun 27, 2024
@colesbury colesbury mentioned this issue Jun 27, 2024
Open
@encukou
Copy link
Member
encukou commented Jul 1, 2024

Debug builds have d as part of their ABI suffix (and LDVERSION), so this would incidentally affect installations of the debug configuration of the non-free-threaded build.

That's the catch. The debug build is ABI-compatible; you do want this kind of cross-installation for the debug build.

Perhaps there needs to be a new variable with just the t flag :/

@colesbury
Copy link
Contributor Author

Yeah, I think just the t flag is probably a good choice to avoid affecting the default build.

colesbury added a commit to colesbury/cpython that referenced this issue Jul 2, 2024
@colesbury
pythongh-121103: Put free-threaded libraries in lib/python3.14t
03772aa 
On POSIX systems, excluding macOS framework installs, the lib directory
for the free-threaded build now includes a "t" suffix to avoid conflicts
with a co-located default build installation.
@bedevere-app bedevere-app bot mentioned this issue Jul 2, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.13 bugs and security fixes 3.14 new features, bugs and security fixes release-blocker topic-free-threading type-bug An unexpected behavior, bug, or error
Projects
Release and Deferred blockers 🚫
Status: Todo
Milestone
No milestone
Development

No branches or pull requests
3 participants
@encukou @colesbury @Yhg1s