3.13.0rc1 regression in PyEval_GetLocals(): `SystemError: Objects/dictobject.c:3774: bad argument to internal function` #122728

mgorny · 2024-08-06T08:23:00Z

Crash report

What happened?

After upgrading to CPython 3.13.0rc1, the gpgme test suite started failing. The tests fail with errors resembling:

Traceback (most recent call last):
  File "/tmp/gpgme-1.23.2/lang/python/tests/./final.py", line 24, in <module>
    import support
  File "/tmp/gpgme-1.23.2/lang/python/tests/support.py", line 60, in <module>
    assert_gpg_version((2, 1, 12))
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "/tmp/gpgme-1.23.2/lang/python/tests/support.py", line 35, in assert_gpg_version
    c.engine_info.version).group(0)
    ^^^^^^^^^^^^^
  File "/tmp/gpgme-1.23.2/lang/python/python3.13-gpg/lib.linux-x86_64-cpython-313/gpg/core.py", line 1352, in engine_info
    infos = [i for i in self.get_engine_info() if i.protocol == p]
                        ~~~~~~~~~~~~~~~~~~~~^^
  File "/tmp/gpgme-1.23.2/lang/python/python3.13-gpg/lib.linux-x86_64-cpython-313/gpg/core.py", line 1366, in get_engine_info
    return gpgme.gpgme_ctx_get_engine_info(self.wrapped)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/tmp/gpgme-1.23.2/lang/python/python3.13-gpg/lib.linux-x86_64-cpython-313/gpg/gpgme.py", line 880, in gpgme_ctx_get_engine_info
    return _gpgme.gpgme_ctx_get_engine_info(ctx)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "<frozen importlib._bootstrap>", line 649, in parent
SystemError: Objects/dictobject.c:3774: bad argument to internal function
FAIL: final.py

I've been able to bisect it to 233ed46 (#121869). However, on this commit I'm getting also a bunch of python3.13: Objects/typeobject.c:5257: _PyType_LookupRef: Assertion '!PyErr_Occurred()' failed. — they are fixed in some subsequent commit. But the final.py failure occurs from that commit to tip of 3.13 (fe65a8b).

Now, gpgme's build system is horrible. To reproduce, I've done the following:

Built CPython with --with-assertions.
Created ln -s python python3.13 symlink in the build directory.
Installed the gpgme C library 1.23.2 using the system package manager.
Then:

wget https://www.gnupg.org/ftp/gcrypt/gpgme/gpgme-1.23.2.tar.bz2
tar -xf gpgme-1.23.2.tar.bz2
cd gpgme-1.23.2
export PATH=${CPYTHON_BUILD_DIRECTORY}:$PATH
./configure  # will probably need some -dev packages
cd lang/python
make PYTHONS=python3.13 check

Note that in order to rebuild, you need to rm -rf python3.13-gpg.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.13.0rc1 (main, Aug 2 2024, 18:56:30) [GCC 14.2.0]

Linked PRs

The text was updated successfully, but these errors were encountered:

thesamesam · 2024-08-06T08:32:29Z

cc @gaogaotiantian @ncoghlan

hroncok · 2024-08-06T12:02:59Z

We see this in Fedora as well.

https://bugzilla.redhat.com/show_bug.cgi?id=2303107

gpgme built with Python 3.13.0b4 exhibits the same behavior when run on Python 3.13.0rc1.

hroncok · 2024-08-06T12:04:40Z

Built CPython with --with-assertions.

Should not be necessary. We build Python without and the same SystemError happens.

vstinner · 2024-08-06T12:15:31Z

The problem is that PyDict_Update() is called with NULL as the first argument. It's a regression in PyEval_GetLocals().

gdb traceback:

(gdb) where
#0  dict_merge (interp=0x983f60 <_PyRuntime+88192>, a=a@entry=0x0, b=b@entry=0x7ffff7b34b50, override=override@entry=1) at Objects/dictobject.c:3796
#1  0x00000000004ece12 in PyDict_Update (a=a@entry=0x0, b=b@entry=0x7ffff7b34b50) at Objects/dictobject.c:3887
#2  0x0000000000607cbe in PyEval_GetLocals () at Python/ceval.c:2492
#3  0x00007ffff77ce622 in _gpg_wrap_result (fragile=fragile@entry=0x7ffff7835940, classname=classname@entry=0x7ffff77d05ab "EngineInfo") at python3.13-gpg/helpers.c:318

(gdb) frame 3
#3  0x00007ffff77ce622 in _gpg_wrap_result (fragile=fragile@entry=0x7ffff7835940, classname=classname@entry=0x7ffff77d05ab "EngineInfo") at python3.13-gpg/helpers.c:318
318	      results = PyImport_ImportModuleLevel("results", PyEval_GetGlobals(),
319	                                           PyEval_GetLocals(), from_list, 1);

(gdb) frame 2
#2  0x0000000000607cbe in PyEval_GetLocals () at Python/ceval.c:2492
2492	        if (PyDict_Update(ret, locals) < 0) {

(gdb) l 2488
2483	        PyObject *ret = f->f_locals_cache;
2484	        if (ret == NULL) {
2485	            PyObject *ret = PyDict_New();
2486	            if (ret == NULL) {
2487	                Py_DECREF(locals);
2488	                return NULL;
2489	            }
2490	            f->f_locals_cache = ret;
2491	        }
2492	        if (PyDict_Update(ret, locals) < 0) {

(gdb) p ret
$1 = (PyObject *) 0x0
(gdb) p locals
$2 = (PyObject *) 0x7ffff7b34b50

Fix PyEval_GetLocals() to avoid SystemError ("bad argument to internal function"). Don't redefine the 'ret' variable in the if block. Add an unit test on PyEval_GetLocals().

Fix PyEval_GetLocals() to avoid SystemError ("bad argument to internal function"). Don't redefine the 'ret' variable in the if block. Add an unit test on PyEval_GetLocals(). (cherry picked from commit 4767a6e) Co-authored-by: Victor Stinner <vstinner@python.org>

Fix PyEval_GetLocals() to avoid SystemError ("bad argument to internal function"). Don't redefine the 'ret' variable in the if block. Add an unit test on PyEval_GetLocals(). (cherry picked from commit 4767a6e)

…122757) gh-122728: Fix SystemError in PyEval_GetLocals() (GH-122735) Fix PyEval_GetLocals() to avoid SystemError ("bad argument to internal function"). Don't redefine the 'ret' variable in the if block. Add an unit test on PyEval_GetLocals(). (cherry picked from commit 4767a6e) Co-authored-by: Victor Stinner <vstinner@python.org>

vstinner · 2024-08-06T21:37:42Z

Fixed. Thanks @mgorny and @hroncok for the bug report.

Fix PyEval_GetLocals() to avoid SystemError ("bad argument to internal function"). Don't redefine the 'ret' variable in the if block. Add an unit test on PyEval_GetLocals().

mgorny added the type-crash A hard crash of the interpreter, possibly with a core dump label Aug 6, 2024

mgorny mentioned this issue Aug 6, 2024

PyEval_GetLocals() leaks locals #118934

Closed

bedevere-app bot mentioned this issue Aug 6, 2024

gh-122728: Fix SystemError in PyEval_GetLocals() #122735

Merged

vstinner changed the title ~~3.13.0rc1 regression in gpgme: SystemError: Objects/dictobject.c:3774: bad argument to internal function~~ 3.13.0rc1 regression in PyEval_GetLocals(): SystemError: Objects/dictobject.c:3774: bad argument to internal function Aug 6, 2024

Eclips4 added 3.13 bugs and security fixes 3.14 new features, bugs and security fixes labels Aug 6, 2024

bedevere-app bot mentioned this issue Aug 6, 2024

[3.13] gh-122728: Fix SystemError in PyEval_GetLocals() (GH-122735) #122757

Merged

vstinner closed this as completed Aug 6, 2024

edgarrmondragon mentioned this issue Aug 7, 2024

Support Python 3.13 jelmer/dulwich#1353

Merged

jelmer mentioned this issue Aug 13, 2024

Support Python 3.13 jelmer/dulwich#1352

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

3.13.0rc1 regression in PyEval_GetLocals(): `SystemError: Objects/dictobject.c:3774: bad argument to internal function` #122728

3.13.0rc1 regression in PyEval_GetLocals(): `SystemError: Objects/dictobject.c:3774: bad argument to internal function` #122728

3.13.0rc1 regression in PyEval_GetLocals(): SystemError: Objects/dictobject.c:3774: bad argument to internal function #122728

3.13.0rc1 regression in PyEval_GetLocals(): SystemError: Objects/dictobject.c:3774: bad argument to internal function #122728

Comments

Crash report

What happened?

CPython versions tested on:

Operating systems tested on:

Output from running 'python -VV' on the command line:

Linked PRs

3.13.0rc1 regression in PyEval_GetLocals(): `SystemError: Objects/dictobject.c:3774: bad argument to internal function` #122728

3.13.0rc1 regression in PyEval_GetLocals(): `SystemError: Objects/dictobject.c:3774: bad argument to internal function` #122728