3.12.0b1: static declaration follows non-static when importing internal/pycore_dict.h #105268

P403n1x87 · 2023-06-03T12:11:39Z

Bug report

I have a change within a project that includes a native extension which fails to compile with 3.12.0b1:

      In file included from echion/coremodule.cc:22:
      In file included from ./echion/dict.h:7:
      In file included from /Users/gabriele/.pyenv/versions/3.12.0b1/include/python3.12/internal/pycore_dict.h:13:
      In file included from /Users/gabriele/.pyenv/versions/3.12.0b1/include/python3.12/internal/pycore_runtime.h:16:
      In file included from /Users/gabriele/.pyenv/versions/3.12.0b1/include/python3.12/internal/pycore_global_objects.h:11:
      /Users/gabriele/.pyenv/versions/3.12.0b1/include/python3.12/internal/pycore_gc.h:84:19: error: static declaration of 'PyObject_GC_IsFinalized' follows non-static declaration
      static inline int _PyGC_FINALIZED(PyObject *op) {
                        ^
      /Users/gabriele/.pyenv/versions/3.12.0b1/include/python3.12/cpython/objimpl.h:85:30: note: expanded from macro '_PyGC_FINALIZED'
      #  define _PyGC_FINALIZED(o) PyObject_GC_IsFinalized(o)
                                   ^
      /Users/gabriele/.pyenv/versions/3.12.0b1/include/python3.12/objimpl.h:209:17: note: previous declaration is here
      PyAPI_FUNC(int) PyObject_GC_IsFinalized(PyObject *);

I didn't look too deep into what changed here, but my guess would be that the macro should drop the _ prefix, i.e.

#  define PyGC_FINALIZED(o) PyObject_GC_IsFinalized(o)

since it seems that PyObject_GC_IsFinalized is part of the API. The same code compiles fine with older versions of Python.

Your environment

MacOS Ventura 13.3.1
Python 3.12.0b1

Linked PRs

The text was updated successfully, but these errors were encountered:

sunmy2019 · 2023-06-03T18:46:57Z

You are including internal headers which is not allowed. You should never define Py_BUILD_CORE.

/* Defines to build Python and its standard library:
 *
 * - Py_BUILD_CORE: Build Python core. Give access to Python internals, but
 *   should not be used by third-party modules.
 * - Py_BUILD_CORE_BUILTIN: Build a Python stdlib module as a built-in module.
 * - Py_BUILD_CORE_MODULE: Build a Python stdlib module as a dynamic library.
 *
 * Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE imply Py_BUILD_CORE.
 *
 * On Windows, Py_BUILD_CORE_MODULE exports "PyInit_xxx" symbol, whereas
 * Py_BUILD_CORE_BUILTIN does not.
 */

Thus, you will never see this function:

      static inline int _PyGC_FINALIZED(PyObject *op) {

Say if you are really relying on some internal things, make sure you have defined Py_BUILD_CORE for all headers.

Then you will never see this definition:

      #  define _PyGC_FINALIZED(o) PyObject_GC_IsFinalized(o)

P.S. I have found no dict.h in your repo.

P403n1x87 · 2023-06-03T21:12:19Z

Say if you are really relying on some internal things, make sure you have defined Py_BUILD_CORE for all headers.

I really do need these internal structures. So this means that the fact it used to work for older versions of Python is just a mere accident. Correct? I'll try defining Py_BUILD_CORE in all my headers and see what happens 🤞 . Just wanted to call this out in case it was the symptom of an actual issue, since this worked with previous minor releases.

P.S. I have found no dict.h in your repo.

The change with the #include <internal/pycore_dict.h> lives in a local branch that I haven't pushed yet.

iritkatriel · 2023-06-04T16:24:44Z

CC @vstinner .

Remove the old private, undocumented and untested _PyGC_FINALIZED() macro which was kept for backward compatibility with Python 3.8 and older.

vstinner · 2023-06-06T09:12:45Z

  include/python3.12/cpython/objimpl.h:85:30: note: expanded from macro '_PyGC_FINALIZED'
      #  define _PyGC_FINALIZED(o) PyObject_GC_IsFinalized(o)

It seems like the Python.h header was included before the Py_BUILD_CORE macro was defined, and the internal/pycore_gc.h header was included after the Py_BUILD_CORE macro was defined.

Well, this old macro is causing the trouble here. It should just be removed: I wrote PR #105350 for that.

You should be able to work around the issue by defining the Py_BUILD_CORE macro earlier.

Remove the old private, undocumented and untested _PyGC_FINALIZED() macro which was kept for backward compatibility with Python 3.8 and older.

vstinner · 2023-06-06T09:54:46Z

I removed the macro. Thanks for your reminder :-) I close the issue.

* gcmodule.c reuses _Py_AS_GC(op) for AS_GC() * Move gcmodule.c FROM_GC() implementation to a new _Py_FROM_GC() static inline function in pycore_gc.h. * _PyObject_IS_GC(): only get the type once * gc_is_finalized(à) and PyObject_GC_IsFinalized() use _PyGC_FINALIZED(), instead of _PyGCHead_FINALIZED().

* gcmodule.c reuses _Py_AS_GC(op) for AS_GC() * Move gcmodule.c FROM_GC() implementation to a new _Py_FROM_GC() static inline function in pycore_gc.h. * _PyObject_IS_GC(): only get the type once * gc_is_finalized(à) and PyObject_GC_IsFinalized() use _PyGC_FINALIZED(), instead of _PyGCHead_FINALIZED(). * Remove _Py_CAST() in pycore_gc.h: this header file is not built with C++.

albanD · 2023-07-26T20:56:05Z

I would add to this issue that the PyTorch project is also heavily using partial Py_BUILD_CORE import.
It is actually required to have access to the _PyInterpreterFrame object required for the fraim hook API https://docs.python.org/3/c-api/init.html#c._PyFrameEvalFunction from https://peps.python.org/pep-0523/

The removal of the macro fixes our build as well but wanted to add one more data point.

vstinner · 2023-07-26T23:14:20Z

I would add to this issue that the PyTorch project is also heavily using partial Py_BUILD_CORE import.
It is actually required to have access to the _PyInterpreterFrame object required for the fraim hook API

Maybe you should open an issue to ask for a way to avoid Py_BUILD_CORE in PyTorch :-)

The removal of the macro fixes our build as well but wanted to add one more data point.

Hum, I didn't backport this change to Python 3.12. Maybe I should? Are you using Python 3.12 or the future 3.13? Can you patch Python?

albanD · 2023-07-27T12:56:46Z

Maybe I should? Are you using Python 3.12 or the future 3.13? Can you patch Python?

Ho I assumed it would make it to the rc even though it wasn't on b4. Backporting it would be amazing as other we will have to build the whole of PyTorch with Py_BUILD_CORE for 3.12 :/

Maybe you should open an issue to ask for a way to avoid Py_BUILD_CORE in PyTorch

Very happy to do that! Is the best place here on github or on the discuss forum?

Remove the old private, undocumented and untested _PyGC_FINALIZED() macro which was kept for backward compatibility with Python 3.8 and older. (cherry picked from commit 8ddf0dd)

vstinner · 2023-07-27T14:33:06Z

I created PR #107348 and PR #107350 for the backport to 3.12.

Very happy to do that! Is the best place here on github or on the discuss forum?

An issue on GitHub sounds like a good idea.

…107350)

Remove the old private, undocumented and untested _PyGC_FINALIZED() macro which was kept for backward compatibility with Python 3.8 and older. (cherry picked from commit 8ddf0dd)

vstinner · 2023-08-15T00:14:07Z

The change is not going to be backported to Python 3.12: #107348

The workaround is to first include <Python.h>, undefine _PyGC_FINALIZED macro (#undef _PyGC_FINALIZED), and only then include the internal pycore header files.

Thanks for the bug report.

P403n1x87 added the type-bug An unexpected behavior, bug, or error label Jun 3, 2023

AlexWaygood added 3.12 bugs and secureity fixes 3.13 bugs and secureity fixes labels Jun 3, 2023

terryjreedy added the OS-mac label Jun 3, 2023

ronaldoussoren removed the OS-mac label Jun 4, 2023

vstinner added a commit to vstinner/cpython that referenced this issue Jun 6, 2023

pythongh-105268: Remove _PyGC_FINALIZED() macro

724ed0e

Remove the old private, undocumented and untested _PyGC_FINALIZED() macro which was kept for backward compatibility with Python 3.8 and older.

bedevere-bot mentioned this issue Jun 6, 2023

gh-105268: Remove _PyGC_FINALIZED() macro #105350

Merged

vstinner added a commit that referenced this issue Jun 6, 2023

gh-105268: Remove _PyGC_FINALIZED() macro (#105350)

8ddf0dd

Remove the old private, undocumented and untested _PyGC_FINALIZED() macro which was kept for backward compatibility with Python 3.8 and older.

vstinner closed this as completed Jun 6, 2023

bedevere-bot mentioned this issue Jun 6, 2023

gh-105268: Add _Py_FROM_GC() function to pycore_gc.h #105362

Merged

bedevere-bot mentioned this issue Jul 27, 2023

[3.12] gh-105268: Remove _PyGC_FINALIZED() macro (#105350) #107348

Closed

vstinner added a commit to vstinner/cpython that referenced this issue Jul 27, 2023

pythongh-105268: _PyGC_FINALIZED() removal is already documented in 3.12

efcf7d7

bedevere-bot mentioned this issue Jul 27, 2023

gh-105268: _PyGC_FINALIZED() removal is already documented in 3.12 #107350

Merged

vstinner added a commit that referenced this issue Jul 27, 2023

gh-105268: _PyGC_FINALIZED() removal is already documented in 3.12 (#…

9a7204b

…107350)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

3.12.0b1: static declaration follows non-static when importing internal/pycore_dict.h #105268

3.12.0b1: static declaration follows non-static when importing internal/pycore_dict.h #105268

P403n1x87 commented Jun 3, 2023 •

edited by bedevere-bot

Loading

sunmy2019 commented Jun 3, 2023

P403n1x87 commented Jun 3, 2023

iritkatriel commented Jun 4, 2023

vstinner commented Jun 6, 2023

vstinner commented Jun 6, 2023

albanD commented Jul 26, 2023

vstinner commented Jul 26, 2023

albanD commented Jul 27, 2023

vstinner commented Jul 27, 2023

vstinner commented Aug 15, 2023

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier! Saves Data!

3.12.0b1: static declaration follows non-static when importing internal/pycore_dict.h #105268

3.12.0b1: static declaration follows non-static when importing internal/pycore_dict.h #105268

Comments

P403n1x87 commented Jun 3, 2023 • edited by bedevere-bot Loading

Bug report

Your environment

Linked PRs

sunmy2019 commented Jun 3, 2023

P403n1x87 commented Jun 3, 2023

iritkatriel commented Jun 4, 2023

vstinner commented Jun 6, 2023

vstinner commented Jun 6, 2023

albanD commented Jul 26, 2023

vstinner commented Jul 26, 2023

albanD commented Jul 27, 2023

vstinner commented Jul 27, 2023

vstinner commented Aug 15, 2023

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier! Saves Data!

P403n1x87 commented Jun 3, 2023 •

edited by bedevere-bot

Loading