gh-111506: Add _Py_OPAQUE_PYOBJECT to hide PyObject layout & related API #136505
Conversation
API that's removed when _Py_OPAQUE_PYOBJECT is defined:
- PyObject_HEAD
- _PyObject_EXTRA_INIT
- PyObject_HEAD_INIT
- PyObject_VAR_HEAD
- struct _object (i.e. PyObject) (opaque)
- struct PyVarObject (opaque)
- Py_SIZE
- Py_SET_TYPE
- Py_SET_SIZE
- PyModuleDef_Base (opaque)
- PyModuleDef_HEAD_INIT
- PyModuleDef (opaque)
- _Py_IsImmortal
- _Py_IsStaticImmortal
Note that the `_Py_IsImmortal` removal means _Py_OPAQUE_PYOBJECT
only works with limited API 3.14+ now.
|
We can also test this (with a workaround/hack in the test for the missing part). |
|
🤖 New build scheduled with the buildbot fleet by @encukou for commit 33d9ae7 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136505%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again. |
|
The tests aren't that straightforward, but the workaround can still be removed if we either scrap the “opaque PyObject” idea or implement PEP 793. |
Lib/test/test_cext/__init__.py
Outdated
| # Test with _Py_OPAQUE_PYOBJECT | ||
| self.check_build('_test_limited_opaque_cext', limited=True, opaque=True) | ||
|
|
||
| def check_build(self, extension_name, std=None, limited=False, opaque=False): |
There was a problem hiding this comment.
I suggest to rename "opaque" to "opaque_pyobject" (also in _check_build).
Lib/test/test_cext/setup.py
Outdated
| @@ -45,6 +45,9 @@ def main(): | |||
| std = os.environ.get("CPYTHON_TEST_STD", "") | |||
| module_name = os.environ["CPYTHON_TEST_EXT_NAME"] | |||
| limited = bool(os.environ.get("CPYTHON_TEST_LIMITED", "")) | |||
| opaque = bool(os.environ.get("CPYTHON_TEST_OPAQUE_PYOBJECT", "")) | |||
There was a problem hiding this comment.
I suggest to rename opaque to opaque_pyobject.
Lib/test/test_cext/setup.py
Outdated
| @@ -75,6 +78,12 @@ def main(): | |||
| version = sys.hexversion | |||
| cflags.append(f'-DPy_LIMITED_API={version:#x}') | |||
|
|
|||
| # Define _Py_OPAQUE_PYOBJECT macro | |||
| if opaque: | |||
| version = sys.hexversion | |||
There was a problem hiding this comment.
version is not used, it can be removed.
Make it so that defining
_Py_OPAQUE_PYOBJECTbefore including<Python.h>will make thePyObjectstruct opaque, and hide all limited-API structs & functions that need access to it.Also, allow
Py_LIMITED_APIwithPy_GIL_DISABLEDif_Py_OPAQUE_PYOBJECTis defined.Opaque PyObject will be needed for most alternatives we have of defining a Stable ABI for free-threaded builds. See Discourse thread for discussion. (We may want to either expose this to users as
Py_OPAQUE_PYOBJECT, or define it implicitly for limited-API free-threaded builds.)And if I'm wrong -- this is easy to remove, like like
Py_GIL_DISABLED.As I mentioned, I'd like to put this in
mainto avoid rebasing a fork and to make it easier to build on top.--disable-gilbuilds are not compatible with the limited API #111506