Skip to content

gh-135906: Test the internal C API in test_cext #136247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jul 11, 2025

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Jul 3, 2025

@vstinner
Copy link
Member Author

vstinner commented Jul 3, 2025

I tried to check for C++ compiler warnings in internal Python C headers, but I got many C++ compiler errors in mimalloc and I don't know how to handle them. Examples:

In file included from /usr/include/features.h:524,
                 from /usr/include/assert.h:35,
                 from /home/vstinner/python/main/Include/Python.h:19,
                 from extension.cpp:13:
/home/vstinner/python/main/Include/internal/mimalloc/mimalloc/types.h:73:33: error: expected unqualified-id before 'noexcept'
   73 | #define mi_decl_throw           __THROW
      |                                 ^~~~~~~
/home/vstinner/python/main/Include/internal/mimalloc/mimalloc/types.h:611:31: note: in expansion of macro 'mi_decl_throw'
  611 | mi_decl_noreturn mi_decl_cold mi_decl_throw
      |                               ^~~~~~~~~~~~~

and

In file included from /home/vstinner/python/main/Include/internal/pycore_mimalloc.h:44,
                 from /home/vstinner/python/main/Include/internal/pycore_tstate.h:13,
                 from /home/vstinner/python/main/Include/internal/pycore_pystate.h:13,
                 from extension.cpp:20:
/home/vstinner/python/main/Include/internal/mimalloc/mimalloc/internal.h: In function 'bool _mi_is_aligned(void*, size_t)':
/home/vstinner/python/main/Include/internal/mimalloc/mimalloc/types.h:613:49: error: '_mi_assert_fail' was not declared in this scope; did you mean '__assert_fail'?
  613 | #define mi_assert(expr)     ((expr) ? (void)0 : _mi_assert_fail(#expr,__FILE__,__LINE__,__func__))
      |                                                 ^~~~~~~~~~~~~~~
/home/vstinner/python/main/Include/internal/mimalloc/mimalloc/types.h:619:31: note: in expansion of macro 'mi_assert'
  619 | #define mi_assert_internal    mi_assert
      |                               ^~~~~~~~~
/home/vstinner/python/main/Include/internal/mimalloc/mimalloc/internal.h:273:3: note: in expansion of macro 'mi_assert_internal'
  273 |   mi_assert_internal(alignment != 0);
      |   ^~~~~~~~~~~~~~~~~~

@vstinner
Copy link
Member Author

vstinner commented Jul 3, 2025

I updated test_cppext to have a bare minimum test of the internal C API for now: test including pycore_frame.h. We can extend the test later.

@vstinner
Copy link
Member Author

vstinner commented Jul 3, 2025

Note to mysel: GitHub Action skips these tests which require the cpu test resource (-u cpu or -u all). Only buildbots run these slow tests.

@vstinner vstinner added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 3, 2025
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 69588da 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136247%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 3, 2025
@vstinner
Copy link
Member Author

vstinner commented Jul 3, 2025

test_cext fails on on multiple buildbots (s390x Fedora Stable Clang PR, AMD64 FreeBSD PR, x86-64 macOS PR, ...) with:

  In file included from extension.c:21:
  In file included from /home/buildbot/buildarea/pull_request.cstratak-fedora-stable-x86_64.clang/build/Include/internal/pycore_gc.h:11:
  /home/buildbot/buildarea/pull_request.cstratak-fedora-stable-x86_64.clang/build/Include/internal/pycore_interp_structs.h:132:16: error: redefinition of typedef 'atexit_datacallbackfunc' is a C11 feature [-Werror,-Wtypedef-redefinition]
    132 | typedef void (*atexit_datacallbackfunc)(void *);
        |                ^
  /home/buildbot/buildarea/pull_request.cstratak-fedora-stable-x86_64.clang/build/Include/cpython/pylifecycle.h:87:16: note: previous definition is here
     87 | typedef void (*atexit_datacallbackfunc)(void *);
        |                ^

atexit_datacallbackfunc type is already defined by
Include/cpython/pylifecycle.h.
@vstinner vstinner added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 3, 2025
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 49faa0c 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136247%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 3, 2025
@vstinner
Copy link
Member Author

vstinner commented Jul 4, 2025

test_cext fails on ARM64 Windows Non-Debug PR:

  "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\bin\HostX86\ARM64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\PCbuild\ARM64 -IC:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\build\test_python_28188�\tempcwd\env\include -IC:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\include -IC:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\PC -IC:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\PC "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.19041.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.19041.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.19041.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.19041.0\\cppwinrt" /Tcextension.c /Fobuild\temp.win-arm64-cpython-315\Release\extension.obj /W4 /WX -DMODULE_NAME=_test_cext -DTEST_INTERNAL_C_API=1
  cl : Command line warning D9025 : overriding '/W3' with '/W4'
  extension.c
  C:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\include\internal\mimalloc/mimalloc/types.h(509): error C2220: the following warning is treated as an error
  C:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\include\internal\mimalloc/mimalloc/types.h(509): warning C4324: 'mi_abandoned_pool_s': structure was padded due to alignment specifier
  C:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\include\internal\mimalloc/mimalloc/types.h(512): warning C4324: 'mi_abandoned_pool_s': structure was padded due to alignment specifier
  C:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\include\internal\mimalloc/mimalloc/types.h(513): warning C4324: 'mi_abandoned_pool_s': structure was padded due to alignment specifier
  C:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\include\internal\mimalloc/mimalloc/types.h(518): warning C4324: 'mi_abandoned_pool_s': structure was padded due to alignment specifier
  C:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\include\internal\pycore_qsbr.h(75): warning C4200: nonstandard extension used: zero-sized array in struct/union
  C:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\include\internal\pycore_backoff.h(68): warning C4244: 'function': conversion from 'int' to 'uint16_t', possible loss of data
  C:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\include\internal\pycore_code.h(114): warning C4201: nonstandard extension used: nameless struct/union
  C:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\include\internal\pycore_object_deferred.h(20): warning C4100: 'op': unreferenced formal parameter
  C:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\include\internal\pycore_stackref.h(792): warning C4100: 'tstate': unreferenced formal parameter
  C:\Workspace\buildarea\pull_request.linaro-win-arm64.nondebug\build\include\internal\pycore_stackref.h(803): warning C4100: 'tstate': unreferenced formal parameter
  error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.31.31103\\bin\\HostX86\\ARM64\\cl.exe' failed with exit code 2
  error: subprocess-exited-with-error

@vstinner vstinner added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 10, 2025
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 7870668 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136247%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 10, 2025
@vstinner
Copy link
Member Author

ARM64 Windows Non-Debug PR failed. .TestInteralCAPI.test_build() failed with:

  extension.obj : error LNK2001: unresolved external symbol PyLong_FromLong
  extension.obj : error LNK2001: unresolved external symbol PyModuleDef_Init
  extension.obj : error LNK2001: unresolved external symbol PyArg_ParseTuple
  build\lib.win-arm64-cpython-315\_test_cext.cp315-win_arm64.pyd : fatal error LNK1120: 3 unresolved externals

and TestInteralCAPI.test_build_c11() fails with:

  extension.c
  C:\Program Files (x86)\Windows Kits\10\\include\10.0.19041.0\\um\winbase.h(9531): error C2220: the following warning is treated as an error
  C:\Program Files (x86)\Windows Kits\10\\include\10.0.19041.0\\um\winbase.h(9531): warning C5105: macro expansion producing 'defined' has undefined behavior
  error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.31.31103\\bin\\HostX86\\ARM64\\cl.exe' failed with exit code 2

@vstinner
Copy link
Member Author

!buildbot Windows

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 30c6265 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136247%2Fmerge

The command will test the builders whose names match following regular expression: Windows

The builders matched are:

  • AMD64 Windows PGO NoGIL PR
  • AMD64 Windows Server 2022 NoGIL PR
  • AMD64 Windows10 PR
  • ARM64 Windows Non-Debug PR
  • AMD64 Windows11 Refleaks PR
  • AMD64 Windows11 Non-Debug PR
  • AMD64 Windows11 Bigmem PR
  • AMD64 Windows PGO PR
  • ARM64 Windows PR

@vstinner
Copy link
Member Author

On ARM64 Windows Non-Debug PR, TestInteralCAPI.test_build_c11() failed with:

  extension.c
  C:\Program Files (x86)\Windows Kits\10\\include\10.0.19041.0\\um\winbase.h(9531): error C2220: the following warning is treated as an error
  C:\Program Files (x86)\Windows Kits\10\\include\10.0.19041.0\\um\winbase.h(9531): warning C5105: macro expansion producing 'defined' has undefined behavior
  error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.31.31103\\bin\\HostX86\\ARM64\\cl.exe' failed with exit code 2

@vstinner vstinner added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 11, 2025
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 9d4d697 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136247%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 11, 2025
@vstinner vstinner marked this pull request as ready for review July 11, 2025 14:47
@vstinner vstinner merged commit 2c9a801 into python:main Jul 11, 2025
120 of 122 checks passed
@vstinner vstinner deleted the test_internal_capi branch July 11, 2025 14:48
Pranjal095 pushed a commit to Pranjal095/cpython that referenced this pull request Jul 12, 2025
Remove duplicated definition: atexit_datacallbackfunc type
is already defined by Include/cpython/pylifecycle.h.
picnixz pushed a commit to picnixz/cpython that referenced this pull request Jul 13, 2025
Remove duplicated definition: atexit_datacallbackfunc type
is already defined by Include/cpython/pylifecycle.h.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy