Skip to content

tests/run-tests.py: Auto detect whether the target has threading and the GIL or not. #17655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

dpgeorge
Copy link
Member

@dpgeorge dpgeorge commented Jul 10, 2025

Summary

This PR makes the following changes to the test runner and CI:

  • detect if the target has the _thread module and, if so, add the threading tests to the set of tests to run
  • if the target has threading, detect if the target has the GIL enabled or not, and use that information to exclude thread mutating tests if the GIL is disabled
  • work around threading bug in tests/extmod/select_poll_eintr.py test
  • make tests/thread/stress_aes.py test do less work on stackless threading builds, so it finishes before the test timeout limit
  • increase test timeout for unix qemu test runs
  • skip some threading tests on macOS and on unix qemu

Currently the information whether a port has the GIL is hard-coded: unix/PC targets and the rp2 port are considered to be GIL-less, and all other targets have the GIL enabled (if they have threading enabled). With the change here, some code is run to detect the GIL. That uses the fact that native code won't bounce the GIL and can hog it, and times how long code runs.

Overall the changes here mean that the threading tests are run in many more configurations:

  • on the unix port with the native emitter
  • on the unix port with single precision float
  • on the unix port with stackless mode enabled
  • on the unix port using clang as a compiler
  • on unix qemu, the architectures MIPS, ARM and RISCV-64
  • on macOS

Testing

Tested locally on the unix port, esp32 and rp2. Let's also see how the CI works.

Trade-offs and Alternatives

This uses a tricky technique to determine GIL/non-GIL. An alternative would be to add some indication in a function whether the GIL is enabled, eg sys.implementation._thread_info. But that will increase code size.

@dpgeorge dpgeorge added the tests Relates to tests/ directory in source label Jul 10, 2025
Copy link

codecov bot commented Jul 10, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.40%. Comparing base (8f8f853) to head (3c1c9ab).

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #17655      +/-   ##
==========================================
- Coverage   98.44%   98.40%   -0.04%     
==========================================
  Files         171      171              
  Lines       22192    22192              
==========================================
- Hits        21847    21839       -8     
- Misses        345      353       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

github-actions bot commented Jul 10, 2025

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:   +88 +0.010% standard[incl +64(data)]
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:   +24 +0.003% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

@dpgeorge dpgeorge force-pushed the tests-detect-thread-gil branch from d63cbda to b90d11e Compare July 10, 2025 15:40
@dpgeorge dpgeorge changed the title tests/run-tests.py: Auto detect whether the target has the GIL or not. tests/run-tests.py: Auto detect whether the target has threading and the GIL or not. Jul 10, 2025
Comment on lines 271 to 276
try:

def stackless():
pass

micropython.heap_lock()
stackless()
micropython.heap_unlock()
except RuntimeError:
micropython.heap_unlock()
is_stackless = True
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
try:
def stackless():
pass
micropython.heap_lock()
stackless()
micropython.heap_unlock()
except RuntimeError:
micropython.heap_unlock()
is_stackless = True
def stackless():
pass
micropython.heap_lock()
try:
stackless()
except RuntimeError:
is_stackless = True
finally:
micropython.heap_unlock()

does it work to make the try/except a bit more focused on where the actual exception is expected?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, it's probably better to be more focused like that. I will change it.

(I was just copying existing code for this from eg tests/micropython/heapalloc.py.)

@@ -56,7 +56,7 @@
#endif

// This value seems to be about right for both 32-bit and 64-bit builds.
#define THREAD_STACK_OVERFLOW_MARGIN (8192)
#define THREAD_STACK_OVERFLOW_MARGIN (4 * 8192)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#define THREAD_STACK_OVERFLOW_MARGIN (4 * 8192)
#define THREAD_STACK_OVERFLOW_MARGIN (16 * 1024)

Also, the commit message could use an explanation of why this is changed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm currently trying to work out if this change is even needed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is not needed, it doesn't fix the issue I thought it would fix. So I took it out.

thread_done = True


global thread_done
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we actually need global statement at the top level?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, now fixed.

busy()
while not thread_done:
time.sleep(0)
dt = time.ticks_diff(time.ticks_ms(), t0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depending on time measurements seems like it could be flaky, especially on desktop systems. I'm not awake enough yet though to try to come up with an alternative that doesn't involve creating a deadlock.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it could be flaky. But from testing on bare-metal and PC, it looks reasonable. The timing is usually either very close to T or 2*T (depending if GIL is disable or enabled).

If it fails due to timing errors (running for too long), it will incorrectly think the GIL is enabled when it is not. Then it will run the mutate tests and they'll probably all fail. In such a case we can just rerun CI...

If it gets to be a real issue, could increase the timing here, eg double it (I just don't want the auto detection to take too long, because it's run at the start of each run of run-tests.py.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some similar concerns, and generally prefer the approach of explicitly adding this information in sys.implementation or similar. Suggest we keep this approach now, but keep the explicit option in mind if it turns out to be flaky.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having some information in sys.implementation could actually be useful for users, instead of them having to run a tricky detection test like this (if they ever need to know the threading implementation...).

And I guess if you have threading enabled you have some amount of flash to spare (although not necessarily on cc3200 which does enable threading...).

So maybe it's worth adding sys.implementation._thread after all?

Copy link
Contributor

@projectgus projectgus Jul 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'd be useful to add. Especially if we don't add any _thread property on builds without threading support.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've now changed this GIL detection to use sys.implementation._thread. It's a lot simpler.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And retested on PICO_W, ESP32_GENERIC and unix port. They detect the GIL correctly and the relevant tests run.

@dpgeorge
Copy link
Member Author

Thanks @dlech for the review here. After I initially submitted this PR and all the CI failed on things I couldn't test locally, the scope of the PR increased quite a bit... so there was a lot of back and forth getting it to work.

@dpgeorge dpgeorge force-pushed the tests-detect-thread-gil branch 3 times, most recently from bf0c6cb to 1266375 Compare July 11, 2025 14:22
@dpgeorge dpgeorge force-pushed the tests-detect-thread-gil branch 7 times, most recently from 05a9ceb to 28535b0 Compare July 13, 2025 12:21
@dpgeorge
Copy link
Member Author

After a lot of back-and-forth with the CI, this PR is now in its final state. I updated the comment at the top with a better description of what this PR is aiming for.

dpgeorge added 7 commits July 15, 2025 13:36
Signed-off-by: Damien George <damien@micropython.org>
This is a workaround for the case where threading is enabled without a GIL.
In such a configuration, creating a new global variable is not atomic and
threads have race conditions resizing/accessing the global dict.

Signed-off-by: Damien George <damien@micropython.org>
Builds with stackless enabled are slower than non-stackless, and this test
takes around 2 minutes on the unix port of MicroPython with the standard
unix parameters.  So detect stackless mode and reduce the time of the test.

Signed-off-by: Damien George <damien@micropython.org>
When detecting the target platform, also check if it has threading and
whether the GIL is enabled or not (using the new attribute
`sys.implementation._thread`).  If threading is available, add the thread
tests to the set of tests to run (unless the set of tests is explicitly
given).

With this change, the unix port no longer needs to explicitly run the set
of thread tests, so that line has been removed from the Makefile.

This change will make sure thread tests are run with other testing
combinations.  In particular, thread tests are now run:
- on the unix port with the native emitter
- on macOS builds
- on unix qemu, the architectures MIPS, ARM and RISCV-64

Signed-off-by: Damien George <damien@micropython.org>
The qemu emulation introduces enough overhead that the
`tests/thread/stress_aes.py` test overruns the default timeout.  So
increase it to allow this test to pass.

Signed-off-by: Damien George <damien@micropython.org>
This test passes sometimes and fails other times.  Eventually that should
be fixed, but for now just skip this test.

Signed-off-by: Damien George <damien@micropython.org>
This test passes sometimes and fails other times.  Eventually that should
be fixed, but for now just skip this test.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge dpgeorge force-pushed the tests-detect-thread-gil branch from 28535b0 to 3c1c9ab Compare July 15, 2025 03:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tests Relates to tests/ directory in source
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy