Content-Length: 374197 | pFad | http://github.com/micropython/micropython/pull/17734

99 py/objint_longlong: Fix left shift of negative values. by projectgus · Pull Request #17734 · micropython/micropython · GitHub
Skip to content

py/objint_longlong: Fix left shift of negative values. #17734

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

projectgus
Copy link
Contributor

@projectgus projectgus commented Jul 22, 2025

Summary

Closes #17702, thanks to @jepler for running these additional checks.

  • The overflow check for 'longlong' representation left shift added in py: Fixes and test coverage for 64-bit big integer representations. #16953 was wrong, and depended on UB. Rewrote to be the same as the small int left shift overflow check.
  • Skip the viper "boundary" store tests if arbitrary integer support is disabled, these tests depend on generating very long integers and then truncating them.

Note: First version of this PR enabled UBSan for longlong tests, but this surfaced another issue so has been spun out to #17735.

Testing

  • Re-ran unix port (with UBSan), observed tests pass and no errors.

Trade-offs and Alternatives

  • The viper "boundary" store tests could probably be rewritten to also work withouth arbitrary precision integers, but relatively fiddly to do so.

@projectgus projectgus added this to the release-1.26.0 milestone Jul 22, 2025
@projectgus projectgus added the py-core Relates to py/ directory in source label Jul 22, 2025
@projectgus
Copy link
Contributor Author

Hmm, so running the longlong tests with UBSan I get two failures like this:

 --- /home/runner/work/micropython/micropython/tests/results/extmod_vfs_rom.py.exp	2025-07-22 01:24:29.075299604 +0000
+++ /home/runner/work/micropython/micropython/tests/results/extmod_vfs_rom.py.out	2025-07-22 01:24:29.075299604 +0000
@@ -1 +1,13 @@
-SKIP
+Traceback (most recent call last):
+  File "/home/runner/work/micropython/micropython/tests/extmod/vfs_rom.py", line 489, in <module>
+  File "/home/runner/work/micropython/micropython/lib/micropython-lib/python-stdlib/unittest/unittest/__init__.py", line 464, in main
+  File "/home/runner/work/micropython/micropython/lib/micropython-lib/python-stdlib/unittest/unittest/__init__.py", line 269, in run
+  File "/home/runner/work/micropython/micropython/lib/micropython-lib/python-stdlib/unittest/unittest/__init__.py", line 254, in run
+  File "/home/runner/work/micropython/micropython/lib/micropython-lib/python-stdlib/unittest/unittest/__init__.py", line 420, in _run_suite
+  File "/home/runner/work/micropython/micropython/tests/extmod/vfs_rom.py", line 166, in setUpClass
+  File "/home/runner/work/micropython/micropython/tests/extmod/vfs_rom.py", line 145, in make_romfs
+  File "/home/runner/work/micropython/micropython/tests/extmod/vfs_rom.py", line 136, in _make_romfs
+  File "/home/runner/work/micropython/micropython/tests/extmod/vfs_rom.py", line 115, in mkfile
+  File "/home/runner/work/micropython/micropython/tests/extmod/vfs_rom.py", line 80, in _encode_uint
+RuntimeError: maximum recursion depth exceeded

(I don't see these locally, but I have a different gcc locally.)

This looks a lot like the problem @koxudaxi was describing in #17649...

Copy link

codecov bot commented Jul 22, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.41%. Comparing base (e993f53) to head (09c6e9b).

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #17734   +/-   ##
=======================================
  Coverage   98.41%   98.41%           
=======================================
  Files         171      171           
  Lines       22210    22210           
=======================================
  Hits        21857    21857           
  Misses        353      353           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

Previous comment was wrong, left shifting a negative value is UB in C. Use
the same approach as small int shifts (from runtime.c).

Signed-off-by: Angus Gratton <angus@redyak.com.au>
Skip the tests if MicroPython can't represent >64-bit integers
(these tests repeatedly left shift values by 8 bits).

It would be possible to have these tests work in this case I think,
as the results are always masked to shorter values. But quite fiddly.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
@projectgus projectgus force-pushed the bugfix/longlong_shift branch from d2517f1 to 09c6e9b Compare July 22, 2025 01:48
@projectgus projectgus changed the title py/objint_longlong: Fix left shift of negative values, add sanitizer to CI build. py/objint_longlong: Fix left shift of negative values. Jul 22, 2025
@projectgus
Copy link
Contributor Author

Have pulled out the CI change to its own PR.

|| lhs_val > (LLONG_MAX >> rhs_val)
|| lhs_val < (LLONG_MIN >> rhs_val);
if (!overflow) {
result = (unsigned long long)lhs_val << rhs_val;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need a cast to unsigned? lhs_val can still be negative here, but that should be OK.

int("0x10000000000000000", 16)
except:
print("SKIP") # No support for >64-bit integers
raise SystemExit
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this, could rename the test to end with _intbig.py. Eg following tests/micropython/viper_misc_intbig.py and tests/micropython/viper_const_intbig.py etc.

@yoctopuce
Copy link
Contributor

yoctopuce commented Jul 22, 2025

Another option to fix the overflow check would be:

if ((int)rhs_val > 62) {
    overflow = 1;
} else {
    rhs_val = 1ll << (int)rhs_val;
    overflow = mp_mul_ll_overflow(lhs_val, rhs_val, &result);                
}

I am not sure which one would generate the shortest code, given that the compiler might be able to merge the call to mp_mul_ll_overflow with the code in MP_BINARY_OP_MULTIPLY

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
py-core Relates to py/ directory in source
Projects
None yet
Development

Successfully merging this pull request may close these issues.

-fsanitizer=undefined diagnostics with longlong
3 participants








ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: http://github.com/micropython/micropython/pull/17734

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy