Closed
Description
Bug report
Bug description:
Environments:
macOS 15.2
Homebrew 4.4.11
Homebrew clang version 19.1.5
Target: arm64-apple-darwin24.2.0
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/llvm/19.1.5/bin
Configuration file: /opt/homebrew/etc/clang/arm64-apple-darwin24.cfg
Ubuntu 22.04.3 LTS
Codename: jammy
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Address sanitizer report
==114762==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x616000005982 at pc 0x5634dd2183f6 bp 0x7fff2ea0d040 sp 0x7fff2ea0d030
READ of size 2 at 0x616000005982 thread T0
#0 0x5634dd2183f5 in ucs2lib_default_find Objects/stringlib/fastsearch.h:600
#1 0x5634dd2183f5 in ucs2lib_fastsearch Objects/stringlib/fastsearch.h:775
#2 0x5634dd2183f5 in ucs2lib_find Objects/stringlib/find.h:18
#3 0x5634dd2183f5 in ucs2lib_find Objects/stringlib/find.h:8
#4 0x5634dd2183f5 in anylib_find Objects/unicodeobject.c:10226
#5 0x5634dd23d120 in replace Objects/unicodeobject.c:10384
#6 0x5634dd0b49b2 in method_vectorcall_FASTCALL Objects/descrobject.c:408
#7 0x5634dd08d80f in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92
#8 0x5634dd08d80f in PyObject_Vectorcall Objects/call.c:325
#9 0x5634dcf3e5ba in _PyEval_EvalFrameDefault Python/bytecodes.c:2715
#10 0x5634dd345626 in _PyEval_EvalFrame Include/internal/pycore_ceval.h:89
#11 0x5634dd345626 in _PyEval_Vector Python/ceval.c:1683
#12 0x5634dd345626 in PyEval_EvalCode Python/ceval.c:578
#13 0x5634dd4451a2 in run_eval_code_obj Python/pythonrun.c:1716
#14 0x5634dd44535c in run_mod Python/pythonrun.c:1737
#15 0x5634dd44ba83 in pyrun_file Python/pythonrun.c:1637
#16 0x5634dd44ba83 in _PyRun_SimpleFileObject Python/pythonrun.c:433
#17 0x5634dd44c39b in _PyRun_AnyFileObject Python/pythonrun.c:78
#18 0x5634dd4b3223 in pymain_run_file_obj Modules/main.c:360
#19 0x5634dd4b3223 in pymain_run_file Modules/main.c:379
#20 0x5634dd4b3223 in pymain_run_python Modules/main.c:633
#21 0x5634dd4b482f in Py_RunMain Modules/main.c:713
#22 0x5634dd4b482f in pymain_main Modules/main.c:743
#23 0x5634dd4b482f in Py_BytesMain Modules/main.c:767
#24 0x7f8c83883d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
#25 0x7f8c83883e3f in __libc_start_main_impl ../csu/libc-start.c:392
#26 0x5634dcf6c5e4 in _start (/space/src/python-heap-buffer-overflow/cpython/python+0x2ed5e4)
0x616000005982 is located 0 bytes to the right of 514-byte region [0x616000005780,0x616000005982)
allocated by thread T0 here:
#0 0x7f8c83c1e887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
#1 0x5634dd2002d1 in unicode_askind Objects/unicodeobject.c:2401
#2 0x5634dd23ceeb in replace Objects/unicodeobject.c:10363
#3 0x5634dd0b49b2 in method_vectorcall_FASTCALL Objects/descrobject.c:408
#4 0x5634dd08d80f in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92
#5 0x5634dd08d80f in PyObject_Vectorcall Objects/call.c:325
#6 0x5634dcf3e5ba in _PyEval_EvalFrameDefault Python/bytecodes.c:2715
#7 0x5634dd345626 in _PyEval_EvalFrame Include/internal/pycore_ceval.h:89
#8 0x5634dd345626 in _PyEval_Vector Python/ceval.c:1683
#9 0x5634dd345626 in PyEval_EvalCode Python/ceval.c:578
#10 0x5634dd4451a2 in run_eval_code_obj Python/pythonrun.c:1716
#11 0x5634dd44535c in run_mod Python/pythonrun.c:1737
#12 0x5634dd44ba83 in pyrun_file Python/pythonrun.c:1637
#13 0x5634dd44ba83 in _PyRun_SimpleFileObject Python/pythonrun.c:433
#14 0x5634dd44c39b in _PyRun_AnyFileObject Python/pythonrun.c:78
#15 0x5634dd4b3223 in pymain_run_file_obj Modules/main.c:360
#16 0x5634dd4b3223 in pymain_run_file Modules/main.c:379
#17 0x5634dd4b3223 in pymain_run_python Modules/main.c:633
#18 0x5634dd4b482f in Py_RunMain Modules/main.c:713
#19 0x5634dd4b482f in pymain_main Modules/main.c:743
#20 0x5634dd4b482f in Py_BytesMain Modules/main.c:767
#21 0x7f8c83883d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
SUMMARY: AddressSanitizer: heap-buffer-overflow Objects/stringlib/fastsearch.h:600 in ucs2lib_default_find
Shadow bytes around the buggy address:
0x0c2c7fff8ae0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c2c7fff8af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c2c7fff8b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c2c7fff8b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c2c7fff8b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c2c7fff8b30:[02]fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c2c7fff8b40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c2c7fff8b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c2c7fff8b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c2c7fff8b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c2c7fff8b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==114762==ABORTING
Python code to reproduce
# reproduce.py
any_three_nonblank_codepoints = '!!!'
seven_codepoints = any_three_nonblank_codepoints + ' ' + any_three_nonblank_codepoints
a = (' ' * 243) + seven_codepoints + (' ' * 7)
#b = ' ' * 6 + chr(0) # OK
#b = ' ' * 6 + chr(255) # OK
b = ' ' * 6 + chr(256) # heap-buffer-overflow
a.replace(seven_codepoints, b)Shell script to git clone CPython, build, and elicit the warning from ASan:
#!/bin/bash
git clone -b 3.12 https://github.com/python/cpython.git
cd cpython
./configure --with-address-sanitizer
make
cd ..
if [[ -x "cpython/python" ]]; then
cpython/python reproduce.py
elif [[ -x "cpython/python.exe" ]]; then
cpython/python.exe reproduce.py
else
echo "error: no CPython binary"
exit 1
fiVerified to happen under 3.12 and 3.13; I did not try any earlier Python version.
CPython versions tested on:
3.12, 3.13
Operating systems tested on:
Linux, macOS
Linked PRs
- gh-127971: fix off-by-one read beyond the end of a string during search #132574
- [3.14] gh-127971: fix off-by-one read beyond the end of a string during search (GH-132574) #136628
- [3.13] gh-127971: fix off-by-one read beyond the end of a string during search (#132574) #136645
- [3.13] gh-127971: fix off-by-one read beyond the end of a string during search (#132574) #136648