gh-135700: Fix instructions in __annotate__ have incorrect code positions #136543
Conversation
…e-135700.0qdtCl.rst Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Frank Hoffmann <15r10nk@users.noreply.github.com>
| c | ||
| """ | ||
| result = self.run_pdb_script(script, commands) | ||
| self.assertNotIn("(1)__annotate__()", result[0]) |
There was a problem hiding this comment.
This test passes for me on main (without this PR).
There was a problem hiding this comment.
The problem is the indentation. @AndPuQing indented my code in the strings which (for some unknown reason for me) caused the test to pass. a textwrap.dedent should help.
There was a problem hiding this comment.
Yes, I fixed the indent problem, but it seems that the current fix also cannot pass the test.
======================================================================
FAIL: test_issue135700 (test.test_pdb.PdbTestCase.test_issue135700)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/mnt/data/xxx/dev/cpython/Lib/test/test_pdb.py", line 3842, in test_issue135700
self.assertNotIn("(1)__annotate__()", result[0])
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '(1)__annotate__()' unexpectedly found in '> ...../cpython/python -m dis testmod.py (base)
-- MAKE_CELL 0 (__conditional_annotations__)
0 RESUME 0
1 LOAD_CONST 2 (<code object __annotate__ at 0x798bd679cd50, file "testmod.py", line 1>)
MAKE_FUNCTION
STORE_NAME 2 (__annotate__)
BUILD_SET 0
STORE_NAME 0 (__conditional_annotations__)
3 LOAD_BUILD_CLASS
PUSH_NULL
LOAD_CONST 0 (<code object ClassVar at 0x798bd6bb1940, file "testmod.py", line 3>)
MAKE_FUNCTION
LOAD_CONST 1 ('ClassVar')
CALL 2
STORE_NAME 1 (ClassVar)
5 LOAD_NAME 0 (__conditional_annotations__)
LOAD_SMALL_INT 0
SET_ADD 1
POP_TOP
LOAD_CONST 3 (None)
RETURN_VALUE
Disassembly of <code object ClassVar at 0x798bd6bb1940, file "testmod.py", line 3>:
3 RESUME 0
LOAD_NAME 0 (__name__)
STORE_NAME 1 (__module__)
LOAD_CONST 0 ('ClassVar')
STORE_NAME 2 (__qualname__)
LOAD_SMALL_INT 3
STORE_NAME 3 (__firstlineno__)
4 LOAD_CONST 1 (())
STORE_NAME 4 (__static_attributes__)
LOAD_CONST 2 (None)
RETURN_VALUE
Disassembly of <code object __annotate__ at 0x798bd679cd50, file "testmod.py", line 1>:
1 RESUME 0
LOAD_FAST_BORROW 0 (format)
LOAD_SMALL_INT 2
COMPARE_OP 132 (>)
POP_JUMP_IF_FALSE 3 (to L1)
NOT_TAKEN
LOAD_COMMON_CONSTANT 1 (NotImplementedError)
RAISE_VARARGS 1
L1: BUILD_MAP 0
5 LOAD_SMALL_INT 0
LOAD_GLOBAL 0 (__conditional_annotations__)
CONTAINS_OP 0 (in)
POP_JUMP_IF_FALSE 10 (to L2)
NOT_TAKEN
LOAD_GLOBAL 2 (ClassVar)
COPY 2
LOAD_CONST 1 ('__dataclass_fields__')
1 STORE_SUBSCR
L2: RETURN_VALUEThere was a problem hiding this comment.
hmm, @AndPuQing your fix solved the problem which I have reported in the issue (the positions in the bytecode and column information) but not the problem with pdb because the line number of the synthetic code in __annotate__ is still 1. I don't know if there is a way to express an non existing code location (-1 maybe), but this is what would be needed here. Another solution might be to use the location of the first ast-node which is handled by the code (line 5 in this case). This would cause no line change when the code is executed. I thing this might be the most practical solution here.
There was a problem hiding this comment.
Why do we have a pdb test for it? From the original issue I don't see how pdb is involved. Will this problem affect pdb in some way? I don't think we should write a regression test in test_pdb if pdb is just a convenient tool to check a certain issue. If the issue breaks pdb, then that's fine.
To be honest I did not completely get the issue, but if the symptom is end_col_offset is wrong, how it impacts pdb?
There was a problem hiding this comment.
@gaogaotiantian I think this is the context you are missing #135814 (comment)
There was a problem hiding this comment.
Okay I think I got the background of this issue.
So this is introduced by PEP 649 right? We need an expert there. We need to define what's the correct line number for this. I don't believe the line number below is correct:
# <--- Please let me know if this position is weird or correct,
8 LOAD_CONST 2 (<code object __annotate__ at 0x7d090a29cd50, file "testmod.py", line 8>)
MAKE_FUNCTION
STORE_NAME 2 (__annotate__)
BUILD_SET 0
STORE_NAME 0 (__conditional_annotations__) This is some global preparation for the future lazy evaluation for annotation, so the line number should be 1. On the other hand, the __annotation__ code object should probably have a single line number which is the line number of the actual annotation.
We should mark the target then shoot on it, not trying to fix something and see if the result is acceptable. (That's why I think we should involve the experts on PEP 647, @larryhastings maybe?).
As for pdb, this test itself is a bit confusing - not unacceptable, but we can probably make it easier to avoid using a separate module. I believe class annotation has the similar effect to reproduce this.
Also, even though this messes up with pdb, fundamentally it's a code object issue. We should also have tests in test_dis or something.
There was a problem hiding this comment.
I feel this is more of a question for experts on tracing (you :D ). From the annotations perspective, the code here isn't directly connected to any code the user wrote; it seems reasonable to me to pretend it's on the first line of the module, or on the first annotation in the module, whatever works better for tools that use the data. But maybe Larry has a stronger opinion.
There was a problem hiding this comment.
Personally I think the best solution here is to set the line number to invalid so it won't trigger a line event at all - there's no corresponding source code for it. Any unexpected side effects for this solution @iritkatriel ?
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
cc @15r10nk @JelleZijlstra @iritkatriel