Skip to content

gh-132661: Disallow Template/str concatenation after PEP 750 spec update #135996

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 34 commits into
base: main
Choose a base branch
from

Conversation

davepeck
Copy link
Contributor

@davepeck davepeck commented Jun 26, 2025

Following the steering council decision and corresponding update to PEP750, we are removing support for both implicit and explicit Template/str concatenation.

Copy link
Member

@sobolevn sobolevn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@lysnikolaou
Copy link
Member

Happy to do this with your input, or happy to hand it off to you!

Happy to help out / review this PR if you wanna take a stab at it. Feel free to reach out if you have any specific questions.

@davepeck davepeck marked this pull request as ready for review July 8, 2025 23:27
@davepeck
Copy link
Contributor Author

davepeck commented Jul 8, 2025

@lysnikolaou Think this is ready. The only question for me is whether we want to go further with _PyPegen_concatenate_strings().

Copy link
Member

@sobolevn sobolevn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Several small nitpicks :)

Copy link
Member

@lysnikolaou lysnikolaou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @davepeck for working on this! Looks great in general. Left a few unimportant comments and one more significant one about how to implement this in the parser. Also, we'll need to change ast_unparse.c around https://github.com/python/cpython/blob/main/Python/ast_unparse.c#L715.

davepeck and others added 5 commits July 9, 2025 15:16
…e-132661.34ftJl.rst

Co-authored-by: sobolevn <mail@sobolevn.me>
Squashed commit of the following:

commit 0941a3088578c1a8002e58ea6e11527bedf0e81e
Author: Dave <davepeck@gmail.com>
Date:   Thu Jul 10 01:49:24 2025 +0000

    Maybe cleaner?

commit e5a69c851a0f43a4e84325051a37de20282dd810
Author: Dave <davepeck@gmail.com>
Date:   Wed Jul 9 23:17:58 2025 +0000

    In progress: update grammar
@pablogsal
Copy link
Member

Screenshot 2025-07-10 at 16 31 27

The universe is at peace :)

@davepeck
Copy link
Contributor Author

davepeck commented Jul 10, 2025

The universe is at peace :)

Getting closer:
Screenshot 2025-07-10 at 9 20 46 AM

I will not rest until it is perfectly at peace, even I if have to commit uncompilable atrocities. :-P

@davepeck
Copy link
Contributor Author

davepeck commented Jul 10, 2025

@lysnikolaou

Okay, I think this is ready! Updated the grammar and actions; fixed up ast_unparse.c.

Two questions:

  1. I suppose we could break _build_concatenated_str() in action_helpers.c into two separate routines. But given how much duplicated logic there would be around flattening/concatenating/re-folding, I decided to leave it untouched. Let me know if you'd like me to tease this apart further.

  2. Because t-string/non-t-string concat errors are handled in the grammar, but bytes/nonbytes are handled in action_helpers.c, t"foo" b"bar" results in the syntax error "cannot mix t-strings with strings or f-strings". Are we okay with that?

(PS: dear Bedevere: I have made the requested changes; please review again)

@bedevere-app
Copy link

bedevere-app bot commented Jul 10, 2025

Thanks for making the requested changes!

@lysnikolaou, @JelleZijlstra: please review the changes made to this pull request.

Copy link
Member

@lysnikolaou lysnikolaou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks even better this time. Thanks @davepeck! Left a few comments inline again. It'll probably be good to go once these are addressed.

Regarding your questions:

  1. I suppose we could break _build_concatenated_str() in action_helpers.c into two separate routines. But given how much duplicated logic there would be around flattening/concatenating/re-folding, I decided to leave it untouched. Let me know if you'd like me to tease this apart further.

Probably okay to leave _build_concatenated_str as-is. There would be too much duplication for us to break it into two separate functions, and even adding more asserts would make things more complicated IMO.

  1. Because t-string/non-t-string concat errors are handled in the grammar, but bytes/nonbytes are handled in action_helpers.c, t"foo" b"bar" results in the syntax error "cannot mix t-strings with strings or f-strings". Are we okay with that?

Addressed inline.

dylwil3 added a commit to astral-sh/ruff that referenced this pull request Jul 14, 2025
I used a script to attempt to identify those rules with the following
property: changing f-strings to t-strings in the corresponding fixture
altered the number of lint errors emitted. In other words, those rules
for which f-strings and t-strings are not treated the same in the
current implementation.

This PR documents the subset of such rules where this is fine and no
changes need to be made to the implementation of the rule. Mostly these
are the rules where it is relevant that an f-string evaluates to type
`str` at runtime whereas t-strings do not.

In theory many of these fixtures are not super necessary - it's unlikely
t-strings would be used for most of these. However, the internal
handling of t-strings is tightly coupled with that of f-strings, and may
become even more so as we implement the upcoming changes due to
python/cpython#135996 . So I'd like to keep
these around as regression tests.

Note: The `flake8-bandit` fixtures were already added during the
original t-string implementation.

| Rule(s) | Reason |
| --- | --- |
| [`unused-method-argument`
(`ARG002`)](https://docs.astral.sh/ruff/rules/unused-method-argument/#unused-method-argument-arg002)
| f-strings exempted for msg in `NotImplementedError` not relevant for
t-strings |
| [`logging-f-string`
(`G004`)](https://docs.astral.sh/ruff/rules/logging-f-string/#logging-f-string-g004)
| t-strings cannot be used here |
| [`f-string-in-get-text-func-call`
(`INT001`)](https://docs.astral.sh/ruff/rules/f-string-in-get-text-func-call/#f-string-in-get-text-func-call-int001)
| rule justified by eager evaluation of interpolations |
| [`flake8-bandit`](https://docs.astral.sh/ruff/rules/#flake8-bandit-s)|
rules justified by eager evaluation of interpolations |
| [`single-string-slots`
(`PLC0205`)](https://docs.astral.sh/ruff/rules/single-string-slots/#single-string-slots-plc0205)
| t-strings cannot be slots in general |
| [`unnecessary-encode-utf8`
(`UP012`)](https://docs.astral.sh/ruff/rules/unnecessary-encode-utf8/#unnecessary-encode-utf8-up012)
| cannot encode t-strings |
| [`no-self-use`
(`PLR6301`)](https://docs.astral.sh/ruff/rules/no-self-use/#no-self-use-plr6301)
| f-strings exempted for msg in NotImplementedError not relevant for
t-strings |
| [`pytest-raises-too-broad`
(`PT011`)](https://docs.astral.sh/ruff/rules/pytest-raises-too-broad/) /
[`pytest-fail-without-message`
(`PT016`)](https://docs.astral.sh/ruff/rules/pytest-fail-without-message/#pytest-fail-without-message-pt016)
/ [`pytest-warns-too-broad`
(`PT030`)](https://docs.astral.sh/ruff/rules/pytest-warns-too-broad/#pytest-warns-too-broad-pt030)
| t-strings cannot be empty or used as messages |
| [`assert-on-string-literal`
(`PLW0129`)](https://docs.astral.sh/ruff/rules/assert-on-string-literal/#assert-on-string-literal-plw0129)
| t-strings are not strings and cannot be empty |
| [`native-literals`
(`UP018`)](https://docs.astral.sh/ruff/rules/native-literals/#native-literals-up018)
| t-strings are not native literals |
@davepeck
Copy link
Contributor Author

davepeck commented Jul 15, 2025

Looks even better this time. Thanks @davepeck! Left a few comments inline again. It'll probably be good to go once these are addressed.

👍 Thanks, @lysnikolaou for all your patience and help here -- think I took care of the rest.

Comment on lines +163 to +165
expected_msg = 'can only concatenate Template (not "str") to Template'
with self.assertRaises(TypeError, msg=expected_msg):
t1 + ", world"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
expected_msg = 'can only concatenate Template (not "str") to Template'
with self.assertRaises(TypeError, msg=expected_msg):
t1 + ", world"
expected_msg = 'can only concatenate string.templatelib.Template ' \
'\\(not "str"\\) to string.templatelib.Template'
with self.assertRaisesRegex(TypeError, expected_msg):
t1 + ", world"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current self.assertRaises call just checks presence of TypeError, not its message.

Comment on lines +176 to +178
expected_msg = 'can only concatenate str (not "string.templatelib.Template") to str'
with self.assertRaises(TypeError, msg=expected_msg):
"Hello, " + t"{name}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
expected_msg = 'can only concatenate str (not "string.templatelib.Template") to str'
with self.assertRaises(TypeError, msg=expected_msg):
"Hello, " + t"{name}"
expected_msg = 'can only concatenate str ' \
'\\(not "string.templatelib.Template"\\) to str'
with self.assertRaisesRegex(TypeError, expected_msg):
"Hello, " + t"{name}"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy