Skip to content

[mypyc] Speed up for loop over native generator #19415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 9, 2025

Conversation

JukkaL
Copy link
Collaborator

@JukkaL JukkaL commented Jul 9, 2025

Call the generator helper method directly instead of calling PyIter_Next when calling a native generator from a native function. This way we can avoid raising StopIteration when the generator is exhausted. The approach is similar to what I used to speed up calls using await in #19398. Refer to that PR for a more detailed explanation.

This helps mostly when a generator produces a small number of values, which is quite common.

This PR improves the performance of this microbenchmark, which is a close to the ideal use case, by about 2.6x (now 5.7x faster than interpreted):

from typing import Iterator

def foo(x: int) -> Iterator[int]:
    for a in range(x):
        yield a

def bench(n: int) -> None:
    for i in range(n):
        for a in foo(1):
            pass

from time import time
bench(1000 * 1000)
t0 = time()
bench(50 * 1000 * 1000)
print(time() - t0)

@JukkaL JukkaL merged commit 6f23e47 into master Jul 9, 2025
13 checks passed
@JukkaL JukkaL deleted the mypyc-generator-optimize branch July 9, 2025 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy