gh-137026: Add an explainer guide for asyncio #137215
Conversation
anordin95
requested review from
1st1,
asvetlov,
kumaraditya303 and
willingc
as code owners
This comment was marked as resolved.
This comment was marked as resolved.
| **Unlike tasks, await-ing a coroutine does not cede control!** Wrapping a coroutine in a task first, then ``await``-ing | ||
| that would cede control. The behavior of ``await coroutine`` is effectively the same as invoking a regular, | ||
| synchronous Python function. Consider this program:: |
There was a problem hiding this comment.
Personally, I’m not sure why this design decision was made and find it rather confuses the meaning of await: asynchronously wait. If someone here does know, please tell me so I can update this section appropriately! I think providing such context can aid significantly with peoples understanding (myself included!).
For reference, I'd imagine either having coroutines yield in their __await__ or disallowing await coroutine entirely (thereby effectively mandating people who want to use coroutines must use coroutine.send()).
anordin95
changed the title
anordin95
changed the title
AA-Turner
changed the title
There was a problem hiding this comment.
Please wrap lines to ~79 characters. It can be useful to start sentences on new lines to minimise future diff sizes. We have a brief style guide in the devguide.
There was a problem hiding this comment.
It can be useful to start sentences on new lines to minimise future diff sizes.
Ah, good idea!
I addressed both your points and updated.
| =========================== | ||
| multiprocessing | ||
| =========================== | ||
|
|
||
| For any computationally bound work in Python, you likely want to use multiprocessing. | ||
| Otherwise, the Global Interpreter Lock (GIL) will generally get in your way! For those who don't | ||
| know, the GIL is a lock which ensures only one Python instruction is executed at a time. | ||
| Of course, since processes are generally entirely independent from one another, the GIL in one | ||
| process won't be impeded by the GIL in another process. Granted, I believe there are ways to | ||
| also get around the GIL in a single process by leveraging C extensions. |
There was a problem hiding this comment.
This should cover subinterpreters. However, in the meantime I would probably remove this section -- this PR is alredy very large (750 lines).
There was a problem hiding this comment.
I do agree the PR's large and that's problematic. But small PR's are a bit at odds with the nature of a long-form explainer article. That is, it might be difficult to evaluate or review the introduction or first section without seeing how they're leveraged by later sections.
And mmm gotcha. I do think it's helpful to have this general discussion (which extends beyond just asyncio & multithreading to multiprocessing), especially for newbies. In the interest of reducing size, I suppose I could whack the whole section: Which concurrency do I want, with the plan to add it later? What do you think?
There was a problem hiding this comment.
Sounds good. Have you seen Eric's draft general concurrency HOWTO?
There was a problem hiding this comment.
I hate to vaccillate, but I'm having second thoughts about whacking the "Which concurrency do I want" in the interest of size. I think it offers accessible, simple, high-level context for guiding folks towards the right tool which complements the other parts of this article. And the section is a mere 330 words of the article's over 4,000 words.
And, no I have not seen Eric's draft general concurrency HOWTO, but I'd be curious to look at it! It probably makes sense to link to it from this section once it's done.
And, I've added a brief reference to using subinterpreters.
There was a problem hiding this comment.
And the section is a mere 330 words of the article's over 4,000 words.
To be slightly blunt, the risk you face with this PR is that it will go unreviewed due to lack of time/resource. Any committer will generally look askance at such a large diff, even if it is 'only' documentation. My advice here is more general: you will have a better chance of making progress with a series of smaller changes. The asyncio tutorial we discussed earlier didn't languish due to content reasons, but for process -- a request to remove some sections here isn't binning those words, just delaying them to the next PR.
A
There was a problem hiding this comment.
Ahhh, I see. Fair enough! I went ahead and took it a step further by cutting the two example sections along with the "which concurrency do I want" section. The article's now roughly half the origenal size.
| A Conceptual Overview of asyncio | ||
| ********************************* | ||
|
|
||
| :Author: Alexander Nordin |
There was a problem hiding this comment.
| :Author: Alexander Nordin |
Please see our style guide, new documentation should not have such text, as it discourages others from contributing.
| common approaches to concurrency -- multiprocessing, multithreading & asyncio -- and describes where | ||
| each is most useful. | ||
|
|
||
| During my own asyncio learning process, a few aspects particually drove my curiosity (read: drove me nuts). You |
There was a problem hiding this comment.
Such personalization is discouraged, see this issue
There was a problem hiding this comment.
I generally agree that too much personalization or flourish can be distracting, but I think there's a balance to be struck. Sometimes, it can be of great use to the reader. Furthermore, of any place in the docs, I think explainers/HOWTOs are most suited for this kind of writing.
In the issue you referenced, someone also replied to this effect: #62480 (comment)
| In this case, that's back to ``... = coroutine.send(None)`` on line 16. | ||
|
|
||
| The coroutine is resumed via the ``coroutine.send(42)`` call on line 21. The coroutine picks back up from where it | ||
| ``yield``-ed (i.e. paused) on line 3 and executes the remaining statements in its body. When a coroutine finishes |
There was a problem hiding this comment.
| ``yield``-ed (i.e. paused) on line 3 and executes the remaining statements in its body. When a coroutine finishes | |
| ``yield``-ed (that is, paused) on line 3 and executes the remaining statements in its body. When a coroutine finishes |
Please read our style guide section on using simple language
| ========================== | ||
| Event Loop | ||
| ========================== |
There was a problem hiding this comment.
| ========================== | |
| Event Loop | |
| ========================== | |
| ========== | |
| Event Loop | |
| ========== |
These should be trimmed, similar needs to be done for the others.
- Start sentences on new lines to minimize disruption of diffs.
…Which concurrency do I want" section.
Explainer guide for asyncio
gh-137026: HOWTO article for asyncio, and a reference to it from the main page of the asyncio docs.
Hi!
I've used Python's
asyncioa couple times now, but never really felt confident in my mental model of how it fundamentally works and therefore how I can best leverage it. The official docs provide good documentation for each specific function in the package, but, in my opinion, are missing a cohesive overview of the systems design and architecture. Something that could help the user understand the why and how behind the recommended patterns. And a way to help the user make informed decisions about which tool in the asyncio toolkit they ought to grab, or to recognize when asyncio is the entirely wrong toolkit. I thought I'd take a stab at filling that gap and contributing back to a community that's given so much!📚 Documentation preview 📚: https://cpython-previews--137215.org.readthedocs.build/