gh-85222: Document the side effect in multiprocessing #136426

aisk · 2025-07-08T16:13:29Z

Issue: class multiprocessing.Value calls set_start_method #85222

📚 Documentation preview 📚: https://cpython-previews--136426.org.readthedocs.build/

LamentXU123 · 2025-07-08T17:23:28Z

Doc/library/multiprocessing.rst

@@ -867,6 +867,10 @@ For an example of the usage of queues for interprocess communication see
   locks/semaphores.  When a process first puts an item on the queue a feeder
   thread is started which transfers objects from a buffer into the pipe.

+   If the global start method has not been set, calling this function will
+   have the side effect of setting the current global start method.
+   See the :func:`get_context` function.


I think repeating the same words here may be a bit 'noisy'. Could we just mention it once?

My concern is that if a user only uses one of these functions and checks that function's documentation, and unfortunately that is not the function that contains this document, they may mess it up.

But I'm unsure about this.

I may be biased but I don't think the extra clarity hurts... I'm with @aisk

yeah, I asked for this to be mentioned everywhere for that reason as this is non-obvious behavior of multiprocessing. This could probably be refined a bit, i'll point some docs focused reviewers at it.

My concern is that if a user only uses one of these functions and checks that function's documentation, and unfortunately that is not the function that contains this document, they may mess it up.

But I'm unsure about this.

IMO it may be better if we mentioned it once and create a reference link to the mention in every function. I'm not sure too (｡･ω･｡)

The repeated part could be a single sentence like Ensures that the current global start method is set., with ”global start method” linking to a longer explanation in a dedicated section.

I'm not sure it needs to be mentioned in every method. There's a section at the top about contexts that can say the global start method is set by any function that does real work (or some appropriate language). I find we often have overarching concerns that apply throughout a module and don't repeat them. Sometime people have to read widely in a module page in order to understand all the nuances.

Maybe I missed the discussion: why is this particular caveat important enough to sprinkle everywhere? What footgun are we preventing?

The callout about the surprising behavior is also (in addition to the issue for this PR) due to #109070 which only just added the caveat about .get_context() having the setting side effect behavior via https://github.com/python/cpython/pull/136341/files.

I agree that the https://docs.python.org/3.15/library/multiprocessing.html#contexts-and-start-methods section up top should be more clear about this. It really only ever mentions get_context() as an alternative today and never highlights that the context is implicitly set at instantiation time by all sorts of APIs... We do say To select a start method you use the set_start_method() in the if __name__ == '__main__' clause of the main module. but we never actually explain the restrictions leading to that advice, the why, there... That would help.

After that these mentions about context setting could turn into a single sentence similar to what encoku suggested that ref's back to that section.

(the other part of this PR is documenting which things accept a ctx= parameter at all)

Hi, thank you all for review this PR, I updated the document and added a new section to explain the implicitly set of start method, and changed other methods/types to link to this section. Please review it to see if this is suitable.

Doc/library/multiprocessing.rst

sharktide

I noticed we say

Calling this may set the global start method. See
:ref:`global-start-method` for more details.

And we say this nearly all the time

But wouldn't it make more sense to say the following in some cases as we are dealing with classes ↓

Instantiating this class may set the global start method. See
:ref:`global-start-method` for more details.

Anyway that's what I think... feel free to correct me if I'm wrong :)

sharktide

I probably should have mentioned this before but I forgot.

As of now we say something like this

Instantiating this class may set the global start method. See
:ref:`global-start-method` for more details.

But if you prefer we could also say something like this:

Instantiating this class may override the global start method. See
:ref:`global-start-method` for more details

Set --> override

But again up to you

gpshead · 2025-07-11T16:35:58Z

Set --> override

I wouldn't use the word override for this behavior. If the global start method has already been set, it will be used. It isn't changed at that point. it's just that it is a set-once before first use global. So if it has not yet been explicitly set, it will be set to the default by the APIs doing this as having the right mp context is required.

sharktide · 2025-07-11T16:39:12Z

@gpshead Do you think that then maybe saying "either set or alter" instead is better?

Something like

... this class may result in either setting or altering the global start method see ...

Doc/library/multiprocessing.rst

willingc · 2025-07-12T00:19:06Z

Do you think that then maybe saying "either set or alter" instead is better?

@sharktide I believe set is the correct term since you are not able to alter after it is set.

I believe the requested change has been addressed.

sharktide

Small suggestion, this probably doesn't matter too much
Around line 1142, we say

If *method* is ``None`` then the default context is returned. Note that if the global start method has not been set, this will set it.
See :ref:`global-start-method` for more details.

Earlier it used to say:

 If *method* is ``None`` then the default context is returned. Note that if
 the global start method has not been set, this will set it to the
 default method.

Just asking, why did we remove this will set it to the default method and instead just say this will set it.?

I think it might be a little better to say

If *method* is ``None`` then the default context is returned. Note that if the global start method has not been set, this will set it to the default method
See :ref:`global-start-method` for more details.

But this doesn't really matter. Up to you all if you think this is worth it.

willingc · 2025-07-12T04:53:20Z

Small suggestion, this probably doesn't matter too much Around line 1142, we say
If *method* is ``None`` then the default context is returned. Note that if the global start method has not been set, this will set it.
See :ref:`global-start-method` for more details.
Earlier it used to say:
 If *method* is ``None`` then the default context is returned. Note that if
 the global start method has not been set, this will set it to the
 default method.
Just asking, why did we remove this will set it to the default method and instead just say this will set it.?

I think it might be a little better to say
If *method* is ``None`` then the default context is returned. Note that if the global start method has not been set, this will set it to the default method
See :ref:`global-start-method` for more details.
But this doesn't really matter. Up to you all if you think this is worth it.

@aisk Thanks for your work on this PR. 🎉 I'm fine with any of the above options.

@sharktide Thanks for taking the time to review and make suggestions. Usually, we will only mark a review "required changes" if there is technically something wrong and it is important to change the PR. For things that are suggestions, we typically use the comment function in the review. Keep up the good work. ☀️

The latest review comment is a suggested change so the requested change is not necessary.

sharktide · 2025-07-12T04:54:47Z

Noted. Thanks!

Co-authored-by: Carol Willing <carolcode@willingconsulting.com>

sharktide

Small grammatical error

The global start method can only be done once. If you need to change the
start method from the system default, you must proactively set the global start method
before calling functions or methods, or creating these objects.

it should instead say Setting the global start method… because without the word Setting the sentence doesn’t make a lot of sense if you think about it. Sorry for being nitty :)

aisk · 2025-07-12T11:47:40Z

Doc/library/multiprocessing.rst

+
+Several multiprocessing functions and methods, as well as creating some objects, will implicitly
+set the global start method to the system's default, if the global start method is not already
+set. The global start method can only be done once. If you need to change the


Suggested change

set. The global start method can only be done once. If you need to change the

set. The global start method can only be set once. If you need to change the

@sharktide Hi, how about this?

sure !

EDIT: my other suggestion includes this

sharktide

This time I left a bunch of suggestions at once so I don't bother anyone too much :)

sharktide · 2025-07-12T11:50:10Z

Doc/library/multiprocessing.rst

+
+Several multiprocessing functions and methods, as well as creating some objects, will implicitly
+set the global start method to the system's default, if the global start method is not already
+set. The global start method can only be done once. If you need to change the


sure !

EDIT: my other suggestion includes this

Doc/library/multiprocessing.rst

sharktide · 2025-07-12T13:00:56Z

Doc/library/multiprocessing.rst

@@ -2313,7 +2367,9 @@ with the :class:`Pool` class.
   the worker processes.  Usually a pool is created using the
   function :func:`multiprocessing.Pool` or the :meth:`Pool` method
   of a context object.  In both cases *context* is set
-   appropriately.
+   appropriately. If ``None``, calling this function will have the side effect
+   of setting the current global start method if it has not been set already.


Suggested change

of setting the current global start method if it has not been set already.

of setting the current global start method to the

system default if it has not been set already.

More small suggestions to improve reading flow

Up to you if you wanna take this one

Doc/library/multiprocessing.rst

Co-authored-by: R Chintan Meher <meherrihaan@gmail.com>

sharktide · 2025-07-12T13:28:24Z

Ahh linter… I’ll give you a fix for that

sharktide

This should make the linter happy

Doc/library/multiprocessing.rst

Co-authored-by: R Chintan Meher <meherrihaan@gmail.com>

sharktide

I just had a thought

Where we say

Instantiating this class may set the global start method. See…

We could explicitly tell the user that it’ll be set to the system default for 2 reasons:

Reading flow —> the period seems a bit abrupt

Clarity: people may not want to click the link for more info; this would give them a basic overview for most scenarios so as not to interrupt reading in cases that this doesn’t really matter

So maybe something like


Instantiating this class may set the global start method to the system default. See …

@aisk feel free to adapt this if you wish

aisk · 2025-07-12T16:23:40Z

Hi @sharktide, thank you for your review. I think we shouldn't mention too many details in every function's documentation. I think users can guess that the start method will be set to the system's default, or if they are confused, they can go to the link to check the details.

sharktide · 2025-07-12T17:00:20Z

All right
I'll finish reviewing during tmrw (I live in USA but im visiting family in another time zone that's why I'm up in the American night. Silence your phones if you live in the US :)

sharktide · 2025-07-12T17:00:40Z

Hi @sharktide, thank you for your review. I think we shouldn't mention too many details in every function's documentation. I think users can guess that the start method will be set to the system's default, or if they are confused, they can go to the link to check the details.

Fine by me! Thanks for the clarification

Document the ctx parameter in some types in multiprocessing

646225c

aisk requested a review from gpshead as a code owner July 8, 2025 16:13

bedevere-app bot added awaiting review docs Documentation in the Doc dir skip news labels Jul 8, 2025

bedevere-app bot mentioned this pull request Jul 8, 2025

class multiprocessing.Value calls set_start_method #85222

Open

github-project-automation bot added this to Docs PRs Jul 8, 2025

github-project-automation bot moved this to Todo in Docs PRs Jul 8, 2025

aisk added 3 commits July 9, 2025 01:30

Update side effect document

1b7a0ea

Update document

29b5cf3

Fix marker

7962657

LamentXU123 reviewed Jul 8, 2025

View reviewed changes

gpshead self-assigned this Jul 10, 2025

StanFromIreland reviewed Jul 10, 2025

View reviewed changes

Doc/library/multiprocessing.rst Outdated Show resolved Hide resolved

Doc/library/multiprocessing.rst Show resolved Hide resolved

encukou reviewed Jul 10, 2025

View reviewed changes

Doc/library/multiprocessing.rst Outdated Show resolved Hide resolved

aisk added 2 commits July 10, 2025 23:11

argument -> parameters

ed60127

Reduce duplication

881a347

sharktide previously requested changes Jul 10, 2025

View reviewed changes

bedevere-app bot added awaiting core review and removed awaiting review labels Jul 10, 2025

calling -> instantiating

de61926

sharktide reviewed Jul 11, 2025

View reviewed changes

willingc reviewed Jul 12, 2025

View reviewed changes

Doc/library/multiprocessing.rst Outdated Show resolved Hide resolved

bedevere-app bot added awaiting review and removed awaiting core review labels Jul 12, 2025

sharktide previously requested changes Jul 12, 2025

View reviewed changes

bedevere-app bot added awaiting core review and removed awaiting review labels Jul 12, 2025

bedevere-app bot added awaiting review and removed awaiting core review labels Jul 12, 2025

Update Doc/library/multiprocessing.rst

982c7c2

Co-authored-by: Carol Willing <carolcode@willingconsulting.com>

sharktide reviewed Jul 12, 2025

View reviewed changes

aisk commented Jul 12, 2025

View reviewed changes

sharktide reviewed Jul 12, 2025

View reviewed changes

Apply suggestions from code review

c85562f

Co-authored-by: R Chintan Meher <meherrihaan@gmail.com>

sharktide reviewed Jul 12, 2025

View reviewed changes

Doc/library/multiprocessing.rst Outdated Show resolved Hide resolved

aisk and others added 2 commits July 12, 2025 22:34

Update Doc/library/multiprocessing.rst

8a9ae53

Co-authored-by: R Chintan Meher <meherrihaan@gmail.com>

Fix line-break

a62a4fe

sharktide reviewed Jul 12, 2025

View reviewed changes

	set. The global start method can only be done once. If you need to change the
	set. The global start method can only be set once. If you need to change the

	of setting the current global start method if it has not been set already.
	of setting the current global start method to the
	system default if it has not been set already.

Uh oh!

gh-85222: Document the side effect in multiprocessing #136426

Are you sure you want to change the base?

gh-85222: Document the side effect in multiprocessing #136426

Conversation

aisk commented Jul 8, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as off-topic.

Uh oh!

LamentXU123 Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sharktide left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sharktide left a comment

Choose a reason for hiding this comment

Uh oh!

gpshead commented Jul 11, 2025

Uh oh!

sharktide commented Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

willingc commented Jul 12, 2025

Uh oh!

sharktide left a comment

Choose a reason for hiding this comment

Uh oh!

willingc commented Jul 12, 2025

Uh oh!

sharktide commented Jul 12, 2025

Uh oh!

sharktide left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sharktide left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sharktide commented Jul 12, 2025

Uh oh!

sharktide left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sharktide left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

aisk commented Jul 8, 2025 •

edited by github-actions bot

Loading

LamentXU123 Jul 10, 2025 •

edited

Loading

sharktide left a comment •

edited

Loading

sharktide commented Jul 11, 2025 •

edited

Loading

sharktide left a comment •

edited

Loading

sharktide left a comment •

edited

Loading