[Python-Dev] PEP 554 v3 (new interpreters module)

[Nick Coghlan]

On 14 September 2017 at 11:44, Eric Snow <ericsnowcurrently at gmail.com> wrote:
> About Subinterpreters
> =====================
>
> Shared data
> -----------

[snip]

> To make this work, the mutable shared state will be managed by the
> Python runtime, not by any of the interpreters.  Initially we will
> support only one type of objects for shared state: the channels provided
> by ``create_channel()``.  Channels, in turn, will carefully manage
> passing objects between interpreters.

Something I think you may want to explicitly call out as *not* being
shared is the thread objects in threading.enumerate(), as the way that
works in the current implementation makes sense, but isn't
particularly obvious (what I have below comes from experimenting with
your branch at https://github.com/python/cpython/pull/1748).

Specifically, what happens is that the operating system thread
underlying the existing interpreter thread that calls interp.run()
gets borrowed as the operating system thread underlying the MainThread
object in the called interpreter. That MainThread object then gets
preserved in the interpreter's interpreter state, but the mapping to
an underlying OS thread will change freely based on who's calling into
it. From outside an interpreter, you *can't* request to run code in
subthreads directly - you'll always run your given code in the main
thread, and it will be up to that to dispatch requests to subthreads.

Beyond the thread lending that happens when you call interp.run()
(where one of your threads gets borrowed as the other interpreter's
main thread), each interpreter otherwise maintains a completely
disjoint set of thread objects that it is solely responsible for.

This also clarifies for me what it means for an interpreter to be a
"main" interpreter: it's the interpreter who's main thread actually
corresponds to the main thread of the overall operating system
process, rather than being temporarily borrowed from another
interpreter.

We're going to have to put some thought into how we want that to
interact with the signal handling logic - right now, I believe *any*
main thread will consider it its responsibility to process signals
delivered to the runtime (and embedding application avoid the
potential problems arising from that by simply not installing the
CPython signal handlers in the first place), and we probably want to
change that condition to be "the main thread in the main interpreter".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia