Skip to content

gh-135552: Make the GC clear weakrefs later. #136189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Ensure weakrefs with callbacks are cleared early.
We need to clear those before executing the callback.  Since this
ensures they can't run a second time, we don't need
_PyGC_SET_FINALIZED().  Revise comments.
  • Loading branch information
nascheme committed Jul 3, 2025
commit 123bc251b60de9029e653d6a30e8d917a399b8fa
48 changes: 23 additions & 25 deletions Python/gc.c
Original file line number Diff line number Diff line change
Expand Up @@ -901,23 +901,30 @@ handle_weakref_callbacks(PyGC_Head *unreachable, PyGC_Head *old)
for (PyWeakReference *wr = *wrlist; wr != NULL; wr = next_wr) {
next_wr = wr->wr_next;

// Since Python 2.3, weakrefs to cyclic garbage have been cleared
// *before* calling finalizers. However, since tp_subclasses
// started being necessary to invalidate caches (e.g. by
// PyType_Modified()), that clearing has created a bug. If the
// weakref to the subclass is cleared before a finalizer is
// called, the cache may not be correctly invalidated. That can
// lead to segfaults since the caches can refer to deallocated
// Weakrefs with callbacks always need to be cleared before
// executing the callback. Sometimes the callback will call
// the ref object, to check if it's actually a dead reference
// (KeyedRef does this, for example). We want to indicate that it
// is dead, even though it is possible a finalizer might resurrect
// it. Clearing also prevents the callback from being executing
// more than once.
//
// Since Python 2.3, all weakrefs to cyclic garbage have
// been cleared *before* calling finalizers. However, since
// tp_subclasses started being necessary to invalidate caches
// (e.g. by PyType_Modified()), that clearing has created a bug.
// If the weakref to the subclass is cleared before a finalizer
// is called, the cache may not be correctly invalidated. That
// can lead to segfaults since the caches can refer to deallocated
// objects. Delaying the clear of weakrefs until *after*
// finalizers have been called fixes that bug. However, that can
// introduce other problems since some finalizer code expects that
// the weakrefs will be cleared first. The "multiprocessing"
// package contains finalizer logic like this, for example. So,
// we have the PyType_Check() test above and only defer the clear
// of types. That solves the issue for tp_subclasses. In a
// future version of Python, we should likely defer the weakref
// clear for all objects, not just types.
if (!PyType_Check(wr->wr_object)) {
// finalizers have been called fixes that bug. However, that
// deferral could introduce other problems if some finalizer
// code expects that the weakrefs will be cleared first. So, we
// have the PyType_Check() test below to only defer the clear of
// weakrefs to types. That solves the issue for tp_subclasses.
// In a future version of Python, we should likely defer the
// weakref clear for all objects, not just types.
if (wr->wr_callback != NULL || !PyType_Check(wr->wr_object)) {
// _PyWeakref_ClearRef clears the weakref but leaves the
// callback pointer intact. Obscure: it also changes *wrlist.
_PyObject_ASSERT((PyObject *)wr, wr->wr_object == op);
Expand Down Expand Up @@ -962,11 +969,6 @@ handle_weakref_callbacks(PyGC_Head *unreachable, PyGC_Head *old)
continue;
}

if (_PyGC_FINALIZED((PyObject *)wr)) {
/* Callback was already run (weakref must have been resurrected). */
continue;
}

/* Create a new reference so that wr can't go away
* before we can process it again.
*/
Expand Down Expand Up @@ -995,10 +997,6 @@ handle_weakref_callbacks(PyGC_Head *unreachable, PyGC_Head *old)
callback = wr->wr_callback;
_PyObject_ASSERT(op, callback != NULL);

/* Ensure we don't execute the callback again if the weakref is
* resurrected. */
_PyGC_SET_FINALIZED(op);

/* copy-paste of weakrefobject.c's handle_callback() */
temp = PyObject_CallOneArg(callback, (PyObject *)wr);
if (temp == NULL) {
Expand Down
48 changes: 23 additions & 25 deletions Python/gc_free_threading.c
Original file line number Diff line number Diff line change
Expand Up @@ -1513,23 +1513,30 @@ find_weakref_callbacks(struct collection_state *state)
for (PyWeakReference *wr = *wrlist; wr != NULL; wr = next_wr) {
next_wr = wr->wr_next;

// Since Python 2.3, weakrefs to cyclic garbage have been cleared
// *before* calling finalizers. However, since tp_subclasses
// started being necessary to invalidate caches (e.g. by
// PyType_Modified()), that clearing has created a bug. If the
// weakref to the subclass is cleared before a finalizer is
// called, the cache may not be correctly invalidated. That can
// lead to segfaults since the caches can refer to deallocated
// Weakrefs with callbacks always need to be cleared before
// executing the callback. Sometimes the callback will call
// the ref object, to check if it's actually a dead reference
// (KeyedRef does this, for example). We want to indicate that it
// is dead, even though it is possible a finalizer might resurrect
// it. Clearing also prevents the callback from being executing
// more than once.
//
// Since Python 2.3, all weakrefs to cyclic garbage have
// been cleared *before* calling finalizers. However, since
// tp_subclasses started being necessary to invalidate caches
// (e.g. by PyType_Modified()), that clearing has created a bug.
// If the weakref to the subclass is cleared before a finalizer
// is called, the cache may not be correctly invalidated. That
// can lead to segfaults since the caches can refer to deallocated
// objects. Delaying the clear of weakrefs until *after*
// finalizers have been called fixes that bug. However, that can
// introduce other problems since some finalizer code expects that
// the weakrefs will be cleared first. The "multiprocessing"
// package contains finalizer logic like this, for example. So,
// we have the PyType_Check() test above and only defer the clear
// of types. That solves the issue for tp_subclasses. In a
// future version of Python, we should likely defer the weakref
// clear for all objects, not just types.
if (!PyType_Check(wr->wr_object)) {
// finalizers have been called fixes that bug. However, that
// deferral could introduce other problems if some finalizer
// code expects that the weakrefs will be cleared first. So, we
// have the PyType_Check() test below to only defer the clear of
// weakrefs to types. That solves the issue for tp_subclasses.
// In a future version of Python, we should likely defer the
// weakref clear for all objects, not just types.
if (wr->wr_callback != NULL || !PyType_Check(wr->wr_object)) {
// _PyWeakref_ClearRef clears the weakref but leaves the
// callback pointer intact. Obscure: it also changes *wrlist.
_PyObject_ASSERT((PyObject *)wr, wr->wr_object == op);
Expand All @@ -1546,11 +1553,6 @@ find_weakref_callbacks(struct collection_state *state)
continue;
}

if (_PyGC_FINALIZED((PyObject *)wr)) {
// Callback was already run (weakref must have been resurrected).
continue;
}

// Create a new reference so that wr can't go away before we can
// process it again.
merge_refcount((PyObject *)wr, 1);
Expand Down Expand Up @@ -1610,10 +1612,6 @@ call_weakref_callbacks(struct collection_state *state)
PyObject *callback = wr->wr_callback;
_PyObject_ASSERT(op, callback != NULL);

/* Ensure we don't execute the callback again if the weakref is
* resurrected. */
_PyGC_SET_FINALIZED(op);

/* copy-paste of weakrefobject.c's handle_callback() */
PyObject *temp = PyObject_CallOneArg(callback, (PyObject *)wr);
if (temp == NULL) {
Expand Down
Loading
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy