This roughly follows what was done for dictobject to make a lock-free lookup operation. On a benchmark running set.__contains__ in a tight loop, this is 1.5x faster on my computer. In the bm_deepcopy benchmark, the gains are very modest, between 1 to 2% faster. On the "set_contains" scaling benchmark, the results are much better. Also, the multi-threaded scaling of "copy" and "deepcopy" seem to be measurably improved.

Summary of changes:

• refactor set_lookkey() into set_do_lookup() which now takes a function pointer that does the entry comparison. This is similar to dictobject and do_lookup(). In an optimized build, the comparison function is inlined and there should be no performance cost to this.
• change set_do_lookup to return a status separately from the entry value.
• add set_compare_frozenset() and use if the object is a frozenset. For the free-threaded build, this avoids some overhead (locking, atomic operations, incref/decref on key)
• use FT_ATOMIC_* macros as needed for atomic loads and stores
• use a deferred free on the set table array, if shared (only on free-threaded build, normal build always does an immediate free)
• for free-threaded build, use explicit for loop to zero the table, rather than memcpy().
• when mutating the set, assign so->table to NULL while the change is a happening. Assign the real table array after the change is done.

Free-threading scaling benchmark results from the attached scripts (result for 6 cores in parallel). This is a modified version of the ftscalingbenchmark.py script.

ftscaling_set.py.txt

• Issue: copy.copy and copy.deepcopy scale poorly with free-threading #132657