Handle `hash = None` in somewhat more intuitive fashion #19168

sterliakov · 2025-05-29T16:37:44Z

This is the first step towards checking hashability of dict keys (and Hashable compatibility in general). This PR implements the following rules:

__hash__ = None is only allowed on classes who inherited __hash__ directly from object (no redefinitions in MRO)
def __hash__ is allowed on subclasses whose parents declare __hash__ = None - it is a relatively common solution to derive a hashable subclass, and LSP violation only means assert SomeCls.__hash__ is None will not pass for subclasses. This sounds safe enough.
Classes defining __eq__ without __hash__ get their __hash__ assigned to None to match runtime semantics. Protocols are exempt from this as defining __eq__ on protocol just requires classes that support __eq__, not necessarily unhashable ones.

Refs #19043.

…k in subclasses, only allowed to override `object.__hash__`

…-dunder-hash-consistently

sterliakov · 2025-07-04T01:52:52Z

xarray should either always use __hash__ = None or __hash__: ClassVar[None]. Using __hash__: None everywhere should also work, I suppose, but doesn't have proper semantics. All errors are correct, and "cannot override instance variable..." will be fixed by using ClassVar[None] everywhere. Another one (argument "name"...) is a true positive, Mapping is not Hashable, the call should be rejected.
pandas: ByT is not hashable because its member Period is not. And it's a bug, at runtime Period is hashable, they're simply missing a __hash__ definition on it. If we can contact pandas-stubs maintainers, would be nice to let them know.
I'm lost in static_frame, this seems reasonable, there's also pyright-ignore on that line, but I'm failing to trace precise types in a 10 KLOC file (and not even self-contained in that file).
All other primer hits are correct; noticing that str and Mapping/MutableMapping are incompatible is also a great outcome.

github-actions · 2025-07-14T15:42:57Z

Diff from mypy_primer, showing the effect of this PR on open source code:

schema_salad (https://github.com/common-workflow-language/schema_salad)
+ schema_salad/jsonld_context.py: note: In function "pred":
+ schema_salad/jsonld_context.py:53:31: error: Subclass of "str" and "MutableMapping[Any, Any]" cannot exist: would have incompatible method signatures  [unreachable]
+ schema_salad/jsonld_context.py:54:21: error: Statement is unreachable  [unreachable]
+ schema_salad/schema.py: note: In function "get_anon_name":
+ schema_salad/schema.py:422:27: error: Subclass of "str" and "Mapping[Any, Any]" cannot exist: would have incompatible method signatures  [unreachable]
+ schema_salad/schema.py:423:17: error: Statement is unreachable  [unreachable]

static-frame (https://github.com/static-frame/static-frame)
+ static_frame/core/series.py:3346: error: Incompatible return value type (got "IndexBase", expected "TLabel | ndarray[Any, Any]")  [return-value]
+ static_frame/core/frame.py:7056: error: Argument 1 to "index_from_optional_constructors" has incompatible type "TypeBlocks"; expected "TIndexInitializer"  [arg-type]
+ static_frame/core/frame.py:7056: note: Following member(s) of "TypeBlocks" have conflicts:
+ static_frame/core/frame.py:7056: note:     Expected:
+ static_frame/core/frame.py:7056: note:         def __iter__(self) -> Iterator[TLabel]
+ static_frame/core/frame.py:7056: note:     Got:
+ static_frame/core/frame.py:7056: note:         def __iter__(self) -> Iterator[TypeBlocks]
+ static_frame/core/frame.py:7056: note: Following member(s) of "TypeBlocks" have conflicts:
+ static_frame/core/frame.py:7056: note:     Expected:
+ static_frame/core/frame.py:7056: note:         def __iter__(self) -> Iterator[Sequence[TLabel]]
+ static_frame/core/frame.py:7056: note:     Got:
+ static_frame/core/frame.py:7056: note:         def __iter__(self) -> Iterator[TypeBlocks]
+ static_frame/core/frame.py:7298: error: Argument 1 to "index_from_optional_constructors" has incompatible type "TypeBlocks"; expected "TIndexInitializer"  [arg-type]
+ static_frame/core/frame.py:7298: note: Following member(s) of "TypeBlocks" have conflicts:
+ static_frame/core/frame.py:7298: note:     Expected:
+ static_frame/core/frame.py:7298: note:         def __iter__(self) -> Iterator[TLabel]
+ static_frame/core/frame.py:7298: note:     Got:
+ static_frame/core/frame.py:7298: note:         def __iter__(self) -> Iterator[TypeBlocks]
+ static_frame/core/frame.py:7298: note: Following member(s) of "TypeBlocks" have conflicts:
+ static_frame/core/frame.py:7298: note:     Expected:
+ static_frame/core/frame.py:7298: note:         def __iter__(self) -> Iterator[Sequence[TLabel]]
+ static_frame/core/frame.py:7298: note:     Got:
+ static_frame/core/frame.py:7298: note:         def __iter__(self) -> Iterator[TypeBlocks]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/arrays/base.py:2207: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/generic.py:1867: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/indexes/base.py:5209: error: Unused "type: ignore" comment  [unused-ignore]

asynq (https://github.com/quora/asynq)
+ asynq/tests/test_tools.py:183: error: Unused "type: ignore" comment  [unused-ignore]

jax (https://github.com/google/jax)
+ jax/_src/core.py:525: error: Unused "type: ignore" comment  [unused-ignore]
+ jax/_src/core.py:837: error: Unused "type: ignore" comment  [unused-ignore]
+ jax/_src/earray.py:33: error: Unused "type: ignore" comment  [unused-ignore]
+ jax/_src/prng.py:302: error: Unused "type: ignore" comment  [unused-ignore]
+ jax/experimental/sparse/_base.py:34: error: Unused "type: ignore" comment  [unused-ignore]

werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/datastructures/structures.py:648: error: Unused "type: ignore" comment  [unused-ignore]
+ src/werkzeug/datastructures/headers.py:108: error: Unused "type: ignore" comment  [unused-ignore]
+ src/werkzeug/routing/rules.py:912: error: Unused "type: ignore" comment  [unused-ignore]

prefect (https://github.com/PrefectHQ/prefect)
+ src/prefect/_internal/concurrency/calls.py:273: error: Unused "type: ignore" comment  [unused-ignore]

mkdocs (https://github.com/mkdocs/mkdocs)
+ mkdocs/utils/templates.py:47: error: Subclass of "ExtraScriptValue" and "str" cannot exist: would have incompatible method signatures  [unreachable]

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
+ pandas-stubs/core/generic.pyi:68: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas-stubs/core/frame.pyi:312: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas-stubs/core/indexes/base.pyi:70: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas-stubs/core/groupby/generic.pyi:206: error: Return type "Iterator[tuple[ByT, Series[S2]]]" of "__iter__" incompatible with return type "Iterator[tuple[Hashable, Series[S2]]]" in supertype "pandas.core.groupby.groupby.BaseGroupBy"  [override]
+ pandas-stubs/core/groupby/generic.pyi:446: error: Return type "Iterator[tuple[ByT, DataFrame]]" of "__iter__" incompatible with return type "Iterator[tuple[Hashable, DataFrame]]" in supertype "pandas.core.groupby.groupby.BaseGroupBy"  [override]

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/python_api.py:88: error: Unused "type: ignore" comment  [unused-ignore]
+ src/_pytest/_code/source.py:50: error: Unused "type: ignore" comment  [unused-ignore]
+ src/_pytest/_code/code.py:77: error: Unused "type: ignore" comment  [unused-ignore]

xarray (https://github.com/pydata/xarray)
+ xarray/core/_typed_ops.py:93: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/_typed_ops.py:362: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/_typed_ops.py:736: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/_typed_ops.py:1164: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/_typed_ops.py:1376: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/_typed_ops.py:1502: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/dataset.py:1430: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/dataset.py: note: In class "Dataset":
+ xarray/core/dataset.py:1430: error: Cannot override instance variable (previously declared on base class "DatasetOpsMixin") with class variable  [misc]
+ xarray/core/dataset.py:1430: note: Error code "misc" not covered by "type: ignore" comment
+ xarray/core/dataset.py: note: At top level:
+ xarray/core/dataarray.py:1312: error: Unused "type: ignore" comment  [unused-ignore]
+ xarray/core/dataarray.py: note: In member "rename" of class "DataArray":
+ xarray/core/dataarray.py:2549: error: Argument "name" to "_replace" of "DataArray" has incompatible type "Hashable | Mapping[Any, Hashable]"; expected "Hashable | Default | None"  [arg-type]
+ xarray/core/dataarray.py: note: At top level:
+ xarray/core/datatree.py:301: error: Unused "type: ignore" comment  [unused-ignore]

beartype (https://github.com/beartype/beartype)
+ beartype/_check/metadata/metadecor.py:289: error: Unused "type: ignore" comment  [unused-ignore]
+ beartype/_check/metadata/metacheck.py:97: error: Unused "type: ignore" comment  [unused-ignore]

mongo-python-driver (https://github.com/mongodb/mongo-python-driver)
+ bson/regex.py:112: error: Unused "type: ignore" comment  [unused-ignore]

ibis (https://github.com/ibis-project/ibis)
- ibis/common/bases.py:167: error: Incompatible types in assignment (expression has type "None", base class "object" defined the type as "Callable[[], int]")  [assignment]
- ibis/common/bases.py:205: error: Incompatible types in assignment (expression has type "None", base class "object" defined the type as "Callable[[], int]")  [assignment]
- ibis/common/bases.py:249: error: Signature of "__hash__" incompatible with supertype "Slotted"  [override]
- ibis/common/bases.py:249: note:      Superclass:
- ibis/common/bases.py:249: note:          None
- ibis/common/bases.py:249: note:      Subclass:
- ibis/common/bases.py:249: note:          def __hash__(self) -> int
- ibis/common/collections.py:153: error: Incompatible types in assignment (expression has type "None", base class "object" defined the type as "Callable[[], int]")  [assignment]
- ibis/common/collections.py:291: error: Signature of "__hash__" incompatible with supertype "Mapping"  [override]
- ibis/common/collections.py:291: note:      Superclass:
- ibis/common/collections.py:291: note:          None
- ibis/common/collections.py:291: note:      Subclass:
- ibis/common/collections.py:291: note:          def __hash__(self) -> int
- ibis/common/collections.py:313: note:      Superclass:
- ibis/common/collections.py:313: note:          None
- ibis/common/collections.py:313: note:      Subclass:
- ibis/common/collections.py:313: note:          def __hash__(self) -> int
- ibis/common/collections.py:313: error: Signature of "__hash__" incompatible with supertype "Mapping"  [override]
- ibis/common/deferred.py:180: error: Incompatible types in assignment (expression has type "None", base class "object" defined the type as "Callable[[], int]")  [assignment]
- ibis/common/patterns.py:660: error: Signature of "__hash__" incompatible with supertype "Slotted"  [override]
- ibis/common/patterns.py:660: note:      Superclass:
- ibis/common/patterns.py:660: note:          None
- ibis/common/patterns.py:660: note:      Subclass:
- ibis/common/patterns.py:660: note:          def __hash__(self) -> int
- ibis/common/patterns.py:937: error: Signature of "__hash__" incompatible with supertype "Slotted"  [override]
- ibis/common/patterns.py:937: note:      Superclass:
- ibis/common/patterns.py:937: note:          None
- ibis/common/patterns.py:937: note:      Subclass:
- ibis/common/patterns.py:937: note:          def __hash__(self) -> int
- ibis/common/grounds.py:164: error: Incompatible types in assignment (expression has type "None", base class "object" defined the type as "Callable[[], int]")  [assignment]
- ibis/common/grounds.py:220: error: Signature of "__hash__" incompatible with supertype "Comparable"  [override]
- ibis/common/grounds.py:220: note:      Superclass:
- ibis/common/grounds.py:220: note:          None
- ibis/common/grounds.py:220: note:      Subclass:
- ibis/common/grounds.py:220: note:          def __hash__(self) -> int
- ibis/common/grounds.py:220: error: Signature of "__hash__" incompatible with supertype "Annotable"  [override]
- ibis/common/grounds.py:220: note:      Superclass:
- ibis/common/grounds.py:220: note:          None
- ibis/common/grounds.py:220: note:      Subclass:
- ibis/common/grounds.py:220: note:          def __hash__(self) -> int
- ibis/common/egraph.py:124: error: Incompatible types in assignment (expression has type "None", base class "object" defined the type as "Callable[[], int]")  [assignment]
- ibis/expr/datatypes/core.py:932: error: Definition of "__hash__" in base class "Concrete" is incompatible with definition in base class "Mapping"  [misc]
- ibis/expr/schema.py:25: error: Definition of "__hash__" in base class "Concrete" is incompatible with definition in base class "Mapping"  [misc]
- ibis/common/tests/test_grounds.py:142: error: Definition of "__hash__" in base class "Concrete" is incompatible with definition in base class "Mapping"  [misc]

sterliakov added 2 commits May 29, 2025 18:13

Make behaviour of __hash__ = None more intuitive: can be rolled bac…

25d10b1

…k in subclasses, only allowed to override `object.__hash__`

Missing get_proper_type

a17a313

This comment has been minimized.

Sign in to view

sterliakov added 3 commits June 1, 2025 16:31

Allow any such overrides

31a5c2a

Fix stub for mypyc test

22a2729

Adjust remaining test, replace ellipsis with pass for consistency

eccc702

This comment has been minimized.

Sign in to view

Do not add implicit __hash__ = None to protocols

fad22f2

This comment has been minimized.

Sign in to view

sterliakov added 3 commits June 3, 2025 20:53

Merge remote-tracking branch 'upstream/master' into feature/st-handle…

320effb

…-dunder-hash-consistently

Support in assignments too

bd953b4

Handle allow_incompatible_override consistenly instead

18815c7

This comment has been minimized.

Sign in to view

sterliakov marked this pull request as ready for review June 6, 2025 22:39

sterliakov changed the title ~~experiment: handle __hash__ = None in somewhat more intuitive fashion~~ Handle __hash__ = None in somewhat more intuitive fashion Jun 7, 2025

sterliakov mentioned this pull request Jun 7, 2025

Fix and simplify error de-duplication #19247

Merged

sterliakov and others added 2 commits June 11, 2025 03:52

Merge branch 'master' into feature/st-handle-dunder-hash-consistently

be5d7e3

Sync test after merge

3f9493d

This comment has been minimized.

Sign in to view

ktbarrett mentioned this pull request Jun 22, 2025

Various fixes cocotb/cocotb#4765

Merged

Merge branch 'master' into feature/st-handle-dunder-hash-consistently

9925077

This comment has been minimized.

Sign in to view

Merge branch 'master' into feature/st-handle-dunder-hash-consistently

236294f

sterliakov requested review from A5rocks and JukkaL July 4, 2025 01:53

This comment has been minimized.

Sign in to view

Merge branch 'master' into feature/st-handle-dunder-hash-consistently

ed7dba5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Handle `hash = None` in somewhat more intuitive fashion #19168

Handle `hash = None` in somewhat more intuitive fashion #19168

Uh oh!

sterliakov commented May 29, 2025 •

edited

Loading

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

sterliakov commented Jul 4, 2025

Uh oh!

This comment has been minimized.

github-actions bot commented Jul 14, 2025

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Uh oh!

Handle __hash__ = None in somewhat more intuitive fashion #19168

Are you sure you want to change the base?

Handle __hash__ = None in somewhat more intuitive fashion #19168

Uh oh!

Conversation

sterliakov commented May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

sterliakov commented Jul 4, 2025

Uh oh!

This comment has been minimized.

github-actions bot commented Jul 14, 2025

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Handle `hash = None` in somewhat more intuitive fashion #19168

Handle `hash = None` in somewhat more intuitive fashion #19168

sterliakov commented May 29, 2025 •

edited

Loading