False narrowing with repeated assignment inside if resulting in bad false negative #18492

MaxG87 · 2025-01-20T10:37:52Z

Bug Report

In a function, mypy fails to detect an incompatible assignment in a function, if that is after a __get_item__ access inside a branch. Weirdly, mypy correctly deduces the types, as shown with reveal_type but does not complain about the incorrect assignment anyways.

To Reproduce

I would have liked to give a playground link, but the MWE involves a pandas.DataFrame which seems not to work on the playground.

import pandas as pd

# fails as expected
some_condition: bool = "123".isnumeric()
list_of_dicts = [{"some": 1, "columns": 2, "will": 3, "go": 4, "here": 5}]
if some_condition > 0:
    output_df = pd.DataFrame(list_of_dicts)
    output_df = output_df[["some", "columns", "here"]]
else:
    output_df = None  # fails here


def fails_as_expected_1(some_condition: bool) -> pd.DataFrame:
    list_of_dicts = [{"some": 1, "columns": 2, "will": 3, "go": 4, "here": 5}]
    output_df = pd.DataFrame(list_of_dicts)
    output_df = output_df[["some", "columns", "here"]]
    output_df = None  # fails here
    other_df: pd.DataFrame = output_df
    return other_df


def fails_as_expected_2(some_condition: bool) -> pd.DataFrame:
    list_of_dicts = [{"some": 1, "columns": 2, "will": 3, "go": 4, "here": 5}]
    if some_condition > 0:
        output_df = pd.DataFrame(list_of_dicts)
    else:
        output_df = None
    other_df: pd.DataFrame = output_df  # fails here
    return other_df


def does_not_fail_but_should(some_condition: bool) -> pd.DataFrame:
    list_of_dicts = [{"some": 1, "columns": 2, "will": 3, "go": 4, "here": 5}]
    if some_condition > 0:
        output_df = pd.DataFrame(list_of_dicts)
        reveal_type(output_df)
        output_df = output_df[["some", "columns", "here"]]
        reveal_type(output_df)
    else:
        output_df = None
    other_df: pd.DataFrame = output_df
    return other_df

Expected Behavior
Each block and function should contain a mypy error regarding an incompatible assignment.

Actual Behavior

For the last function, no such error is produced:

mwe2.py:10: error: Incompatible types in assignment (expression has type "None", variable has type "DataFrame")  [assignment]
mwe2.py:17: error: Incompatible types in assignment (expression has type "None", variable has type "DataFrame")  [assignment]
mwe2.py:28: error: Incompatible types in assignment (expression has type "DataFrame | None", variable has type "DataFrame")  [assignment]
mwe2.py:36: note: Revealed type is "pandas.core.frame.DataFrame"
mwe2.py:38: note: Revealed type is "pandas.core.frame.DataFrame"
Found 3 errors in 1 file (checked 1 source file)

Your Environment

Mypy version used: mypy 1.14.1 (compiled: yes)
Mypy command-line flags: only the filename
Mypy configuration options from pyproject.toml:

[tool.mypy]
warn_unreachable = true
enable_error_code = [
    "possibly-undefined"
]
strict = true

Python version used: Python 3.12.8 from Debian Testing

The text was updated successfully, but these errors were encountered:

hauntsaninja · 2025-01-20T21:59:29Z

Thanks! Pretty bad bug. Looks like it's just some issue if there's a double assignment (and specifically with None and without an explicit annotation, so likely related to partial types):

class A: ...

def does_not_fail_but_should(some_condition: bool) -> A:
    if some_condition > 0:
        var = A()
        reveal_type(var)
        # Commenting out this assignment makes mypy correctly error
        var = A()
        reveal_type(var)
    else:
        var = None

    reveal_type(var)
    other: A = var
    return other

MaxG87 · 2025-01-21T09:45:51Z

I can reproduce the minified example with the same toolchain as above.

Prabhat-Thapa45 · 2025-01-31T05:53:12Z

@hauntsaninja While trying to reproduce this I encountered an issue with mypy where it fails to correctly identify the type of variable and ultimately raises an error at an undesired place

def gives_wrong_output(some_condition: bool) -> None:
    if some_condition:
        a = 3
    else:
        a = 's' # fails here
    reveal_type(a)
    b: int = a # should have failed here

main.py:5: error: Incompatible types in assignment (expression has type "str", variable has type "int")  [assignment]
main.py:6: note: Revealed type is "builtins.int"

east825 · 2025-05-19T16:07:14Z

I've tried reproducing the original example and in the latest nightly (301c3b6) and it no longer reproduces. In the function does_not_fail_but_should there is a true positive error about assigning a value of type DataFrame | None to a variable expecting DataFrame.

original.py:43: error: Incompatible types in assignment (expression has type "DataFrame | None", variable has type "DataFrame")

The problem in the simplified example by @Prabhat-Thapa45 is still reproducible, though (mypy doesn't take into assignment a reachable assignment in the false branch).

However, with the experimental options --allow-redefinition-new --local-partial-types the false negative in the simplified example disappears and the behavior in the first example becomes better. The reassignment of output_df on line 12 becomes possible and in fails_as_expected_1 mypy reports assignment of a None value to other_df on line 20 instead of the reassigment of output_df on line 19.

some_condition: bool = "123".isnumeric()
list_of_dicts = [{"some": 1, "columns": 2, "will": 3, "go": 4, "here": 5}]
if some_condition > 0:
    output_df = pd.DataFrame(list_of_dicts)
    output_df = output_df[["some", "columns", "here"]]
else:
    output_df = None  # warning (false positive?) without --allow-redefinition-new --local-partial-type


def fails_as_expected_1(some_condition: bool) -> pd.DataFrame:
    list_of_dicts = [{"some": 1, "columns": 2, "will": 3, "go": 4, "here": 5}]
    output_df = pd.DataFrame(list_of_dicts)
    output_df = output_df[["some", "columns", "here"]]
    output_df = None  # warning here (false positive?) without --allow-redefinition-new --local-partial-type
    other_df: pd.DataFrame = output_df  # warning here with --allow-redefinition-new
    return other_df

hauntsaninja · 2025-05-19T19:51:55Z

Looks like this got fixed by #18538

MaxG87 added the bug mypy got something wrong label Jan 20, 2025

hauntsaninja added good-second-issue topic-type-narrowing Conditional type narrowing / binder labels Jan 20, 2025

hauntsaninja changed the title ~~Incompatible Assignment of None to DataFrame in Function not Caught~~ False narrowing with repeated assignment inside if resulting in bad false negative Jan 23, 2025

JukkaL closed this as completed May 19, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

False narrowing with repeated assignment inside if resulting in bad false negative #18492

False narrowing with repeated assignment inside if resulting in bad false negative #18492

MaxG87 commented Jan 20, 2025

hauntsaninja commented Jan 20, 2025 •

edited

Loading

Uh oh!

MaxG87 commented Jan 21, 2025

Uh oh!

Prabhat-Thapa45 commented Jan 31, 2025

Uh oh!

east825 commented May 19, 2025

Uh oh!

hauntsaninja commented May 19, 2025

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Uh oh!

False narrowing with repeated assignment inside if resulting in bad false negative #18492

False narrowing with repeated assignment inside if resulting in bad false negative #18492

Comments

MaxG87 commented Jan 20, 2025

hauntsaninja commented Jan 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MaxG87 commented Jan 21, 2025

Uh oh!

Prabhat-Thapa45 commented Jan 31, 2025

Uh oh!

east825 commented May 19, 2025

Uh oh!

hauntsaninja commented May 19, 2025

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

hauntsaninja commented Jan 20, 2025 •

edited

Loading