Error condition in _datetimemodule.c is unchecked with datetime.fromtimestamp on local times > Y10k
#91581
Comments
pganssle
added a commit
to pganssle/cpython
that referenced
this issue
May 12, 2022
The `utc_to_seconds` call can fail, here's a minimal reproducer on Linux: TZ=UTC python -c "from datetime import *; datetime.fromtimestamp(253402300799 + 1)" The old behavior still raised an error in a similar way, but only because subsequent calculations happened to fail as well. Better to fail fast. This also refactors the tests to split out the `fromtimestamp` and `utcfromtimestamp` tests, and to get us closer to the actual desired limits of the functions. As part of this, we also changed the way we detect platforms where the same limits don't necessarily apply (e.g. Windows). As part of refactoring the tests to hit this condition explicitly (even though the user-facing behvior doesn't change in any way we plan to guarantee), I noticed that there was a difference in the places that `datetime.utcfromtimestamp` fails in the C and pure Python versions, which was fixed by skipping the "probe for fold" logic for UTC specifically — since UTC doesn't have any folds or gaps, we were never going to find a fold value anyway. This should prevent some failures in the pure python `utcfromtimestamp` method on timestamps close to 0001-01-01. There are two separate news entries for this because one is a potentially user-facing change, the other is an internal code correctness change that, if anything, changes some error messages. The two happen to be coupled because of the test refactoring, but they are probably best thought of as independent changes. Fixes pythonGH-91581
pganssle
added a commit
that referenced
this issue
May 12, 2022
…1582) The `utc_to_seconds` call can fail, here's a minimal reproducer on Linux: TZ=UTC python -c "from datetime import *; datetime.fromtimestamp(253402300799 + 1)" The old behavior still raised an error in a similar way, but only because subsequent calculations happened to fail as well. Better to fail fast. This also refactors the tests to split out the `fromtimestamp` and `utcfromtimestamp` tests, and to get us closer to the actual desired limits of the functions. As part of this, we also changed the way we detect platforms where the same limits don't necessarily apply (e.g. Windows). As part of refactoring the tests to hit this condition explicitly (even though the user-facing behvior doesn't change in any way we plan to guarantee), I noticed that there was a difference in the places that `datetime.utcfromtimestamp` fails in the C and pure Python versions, which was fixed by skipping the "probe for fold" logic for UTC specifically — since UTC doesn't have any folds or gaps, we were never going to find a fold value anyway. This should prevent some failures in the pure python `utcfromtimestamp` method on timestamps close to 0001-01-01. There are two separate news entries for this because one is a potentially user-facing change, the other is an internal code correctness change that, if anything, changes some error messages. The two happen to be coupled because of the test refactoring, but they are probably best thought of as independent changes. Fixes GH-91581
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this issue
May 12, 2022
…thonGH-91582) The `utc_to_seconds` call can fail, here's a minimal reproducer on Linux: TZ=UTC python -c "from datetime import *; datetime.fromtimestamp(253402300799 + 1)" The old behavior still raised an error in a similar way, but only because subsequent calculations happened to fail as well. Better to fail fast. This also refactors the tests to split out the `fromtimestamp` and `utcfromtimestamp` tests, and to get us closer to the actual desired limits of the functions. As part of this, we also changed the way we detect platforms where the same limits don't necessarily apply (e.g. Windows). As part of refactoring the tests to hit this condition explicitly (even though the user-facing behvior doesn't change in any way we plan to guarantee), I noticed that there was a difference in the places that `datetime.utcfromtimestamp` fails in the C and pure Python versions, which was fixed by skipping the "probe for fold" logic for UTC specifically — since UTC doesn't have any folds or gaps, we were never going to find a fold value anyway. This should prevent some failures in the pure python `utcfromtimestamp` method on timestamps close to 0001-01-01. There are two separate news entries for this because one is a potentially user-facing change, the other is an internal code correctness change that, if anything, changes some error messages. The two happen to be coupled because of the test refactoring, but they are probably best thought of as independent changes. Fixes pythonGH-91581 (cherry picked from commit 83c0247) Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com>
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this issue
May 12, 2022
…thonGH-91582) The `utc_to_seconds` call can fail, here's a minimal reproducer on Linux: TZ=UTC python -c "from datetime import *; datetime.fromtimestamp(253402300799 + 1)" The old behavior still raised an error in a similar way, but only because subsequent calculations happened to fail as well. Better to fail fast. This also refactors the tests to split out the `fromtimestamp` and `utcfromtimestamp` tests, and to get us closer to the actual desired limits of the functions. As part of this, we also changed the way we detect platforms where the same limits don't necessarily apply (e.g. Windows). As part of refactoring the tests to hit this condition explicitly (even though the user-facing behvior doesn't change in any way we plan to guarantee), I noticed that there was a difference in the places that `datetime.utcfromtimestamp` fails in the C and pure Python versions, which was fixed by skipping the "probe for fold" logic for UTC specifically — since UTC doesn't have any folds or gaps, we were never going to find a fold value anyway. This should prevent some failures in the pure python `utcfromtimestamp` method on timestamps close to 0001-01-01. There are two separate news entries for this because one is a potentially user-facing change, the other is an internal code correctness change that, if anything, changes some error messages. The two happen to be coupled because of the test refactoring, but they are probably best thought of as independent changes. Fixes pythonGH-91581 (cherry picked from commit 83c0247) Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com>
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this issue
May 12, 2022
…thonGH-91582) The `utc_to_seconds` call can fail, here's a minimal reproducer on Linux: TZ=UTC python -c "from datetime import *; datetime.fromtimestamp(253402300799 + 1)" The old behavior still raised an error in a similar way, but only because subsequent calculations happened to fail as well. Better to fail fast. This also refactors the tests to split out the `fromtimestamp` and `utcfromtimestamp` tests, and to get us closer to the actual desired limits of the functions. As part of this, we also changed the way we detect platforms where the same limits don't necessarily apply (e.g. Windows). As part of refactoring the tests to hit this condition explicitly (even though the user-facing behvior doesn't change in any way we plan to guarantee), I noticed that there was a difference in the places that `datetime.utcfromtimestamp` fails in the C and pure Python versions, which was fixed by skipping the "probe for fold" logic for UTC specifically — since UTC doesn't have any folds or gaps, we were never going to find a fold value anyway. This should prevent some failures in the pure python `utcfromtimestamp` method on timestamps close to 0001-01-01. There are two separate news entries for this because one is a potentially user-facing change, the other is an internal code correctness change that, if anything, changes some error messages. The two happen to be coupled because of the test refactoring, but they are probably best thought of as independent changes. Fixes pythonGH-91581 (cherry picked from commit 83c0247) Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com>
miss-islington
added a commit
that referenced
this issue
May 14, 2022
…-91582) The `utc_to_seconds` call can fail, here's a minimal reproducer on Linux: TZ=UTC python -c "from datetime import *; datetime.fromtimestamp(253402300799 + 1)" The old behavior still raised an error in a similar way, but only because subsequent calculations happened to fail as well. Better to fail fast. This also refactors the tests to split out the `fromtimestamp` and `utcfromtimestamp` tests, and to get us closer to the actual desired limits of the functions. As part of this, we also changed the way we detect platforms where the same limits don't necessarily apply (e.g. Windows). As part of refactoring the tests to hit this condition explicitly (even though the user-facing behvior doesn't change in any way we plan to guarantee), I noticed that there was a difference in the places that `datetime.utcfromtimestamp` fails in the C and pure Python versions, which was fixed by skipping the "probe for fold" logic for UTC specifically — since UTC doesn't have any folds or gaps, we were never going to find a fold value anyway. This should prevent some failures in the pure python `utcfromtimestamp` method on timestamps close to 0001-01-01. There are two separate news entries for this because one is a potentially user-facing change, the other is an internal code correctness change that, if anything, changes some error messages. The two happen to be coupled because of the test refactoring, but they are probably best thought of as independent changes. Fixes GH-91581 (cherry picked from commit 83c0247) Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com>
miss-islington
added a commit
that referenced
this issue
May 14, 2022
…-91582) The `utc_to_seconds` call can fail, here's a minimal reproducer on Linux: TZ=UTC python -c "from datetime import *; datetime.fromtimestamp(253402300799 + 1)" The old behavior still raised an error in a similar way, but only because subsequent calculations happened to fail as well. Better to fail fast. This also refactors the tests to split out the `fromtimestamp` and `utcfromtimestamp` tests, and to get us closer to the actual desired limits of the functions. As part of this, we also changed the way we detect platforms where the same limits don't necessarily apply (e.g. Windows). As part of refactoring the tests to hit this condition explicitly (even though the user-facing behvior doesn't change in any way we plan to guarantee), I noticed that there was a difference in the places that `datetime.utcfromtimestamp` fails in the C and pure Python versions, which was fixed by skipping the "probe for fold" logic for UTC specifically — since UTC doesn't have any folds or gaps, we were never going to find a fold value anyway. This should prevent some failures in the pure python `utcfromtimestamp` method on timestamps close to 0001-01-01. There are two separate news entries for this because one is a potentially user-facing change, the other is an internal code correctness change that, if anything, changes some error messages. The two happen to be coupled because of the test refactoring, but they are probably best thought of as independent changes. Fixes GH-91581 (cherry picked from commit 83c0247) Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com>
ambv
pushed a commit
that referenced
this issue
May 16, 2022
…-91582) (GH-92748) The `utc_to_seconds` call can fail, here's a minimal reproducer on Linux: TZ=UTC python -c "from datetime import *; datetime.fromtimestamp(253402300799 + 1)" The old behavior still raised an error in a similar way, but only because subsequent calculations happened to fail as well. Better to fail fast. This also refactors the tests to split out the `fromtimestamp` and `utcfromtimestamp` tests, and to get us closer to the actual desired limits of the functions. As part of this, we also changed the way we detect platforms where the same limits don't necessarily apply (e.g. Windows). As part of refactoring the tests to hit this condition explicitly (even though the user-facing behvior doesn't change in any way we plan to guarantee), I noticed that there was a difference in the places that `datetime.utcfromtimestamp` fails in the C and pure Python versions, which was fixed by skipping the "probe for fold" logic for UTC specifically — since UTC doesn't have any folds or gaps, we were never going to find a fold value anyway. This should prevent some failures in the pure python `utcfromtimestamp` method on timestamps close to 0001-01-01. There are two separate news entries for this because one is a potentially user-facing change, the other is an internal code correctness change that, if anything, changes some error messages. The two happen to be coupled because of the test refactoring, but they are probably best thought of as independent changes. Fixes GH-91581 (cherry picked from commit 83c0247) Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com>
hello-adam
pushed a commit
to hello-adam/cpython
that referenced
this issue
Jun 2, 2022
…thonGH-91582) (pythonGH-92748) The `utc_to_seconds` call can fail, here's a minimal reproducer on Linux: TZ=UTC python -c "from datetime import *; datetime.fromtimestamp(253402300799 + 1)" The old behavior still raised an error in a similar way, but only because subsequent calculations happened to fail as well. Better to fail fast. This also refactors the tests to split out the `fromtimestamp` and `utcfromtimestamp` tests, and to get us closer to the actual desired limits of the functions. As part of this, we also changed the way we detect platforms where the same limits don't necessarily apply (e.g. Windows). As part of refactoring the tests to hit this condition explicitly (even though the user-facing behvior doesn't change in any way we plan to guarantee), I noticed that there was a difference in the places that `datetime.utcfromtimestamp` fails in the C and pure Python versions, which was fixed by skipping the "probe for fold" logic for UTC specifically — since UTC doesn't have any folds or gaps, we were never going to find a fold value anyway. This should prevent some failures in the pure python `utcfromtimestamp` method on timestamps close to 0001-01-01. There are two separate news entries for this because one is a potentially user-facing change, the other is an internal code correctness change that, if anything, changes some error messages. The two happen to be coupled because of the test refactoring, but they are probably best thought of as independent changes. Fixes pythonGH-91581 (cherry picked from commit 83c0247) Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
On twitter, someone asked about a weird difference in the error messages from
datetime.fromtimestamp()when called with a timestamp that falls on 10000-01-01 when compared to 10000-01-02. Compare:Turns out that this is because in the localtime-specific implementation of
datetime.fromtimestampin_datetimemodule.c, there's an unchecked error condition when callingutc_to_seconds, which fails with-1.As a practical matter, this always results in basically the same error anyway, since in all situations where
result_secondsgets set to-1the subsequent calculations fail anyway, and the only difference is that the error message has a slightly different form, but we may as well fix this.The text was updated successfully, but these errors were encountered: