Skip to content

0.4.2 test failures: tests/integration/test_validators.py::TestOAS{30,31}Validator::test_format_checkers #61

New issue
New issue
Closed
#62
Closed
0.4.2 test failures: tests/integration/test_validators.py::TestOAS{30,31}Validator::test_format_checkers#61
#62
@mgorny

Description

@mgorny
mgorny
opened on Jan 31, 2023

When running the test suite in 0.4.2, I get the following failures:

========================================================= test session starts =========================================================
platform linux -- Python 3.10.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- /tmp/openapi-schema-validator/.venv/bin/python
cachedir: .pytest_cache
rootdir: /tmp/openapi-schema-validator, configfile: pyproject.toml
plugins: flake8-1.1.0, cov-3.0.0
collected 58 items                                                                                                                    

tests/integration/test_validators.py::TestOAS30Validator::test_format_checkers FAILED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_null[boolean] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_null[array] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_null[integer] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_null[number] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_null[string] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable_untyped[True] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable_untyped[False] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable[boolean] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable[array] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable[integer] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable[number] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable[string] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable_enum_without_none PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable_enum_with_none PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_string_format_byte_valid[c3RyaW5n0] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_string_format_byte_valid[c3RyaW5n1] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_string_format_byte_invalid[string0] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_string_format_byte_invalid[string1] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_allof_required PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_required PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_read_only PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_write_only PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_required_read_only PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_required_write_only PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_oneof_required PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_oneof_discriminator[oneOf] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_oneof_discriminator[anyOf] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_oneof_discriminator[allOf] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable_ref[True] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable_ref[False] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable_schema_combos[True-oneOf-None is not valid under any of the given schemas] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable_schema_combos[True-anyOf-None is not valid under any of the given schemas] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable_schema_combos[True-allOf-None for not nullable] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable_schema_combos[False-oneOf-None is not valid under any of the given schemas] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable_schema_combos[False-anyOf-None is not valid under any of the given schemas] PASSED
tests/integration/test_validators.py::TestOAS30ValidatorValidate::test_nullable_schema_combos[False-allOf-None for not nullable] PASSED
tests/integration/test_validators.py::TestOAS31Validator::test_format_checkers FAILED
tests/integration/test_validators.py::TestOAS30ReadWriteValidatorValidate::test_read_only PASSED
tests/integration/test_validators.py::TestOAS30ReadWriteValidatorValidate::test_write_only PASSED
tests/integration/test_validators.py::TestOAS30ReadWriteValidatorValidate::test_required_read_only PASSED
tests/integration/test_validators.py::TestOAS30ReadWriteValidatorValidate::test_required_write_only PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_null[boolean] PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_null[array] PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_null[integer] PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_null[number] PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_null[string] PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_nullable[boolean] PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_nullable[array] PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_nullable[integer] PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_nullable[number] PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_nullable[string] PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_schema_validation PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_schema_ref PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_array_prefixitems[value0] PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_array_prefixitems[value1] PASSED
tests/integration/test_validators.py::TestOAS31ValidatorValidate::test_array_prefixitems_invalid[value0] PASSED
tests/unit/test_shortcut.py::ValidateTest::test_validate_does_not_mutate_schema_adding_nullable_key PASSED

============================================================== FAILURES ===============================================================
_______________________________________________ TestOAS30Validator.test_format_checkers _______________________________________________

self = <test_validators.TestOAS30Validator object at 0x7f0996aea470>

    def test_format_checkers(self):
>       assert set(OAS30Validator.FORMAT_CHECKER.checkers.keys()) == set([
            # standard formats
            "int32", "int64", "float", "double", "byte", "binary",
            "date", "date-time", "password",
            # extra formats
            "uuid", "regex",
            "ipv4", "ipv6", "email", "idn-email", "time"
        ])
E       AssertionError: assert {'binary',\n 'byte',\n 'date',\n 'double',\n 'email',\n 'float',\n 'idn-email',\n 'idn-hostname',\n 'int32',\n 'int64',\n 'ipv4',\n 'ipv6',\n 'password',\n 'regex',\n 'time',\n 'uuid'} == {'binary',\n 'byte',\n 'date',\n 'date-time',\n 'double',\n 'email',\n 'float',\n 'idn-email',\n 'int32',\n 'int64',\n 'ipv4',\n 'ipv6',\n 'password',\n 'regex',\n 'time',\n 'uuid'}
E         Extra items in the left set:
E         'idn-hostname'
E         Extra items in the right set:
E         'date-time'
E         Full diff:
E           {
E            'binary',
E            'byte',
E            'date',
E         -  'date-time',
E            'double',
E            'email',
E            'float',
E            'idn-email',
E         +  'idn-hostname',
E            'int32',
E            'int64',
E            'ipv4',
E            'ipv6',
E            'password',
E            'regex',
E            'time',
E            'uuid',
E           }

self       = <test_validators.TestOAS30Validator object at 0x7f0996aea470>

tests/integration/test_validators.py:17: AssertionError
_______________________________________________ TestOAS31Validator.test_format_checkers _______________________________________________

self = <test_validators.TestOAS31Validator object at 0x7f09969c2920>

    def test_format_checkers(self):
>       assert set(OAS31Validator.FORMAT_CHECKER.checkers.keys()) == set([
            # standard formats
            "int32", "int64", "float", "double", "password",
            # extra formats
            "date", "date-time", "uuid", "regex",
            "ipv4", "ipv6", "email", "idn-email", "time"
        ])
E       AssertionError: assert {'date',\n 'double',\n 'email',\n 'float',\n 'idn-email',\n 'idn-hostname',\n 'int32',\n 'int64',\n 'ipv4',\n 'ipv6',\n 'password',\n 'regex',\n 'time',\n 'uuid'} == {'date',\n 'date-time',\n 'double',\n 'email',\n 'float',\n 'idn-email',\n 'int32',\n 'int64',\n 'ipv4',\n 'ipv6',\n 'password',\n 'regex',\n 'time',\n 'uuid'}
E         Extra items in the left set:
E         'idn-hostname'
E         Extra items in the right set:
E         'date-time'
E         Full diff:
E           {
E            'date',
E         -  'date-time',
E            'double',
E            'email',
E            'float',
E            'idn-email',
E         +  'idn-hostname',
E            'int32',
E            'int64',
E            'ipv4',
E            'ipv6',
E            'password',
E            'regex',
E            'time',
E            'uuid',
E           }

self       = <test_validators.TestOAS31Validator object at 0x7f09969c2920>

tests/integration/test_validators.py:474: AssertionError
--------------------------------- generated xml file: /tmp/openapi-schema-validator/reports/junit.xml ---------------------------------

---------- coverage: platform linux, python 3.10.9-final-0 -----------
Name                                      Stmts   Miss Branch BrPart  Cover   Missing
-------------------------------------------------------------------------------------
openapi_schema_validator/__init__.py         13      0      0      0   100%
openapi_schema_validator/_format.py          41      7      6      1    79%   19, 25, 29, 36, 43-46
openapi_schema_validator/_types.py           12      0      0      0   100%
openapi_schema_validator/_validators.py     125     25     72     10    73%   37, 47-53, 62, 131, 133->exit, 146-150, 160, 184, 186->185, 188->195, 205, 207->206, 209->213, 222-237
openapi_schema_validator/shortcuts.py        13      1      2      1    87%   23
openapi_schema_validator/validators.py       37      0      6      0   100%
-------------------------------------------------------------------------------------
TOTAL                                       241     33     86     12    80%
Coverage XML written to file reports/coverage.xml

======================================================= short test summary info =======================================================
FAILED tests/integration/test_validators.py::TestOAS30Validator::test_format_checkers - AssertionError: assert {'binary',\n 'byte',\...
FAILED tests/integration/test_validators.py::TestOAS31Validator::test_format_checkers - AssertionError: assert {'date',\n 'double',\...
==================================================== 2 failed, 56 passed in 0.55s =====================================================

I can reproduce it reliably in a venv via:

git clone https://github.com/p1c2u/openapi-schema-validator/
cd openapi-schema-validator/
git checkout 0.4.2
poetry install
pytest -vv

On a production system, I get even more mismatches:

_______________________________________________ TestOAS30Validator.test_format_checkers _______________________________________________

self = <test_validators.TestOAS30Validator object at 0x7f6ee63cceb0>

    def test_format_checkers(self):
>       assert set(OAS30Validator.FORMAT_CHECKER.checkers.keys()) == set([
            # standard formats
            "int32", "int64", "float", "double", "byte", "binary",
            "date", "date-time", "password",
            # extra formats
            "uuid", "regex",
            "ipv4", "ipv6", "email", "idn-email", "time"
        ])
E       AssertionError: assert {'color', 'byte', 'idn-hostname', 'email', 'hostname', 'float', 'ipv6', 'idn-email', 'time', 'double', 'uri-template', 'password', 'uri-reference', 'duration', 'regex', 'uri', 'date', 'ipv4', 'int32', 'iri-reference', 'date-time', 'relative-json-pointer', 'iri', 'binary', 'uuid', 'json-pointer', 'int64'} == {'byte', 'time', 'ipv6', 'date', 'regex', 'double', 'email', 'float', 'int32', 'binary', 'password', 'uuid', 'ipv4', 'date-time', 'in
t64', 'idn-email'}
E         Extra items in the left set:
E         'color'
E         'uri'
E         'relative-json-pointer'
E         'idn-hostname'
E         'hostname'
E         'iri'
E         'uri-template'
E         'uri-reference'
E         'iri-reference'
E         'duration'
E         'json-pointer'
E         Full diff:
E           {
E            'binary',
E            'byte',
E         +  'color',
E            'date',
E            'date-time',
E            'double',
E         +  'duration',
E            'email',
E            'float',
E         +  'hostname',
E            'idn-email',
E         +  'idn-hostname',
E            'int32',
E            'int64',
E            'ipv4',
E            'ipv6',
E         +  'iri',
E         +  'iri-reference',
E         +  'json-pointer',
E            'password',
E            'regex',
E         +  'relative-json-pointer',
E            'time',
E         +  'uri',
E         +  'uri-reference',
E         +  'uri-template',
E            'uuid',
E           }

self       = <test_validators.TestOAS30Validator object at 0x7f6ee63cceb0>

tests/integration/test_validators.py:17: AssertionError
_______________________________________________ TestOAS31Validator.test_format_checkers _______________________________________________

self = <test_validators.TestOAS31Validator object at 0x7f6ee642d6f0>

    def test_format_checkers(self):
>       assert set(OAS31Validator.FORMAT_CHECKER.checkers.keys()) == set([
            # standard formats
            "int32", "int64", "float", "double", "password",
            # extra formats
            "date", "date-time", "uuid", "regex",
            "ipv4", "ipv6", "email", "idn-email", "time"
        ])
E       AssertionError: assert {'color', 'idn-hostname', 'email', 'hostname', 'float', 'ipv6', 'idn-email', 'time', 'double', 'uri-template', 'password', 'uri-reference', 'duration', 'regex', 'uri', 'date', 'ipv4', 'int32', 'iri-reference', 'date-time', 'relative-json-pointer', 'iri', 'uuid', 'json-pointer', 'int64'} == {'time', 'ipv6', 'date', 'regex', 'double', 'email', 'float', 'password', 'int32', 'uuid', 'ipv4', 'date-time', 'int64', 'idn-email'}
E         Extra items in the left set:
E         'color'
E         'uri'
E         'relative-json-pointer'
E         'idn-hostname'
E         'hostname'
E         'iri'
E         'uri-template'
E         'uri-reference'
E         'iri-reference'
E         'duration'
E         'json-pointer'
E         Full diff:
E           {
E         +  'color',
E            'date',
E            'date-time',
E            'double',
E         +  'duration',
E            'email',
E            'float',
E         +  'hostname',
E            'idn-email',
E         +  'idn-hostname',
E            'int32',
E            'int64',
E            'ipv4',
E            'ipv6',
E         +  'iri',
E         +  'iri-reference',
E         +  'json-pointer',
E            'password',
E            'regex',
E         +  'relative-json-pointer',
E            'time',
E         +  'uri',
E         +  'uri-reference',
E         +  'uri-template',
E            'uuid',
E           }

self       = <test_validators.TestOAS31Validator object at 0x7f6ee642d6f0>

tests/integration/test_validators.py:474: AssertionError
-- generated xml file: /tmp/portage/dev-python/openapi-schema-validator-0.4.2/work/openapi-schema-validator-0.4.2/reports/junit.xml ---
======================================================= short test summary info =======================================================

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions
      pFad - Phonifier reborn

      Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

      Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


      Alternative Proxies:

      Alternative Proxy

      pFad Proxy

      pFad v3 Proxy

      pFad v4 Proxy