Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate oneOf usage in API tests #6187

Closed
jeluard opened this issue Dec 13, 2023 · 1 comment · Fixed by #6354
Closed

Investigate oneOf usage in API tests #6187

jeluard opened this issue Dec 13, 2023 · 1 comment · Fixed by #6354
Labels
meta-investigate Issues found that require further investigation and may not have a specific resolution/fix scope-testing Issues for adding test coverage, fixing existing tests or testing strategies.

Comments

@jeluard
Copy link
Contributor

jeluard commented Dec 13, 2023

JSON Schema oneOf usage in API tests is problematic and lead to some workaround:

Investigate the underlying cause as it might lead to false negative or ignored failures.

@jeluard jeluard added scope-testing Issues for adding test coverage, fixing existing tests or testing strategies. meta-investigate Issues found that require further investigation and may not have a specific resolution/fix labels Dec 13, 2023
@jeluard
Copy link
Contributor Author

jeluard commented Dec 15, 2023

Note that the OpenAPI description of the beacon-APIs is not valid (see ethereum/beacon-APIs#393). Reported errors looks related to this issue:

{
    "schemaValidationMessages": [
        {
            "level": "error",
            "domain": "validation",
            "keyword": "oneOf",
            "message": "instance failed to match exactly one schema (matched 0 out of 2)",
            "schema": {
                "loadingURI": "#",
                "pointer": "/definitions/Components/properties/schemas/patternProperties/^[a-zA-Z0-9\\.\\-_]+$"
            },
            "instance": {
                "pointer": "/components/schemas/Bellatrix.BeaconBlock"
            }
        },
        {
            "level": "error",
            "domain": "validation",
            "keyword": "oneOf",
            "message": "instance failed to match exactly one schema (matched 0 out of 2)",
            "schema": {
                "loadingURI": "#",
                "pointer": "/definitions/Components/properties/schemas/patternProperties/^[a-zA-Z0-9\\.\\-_]+$"
            },
            "instance": {
                "pointer": "/components/schemas/Bellatrix.SignedBeaconBlock"
            }
        },
        {
            "level": "error",
            "domain": "validation",
            "keyword": "oneOf",
            "message": "instance failed to match exactly one schema (matched 0 out of 2)",
            "schema": {
                "loadingURI": "#",
                "pointer": "/definitions/Components/properties/schemas/patternProperties/^[a-zA-Z0-9\\.\\-_]+$"
            },
            "instance": {
                "pointer": "/components/schemas/Blob"
            }
        },
        {
            "level": "error",
            "domain": "validation",
            "keyword": "oneOf",
            "message": "instance failed to match exactly one schema (matched 0 out of 2)",
            "schema": {
                "loadingURI": "#",
                "pointer": "/definitions/Components/properties/schemas/patternProperties/^[a-zA-Z0-9\\.\\-_]+$"
            },
            "instance": {
                "pointer": "/components/schemas/Capella.BeaconBlock"
            }
        },
        {
            "level": "error",
            "domain": "validation",
            "keyword": "oneOf",
            "message": "instance failed to match exactly one schema (matched 0 out of 2)",
            "schema": {
                "loadingURI": "#",
                "pointer": "/definitions/Components/properties/schemas/patternProperties/^[a-zA-Z0-9\\.\\-_]+$"
            },
            "instance": {
                "pointer": "/components/schemas/Capella.SignedBeaconBlock"
            }
        },
        {
            "level": "error",
            "domain": "validation",
            "keyword": "oneOf",
            "message": "instance failed to match exactly one schema (matched 0 out of 2)",
            "schema": {
                "loadingURI": "#",
                "pointer": "/definitions/Components/properties/schemas/patternProperties/^[a-zA-Z0-9\\.\\-_]+$"
            },
            "instance": {
                "pointer": "/components/schemas/Deneb.BeaconBlock"
            }
        },
        {
            "level": "error",
            "domain": "validation",
            "keyword": "oneOf",
            "message": "instance failed to match exactly one schema (matched 0 out of 2)",
            "schema": {
                "loadingURI": "#",
                "pointer": "/definitions/Components/properties/schemas/patternProperties/^[a-zA-Z0-9\\.\\-_]+$"
            },
            "instance": {
                "pointer": "/components/schemas/Deneb.BlobSidecars"
            }
        },
        {
            "level": "error",
            "domain": "validation",
            "keyword": "oneOf",
            "message": "instance failed to match exactly one schema (matched 0 out of 2)",
            "schema": {
                "loadingURI": "#",
                "pointer": "/definitions/Components/properties/schemas/patternProperties/^[a-zA-Z0-9\\.\\-_]+$"
            },
            "instance": {
                "pointer": "/components/schemas/Deneb.BlockContents"
            }
        },
        {
            "level": "error",
            "domain": "validation",
            "keyword": "oneOf",
            "message": "instance failed to match exactly one schema (matched 0 out of 2)",
            "schema": {
                "loadingURI": "#",
                "pointer": "/definitions/Components/properties/schemas/patternProperties/^[a-zA-Z0-9\\.\\-_]+$"
            },
            "instance": {
                "pointer": "/components/schemas/Deneb.SignedBeaconBlock"
            }
        },
        {
            "level": "error",
            "domain": "validation",
            "keyword": "oneOf",
            "message": "instance failed to match exactly one schema (matched 0 out of 2)",
            "schema": {
                "loadingURI": "#",
                "pointer": "/definitions/Components/properties/schemas/patternProperties/^[a-zA-Z0-9\\.\\-_]+$"
            },
            "instance": {
                "pointer": "/components/schemas/Deneb.SignedBlobSidecar"
            }
        },
        {
            "level": "error",
            "domain": "validation",
            "keyword": "oneOf",
            "message": "instance failed to match exactly one schema (matched 0 out of 2)",
            "schema": {
                "loadingURI": "#",
                "pointer": "/definitions/Components/properties/schemas/patternProperties/^[a-zA-Z0-9\\.\\-_]+$"
            },
            "instance": {
                "pointer": "/components/schemas/Deneb.SignedBlockContents"
            }
        }
    ]
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
meta-investigate Issues found that require further investigation and may not have a specific resolution/fix scope-testing Issues for adding test coverage, fixing existing tests or testing strategies.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant