Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON Schema output definitions attribute name change #311

Merged
merged 5 commits into from
Sep 13, 2021

Conversation

sujaypatil96
Copy link
Member

Per @gaurav's recommendation in issue #305, the following PR seeks to change the JSON Schema generator such that it outputs the JSON Schema with the new $defs attribute rather than using the definitions attribute.

I tested the PR on the examples\organization.yaml schema.

$ gen-json-schema examples/organization.yaml

Here's what the output looks like:

{
   "$defs": {
      "Employee": {
         "additionalProperties": false,
         "description": "A person employed by an organization",
         "properties": {
            "age_in_years": {
               "description": "The age of a person if living or age of death if not",
               "type": "integer"
            },
            "aliases": {
               "description": "An alternative name",
               "items": {
                  "type": "string"
               },
               "type": "array"
            },
            "first_name": {
               "description": "The first name of a person",
               "type": "string"
            },
            "id": {
               "description": "Unique identifier of a person",
               "type": "string"
            },
            "last_name": {
               "type": "string"
            }
         },
         "required": [
            "id",
            "last_name"
         ],
         "title": "Employee",
         "type": "object"
      },
      "Manager": {
         "additionalProperties": false,
         "description": "An employee who manages others",
         "properties": {
            "age_in_years": {
               "description": "The age of a person if living or age of death if not",
               "type": "integer"
            },
            "aliases": {
               "description": "An alternative name",
               "items": {
                  "type": "string"
               },
               "type": "array"
            },
            "first_name": {
               "description": "The first name of a person",
               "type": "string"
            },
            "has_employees": {
               "items": {
                  "$ref": "#/definitions/Employee"
               },
               "type": "array"
            },
            "id": {
               "description": "Unique identifier of a person",
               "type": "string"
            },
            "last_name": {
               "type": "string"
            }
         },
         "required": [
            "id",
            "last_name"
         ],
         "title": "Manager",
         "type": "object"
      },
      "Organization": {
         "additionalProperties": false,
         "description": "An entity comprised of blah blah",
         "properties": {
            "has_boss": {
               "$ref": "#/definitions/Manager"
            },
            "id": {
               "description": "Unique identifier of a person",
               "type": "string"
            },
            "name": {
               "description": "human readable name",
               "type": "string"
            }
         },
         "required": [
            "id"
         ],
         "title": "Organization",
         "type": "object"
      }
   },
   "$id": "http://example.org/sample/organization",
   "$schema": "http://json-schema.org/draft-07/schema#",
   "additionalProperties": true,
   "properties": {},
   "title": "organization",
   "type": "object"
}

@sujaypatil96 sujaypatil96 linked an issue Aug 31, 2021 that may be closed by this pull request
@sujaypatil96 sujaypatil96 changed the title JSON Schema output definitions attribute name change JSON Schema output definitions attribute name change Aug 31, 2021
@sujaypatil96
Copy link
Member Author

Looks like this change is causing a lot of tests to fail.

Copy link
Contributor

@gaurav gaurav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these changes look great! However, do note that you need to also update everywhere that #/definitions/{...} is used as a reference name, such as:

'items': {'$ref': f"#/definitions/{camelcase(c)}"}}

This should be changed to #/$defs/{...}, I think. Test it out, and let me know if I'm wrong!

The reason you're running into test failures is because a lot of the LinkML tests have expected output written into the repository -- so when you change the JSON Schema file, you'll also need to change the expected output files. I don't actually know how to run the tests via tox (maybe @cmungall knows?), but I use pipenv run python setup.py test to run all the tests, or pipenv run python setup.py test -s {test suite/case name} to run a particular test -- for example, I can run tests relating to #/definitions/{...} that are currently failing for me by running pipenv run python setup.py test -s 'tests.test_generators.test_jsonschemagen'. You'll need to run pipenv install to set up the Pipenv first.

@cmungall cmungall added the json label Aug 31, 2021
@cmungall
Copy link
Member

Let's discuss testing on the call on Friday

You can also run the tests directly from pycharm. note if you do run all the tests then you will get changes in the output folders tht shouldn't be committed. This all needs documented better: #237

@sujaypatil96
Copy link
Member Author

@gaurav: you're exactly right, the references do need to be changed to $defs as well.

After this commit, we can see the output as below:

{
   "$defs": {
      "Employee": {
         "additionalProperties": false,
         "description": "A person employed by an organization",
         "properties": {
            "age_in_years": {
               "description": "The age of a person if living or age of death if not",
               "type": "integer"
            },
            "aliases": {
               "description": "An alternative name",
               "items": {
                  "type": "string"
               },
               "type": "array"
            },
            "first_name": {
               "description": "The first name of a person",
               "type": "string"
            },
            "id": {
               "description": "Unique identifier of a person",
               "type": "string"
            },
            "last_name": {
               "type": "string"
            }
         },
         "required": [
            "id",
            "last_name"
         ],
         "title": "Employee",
         "type": "object"
      },
      "Manager": {
         "additionalProperties": false,
         "description": "An employee who manages others",
         "properties": {
            "age_in_years": {
               "description": "The age of a person if living or age of death if not",
               "type": "integer"
            },
            "aliases": {
               "description": "An alternative name",
               "items": {
                  "type": "string"
               },
               "type": "array"
            },
            "first_name": {
               "description": "The first name of a person",
               "type": "string"
            },
            "has_employees": {
               "items": {
                  "$ref": "#/$defs/Employee"
               },
               "type": "array"
            },
            "id": {
               "description": "Unique identifier of a person",
               "type": "string"
            },
            "last_name": {
               "type": "string"
            }
         },
         "required": [
            "id",
            "last_name"
         ],
         "title": "Manager",
         "type": "object"
      },
      "Organization": {
         "additionalProperties": false,
         "description": "An entity comprised of blah blah",
         "properties": {
            "has_boss": {
               "$ref": "#/$defs/Manager"
            },
            "id": {
               "description": "Unique identifier of a person",
               "type": "string"
            },
            "name": {
               "description": "human readable name",
               "type": "string"
            }
         },
         "required": [
            "id"
         ],
         "title": "Organization",
         "type": "object"
      }
   },
   "$id": "http://example.org/sample/organization",
   "$schema": "http://json-schema.org/draft-07/schema#",
   "additionalProperties": true,
   "properties": {},
   "title": "organization",
   "type": "object"
}

Yup, I realized that a few tests would fail considering it's being tested against the previous expected output which used definitions rather than $defs, but there are quite a few of them that are failing as you can see from this action. I'll try to figure out which ones need to be modified exactly.

@gaurav
Copy link
Contributor

gaurav commented Aug 31, 2021 via email

Copy link
Member

@cmungall cmungall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge when tests pass

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Replace definitions with $defs in jsonschemagen.py
3 participants