Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Link's ref path needs an update for all the event schemas #211

Closed
rjalander opened this issue May 8, 2024 · 6 comments · Fixed by #219
Closed

Link's ref path needs an update for all the event schemas #211

rjalander opened this issue May 8, 2024 · 6 comments · Fixed by #219
Assignees
Milestone

Comments

@rjalander
Copy link

The ref path needs to be updated with the correct path to refer links,
From

"$ref": "/schema/links/embeddedlinksarray.json"

To

"$ref": "./links/embeddedlinksarray.json"

@afrittoli
Copy link
Contributor

Thanks, I also realised that while working on the golang SDK.
I'm just waiting to see if there is any more issue, and then I'll make a PR.

@afrittoli afrittoli self-assigned this May 8, 2024
@afrittoli
Copy link
Contributor

I believe a reference "$ref": "links/embeddedlinksarray.json" would work too.

@afrittoli
Copy link
Contributor

According to the jsonschema docs, schema references are relative to the schema baseURI.

In our case, the schema id looks like https://cdevents.dev/0.4.0/schema/artifact-deleted-event, the baseURI would be https://cdevents.dev/ and the $ref for the embedded schema should be /0.4.0/schema/links/embeddedlinksarray.json.

I tried with check-jsonschema and the format /0.4.0/schema/links/embeddedlinksarray.json works with it - the tool fetches the schema from the internet:

➜ check-jsonschema examples/artifact_deleted.json --schemafile schemas/artifactdeleted.json
ok -- validation done

If I used the --base-uri option instead, it works as long as $ref does not start with a / and the combination baseUri + $ref points to a valid location on my local disk. For instance, with $ref == schemas/links/embeddedlinksarray.json, this works:

➜ check-jsonschema examples/artifact_deleted.json --schemafile schemas/artifactdeleted.json --base-uri /git/github.com/cdevents/spec/
ok -- validation done

In CI we need to use local files (not fetched from the internet) because the schemas from PRs have not been published yet.
In the SDK the schemas can be embedded in the code, so they don't have to be fetched every time. The code that generates the actual SDK code should rely on the local versions from disk too.

In the jsonschema docs it even says that the URIs do not necessarily need to be network addressable, they're only identifiers, and that typically the schemas are embedded in the tool that does the validation.

I lean slightly towards having the internet based resolution working (i.e. using a $ref like /0.4.0/schema/links/embeddedlinksarray.json) and have tools like CI and code generation do some magic (like rewrite the refs) to be able to resolve all the references from local disk.

@xibz @e-backmark-ericsson @rjalander WDYT?

@afrittoli afrittoli added this to the v0.4 milestone May 9, 2024
@afrittoli
Copy link
Contributor

Looking at how references are used for the meta schemas:

So one level is stripped from the root schema ID and then the relative path applied

If we followed the same approach for CDEvents:

I think the relative references option would be better because, to use the absolute one, we would include the spec version in the reference, so every new release would require updating all references, even if it wasn't changed.

@rjalander
Copy link
Author

As per the jsonschema docs, the $ref is a URI-reference that is resolved against the schema's Base URI.
So we can not use ref pointing from a classpath or local system path, it always resolved against root schema ID URL,
In this case the schema fails to load without internet,

[main] ERROR com.networknt.schema.JsonSchemaFactory - Failed to load json schema from https://cdevents.dev/0.4.0/schema/links/embeddedlinksarray.json
java.net.UnknownHostException: cdevents.dev

@afrittoli
Copy link
Contributor

@rjalander what I found is that you need to load the schemas from the local disk into your validator, so that it will use its local database of schemas instead of going out to the internet.

afrittoli added a commit to afrittoli/cdevents-spec that referenced this issue May 10, 2024
Add validation of jsonschemas to our CI.
The links and embeddedlinks schemas where invalid,
as well as the references to them from the event
schemas, so fixing all those to let CI pass.

Switch to ajv for example validation, which allows for
validation of both JSON documents as well as schemas.

ajv let users load references schemas from files into
the tool offline databases, which enables offline
validation of unpublished schemas.

Since ajv is javascript, update the GitHub action
workflow to javascript.

Fixes: cdevents#208
Fixes: cdevents#211

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
afrittoli added a commit to afrittoli/cdevents-spec that referenced this issue May 10, 2024
Add validation of jsonschemas to our CI.
The links and embeddedlinks schemas where invalid,
as well as the references to them from the event
schemas, so fixing all those to let CI pass.

Switch to ajv for example validation, which allows for
validation of both JSON documents as well as schemas.

ajv let users load references schemas from files into
the tool offline databases, which enables offline
validation of unpublished schemas.

Since ajv is javascript, update the GitHub action
workflow to javascript.

Fixes: cdevents#208
Fixes: cdevents#211

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
afrittoli added a commit to afrittoli/cdevents-spec that referenced this issue May 10, 2024
Add validation of jsonschemas to our CI.
The links and embeddedlinks schemas where invalid,
as well as the references to them from the event
schemas, so fixing all those to let CI pass.

Switch to ajv for example validation, which allows for
validation of both JSON documents as well as schemas.

ajv let users load references schemas from files into
the tool offline databases, which enables offline
validation of unpublished schemas.

Since ajv is javascript, update the GitHub action
workflow to javascript.

Fixes: cdevents#208
Fixes: cdevents#211

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
afrittoli added a commit to afrittoli/cdevents-spec that referenced this issue May 10, 2024
Add validation of jsonschemas to our CI.
The links and embeddedlinks schemas where invalid,
as well as the references to them from the event
schemas, so fixing all those to let CI pass.

Switch to ajv for example validation, which allows for
validation of both JSON documents as well as schemas.

ajv let users load references schemas from files into
the tool offline databases, which enables offline
validation of unpublished schemas.

Since ajv is javascript, update the GitHub action
workflow to javascript.

Fixes: cdevents#208
Fixes: cdevents#211

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
afrittoli added a commit to afrittoli/cdevents-spec that referenced this issue May 10, 2024
Add validation of jsonschemas to our CI.
The links and embeddedlinks schemas where invalid,
as well as the references to them from the event
schemas, so fixing all those to let CI pass.

Switch to ajv for example validation, which allows for
validation of both JSON documents as well as schemas.

ajv let users load references schemas from files into
the tool offline databases, which enables offline
validation of unpublished schemas.

Since ajv is javascript, update the GitHub action
workflow to javascript.

Fixes: cdevents#208
Fixes: cdevents#211

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
afrittoli added a commit to afrittoli/cdevents-spec that referenced this issue May 10, 2024
Add validation of jsonschemas to our CI.
The links and embeddedlinks schemas where invalid,
as well as the references to them from the event
schemas, so fixing all those to let CI pass.

Switch to ajv for example validation, which allows for
validation of both JSON documents as well as schemas.

ajv let users load references schemas from files into
the tool offline databases, which enables offline
validation of unpublished schemas.

Since ajv is javascript, update the GitHub action
workflow to javascript.

Fixes: cdevents#208
Fixes: cdevents#211

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
afrittoli added a commit to afrittoli/cdevents-spec that referenced this issue May 10, 2024
Add validation of jsonschemas to our CI.
The links and embeddedlinks schemas where invalid,
as well as the references to them from the event
schemas, so fixing all those to let CI pass.

Switch to ajv for example validation, which allows for
validation of both JSON documents as well as schemas.

ajv let users load references schemas from files into
the tool offline databases, which enables offline
validation of unpublished schemas.

Since ajv is javascript, update the GitHub action
workflow to javascript.

Fixes: cdevents#208
Fixes: cdevents#211

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
afrittoli added a commit to afrittoli/cdevents-spec that referenced this issue May 10, 2024
Add validation of jsonschemas to our CI.
The links and embeddedlinks schemas where invalid,
as well as the references to them from the event
schemas, so fixing all those to let CI pass.

Switch to ajv for example validation, which allows for
validation of both JSON documents as well as schemas.

ajv let users load references schemas from files into
the tool offline databases, which enables offline
validation of unpublished schemas.

Since ajv is javascript, update the GitHub action
workflow to javascript.

Fixes: cdevents#208
Fixes: cdevents#211

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
afrittoli added a commit to afrittoli/cdevents-spec that referenced this issue May 10, 2024
Add validation of jsonschemas to our CI.
The links and embeddedlinks schemas where invalid,
as well as the references to them from the event
schemas, so fixing all those to let CI pass.

Switch to ajv for example validation, which allows for
validation of both JSON documents as well as schemas.

ajv let users load references schemas from files into
the tool offline databases, which enables offline
validation of unpublished schemas.

Since ajv is javascript, update the GitHub action
workflow to javascript.

Fixes: cdevents#208
Fixes: cdevents#211

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
afrittoli added a commit to afrittoli/cdevents-spec that referenced this issue May 10, 2024
Add validation of jsonschemas to our CI.
The links and embeddedlinks schemas where invalid,
as well as the references to them from the event
schemas, so fixing all those to let CI pass.

Switch to ajv for example validation, which allows for
validation of both JSON documents as well as schemas.

ajv let users load references schemas from files into
the tool offline databases, which enables offline
validation of unpublished schemas.

Since ajv is javascript, update the GitHub action
workflow to javascript.

Fixes: cdevents#208
Fixes: cdevents#211

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
afrittoli added a commit to afrittoli/cdevents-spec that referenced this issue May 10, 2024
Add validation of jsonschemas to our CI.
The links and embeddedlinks schemas where invalid,
as well as the references to them from the event
schemas, so fixing all those to let CI pass.

Switch to ajv for example validation, which allows for
validation of both JSON documents as well as schemas.

ajv let users load references schemas from files into
the tool offline databases, which enables offline
validation of unpublished schemas.

Since ajv is javascript, update the GitHub action
workflow to javascript.

Fixes: cdevents#208
Fixes: cdevents#211

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
afrittoli added a commit to afrittoli/cdevents-spec that referenced this issue May 10, 2024
Add validation of jsonschemas to our CI.
The links and embeddedlinks schemas where invalid,
as well as the references to them from the event
schemas, so fixing all those to let CI pass.

Switch to ajv for example validation, which allows for
validation of both JSON documents as well as schemas.

ajv let users load references schemas from files into
the tool offline databases, which enables offline
validation of unpublished schemas.

Since ajv is javascript, update the GitHub action
workflow to javascript.

Fixes: cdevents#208
Fixes: cdevents#211

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
afrittoli added a commit that referenced this issue May 14, 2024
Add validation of jsonschemas to our CI.
The links and embeddedlinks schemas where invalid,
as well as the references to them from the event
schemas, so fixing all those to let CI pass.

Switch to ajv for example validation, which allows for
validation of both JSON documents as well as schemas.

ajv let users load references schemas from files into
the tool offline databases, which enables offline
validation of unpublished schemas.

Since ajv is javascript, update the GitHub action
workflow to javascript.

Fixes: #208
Fixes: #211

Signed-off-by: Andrea Frittoli <andrea.frittoli@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants