From 194b545c8a85679e41ee5ff481b52123a7d37720 Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Mon, 26 Feb 2018 18:43:05 -0800 Subject: [PATCH 1/7] Move "$ref" section below "$id" Talking about references and dereferencing makes more sense after we establish base URIs and subschema identfication. This commit just moves the section, subsequent commits will adjust the text and titles accordingly. --- jsonschema-core.xml | 57 ++++++++++++++++++++++++--------------------- 1 file changed, 30 insertions(+), 27 deletions(-) diff --git a/jsonschema-core.xml b/jsonschema-core.xml index 61741cde..60a69bd7 100644 --- a/jsonschema-core.xml +++ b/jsonschema-core.xml @@ -527,33 +527,6 @@ -
- - The "$ref" keyword is used to reference a schema, and provides the ability to - validate recursive structures through self-reference. - - - An object schema with a "$ref" property MUST be interpreted as a "$ref" reference. - The value of the "$ref" property MUST be a URI Reference. - Resolved against the current URI base, it identifies the URI of a schema to use. - All other properties in a "$ref" object MUST be ignored. - - - The URI is not a network locator, only an identifier. A schema need not be - downloadable from the address if it is a network-addressable URL, and - implementations SHOULD NOT assume they should perform a network operation when they - encounter a network-addressable URI. - - - A schema MUST NOT be run into an infinite loop against a schema. For example, if two - schemas "#alice" and "#bob" both have an "allOf" property that refers to the other, - a naive validator might get stuck in an infinite recursive loop trying to validate - the instance. - Schemas SHOULD NOT make use of infinite recursive nesting like this; the behavior is - undefined. - -
-
@@ -652,6 +625,36 @@ urn:uuid:ee564b8a-7a87-4125-8c96-e9f123d6766f +
+ +
+ + The "$ref" keyword is used to reference a schema, and provides the ability to + validate recursive structures through self-reference. + + + An object schema with a "$ref" property MUST be interpreted as a "$ref" reference. + The value of the "$ref" property MUST be a URI Reference. + Resolved against the current URI base, it identifies the URI of a schema to use. + All other properties in a "$ref" object MUST be ignored. + + + The URI is not a network locator, only an identifier. A schema need not be + downloadable from the address if it is a network-addressable URL, and + implementations SHOULD NOT assume they should perform a network operation when they + encounter a network-addressable URI. + + + A schema MUST NOT be run into an infinite loop against a schema. For example, if two + schemas "#alice" and "#bob" both have an "allOf" property that refers to the other, + a naive validator might get stuck in an infinite recursive loop trying to validate + the instance. + Schemas SHOULD NOT make use of infinite recursive nesting like this; the behavior is + undefined. + +
+ +
Schemas can be identified by any URI that has been given to them, including From 1ccab5e46c9cad37abf2d2b0b823534adbea04ce Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Fri, 23 Feb 2018 23:35:45 -0800 Subject: [PATCH 2/7] Expand $id example to show all valid URIs This clarifies that JSON Pointers work from any base URI within a document to any location underneath that base, and shows that this can produce several different valid URIs for a subschema. --- jsonschema-core.xml | 77 ++++++++++++++++++++++++++++++++++----------- 1 file changed, 58 insertions(+), 19 deletions(-) diff --git a/jsonschema-core.xml b/jsonschema-core.xml index 60a69bd7..9be3ae53 100644 --- a/jsonschema-core.xml +++ b/jsonschema-core.xml @@ -584,9 +584,13 @@ matches the above requirements nor is a valid JSON pointer is not defined. - - For example: +
+ + Consider the following schema, which shows "$id" being used to identify + the root schema, change the base URI for subschemas, and assign plain + name fragments to subschemas: +
- - - The schemas at the following URI-encoded JSON - Pointers (relative to the root schema) have the following - base URIs, and are identifiable by either URI in accordance with - Section above: - - - - http://example.com/root.json# - http://example.com/root.json#foo - http://example.com/other.json - http://example.com/other.json#bar - http://example.com/t/inner.json - urn:uuid:ee564b8a-7a87-4125-8c96-e9f123d6766f - - + + The schemas at the following URI-encoded JSON + Pointers (relative to the root schema) have the following + base URIs, and are identifiable by any listed URI in accordance with + Section above: + + + + + + http://example.com/root.json + http://example.com/root.json# + + + + + http://example.com/root.json#foo + http://example.com/root.json#/definitions/A + + + + + http://example.com/other.json + http://example.com/other.json# + http://example.com/root.json#/definitions/B + + + + + http://example.com/other.json#bar + http://example.com/other.json#/definitions/X + http://example.com/root.json#/definitions/B/definitions/X + + + + + http://example.com/t/inner.json + http://example.com/t/inner.json# + http://example.com/other.json#/definitions/Y + http://example.com/root.json#/definitions/B/definitions/Y + + + + + urn:uuid:ee564b8a-7a87-4125-8c96-e9f123d6766f + urn:uuid:ee564b8a-7a87-4125-8c96-e9f123d6766f# + http://example.com/root.json#/definitions/C + + + + +
From 5947aafdc11d9d1740efe7b495103acdff8529a3 Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Tue, 27 Feb 2018 22:24:17 -0800 Subject: [PATCH 3/7] "$id" subsections by use case This re-organizes and expands the existing text for "$id" into subsections based on the three primary use cases: * Identifying the root schema in a schema document * Changing the base URI (with implications for JSON Pointer fragments) * Defining a location-independent name All of these are demonstrated in the subsequent example section. --- jsonschema-core.xml | 78 ++++++++++++++++++++++++++++----------------- 1 file changed, 48 insertions(+), 30 deletions(-) diff --git a/jsonschema-core.xml b/jsonschema-core.xml index 9be3ae53..d301a733 100644 --- a/jsonschema-core.xml +++ b/jsonschema-core.xml @@ -554,36 +554,54 @@ This value SHOULD be normalized, and SHOULD NOT be an empty fragment <#> or an empty string <>. - - The root schema of a JSON Schema document SHOULD contain an "$id" keyword with - a URI (containing a scheme). This URI SHOULD either not have a fragment, or - have one that is an empty string. - - - How should an "$id" URI reference containing a fragment with other components - be interpreted? There are two cases: when the other components match - the current base URI and when they change the base URI. - - - - To name subschemas in a JSON Schema document, - subschemas can use "$id" to give themselves a document-local identifier. - This is done by setting "$id" to a URI reference consisting - only of a plain name fragment (not a JSON Pointer fragment). - The fragment identifier MUST begin with a letter ([A-Za-z]), followed by - any number of letters, digits ([0-9]), hyphens ("-"), underscores ("_"), colons - (":"), or periods ("."). - - - Providing a plain name fragment enables a subschema to be - relocated within a schema without requiring that JSON - Pointer references are updated. - - - The effect of defining a fragment-only "$id" URI reference that neither - matches the above requirements nor is a valid JSON pointer - is not defined. - +
+ + The root schema of a JSON Schema document SHOULD contain an "$id" keyword with + a URI (containing a scheme). This URI SHOULD either not have a fragment, or + have one that is an empty string. + + +
+
+ + When an "$id" sets the base URI, the object containing that "$id" and all of + its subschemas can be identified by using a JSON Pointer fragment starting + from that location. This is true even of subschemas that further change the + base URI. Therefore, a single subschema may be accessible by multiple URIs, + each consisting of base URI declared in the subschema or a parent, along with + a JSON Pointer fragment identifying the path from the schema object that + declares the base to the subschema being identified. Examples of this are + shown in section . + +
+
+ + Using JSON Pointer fragments requires knowledge of the structure of the schema. + When writing schema documents with the intention to provide re-usable + schemas, it is preferable to use a plain name fragment that is not tied to + any particular structural location. This allows a subschema to be relocated + without requiring JSON Pointer references to be updated. + + + To name subschemas in a JSON Schema document, + subschemas can use "$id" to give themselves a document-local identifier. + This is done by setting "$id" to a URI reference consisting + only of a plain name fragment (not a JSON Pointer fragment). + The fragment identifier MUST begin with a letter ([A-Za-z]), followed by + any number of letters, digits ([0-9]), hyphens ("-"), underscores ("_"), colons + (":"), or periods ("."). + + + The effect of defining a fragment-only "$id" URI reference that neither + matches the above requirements nor is a valid JSON pointer + is not defined. + + How should an "$id" URI reference containing a fragment with other components + be interpreted? There are two cases: when the other components match + the current base URI and when they change the base URI. + + +
From 2040d745dd221fb7785aca86a045742ebde8b1a1 Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Tue, 27 Feb 2018 22:40:42 -0800 Subject: [PATCH 4/7] Move and rename the "External References" section Move it up under "$ref", and change the title to be about loading referenced schemas. Some further wording tweaks will be required to the contents, this just moves the paragraph without changing it. --- jsonschema-core.xml | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/jsonschema-core.xml b/jsonschema-core.xml index d301a733..ac372965 100644 --- a/jsonschema-core.xml +++ b/jsonschema-core.xml @@ -709,6 +709,25 @@ Schemas SHOULD NOT make use of infinite recursive nesting like this; the behavior is undefined. +
+ + To differentiate schemas between each other in a vast ecosystem, schemas are + identified by URI. As specified above, this does not necessarily mean + anything is downloaded, but instead JSON Schema implementations SHOULD + already understand the schemas they will be using, including the URIs that + identify them. + + + Implementations SHOULD be able to associate arbitrary URIs with an arbitrary + schema and/or automatically associate a schema's "$id"-given URI, depending + on the trust that the validator has in the schema. + + + A schema MAY (and likely will) have multiple URIs, but there is no way for a + URI to identify more than one schema. When multiple schemas try to identify + with the same URI, validators SHOULD raise an error condition. + +
@@ -758,25 +777,6 @@ resolve the fragment against the base URI.
-
- - To differentiate schemas between each other in a vast ecosystem, schemas are - identified by URI. As specified above, this does not necessarily mean - anything is downloaded, but instead JSON Schema implementations SHOULD - already understand the schemas they will be using, including the URIs that - identify them. - - - Implementations SHOULD be able to associate arbitrary URIs with an arbitrary - schema and/or automatically associate a schema's "$id"-given URI, depending - on the trust that the validator has in the schema. - - - A schema MAY (and likely will) have multiple URIs, but there is no way for a - URI to identify more than one schema. When multiple schemas try to identify - with the same URI, validators SHOULD raise an error condition. - -
From 8ca3ecac33df23f66e433520e8967b4f9fc569f3 Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Tue, 27 Feb 2018 23:01:09 -0800 Subject: [PATCH 5/7] More detail on associating URIs and schemas Note the case of identifying schemas during processing, as shown in the earlier example. This replaces the explicitly called-out concept of "Internal Referencing" which will be removed. --- jsonschema-core.xml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/jsonschema-core.xml b/jsonschema-core.xml index ac372965..bce62f54 100644 --- a/jsonschema-core.xml +++ b/jsonschema-core.xml @@ -720,7 +720,10 @@ Implementations SHOULD be able to associate arbitrary URIs with an arbitrary schema and/or automatically associate a schema's "$id"-given URI, depending - on the trust that the validator has in the schema. + on the trust that the validator has in the schema. Such URIs and schemas + can be supplied to an implementation prior to processing instances, or may + be noted within a schema document as it is processed, producing associations + as shown in section . A schema MAY (and likely will) have multiple URIs, but there is no way for a From bb08511a984c51a12327c2df91bb763c86a1690d Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Tue, 27 Feb 2018 23:31:06 -0800 Subject: [PATCH 6/7] Remove "Internal Referencing" and rework example We know from feedback that the "internal referencing" concept and in particular the "same document" wording is confusing and for some people misleading. Remove the term and focus the wording on normal RFC 3986 URI reference resolution. Cover the same logical ground as "internal referencing" by discussing whether and how the implementation knows of the identified schema, using the same general ideas as covered in the previous section on loading (or not loading) referenced schemas. --- jsonschema-core.xml | 38 ++++++++++++++++++++++++++------------ 1 file changed, 26 insertions(+), 12 deletions(-) diff --git a/jsonschema-core.xml b/jsonschema-core.xml index bce62f54..96591f4c 100644 --- a/jsonschema-core.xml +++ b/jsonschema-core.xml @@ -731,19 +731,19 @@ with the same URI, validators SHOULD raise an error condition.
- - -
-
+
Schemas can be identified by any URI that has been given to them, including - a JSON Pointer or their URI given directly by "$id". + a JSON Pointer or their URI given directly by "$id". In all cases, + dereferencing a "$ref" reference involves first resolving its value as a + URI reference against the current base URI per + RFC 3986. - Tools SHOULD take note of the URIs that schemas, including subschemas, - provide for themselves using "$id". This is known as "Internal referencing". + If the resulting URI identifies a schema within the current document, or + within another schema document that has been made available to the implementation, + then that schema SHOULD be used automatically. - For example, consider this schema: @@ -760,7 +760,8 @@ "definitions": { "single": { "$id": "#item", - "type": "integer" + "type": "object", + "additionalProperties": { "$ref": "other.json" } } } } @@ -775,9 +776,22 @@ When an implementation then looks inside the <#/items> schema, it encounters the <#item> reference, and resolves this to - <http://example.net/root.json#item> which is understood as the schema - defined elsewhere in the same document without needing to - resolve the fragment against the base URI. + <http://example.net/root.json#item>, which it has seen defined in + this same document and can therefore use automatically. + + + When an implementation encounters the reference to "other.json", it resolves + this to <http://example.net/other.json>, which is not defined in this + document. If a schema with that identifier has otherwise been supplied to + the implementation, it can also be used automatically. + + What should implementations do when the referenced schema is not known? + Are there circumstances in which automatic network dereferencing is + allowed? A same origin policy? A user-configurable option? In the + case of an evolving API described by Hyper-Schema, it is expected that + new schemas will be added to the system dynamically, so placing an + absolute requirement of pre-loading schema documents is not feasible. +
From 2fa90f6646d3bd56d451aaf6cbd79f4057776648 Mon Sep 17 00:00:00 2001 From: Henry Andrews Date: Wed, 28 Feb 2018 11:33:40 -0800 Subject: [PATCH 7/7] "may be preferable" instead of "is preferable" Feedback from PR that this should be more of a suggestion. We may strengthen later. --- jsonschema-core.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jsonschema-core.xml b/jsonschema-core.xml index 96591f4c..b74fe22a 100644 --- a/jsonschema-core.xml +++ b/jsonschema-core.xml @@ -578,7 +578,7 @@ Using JSON Pointer fragments requires knowledge of the structure of the schema. When writing schema documents with the intention to provide re-usable - schemas, it is preferable to use a plain name fragment that is not tied to + schemas, it may be preferable to use a plain name fragment that is not tied to any particular structural location. This allows a subschema to be relocated without requiring JSON Pointer references to be updated.