Add "$recurse" for extending recursive schemas

[handrews]

Addresses #558.

The 3rd and 2nd to last two commits rework $ref and $recurse as adjacent subsections in the "Schema Referencing" section, as most of their behavior is identical (particularly once #585 is complete).

The third-to-last commit just moves things around with no content changes other than obvious section titles, while the second-to-last commit updates the text, including minor updates to $ref's subsection.

[philsturgeon]

it MUST be treated as a reference to the schema document where processing was initiated.

vs

Note that even if processing began at a subschema within a document, the "$recurse" target MUST be the root of the document.

The first seems like its saying if we ref into a document at some lower level, it should recurse to THAT level, but the 2nd says it must be the root? Confusing.

[handrews]

Yeah, the key here is the wording "schema document" which is defined in section 4.3 in both draft-07 and the current HEAD commit. A schema document is a document, not a JSON object. So saying that it MUST be treated as a reference to the schema document where processing was initiated means exactly what the clarification says.

But I'm open to suggestion on how to word it better, preferably without entirely re-stating 4.3. Maybe just an <xref /> to it? I suspect the complete list of people who understand the distinction without having to look it consists of is @awwright and me.

[philsturgeon]

Oops?

[handrews]

er... yeah, thanks!

[awwright]

How would I refer to extended schemas from schemas of my own?

For example, suppose I have a media type that tries to represent HTML forms. It's an array of objects, each describing a form field. One of the properties is a schema that can describe a schema, and I want to

{
  type: "array",
  items: {
    type: "object",
    properties: {
      "name": { type: "string" },
      "schema": { $ref: "http://json-schema.org/draft-nnn/hyper-schema" }
    }
  }
}

where http://json-schema.org/draft-nnn/hyper-schema is some future schema for Hyper-schema that uses this feature.

What happens when a JSON Schema validator hits $recurse: true?

[handrews]

@awwright that is an excellent question, and definitely a problem for this proposal as it stands.

Embedding the meta-schema is a different use case than "extending" a schema (I hate to use the word "extending" as it makes people think of OOP, but JSON Schema is a constraint system which is very different- not sure what else to call it, though). But there is no way in this proposal to distinguish the two, so your example would break. And in a non-obvious way.

The simplest fix would be to add something alongside the $ref that controls whether recursion takes the containing schema into account. This draws from your other idea in #558 of changing how a specific $ref is resolved. I'm worried about the added complexity, but $recurse as proposed here can't work so we need some sort of fix.

The thing I want to preserve about $recurse is that it's target is dynamic, while $ref's target is static. I'm open to any proposal that keeps that distinction.

[handrews]

I'm going to move this discussion back to the issue and make some revised proposals there.

[handrews]

Closing per #558 (comment) due to the problems noted by @awwright. I hope to revisit this, but as noted in the issue, it will be a while and we can get by without it if we must.