Add support for draft-06

[erayd]

What

Meta-issue for organising the addition of draft-06 support. Draft-06 is currently in final review and will probably be released sometime in the next few days.

Why

It's the next version of the official spec. We are a validator of the official spec. Need I say more?

Related Issues

• Type validation conflict between draft-03 and later versions #374 (resolve draft-03 version conflicts)

[erayd]

@bighappyface / @shmax Do you have any objection to me adding this as a project dependency?

[shmax]

Will have to get back to you on this tomorrow, but looks interesting...

[shmax]

So where can we see the official draft-06 spec?

[erayd]

It's not released yet, but @handrews says it should be this week (see this comment from four days ago).

In the meantime, the master branch here should be pretty close to the final form of draft-06 :-).

[shmax]

Hmm, okay. And what do you mean about adding your other repo to this one as a dependency?

[shmax]

Also, did we skip a draft? What happened to 5?

[erayd]

Hmm, okay. And what do you mean about adding your other repo to this one as a dependency?

I mean add it to the composer "require" section. It's useful for more than just this project, so my intention is to maintain it separately in that repo. Could also copy / paste it directly into this project if you'd rather avoid the dependency (although that has the disadvantage of no automatic composer updates); the bit that matters is just one file.

Also, did we skip a draft? What happened to 5?

Draft-05 is just a tidy-up of draft-04. There are no significant changes from draft-04, and no meta-schema exists.

[shmax]

So the idea is that we import your spec object, and use it to decide how to handle various rules as we validate? I think that sounds better in principle than what we're doing now, which seems to be to try to make guesses about which schema is in play by examining the schema object itself (eg. checking to see if "required" is an array, rather than checking the draft version). On the other hand, we never really got a chance to review your other project, and I'm sure we would have had feedback (for example, I would have suggested an override architecture instead of what you have, which is an all-in-one). Would you be open to a PR over yonder?

Semi off-topic; my brain is too blurry to figure it out for myself, so maybe you can just tell me: does draft 6 have any mechanism for creating custom types? It's always felt to me like the most obvious deficiency in the spec, and to be honest the only reason I wound up with JSON schema and not RAML is because I couldn't find any full implementations of RAML 1.0 in PHP.

[erayd]

So the idea is that we import your spec object, and use it to decide how to handle various rules as we validate?

Yes. That project is basically intended to provide information in the state of various validation rules, depending on which version of the spec you tell it you're using.

Would you be open to a PR over yonder?

Absolutely. I've made you and @bighappyface contributors too - I see no reason why you guys shouldn't have push access. I care that stuff gets reviewed before it hits master though.

Does draft 6 have any mechanism for creating custom types?

No, but it does allow custom formats. Is there something you're wanting to achieve that custom formats does not cover?

[erayd]

I would have suggested an override architecture...

Can you expand on this a bit? I don't understand what you mean by an override architecture in this context.

I have no objection in principle to changing the architecture, so long as it remains simple to use.

[erayd]

One other thought on the custom types - from a quick look at that RAML page you linked, using $ref and linking from #definitions would seem to cover the same use-case, if you were to implement that with JSON schema.

[shmax]

Okay. I'll look at it in a little more depth in the next few days.

Is there something you're wanting to achieve that custom formats does not cover?

Totally. In my own schemas, I quite often nest sub-schemas. Let's say I have a model I need to validate called "sku", and sku has properties called "frontPhoto" and "backPhoto". I can factor out a new sub-schema called "photo", and it has its own properties ("submitter", "date", etc). Unfortunately, it's impossible to just drop it in and still override little bits and pieces to taste, such as "description" and "title". My options are to sort of build out the inheritance in PHP, or limit "photo" to the "properties" of a photo schema, which looks sort of like this:

{
  description:'a sku record',
  type:'object',
  properties: {
    frontPhoto: {
       description: "A photo of the front of the item",
       type:"object",
       properties: {
          '$ref':"/path/to/photo.json"
       } 
    },
    backPhoto: {
       description: "A photo of the back of the item",
       type:"object",
       properties: {
          '$ref':"/path/to/photo.json"
       } 
    }
  }
}

What I would LIKE to be able to do is this:

{
  description:'a sku record',
  type:'object',
  properties: {
    frontPhoto: {
       description: "A photo of the front of the item",
       type:"photo"
    },
    backPhoto: {
       description: "A photo of the back of the item",
       type:"photo"
    }
  }
}

[shmax]

Can you expand on this a bit? I don't understand what you mean by an override architecture in this context.

We can talk about it in the other repo, but the idea is that you use either class inheritance, or a system of config files that layer on top of each other, such that a lower schema has no knowledge of a higher schema.

[shmax]

One other thought on the custom types - from a quick look at that RAML page you linked, using $ref and linking from #definitions would seem to cover the same use-case, if you were to implement that with JSON schema.

Unfortunately, not. You can use $ref, yes, but when you drop it in it's a solid block; you can't, say, touch up the description or title. See my earlier comment.

[erayd]

What I would LIKE to be able to do is this

OK, yeah you can't do that (as far as I'm aware) with draft-06, but inheritance is something that a lot of people have been asking for - I've seen it crop up in their issue tracker in a number of places. I suspect it'll find its way into the spec sooner or later, but it would IMO be worth adding your voice too... the more people who ask, the more likely it is that they'll do something with it.

[shmax]

Fair enough. Interested in collaborating on a feature request, or issue, or whatever it is they do?

[erayd]

Fair enough. Interested in collaborating on a feature request, or issue, or whatever it is they do?

I think we need to check whether there is already an open issue at the moment, and I think we need to wait until draft-06 is final before asking (that's where their attention is right now). But yes, in principle I'd be happy to collaborate on that.

We can talk about it in the other repo, but the idea is that you use either class inheritance, or a system of config files that layer on top of each other, such that a lower schema has no knowledge of a higher schema.

I like that idea. Can you open an issue for it on the other repo?

[shmax]

Will do. Off to bed, though.

[erayd]

OK - sleep well, and catch you later :-).

[handrews]

@erayd @shmax you're looking for json-schema-org/json-schema-spec#98 (overriding annotation properties at point of use with "$use"). There's still some debate over that vs some other proposals, but that proposal would do what you want and is the only one that has significant support (and relatively weak objections).

I am hoping to revisit it for Draft 07. Please comment on the issue with your use case, even if it is a duplicate of others (no need to read through the whole discussion to check- it's super-long). The more people saying that they need it the more likely it is that it will be added.

Note that "$use" is similar to an older proposal, "$merge", but restricted to annotation keywords. This is because allowing validation keywords produces arbitrary transformations of JSON that makes it very hard for validators to reason about the results. We identified that one of the major use cases is changing "description", "title", "default", or "example", so "$use" addresses that. There are other (less clearly supported) proposals for other use cases such as the perennially problematic "additionalProperties".

[shmax]

@handrews That looks more or less perfect, thank you. I'll read up on it in a bit more depth and add my two cents in the next few days. Thank you!

[anthomps]

We've been discussing the creation of a central data dictionary for multiple APIs.

The requirements would be met by $use. Primarily we'd been thinking about overriding the description of nodes, but I'm sure other cases would have come to us.

I'm struggling a bit to find a definition of annotation properties vs. validation properties. It's intuitive what that means, but are they enumerated anywhere?

[handrews]

draft-06 is out! (as of a few weeks ago- apologies for the delay, some rather involved hyper-schema issues came up at the last minute)

Guide for core and validation
Guide for hyper-schema

[erayd]

@handrews - Awesome - thanks for all the work you guys have put into getting it out :-D.

[dafeder]

Seems like draft-07 might be a better target at this point? Changes from 06 are minimal and its backward compatible. Or is the correct pattern to implement 06 and then once released "top it off" with 07?

[handrews]

@dafeder draft-07 and draft 2020-12 are widely used and/or incorporated into other widely used specifications (OpenAPI 3.1, AsyncAPI). The other drafts (draft-06 and 2019-09) were only briefly current before being updated by draft-07 and 2020-12, respectively.

[dafeder]

@handrews yes that's kind of where I was coming from. I don't see schemas referencing draft-06 much