Proposal: add keyword firstOf

[SorinGFS]

Making my implementation I realized that anyOf keyword produces unneccessary overhead in many cases where there are no benefits. The reason for this is the requirement to parse all the items despite the fact that in 99% of the cases we only need the first one. I tested all the anyOf tests in test-suite with the proposed behaviour and... it actually break only one test. Mean while the actual behaviour of anyOf keyword is also useful and everyone is already used to it. So, in order to obtain the desired behaviour the only choice would be to add a new keyword firstOf which will stop evaluating items when a valid item is encountered.

[gregsdennis]

The spec explicitly states that implementations may shortcut any keyword so long as it doesn't impact annotation collection. The openeing paragraph to Section 7 has this (emphasis added):

Evaluation of a parent schema object can complete once all of its subschemas have been evaluated, although in some circumstances evaluation may be short-circuited due to assertion results. When annotations are being collected, some assertion result short-circuiting is not possible due to the need to examine all subschemas for annotation collection, including those that cannot further change the assertion result.

I think this gives you license to treat anyOf as you're proposing. You might run into some problems with the unevaluated* keywords, though, so be careful about that.

(It does appear we should amend the tests in /tests/draft2020-12/unevaluatedProperties.json to cater to the scenario where the first subschema matches, but a second subschema defines a property that should be considered "known" by unevaluatedProperties. I don't think this kind of thing is covered, and it's a good test case.)

[gregsdennis]

just not agree with that

I can't help this. We're not going to change something that no one else seems to have a problem with because you disagree about how it works. This is how it works. You need to accept that.

Then, unevaluatedProperties keyword came out... namely to solve this kind of problems. So I should understand that it doesn't?

Are you saying that unevaluated* doesn't do what we've defined it to do? That's patently wrong. It does exactly what we set out for it to do.

The ask from our users was to have something like additionalProperties that can "see into" keywords like allOf. That's exactly what unevaluatedProperties does.

The user base asked for a solution to a problem they were having, and we gave them one. Countless StackOverflow questions have been answered with "use unevaluatedProperties instead", and the response is overwhelmingly "thank you, that's exactly what I needed." So you can't tell me that it doesn't do what it's supposed to do. It very clearly does.

Especially because it is possible to get the correct behaviour

This is where I have a problem with your approach, @SorinGFS. You presume that your understanding of a thing is the "correct" way to do that thing, even when it's obvious that you don't understand how that thing actually works. And when you're told how it works, you ignore what you're told because you disagree that it should work that way.

It's disrespectful to the current and historical maintainers.

This is how JSON Schema operates. If you don't like it, or if you think you can do better, you're free to go write your own specification.

If you want to be a part of this community, you'll need a better, more respectful attitude. Start with curiosity. Find out how things work and why. Have an open mind: ask questions instead of making assumptions. Then, and only then, will I be open to discuss making a change.

Our history (yours and mine together) is a rocky one, with plenty of arguments. But I came to this discussion with an open mind, wanting to explore your idea because I saw merit. I saw a potentiality new behavior that couldn't be accomplished with the current keyword set. But you've ruined it, @SorinGFS. I don't want to do it.

I've repeatedly asked for a legitimate use case, and you haven't provided one. Instead, you have the audacity to tell me that we, the authors of the specification, the maintainers of many implementations and tooling, the very admins of the project, are doing JSON Schema wrong.

As no use case has been provided, I see no reason to move forward with this proposal.

[SorinGFS]

I've repeatedly asked for a legitimate use case

I think in algorithmic terms, I saw the gap, I came with the idea. I also build meta-schemas, and I realized that need. Sorry if I didn't came up with specific use cases, I thought this will spark your imagination, same as it was when you said that you can solve schemas by hand. You probably think that I need this for me... no, I don't need anything for me personally. There are vision differences between us, those vision differences made me start working on my implementation, I spent plenty of time on it thinking how to solve different things not solved by json-schema, and what I have today looks much better from the logical point of view. What I really didn't want was to create my own spec, so I tried to convince you to implement some, and step by step to bring you much closer to my vision. Instead what you did is to explain me the reasons for saying no to everything. You asked me to be open minded... while you were conservative to the max. I actually was open minded when I started searching new methods, new ideas since more than 2 years ago. Maybe you didn't notice, but there arn't too many people doing this in the community. Anyway, you're right, if I don't like your specs probably I should do mine, maybe I will.

It's disrespectful to the current and historical maintainers.

I wouldn't consider disrespect having the courage to call things by their name. Anyway, I appologize if this offended you, that wasn't my intent.

[mwadams]

Again, you need to provide this evidence.

I will, I know where the problem is, I know how to construct schemas that fail to provide consistent results, but... I don't have implementations to test! All the implementations I know, including yours, are using... ordered proccessing.

If you wanted, I could switch my implementation to randomize the order it emitted the *Of composition validation and I guarantee it wouldn't make any difference to the validation results (just for test purposes, of course.)

JSON schema is a set-like thing not an imperative thing. It really doesn't matter what order compositional subschema are applied - except relative to their sibling unevaluated* keyword.

[mwadams]

I have no objection in principle to adding a new ordered subschema applicator capability of this kind. It would be fairly straightforward to define behaviour in the face of unevaluated* (I would choose 'only the first valid subschema applies')

These are actually two orthogonal capabilities in that sense.

Thinking of a use case for firstOf.

Maybe it could be a nice way to describe a "priority" matcb.

Consider a situation where there are multiple potential "groups" of matches where there is a (required) discriminator property that distinguishes between subsets of the schema in your firstOf set.

However, several of the schema in each subset match if some optional property is missing that is the discriminator between them, and you want a specific one from that group to "win" in that case.

[notEthan]

@notEthan can you share a playground where I can make some tests?

I'm afraid I don't have that, it's just a Ruby library - I would like to get a web UI set up but haven't yet. But I am pretty certain that results with my implementation would be the same as with Greg's, barring some bug in one or the other.

I did run the JSON Schema Test Suite against my implementation, randomizing the order of evaluation of every anyOf/oneOf/allOf, to ensure that makes no difference. The test suite all still passes.

your implementation is incapable to preserve the evaluated properties from the first rule to beyond, hence unevaluatedProperties is not seeing the "foo" as being evaluated.

It's not supposed to be able to see that - that's not how it works, as I mentioned in the previous comment and Greg has also tried to clarify. This is what is leading you to think that evaluation order must be significant.

I got that, just not agree with that! First, it was additionalProperties a keyword having problems in deep locations. Then, unevaluatedProperties keyword came out... namely to solve this kind of problems. So I should understand that it doesn't? Sorry, but I can't! Especially because it is possible to get the correct behaviour!

I get that you don't agree with the behavior, but that is the specified behavior. It is correct according to the spec, even if not correct by your understanding of how it should be.

P.S- I also didn't understand why you kept the old keywords items/additionalProperties since their function is redundant with unevaluatedItems/unevaluatedProperties... Are we collecting historical mistakes?

They have different behavior, and both behaviors remain useful. They weren't "historical mistakes", they were just not sufficient to address every problem, and adding unevaluated* does address the problem it was meant to.

@gregsdennis If you want to be a part of this community, you'll need a better, more respectful attitude. Start with curiosity. Find out how things work and why. / @SorinGFS what you did is to explain me the reasons for saying no to everything. You asked me to be open minded... while you were conservative to the max. I actually was open minded when I started searching new methods, new ideas

I am loath to get much into attitude, tone, respectfulness. I think, @SorinGFS, you have interesting ideas that are worth considering, and have led me to address things in my own implementation that I hadn't previously thought of. I appreciate your participation here.

You do have strong ideas of the way that things should work and sometimes (often, even) these are not compatible with how JSON Schema has been for a very long time, or with how those who developed JSON Schema intend for it to be. This is inevitably going to mean a lot of push back against what you propose - the maintainers need to be fairly conservative because breaking changes are problematic, and new features need a high bar to make it in, otherwise the spec would just become enormous. I have had a number of my own ideas (improvements, in my opinion) that have been rejected; so it goes. But there are times when you dig your heels in and insist that this is the way that JSON Schema should be or must be, that no other way makes sense. That can get tiresome, when there are usually pretty good reasons for the way things are, that you don't seem open to considering.

I wouldn't consider disrespect having the courage to call things by their name. Anyway, I appologize if this offended you, that wasn't my intent.

This is unhelpful - you aren't courageously telling it like it really is; while your viewpoint is valuable, it is not the only one or the objectively correct one. And the subsequent textbook non-apology apology does not improve anything.

@gregsdennis If you don't like it, or if you think you can do better, you're free to go write your own specification. / @SorinGFS What I really didn't want was to create my own spec, so I tried to convince you to implement some ... if I don't like your specs probably I should do mine, maybe I will.

The suggestion to "go write your own spec" seems dismissive, and also unhelpful ... but I don't know, it may be that when ideas are too incompatible, they need to be done differently. It has happened before - the JSON Type Definition also came about due to different visions of how JSON data should be described. A different dialect, realizing @SorinGFS's ideas, could be an interesting thing to explore.

[jdesrosiers]

This was only posted two days ago, but already I feel very late to the party given the length and intensity of the discussion 😆.

While I agree that the lack of ability to short circuit (in some cases) is a concern, I don't think this is the right solution. First, I don't think it's addressing the root cause of the problem. Second, there's already an existing solution that has even better performance characteristics than firstOf would.

The root of the problem is the way annotation collection works. Annotation collection requires all schemas be evaluated and that's what breaks short circuiting. I don't think creating a new keyword to work around the consequence of that choice is the right thing. If short circuiting is something we want to optimize for, we should be questioning how annotation collection works or maybe questioning whether we should have keywords that rely on dynamic annotation evaluation like the "unevaluated" keywords. I have opinions on both of those, but I'm not going to get into it because I do think it matters.

It doesn't matter because there's already a solution available to schema authors to avoid this problem and get even better performance (and error messaging) than firstOf. Using anyOf (or oneOf) when choosing between a set of schemas has always been sub-optimal. Using if/then instead allows you to optimize the performance (as well as error reporting) by allowing you to be explicit about under what conditions you should be evaluating each option.