General expressions support and cross-field validations

[brunodymios]

What about having a requirement saying that field A must be always equal to field B divided by field C?
Consider the case in which we are managing financial data and we need this information to both show in a form the auto-computed value and validate/recompute this value server-side.

Having this kind of possibility directly in the Json Schema (even through the support for referencing other fields in properties like "minimum" or "multipleOf") could be extremely useful and makes Json Schema even more complete in my opinion.
Such a feature will enable for creating both server-side and client-side validations by only relying on the Json Schema describing the data.

Is this currently being addressed somehow?

[anweiss]

Big +1 to this. Support for co-occurrence constraints in JSON schema would be incredibly advantageous in terms of deferring property and property value interpretation to the schema itself instead of having to build such interpretation into JSON consumption tooling.

[handrews]

@brunodymios @anweiss you might want to take a look at @awwright's data relationship proposal #549 and see if that would meet your needs?

[anweiss]

Thanks @handrews. Per some of your comments, what I'm mostly interested in is data relationship validations in the instance document itself and not references to external data parameters via URIs per the proposed valueRange, valueDefine, valueUniqueKeys, etc keywords

[awwright]

My personal opinion is if JSON Schema is going to support assertions that source multiple data sources, it needs to support references in the same document, values in other documents, and arbitrary data sources (e.g. databases). Because, in a nutshell, JSON Schema shouldn't encourage people to combine too much data into a single document just so that developers can make use of one of its features. I'd only like to encourage documents to contain more data if that's actually the right way to go about designing them.

Also, note there's a slight change in the meaning of the assertion. All the assertions presently are assertions about a single JSON value (the instance). The proposal is about relationships between multiple data points.

This is also really difficult to implement, in general. XForms has the ability to do exactly what you're asking; it's just not widely implemented because it's so comprehensive.

Eventually someone will want to do something really complex. Say, "An array of objects, the qty property must be an integer divisible by the previous object's qty if it exists". Which I think is a perfectly reasonable constraint for certain applications! The only reason I haven't seen this asked yet is because we can't do step one first.

The easy solution is just implement multi-datum assertions in application code. The comprehensive solution is let's copy XForms. Maybe there's a solution in the middle too (only implement simple references), I'm not sure exactly.

(Or, let people go all-out and embed processing languages in JSON Schema like Lua, XForms, XSLT, or ECMAScript. I don't think this is unreasonable either; this is exactly what HTML does to satisfy the "Code-on-demand" part of REST.)

[handrews]

@awwright I'm increasingly convinced that the right way to implement this whole area of functionality, whether $data, this proposal, or any of the other vaguely related proposals, is as an additional vocabulary.

Yeah, I know, vocabularies are my solution to nearly everything, but as you note, this is both difficult to implement yet in demand from a certain subset of users. We've had multiple related issues filed over just the past few months, and $data has come up regularly since draft-04 and been in use in at least one implementation.

I don't think that adding this sort of thing disrupts the existing core spec (including the additions to it already planned for draft-08). It will also continue to be possible to implement existing validation and hyper-schema without it. And as you note, this is "code-on-demand" territory which is an optional constraint for REST in general, which tells us something.

How would you feel about working on this as an additional vocabulary? It can then succeed or fail on its own, like XForms (although hopefully we can learn from XForms' difficulties). And it also would mean that we don't add a huge and burdensome requirement to basic validator implementations when we're trying to stabilize core and validation as an actual standard.

[awwright]

#549 covers the fundamental idea behind what's been missing, I think, but still needs to describe how it covers cases like this issue.

I sort of have an idea on how to fill that in, except this issue asks for one more thing: it's asking for a relationship between three fields. It wants to assert that A = B / C (and presumably that B mod C = 0). I don't know of any way to do this except implement some sort of scripting language or scripting vocabulary, and an array of general assertions. This is very powerful (in the bad sense (1) (2) (3), in that we lose many of the semantic meanings over if a keyword were used).

Further, there's math considerations. JSON Schema is arbitrary precision. But most people will probably want to implement IEEE floating point.

[brunodymios]

I agree on what has been written above. A first step is to introduce relationships between data inside the same JSON Schema definition and define how they relates in order to cross-validate them.

I did not think about data precision before @awwright comment, since the A = B / C expression was used as an example only. Regarding this, I think that the most important thing is to have something that is coherent in both client and server validators.

Remote data sources for validations can of course help very often but I see them as a subsequent step with respect to these "local" cross-validations and I would maintain them separated.

[handrews]

We've consolidated the proposals of this sort into #855. If this conversation is still of interest, please continue it there.