Explain JSON-LD further.

[BigBlueHat]

Addresses #1227.

That issue accumulated many references with many requests for it to address all maner of things. I'm confident I did not address them all. However, I hope this further explains the value of JSON-LD to the VCDM.

---

Preview | Diff

[selfissued]

Most of this text is not normative requirements that affect implementations. Please pare it down to limit what it says to normative requirements.

[BigBlueHat]

Most of this text is not normative requirements that affect implementations. Please pare it down to limit what it says to normative requirements.

Looks like I forgot the "non-normative" intro text. I'll add it. I agree it can use paring down, but given the group continues to attempt to span to ecosystems of parsing approaches--in which there are multiple "normative" ways to do most things--the connection this makes to the JSON-LD ecosystem and the benefits available within it is rather the point and focus of the text.

Happy to entertain specific reduction/simplification requests!

[BigBlueHat]

Most of this text is not normative requirements that affect implementations. Please pare it down to limit what it says to normative requirements.

The text is now marked as "informative". I'll continue to work to pare it down as more reviews come in--but given that "pare down" is not a defined target, if you could re-review or un-minus-one until there's a substantive thing I can address, I'd greatly appreciate it!

[Sakurann]

Suggested change

	mechanisms without altering or damaging the underlying data structure. This
	creates economies of scale which reduces development time, decreases
	retooling cost, decreases time to market, and provides long term stability for
	Issuers, Verifiers, and Wallets.
	mechanisms without altering or damaging the underlying data structure.

[BigBlueHat]

Shared foundational data models and consistent data processing pipelines do quantifiably reduce development time, decrease retooling cost, etc. Could you expand on why you'd like this statement removed?

[BigBlueHat]

@Sakurann this paragraph remains unchanged in the most recent revision, so please let me know if there are specific complaints with the above claims. Thanks!

[nadalin]

not in favor of this PR, suggest that this PR not be merged

[Sakurann]

(still reviewing. I signed up to make concrete changes during TPAC. will do so later)

[TallTed]

[@nadalin] not in favor of this PR, suggest that this PR not be merged

It is impossible to consider modifying my position to match yours without more words from you about why you hold it.

[iherman]

The issue was discussed in a meeting on 2023-09-14

• no resolutions were taken

View the transcript

1.4. Explain JSON-LD further. (pr vc-data-model#1270)

See github pull request vc-data-model#1270.

Brent Zundel: Explain json-ld further. A section that was widely asked for. Has had comments back and forth, no approvals. More work to be done. We'll time box to 10 minutes what changes are needed from folks.

Michael Jones: This PR is a lot of philosophy and not a lot that helps implementers.

Manu Sporny: The group asked us to write this. They asked for "we want to see a section that explains json-ld". We have other places in the spec that explain why some choices were made.

Sebastian Crane: It looks good on a first impression. I would prefer for it to be cut back. Not because it is incorrect, but it includes some things that sounds like opinions. This makes folks uncomfortable. I would suggest som word smithing. I would object to merging now, but ok in the future.

Brent Zundel: Suggestion is to make concrete suggestions.

Michael Jones: There are 14 new paragraphs. Please cut it down accordingly.

Ted Thibodeau Jr.: The entire section is informative. It might be better as an appendix.
� It doesn't talk about JSON, but its use. That's the way it is.
� If Mike really thinks it could be said in a single paragraph, please suggest how to concretely do it.

Kristina Yasuda: we need something in main text as opposed to appendix.

Shigeya Suzuki: +1 kristina.

Brent Zundel: Happy to hear thought about the appendix suggestion.

Manu Sporny: +1 what TallTed said. It's really difficult to understand the changes requested without suggested changes. Please either write a PR, or make concrete github suggestions.

Kristina Yasuda: Re: moving to appendix. I do believe we need a few short and sweet paragraphs in the main text. More details can go in the appendix. Doesn't need to be pre-cr. Bottom line, the goal is to give folks some background on JSON-ld before sections that work off of it.

Manu Sporny: +1 kristina, keep shorter section in the main spec... but don't feel super strongly about it.

Brent Zundel: Since this PR is non-normative, it's not urgent to merge this PR.

[BigBlueHat]

not in favor of this PR, suggest that this PR not be merged

Could you provide something more concrete to review, discuss, or respond to?

[BigBlueHat]

The intro is significantly reduced--with special thanks to @dlongley for his collaboration!

[selfissued]

This PR adds a lot of informative text that isn't actionable by implementers. It's more philosophy than specification.

It was the job of the JSON-LD spec to explain JSON-LD. Please simply cite it, possibly with a section reference, along the lines of "For more about JSON-LD, see [ref]" and stop there. The current contents of this PR doesn't belong in our specification.

[OR13]

I've like to see specific text providing guidance on processing the core data model as RDF.

JSON-LD is a concrete RDF syntax as described in [RDF11-CONCEPTS]. Hence, a JSON-LD document is both an RDF document and a JSON document and correspondingly represents an instance of an RDF data model.

• https://www.w3.org/TR/json-ld11/#relationship-to-rdf

Some ideas to help frame what I would like to see:

• https://www.w3.org/TR/json-ld11/#advanced-context-usage
• https://www.w3.org/TR/json-ld11-framing/
• https://www.w3.org/TR/sparql11-overview/

Basically, we have a lot of text that says "you don't need to process the core data model as JSON-LD".... but we have yet to show any value for processing the core data model as something other than JSON (as JSON-LD, which by definition is also RDF).

I think it would be best to address this directly in the context of related work at W3C, for example:

https://github.com/WICG/identity-credential/blob/main/identity-credential-proposal.md#mdocs

Notice the way the above specification does not use RDF predicates... Why would it be better to use RDF predicates and JSON-LD frames?

If there is no benefit to processing the data model as RDF, we should state that explicitly, and warn implementers that RDF predicates and RDF types are not stable extension or interoperability points.

[BigBlueHat]

Basically, we have a lot of text that says "you don't need to process the core data model as JSON-LD".... but we have yet to show any value for processing the core data model as something other than JSON (as JSON-LD, which by definition is also RDF).

We've had this backwards. Developers always process the core data model as JSON-LD, they may just be using the more limited approach defined in the JSON Processing section. JSON-LD is a syntax and you can access it directly. You can do things like pre-process a context once (or have a trusted party do this for you) and then write your code against supported contexts or context hash values. This is still doing JSON-LD processing…just a more limited form of it which attempts to optimize it based on accepting only specific inputs.

The content we have in the spec today that explains how to do these optimizations in JSON parsing land is unnecessary when you use a JSON-LD library to convert from one context to another or to RDF, but not doing that does not mean you're not doing JSON-LD processing. We should probably rename that section to "Optimizing JSON-LD processing (for specific use)" instead of "JSON Processing". Again, JSON-LD is a syntax and developers should read it as long as it is being provided with a context you accept. JSON-LD tells you the context of the data as currently expressed. This is true even if one chooses to process JSON-LD in a more limited way. It’s still JSON-LD.

Additionally, these more limited JSON parsers do not do the following (at least not without nearly recreating JSON-LD processing...):

• handle term consistency--they always assume the key names mean the same thing unless they're within very narrowly defined media type namespaces
• they do not have constructs for advanced data types
• nor language value objects
• nor a consistent extensibility mechanism (other than one media type per document use)
• nor underlying graph model which enables aggregation, faceted querying, statement signing/extraction/reuse

All of this is expressed in what I've written.

https://github.com/WICG/identity-credential/blob/main/identity-credential-proposal.md#mdocs

Notice the way the above specification does not use RDF predicates... Why would it be better to use RDF predicates and JSON-LD frames?

This happens everywhere there's a belief that English words mean the same thing within all the communities that use them. You can read through the various specifications which they are trying to combine to find that's not the case. Each of those specs have different type systems, value definitions, etc. All of those facts could coexist more easily when mapped to non-conflicting identifiers, value definitions, etc. The reconciliation example becomes far simpler (perhaps using JSON-LD Framing) when JSON-LD is used.

[OR13]

We've had this backwards. Developers always process the core data model as JSON-LD, they may just be using the more limited approach defined in the JSON Processing section. JSON-LD is a syntax and you can access it directly.

This is great, I think the "JSON Processing" section is confusing and should be removed.

The content we have in the spec today that explains how to do these optimizations in JSON parsing land is unnecessary when you use a JSON-LD library to convert from one context to another or to RDF, but not doing that does not mean you're not doing JSON-LD processing.

I'd say its not necessary regardless of which libraries you use.... and the fact that you can be "doing JSON-LD processing" when "not realizing it" is a security and readability issue in the current spec, that needs to be addressed.

Additionally, these more limited JSON parsers do not do the following (at least not without nearly recreating JSON-LD processing...):

• handle term consistency--they always assume the key names mean the same thing unless they're within very narrowly defined media type namespaces
• they do not have constructs for advanced data types
• nor language value objects
• nor a consistent extensibility mechanism (other than one media type per document use)
• nor underlying graph model which enables aggregation, faceted querying, statement signing/extraction/reuse

^ maybe it would be better to define "JSON-LD Processing first", and then define "limited / unsafe JSON processing second".

[OR13]

@BigBlueHat I suggest you pull anything useful that is removed in #1298 into this PR, and say in plain english essentially what you said in #1270 (comment)

[iherman]

The issue was discussed in a meeting on 2023-10-04

• no resolutions were taken

View the transcript

1.1. Explain JSON-LD further. (pr vc-data-model#1270)

See github pull request vc-data-model#1270.

Manu Sporny: so, 1270 has multiple requests for changes.
… selfissued is blocking. Orie is blocking.
… what changes need to be made?

Orie Steele: we have this section about JSON processing...which I don't really like.
… I'd believed we would/should add a JSON-LD section to explain applying @context more completely.
… and these two things differ substantially when you just process JSON.
… I think the text in the JSON processing section is dangerous and misleading.
… that text is mostly information that doesn't help implementers.

Dave Longley: -1 to what Orie is saying :) ... the information is the same, JSON-LD is a syntax that expresses graph information.

Orie Steele: and doesn't help people use RDF or JSON-LD.
… so the changes I want to see are to remove these differences.
… bigbluehat's comments are excellent.
… he says there dangers of doing just JSON processing.
… it's a huge mistake for us to be vague.
… and this will harm interop if these things differ too much.
… the JSON-LD processing section needs to talk more about RDF data model.
… @id, @list, etc.
… basically the stuff that's in the @context that only an advanced processor would use.
… I strongly disagree with the position that dlongley stated.
… that we should avoid the JSON-LD feature to keep it compatible with JSON processing.
… bigbluehat's PR has some of this, but not enough which is why I'm requesting changes.

Kristina Yasuda: +1 to json processing is not compatible with JSON-LD processing (without additional checks on top co json processing).

Michael Jones: this PR kind of runs at the mouth.
… it says a lot of things that aren't useful to implementers.
… we don't need to be an evangelist mouth piece for JSON-LD.
… I will not approve this until it's down to a sentence or a paragraph.

Orie Steele: I would encourage better direct references to the JSON-LD spec, for example: https://www.w3.org/TR/json-ld11/#relationship-to-rdf.

Dave Longley: trying to figure out where to start.
… there are many competing requests.
… many from Orie requesting explanation of the advantages of JSON-LD.

Kristina Yasuda: chairs hat on, i don't think the request was "json-ld advantage" it was about "json-ld processing guide".

Orie Steele: I disagree that the PR captures the advantages on JSON-LD, but I agree that is an objective... and I think citing the JSON-LD spec, would better accomplish the mission.

Dave Longley: new comers are often confused by the ability to process JSON-LD further with libraries.
… but they miss that JSON-LD is a syntax.
… if you understand the @context, then you can move ahead.
… if you don't understand what's in `@context, then you do need to process that and understand it.
… so this section is really talking about that difference.
… JSON is really just a syntax.
… and if you want to do anything with JSON, you need a spec to read and implement.
… whereas JSON-LD is an object-oriented graph model that defines how to be processed.
… JSON-LD imposes additional constraints.
… so you don't have this problem of missing out on this extensibility and data model.

Orie Steele: I agree with what dave is saying, that JSON-LD and JSON claims can't be merged, without understanding RDF... that is the security issue that the group is being unclear regarding.

Dave Longley: the JSON processing model section basically says that if you know it's using these @context definitions that the spec specifies, then you can just process this in JSON--because you trust it implemented the spec.
… on top of that Orie has been asking on elaborating on the additional value of JSON-LD.

Orie Steele: We don't have the section, we have draft text, which I don't feel is acceptable yet... the mission is still ongoing.

Dave Longley: but now we have the counter request to remove all of this--both JSON-LD values that were requested and the JSON processing section.
… many of the new comers have missed the history here.
… that this context ability was chosen on purpose.
… that it provides the clarity of what the data is about referenced from that data.

Orie Steele: Agree with what dave is saying about JSON-LD providing value to W3C Verifiable Credentials, hence the need to communicate the value of JSON-LD and RDF processing as opposed to "JSON Processing", which does not require reading the JSON-LD spec, to do safely.

Orie Steele: +1 to landing the PR, after revising it.

Dave Longley: it's valuable to explain this value and how it helps and has helped this community since the beginning.

Kristina Yasuda: I don't think we don't want to have this section.
… it feels like there's more discussion needed for this PR.
… it's not like this needs huge conversation.
… so lets entertain the queue.
… but keep it sort and sweet to exactly what you want to see here.
… not debate the values of JSON-LD etc.

Ted Thibodeau Jr.: I started off being extremely frustrated by two voices in this conversation.
… both are asking for vague changes with no specific changes.

Orie Steele: If the working group agrees, I am happy to author an alternative PR.

Ted Thibodeau Jr.: I kinda want to lock Orie and selfissued in a room and let you hash it out.
… but I don't think that's going to be useful.
… and others should support you if they agree.
… Orie's been asking for this for months.
… but without any actual contribution to the conversation.

Orie Steele: IMO its better to refer to a spec, than it is to mischaracterize the benefits of a spec, without references.

Ted Thibodeau Jr.: he hasn't put the concrete suggestion forth.
… and selfissued if you want this down to a sentence, then write that sentence.
… otherwise you're just blocking to block.

Kristina Yasuda: this is an inclusive environment.

Ted Thibodeau Jr.: you're spending more time on this than anyone.
… let's move onto the queue.

Kristina Yasuda: please leave the call TallTed.

Brent Zundel: ivan please remove TallTed from the call.

Orie Steele: My offer to author an alternative is sincere, we may find that merging 2 different PRs is an effective path forward.

Manu Sporny: yes, please write something Orie ^.

Orie Steele: will do.

Michael Jones: strangely related to what TallTed was saying.
… I do have a specific action.
… put this in a note, and reference the note.

Ivan Herman: clarification. you mean a separate WG note document, correct?

Michael Jones: yes.

Kristina Yasuda: there is a separate JSON-LD spec...so I'm not sure a separate note gets us much.
… please offer concrete PRs with concrete suggestions Orie.
… and we can move from there.
… Orie is that OK?

Orie Steele: IMO, its worth highlighting the parts of the JSON-LD spec that can provide the most value to Verifiable Credentials.

Orie Steele: yes, i will do a PR.

Dave Longley: +1 to see something from Orie ... but also note, Orie, I think there may be more benefits to VCs than you're thinking of (including all of the ones in the PR of that subtopic).

[iherman]

This comment is generally on the text, regardless of where it ends up (whether in a separate note or in the VCDM spec).

The text uses the analogy/reference to object-oriented modeling as a paradigm "familiar to developers". This familiarity is what worries me, because, usually, object-oriented terminology refers to the "class" feature in languages like C++, Java, Python, TypeScript and, more recently, JavaScript. And this analogy is wrong and misleading when it comes to RDF. RDF uses the term "class", but that is in the sense of ontologies.

In, say, Java one assigns the type of an "object" as a declaration, and that is bound to the object for life. Object declaration is (also) a constraint for the properties (attributes) that one can use in conjunction with that object. The type of an RDF resource can also be assigned at "declaration" (that is what rdf:type does) but, more importantly, it can also be inferred from statements like property domains and ranges (if the RDF environment uses some inference engine for, e.g., RDFS processing). Hence, the domains and ranges in the vocabulary are not constraints, they are licenses to infer. This is in line with the "Ontology" nature of RDFS (and OWL), which is used to categorize knowledge in classes.

As an example, if my dataset contains these two triples:

ex:R ex:p "something" ;
ex:p rdfs:domain ex:Cl;

then I can infer the following statement:

ex:R rdf:type ex:Cl;

The misunderstanding on the nature of domains and ranges has come up several times in this Working Group as well, and this is mostly because people often believe that RDF's classes are analogous to classes in object-oriented languages. We should not reinforce this misunderstanding. I would therefore propose to remove all references to object-oriented modeling from the text in the PR.

cc @pchampin

[OR13]

In case you missed this, w3ctag/design-principles#239 (comment)

Very relevant to the guidance we are attempting on this PR, and on #1302

[dlongley]

@iherman,

The object-oriented references in the text above pertain to how statements about subjects are expressed in JSON (as JSON-LD). They are not about the RDF as much as your comment suggests, but the modeling does enable consistent processing to enable mapping to / from RDF much more simply.

What is trying to be said here, is that, in JSON, there are "objects", which have "keys" (aka "properties") and "values" associated with those keys. Someone designing a JSON document, could, if they wanted (and clearly some often do!), decouple their data model from these primitives, using them any which way.

For example, if we wanted to model a "car", which has a color, an engine, and a number of wheels, we could express this information in JSON any number of ways. For example:

{
  "thing": "car1",
  "engine": true,
  "wheels": true,
  "car1Props": [["color", "red"]],
  "car1Wheels": 4,
  "car1Engine": ["electric"]
}

There's a lot of JSON in the wild that evolves organically and ends up looking like that. Applications that consume it might have bespoke rules that have also evolved over time... "if you see 'thing' and it's value starts with 'car', then it's a car, and its properties will be in the rest of that value + 'Wheels' or + 'Engine', and you'll know if those are there if 'engine' or 'wheels' are true, and if the car has other properties it will be under + 'Props' as nested arrays, etc.".

An alternative expression might be:

{
  "id": "car1",
  "type": "Car",
  "color": "red",
  "wheels": 4,
  "engine": {
    "type": "ElectricEngine"
  }
}

And the rules for processing this data can be more consistent. In fact, you can write consistent processing rules for most things modeled this way because the data model concepts are more tightly coupled to the primitives provided by JSON. It also adds compositional capabilities to your data. JSON-LD requires JSON to be interpreted in this way, such that authors must model their data accordingly as consumers can and will presume this basic modeling is built-in.

This is an important concept to express as an advantage of using JSON-LD (over "vanilla JSON" where anything goes) as it eliminates (or greatly reduces) situations like the first example, where complex bespoke processing rules must be written into applications to even figure out a basic interpretation of the information one is expressing. Requiring "JSON objects" to actually express "objects" in the information model and JSON keys/properties to actually express the properties of those objects is "object oriented" and should be highlighted as a benefit.

[iherman]

@dlongley I do not question those advantages of JSON or of JSON-LD. But "object-oriented", to me, resonates primarily to the programming concepts in object-oriented programming languages. That, plus the obvious relationship to RDF, does bother me.

I won't lie down the road over this, obviously; if this works for everyone else, be it. I just wanted to express my discomfort. Maybe the reason that I never look at JSON-LD in isolation but only through the goggles of RDF...

(And no, I do not have a good alternative term in my mind right now.)

[msporny]

Proposal from the WG call: We should move this text to the VC Implementation Guide in order to "move it forward"... no objections on the call.

[iherman]

The issue was discussed in a meeting on 2023-10-17

• no resolutions were taken

View the transcript

2.1. Explain JSON-LD further. (pr vc-data-model#1270)

See github pull request vc-data-model#1270.

Brent Zundel: The chairs feel that this is prime example of a PR that is unlikely to achieve consensus.

Manu Sporny: +1 to this one not making it in. It has text that some in the group feel strongly about, moving it to the VC implementation guide and linking it is something Mike Jones did not object to.

Brent Zundel: Anyone want to object to that proposal?
… I will mark this pending close and say it will be done in the implementation guide rather than the core data model.

[brentzundel]

I have raised w3c/vc-imp-guide#71 to track adding this text to the implementation guide.

[msporny]

Pending close has been marked on this PR for 2 weeks, the issue is being tracked in the vc-imp-guide repo. Closing.