Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"profile" media type parameter usage #222

Closed
handrews opened this issue Jan 4, 2017 · 27 comments
Closed

"profile" media type parameter usage #222

handrews opened this issue Jan 4, 2017 · 27 comments
Assignees
Labels
clarification Items that need to be clarified in the specification core Priority: High
Milestone

Comments

@handrews
Copy link
Contributor

handrews commented Jan 4, 2017

In mnot/I-D#175 @dret states:

profile was not intended to refer to a schema. it is intended to refer
to a profile, which is defined as extending and/or constraining the
original media type. the spec gives examples.

yet in issue #26 filed here by @dret it looked like we had received his approval on how we are wording our specification around the use of "profile"

@dret can you help us out here? Is what we have acceptable or not? To me it seems like JSON Schema is doing exactly this- further constraining the application/json media type by declaring that only a subset of possible documents are valid rather than all possible expressions of the JSON syntax. But it sounds like you do not view it this way.

Also in the same issue in mnot/I-D:

a profile media type parameter has to be allowed by the media type.
very few media types do this. so while i think that it's possible (and
maybe even better) to use a media type profile parameter, if that
mechanism is required, in many cases the media type simply will not
allow such a parameter to be used.

Since application/json does not explicitly specify profile as a media type parameter, does that mean that even if "profile" is correct we cannot use it with application/json? Do we need to specify an application/instance+json media type in order to add a parameter (whether it is "profile" or not?) Honestly my entire API evolution strategy relies on having schema URIs in a media type parameter so it's a huge problem for me if we can't do this somehow.

The balkanization of IETF standards is immensely frustrating. "Look, here are these two great ideas that do not conflict in any way and perfectly complement each other, but you cannot use them together because the two RFC authors did not coordinate and/or wrote their RFCs at different times!"

Finally:

in my mind, "profile", "type", and "describedBy" are distinct, but i
have heard world views where they are more similar and do have quite
some overlap.

@dret if aspects of this distinction are relevant to JSON Schema and not covered in #26 could you please elaborate?

@awwright
Copy link
Member

It's my understanding we can update the JSON media type registration to add the 'profile' parameter -- with the same semantics as expected, nothing JSON Schema specific.

Let's keep investigating this.

@handrews
Copy link
Contributor Author

@awwright oh cool, glad you've thought of this. Yeah, let's keep this issue open to track that dependency.

Would it make sense to create an "RFC" milestone to group issues that need to be handled before RFC but are not directly part of the draft iteration process?

@dret
Copy link
Contributor

dret commented Jan 16, 2017 via email

@handrews
Copy link
Contributor Author

@dret you and @awwright had this conversation in #26 and you gave your approval. What's changed?

@awwright
Copy link
Member

awwright commented Jan 16, 2017 via email

@handrews
Copy link
Contributor Author

@awwright , @dret's comment about schemas not being profiles (and the blog post he linked) are not specific to link relation or media type parameter. A profile is semantically a profile either way it's expressed, that's the point of allowing both expressions, really.

@handrews
Copy link
Contributor Author

handrews commented Jan 16, 2017

I'm guessing there is one point of view that looks at a schema as defining a (possibly extremely) restricted subset of JSON, and the other that views it as carrying more meaning than that.

Looking @dret's example of I-JSON, that's a generally usable subset of JSON. Schemas are there to constrain the set of documents to a specific use, which is conceptually a very different thing. So if that's an ideal example (and since @dret wrote the profile spec we can assume it is), I'm likely to agree that schemas are not profiles.

I think it is critical to be able to request conformance to a schema through media type negotiation, so we should make that possible. We may need to choose or invent another relation, though. I'd like to use an existing one (and likewise a media type parameter that would be recognizable) if possible, but only if it actually fits.

@dret
Copy link
Contributor

dret commented Jan 16, 2017 via email

@dret
Copy link
Contributor

dret commented Jan 16, 2017 via email

@handrews
Copy link
Contributor Author

@dret said:

personally, i would think that "type" is much more fitting for this scenario, but that spec is equally fuzzy.

Yeah, I noticed that :-/

I agree that it seems to be the closest one. Perhaps we can simply define and register a media type parameter for specifying type analogous to the profile one? I have on idea whether that sort of thing is likely to be embraced or frowned upon in the IETF.

@dret
Copy link
Contributor

dret commented Jan 17, 2017 via email

@handrews
Copy link
Contributor Author

@dret part of the point of REST is to not design your own protocols :-P

What I want out of the media type parameter is to be able to manage API evolution by allowing a client to signal which representations (as indicated by schema URIs which may include a version) they understand. This is not a concept I've been able to test on a large scale yet, so I am also still looking into other options. But this is the one that is most appealing and flexible so far.

  • I don't like versioning resources through the URI. It's still the same resource whether I add a new field or not, and changing the URI guarantees that things will break even if the change does not directly impact the client
  • I don't like resource-specific media types. Media types are generic.
  • I don't like custom headers (for the same reasons that I don't like custom protocols, although custom headers are less burdensome)

A schema media type parameter seems ideal. It lets me version the thing that actually changes: the structural details of the representation. I can put multiple schema versions in if the representation is potentially usable to clients who know different schemas (if it validates, it's usable- if a client isn't impacted by a new field, the new field's presence should not automatically break things).

I am very interested in other ideas on the topic of API evolution, though, and don't consider this a settled solution at all.

@handrews
Copy link
Contributor Author

@dret the solution may be to define application/instance+json as well as application/schema+json, both of which can take a schema media type parameter, which is expected to be of type application/schema+json (this allows schemas to specify meta-schemas by the same mechanism that instances specify schemas). That may be preferable or at least more feasible than adding a parameter type to application/json itself.

@handrews
Copy link
Contributor Author

@dret with regards to your XML comparison, we've really avoided putting any requirements on the content of the JSON instance document aside from what is actually required by the schema. I want JSON Schema to be usable with any JSON document, including JSON document formats that were not initially built with JSON Schema in mind (e.g. legacy API formats).

I am sure I could find a way to make things work without a media type parameter. The content negotiation abilities would not be there, but clients could just look through the link headers (which is already one of the options we offer with "describedBy" links, although the more I look at it the more I think "type" is more appropriate). It makes the client do a bit more work but the fundamental capabilities are all still there. Maybe that's what you meant by defining a protocol?

@dret
Copy link
Contributor

dret commented Jan 17, 2017 via email

@awwright
Copy link
Member

What I see "profile" accomplishing in JSON Schema is exactly what the RFC claims: It's enabling clients to comprehend additional semantics about a document. For example, a podcast profile of JSON.

My understanding of "type" is from the RDF sense, it's the most generic kind of membership you can get: An assertion that some resource is an "instance" in some non-terminated, open set. In this understanding, it seems to be that "profile" is a special kind of "type" relationship.

I'm not suggesting a profile should imply there's a JSON Schema, but rather some profiles on JSON documents will have a document describing that profile (i.e. vocabulary) in a machine readable fashion, which is useful in RESTful applications.

I think having "profile" as a media type parameter is useful because that enables Content-Type negotiation. This is especially useful for a syntax that is very brittle. However, perhaps it could be a separate document altogether to emphasize this isn't a JSON Schema thing.

@handrews
Copy link
Contributor Author

@awwright you seem to have the opposite view of "profile" and "type" as @dret. When you refer to RDF, do you mean that the RFC 5988 "type" link relation is used in RDF, or do you mean a more general usage of the term "type"?

@handrews
Copy link
Contributor Author

@awwright From looking at JSON-LD's use of profile (it defines three for application/ld+json: expanded, compacted, and flattened), it seems to be very much in line with @dret's description rather than the currently described use within JSON Schema.

As we look towards Draft 07 and beyond, I think we should consider the "type" relation, and look for a good way to enable content negotiation based on either that or a more specific "schema" relation and parameter. I really don't see how our use of profile can be squared with its definitions or with examples such as I-JSON and JSON-LD

@awwright
Copy link
Member

I suppose @dret would be the expert on that subject.

To put my viewpoint another way, "type" just doesn't convey the same meaning to me. A resource might have a "type" of Car (an instance of a Car), or be an instance of Process, or something. It just classifies resources.

A profile might be used to express syntactical restrictions sure, but also vocabulary restrictions. HTML uses "profile" in the vocabulary fashion (as seen in HTML 4.01, here's a proposal to add it to HTML5).

There's a lot of nuance here that's probably getting glossed over, let me see if I can take some time to understand it.

@handrews
Copy link
Contributor Author

@awwright I agree that that definition of profile makes sense for use here, but that's not the definition declared in the RFC and registered as a link relation with the IANA.

I'm not dead set on "type", it just happens to be around and close. There is also "describedby" which is pretty vague and people seem to talk about it pointing to docs more than machine readabile thing (that's a casual assertion, not backed up with anything resembling proof).

"schema" would not seem totally out of the question, if we end up defining our own.

It's frustrating that not all standards use the same term in the same way, but as much as the HTML proposal fits (seems to have been abandoned for nearly 7 years, though), the more well-established definition does not.

@dret
Copy link
Contributor

dret commented Jan 31, 2017 via email

@jam01
Copy link

jam01 commented Mar 29, 2017

Hey all,

After a few days going through Dr. Fielding's work, reading up on Media Types, related RFC specs, few blog articles, and github issues I ended up here.

In an effort to implement REST, I'm trying to find concrete ways to achieving the following:
Use Media Type for content negotiation.
Use Media Type to help document representation with hypermedia.

I think a combination of application/json and hyperschema would achieve this. Which I think coincides with @handrews comment #222 (comment)

Here's a couple resoures that helped me get here:

Profile parameter to define the representation
https://www.narwhl.com/2015/03/the-ultimate-solution-to-versioning-rest-apis-content-negotiation and
json-schema/json-schema#179

Media type for content negotiation, though using a vendor tree.
http://www.baeldung.com/spring-rest-custom-media-type

Now it seems the question is whether to use "profile" parameter or create another one, "schema".

Profile:
Pros:
It was already defined in the json-schema spec
It seems to already be getting some adoption
While not specifically what profile was meant to be, I think in a particular interpretation it does fit

Cons:
@dret being the authority, we know that it's not what it was meant for.
Might make interpreation of profile more difficult
Might influence its use with other media types. Pro?

Schema:
Pros:
Completely clear, unambiguous
Cons:
Requires more effort, write and register the spec. @dret pointed out added difficulty as it'd relate to json itself.

Couldn't we register schema to be a media type level parameter, and not specific to json?
What's the reasoning behind not using vendor trees? We could use unregistered x tree.

My vote is with using profile, though I don't think we can go wrong with either option.

@handrews
Copy link
Contributor Author

@jam01 thanks for commenting, this is great!

I am personally leaning more towards trying to register "schema" as a link relation and media type parameter. Content negotiation via schema URI in a media type parameter is essential to the API evolution approach that I want to take, as I've no doubt mentioned above (can't be bothered to re-read all of my comments in detail right now :-)

As for media type parameters and specificity to JSON, the media type has to declare what parameters it accepts. So we can specify that a particular link relation is intended to be used with the same semantics in a media type parameter, but each media type has to choose to use it. Of course, if you make a vendor +json type, you can put whatever parameters you want on it. I'd rather use a parameter with application/json but there are options there.

We're on the verge of publishing another draft in the next few days/week or so, but I'd really like to get this and other core aspects nailed down in the following draft. It would be good to settle on the core stuff and give folks confidence that it will be as stable as a draft can be expected to be.

I also want to explore ideas around how JSON Schema can integrate with JSON-LD. I've discussed this with one of the editors of the JSON-LD spec and we both feel like there's something there, but the details are vague. I'd like the idea of being able to declare JSON Schema vocabularies through JSON-LD instead of (or more likely in addition to?) the "$schema" keyword. It would be much more flexible.

@akuckartz
Copy link

👍 for attempting to use JSON-LD :-)

@handrews handrews added the core label May 16, 2017
@handrews handrews added this to the draft-07 (wright-*-02) milestone May 16, 2017
This was referenced Aug 19, 2017
@handrews
Copy link
Contributor Author

handrews commented Sep 1, 2017

Having surveyed the current state of the world here, I am going to go ahead with #351 application/schema-instance+json as the mechanism for defining media-type-specific mechanisms for connecting instances and schemas.

As noted at various points above, we can propagate media type features such as content negotiation parameters and fragment identifier syntax to application/json at any time in the future. Defining and using them in application/schema-instance+json will give us real-world experience in whether the solutions are useful enough to be of interest to the broader JSON community.

As for "profile":

  • @dret says that this is not how it was intended to be used
  • @awwright points to the HTML5 "profile", but that is not related to @dret's "profile" (the RFC is never mentioned); a coincidence of naming is not relevant here
  • @dret has attempted to get the application/json folks to add "profile" as a parameter, and they have declined
  • Looking at other JSON profile uses, JSON-LD's usage of profile matches @dret's descriptions and not @awwright's

Therefore I will be taking "profile" out while adding application/schema-instance+json. Even though we could define a profile parameter for that media type, it seems clear from the arguments to date, and the lack of relevance of some cited supporting materials, that "profile" is just not the right mechanism.

This will be done as a single PR with #351 as it is a direct replacement of one solution with another.

@handrews handrews self-assigned this Sep 1, 2017
@handrews
Copy link
Contributor Author

handrews commented Sep 1, 2017

@akuckartz for JSON-LD + JSON Schema discussion, please see #309 (or if that is not quite right, please file a new issue). I'm still interested in this, I would just like to consider it separate from "profile"

handrews added a commit to handrews/json-schema-spec that referenced this issue Sep 19, 2017
This change removes "profile", since as noted in issue json-schema-org#222 the
usage is not correct.  It replaces it with "schema", with the
minimal possible change.

We may wish to also remove the "describedby" link relation and
instead mandate using "schema", or a similarly named
extension relation, and allow downloading schemas based on the
parameter.  That is outside of the scope of this change.
@awwright awwright self-assigned this Sep 19, 2017
@handrews
Copy link
Contributor Author

Merged #414.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clarification Items that need to be clarified in the specification core Priority: High
Projects
None yet
Development

No branches or pull requests

6 participants