link hints

[mnot]

See also #55 and #192

This issue is for general discussion about a standalone concept of "link hints."

[mnot]

For background - the current approach to target attributes (which are effectively link hints) in the proposed RFC5988bis document is that they can be defined by one of:

• The link serialisation
• The link relation type itself

So, e.g., HTML can define some that apply to all links (and it does), and individual link relations can define additional ones (and they do).

AFAICT the ideas behind doing more (probably in a separate spec) are coming from roughly two directions:

1. Discomfort that there is potential for conflict/semantic duplication. E.g., two serialisations might define conflicting attributes, or a relation type's attribute might be subsumed by that defined by a serialiation.

The counterargument is that we've got this far without this being a significant problem. I think that's largely because there are a couple of "big" serialisations (HTML and Atom), and their set of target attributes are relatively well-known and stable.

If we had a galaxy of serialisations with a number of potential conflicts, I could see how that would be annoying, but even then, a new serialisation could take steps to isolate its target attributes from those of the link relation types used in it.

Regarding conflicts between target attributes defined by serialisations themselves -- again, it may be a bit annoying, but I don't see big interop problems here.

2. Reuse. If I go to the trouble of defining a target attribute for one relation type (or serialisation), it'd be really nice if it was usable with others.

The counterargument here is that they are; you just need to reference the defined target attribute in the definition of your relation type / link serialisation, or copy it (but probably reference).

Allowing a third party to add semantics (even if just hints) to an existing serialisation or relation type without coordination seems suboptimal, especially if they change the semantics of the link (e.g., overloading how the client interacts with the server).

Anything else? Please feel free to add / challenge / etc., I'm just trying to summarise my current thinking.

Also, please note I'm not against a given serialisation defining its own concept of link hints, or even a generic concept of link hints w/ a registry that serialisations and/or relation types opt into in their definitions; I just don't want to make it a blanket thing.

So, I think it's out of scope for RFC5988bis (as discussed in #36). Regarding json-home, it feels premature to split out the link hints for that format into a generic solution for other serialisations.

cc @dret

[handrews]

@mnot I'm trying to figure out how to best relate this to JSON Hyper-Schema and its Link Description Object (LDO) which is its link serialization mechanism.

My main question is: should URI scheme/protocol specific things, such as an "allow" hint corresponding to the HTTP Allow header, be part of this discussion?

I feel like these are demanded primarily by people writing full API description formats that want all possible information presented from design-time. Hypermedia-driven systems could make some use of these but it feels like less of a match. A link hints standard could provide some guidance on whether and how to include these sorts of hints when designing link serialization formats.

---

For a bit more context on how I'm looking at this, I mostly agree with @dret's hypermedia concepts, particularly the distinction between target resource hints vs traversal hints. My only disagreement is that I consider URI template resolution separate from traversal hints. So I see four questions about a link:

Q: Why should I use this link?
A: link relation type

Q: Which resource out of the possible set should I use?
A: URI template and any guidance on how to resolve the template

Q: What options do I have for consuming the target resource, e.g. content negotiation
A: target resource hints per @dret

Q: How can I interact with this resource, e.g. HTTP usage?
A: traversal hints per @dret, excluding URI templating

... and it's not clear to me whether links and link serialization formats should answer that last question.

[handrews]

@mnot while I think the comment above is pretty general, if this one is getting too down in the weeds of JSON Hyper-Schema I'll be happy to remove this comment.

There is a TL;DR at the bottom

---

Aside from traversal hints, I am also concerned over JSON Hyper-Schema's "targetSchema" LDO keyword. Like all target resource hints, it is advisory. But it is weird in that it is actually hinting the result of following two links. In a fully dynamic system you would:

For a GET (without "targetSchema" in the LDO):

1. Start with resource A and its hyper-schema, and look at a link to B
2. Issue a GET to B
3. follow the "describedBy" link in B's response header to get B's hyper-schema
4. If I want to issue a PUT, I know its request should validate against the same schema used for the GET (per HTTP semantics, both are representations)

For a PUT (without "targetSchema" in the LDO, and avoiding a GET)

1. Start with resource A and its hyper-schema, and look at a link to B
2. Issue a HEAD on B
3. Check its "allow" header to see if PUT is supported, but more importantly
4. follow the "describedBy" link in the response header to get B's hyper-schema
5. Use the hyperschema to build a PUT request for B without issuing a GET

For a PUT that avoids a GET when "targetSchema" is present:

1. Start with resource A and its hyper-schema, and look at a link to B
2. Look at the "targetSchema", which SHOULD be the schema for B and therefore the schema for a PUT request to B
3. Use the link's "targetSchema" to build and validate a PUT request to B

That last one is problematic. There is no inherent guarantee that the "targetSchema" in the link from A is correct for B. Even if it was correct at the time that A's hyper-schema was generated, in a sufficiently dynamic system it may be out of date by the time the PUT request is constructed and sent.

• "targetSchema" feels like a target hint, particularly when you consider the schema to be part of the media type (in some proposals there is a media type parameter for it)
• Implementations treat it as the schema to use for a PUT request, or the schema to use in combination with the media type from Accept-Patch to figure out how to build a PATCH request
• This seems to go against the security considerations based on who owns which schema

---

TL;DR:

Is this "target hint" really an API description / documentation "feature" masquerading as a link hint? This is the sort of thing that I'd like to understand better when it comes to properly designing the LDO for Hyper-Schema.

If the main point of "targetSchema" is to enable documentation and static description, but it MUST or even SHOULD be ignored at runtime for things like building PUT or PATCH request documents, then I think it probably belongs in API description formats (which may build on hyper-schema) but not in hyper-schema's LDOs. Thoughts?

[mnot]

I don't think @dret's implicit and explicit traversal hints are a dichotomy; rather, I tend to see the explicit hints as refining the semantics of the link relation with link-authoring-time information.

The ones that I define in json-home are limited to information that you can discover by interacting with the resource -- and importantly, that information that's already expressed through standard (not resource-specific) mechanisms. It's really just trying to save you a bit of trouble (and RTs) when you interact with those resources.

[handrews]

It's really just trying to save you a bit of trouble (and RTs) when you interact with those resources.

That seems to sum up both "target hints" and "explicit traversal hints".

I think I agree with explicit traversal hints being refinements of the more general (and possibly nonexistent) implicit hints indicated by the link relation. To rephrase to see if I understand this, a link relation might imply the ability to do any number of things, while an explicit traversal hint restricts that to allow a client to avoid attempting those things that are not implemented for this particular link and target resource.

---

I think one reason I'm struggling with target vs traversal hints is that target hints are generally about the target resource's representation (media type, character set, language). These are applicable to pretty much any dereferenceable link, no matter the URI scheme. So adding them to a link representation spec feels pretty natural.

But traversal hints are, by definition, scheme/protocol specific. And then I feel like we're back to one of:

• Add protocol-specific hints directly to the link format, for however many protocols are desired. That seems kinda gross, and easy to get out of hand from sheer volume.
• Add a general pattern from protocol-specific hints, and (somewhere) specify how to map elements of each protocol into the hint syntax. That defers the protocol-specific things to structures that are clearly protocol-specific
• Attempt to generalize traversal hints to a protocol-neutral form. Or at least protocol-neutral names, perhaps allowing protocol-specific values. Which I'm guessing would end up as looking mostly like HTTP and could therefore end up rather limiting at some point.

Am I missing something here? It feels like there needs to be a decision on whether to provide explicit traversal hints, and if so which of these (or some other) approach to use. Is that decision in-scope for this issue?

[mnot]

When you say protocol, do you mean the-thing-roughly-identified-by-the-uri-scheme, or the application-specific protocol in use (e.g,. "e-mail over HTTP")?

[handrews]

@mnot I mean the-thing-roughly-identified-by-the-uri-scheme :-) Is there a better name for that? I realized when I was writing the above comment that it's not quite right but couldn't come up with anything else. Maybe I should have just said URI-scheme-specific...

[mnot]

I guess I'm not feeling too much concern here, because my interests are around / needs are met by HTTP(S), not other protocols (schemes).

What other ones did you have in mind?

[handrews]

@awwright has often brought up the need to be HTTP-agnostic in JSON Hyper-Schema, so perhaps he can comment on this.

For my part, the most obvious other one is CoAP which of course draws heavily from HTTP and is therefore the least problematic alternative anyway.

Another realistic possibility is integrating something like Cap'n Proto RPC into a hypermedia system.
That would require defining a media type for the serialization and a URI scheme for the RPC protocol, but those are solvable problems, and I've floated the idea with capnp's inventor/maintainer in the context of bridging web-oriented hypermedia and capability-based RPC.

[awwright]

I think you just need to define the parameters/hints in an appropriate manner, perhaps (in so many words) "When making HTTP-compatible requests for this resource, optimize the request expecting the specified HTTP methods to be supported by the resource".

Worded this way because, keep in mind, it's technically legal to make a request to an HTTP server for any URI of any scheme, and the semantics of the Web are by and large defined by HTTP even when it's a different protocol (especially see HTTP/2, CoAP).

And then for any resources not retrievable over the network, a definition like this is just a no-op.

[algermissen]

Reading this briefly (so excuse if I end up OT) it sounds as if there is too much desire to somehow establish more contract than is inherently possible in a decentralised system. Over-emphasising hints does not make them any more reliable - after all, they are just hints and the server can still do what it wants.

A developer will best introspect servers anyhow at implementation time and assume reasonable stability - and then code for the unhappy paths. Hints do not make that go away - they just look as if they could. But they can't :-)

Regarding non HTTP-like protocols: I'd probably see JSON Home as for the HTTP family and let other protocols deal with whatever 'home info' approach they deem suitable for themselves.

[handrews]

@algermissen the point of hints is not to establish a further contract (use OpenAPI or RAML or something for that). It is to advertise what is reasonably likely to be supported and therefore worth attempting and coding for at the application level. Yes, you always have to handle all errors, I'm well aware of that thank you.

There's this tendency to act as if the only choices for applications are fully dynamic hypermedia with runtime introspection, or hardcoding everything. This is not the case. The point of hints and other non-authoritative information is narrowing down the set of likely scenarios that are worth attempting and handling. In any kid of programming, you make tradeoffs about how much effort to put into different things, ranging from very thorough handling to exiting at the first sign of something unexpected. There are a great many possible things one can do with HTTP in particular, but not all of them are always reasonable to attempt, and not all of the results are always worth handling in detail.

Also, as noted in the first comment, this is about the standalone concept of "link hints", not JSON Home, specifically. Unless I am really misreading that sentence.

Anyway, if there's no interest in the concept, @awwright and I can just go back to arguing about it for JSON Hyper-Schema and related work. I had thought @dret had some interest in this as well, but if he doesn't see anything worth supporting here, and no one else has shown up with interest in the last three months, I'm not particularly interested in trying to sell the idea myself.

I would find it ideal to base anything in JSON Hyper-Schema off of a more generic approach to link hinting, but we certainly don't need such an approach to move forward and one project is not enough to motivate an independent spec. Link hints aren't even the only possible approach to solving Hyper-Schema's needs here.

[dret]

On 2017-05-20 20:51, Henry Andrews wrote: @algermissen <https://github.com/algermissen> the point of hints is not to establish a further contract (use OpenAPI or RAML or something for that). It is to advertise what is reasonably likely to be supported and therefore worth attempting and coding for at the application level. Yes, you always have to handle all errors, I'm well aware of that thank you.

+1; it's as much about documenting capabilities (i.e., design time) as it is about making them usable for runtime discovery. the more we make these capabilities describable and discoverable in a coherent way, the easier we make it for designers/developers to describe, discover, and use them. because of this, factoring them out of specific formats is a useful, even though of course as @mnot mentioned, there always is the risk of premature optimization/standardization.

Also, as noted in the first comment, this is about the standalone concept of "link hints", not JSON Home, specifically. Unless I am really misreading that sentence.

yes. the connection is that JSON home's inclusion of some form of link hints spawned this discussion. if those were reusable standalone, JSON home described resources could directly reuse them. and anybody else could reuse them as well (such as JSON hyperschema).

Anyway, if there's no interest in the concept, @awwright <https://github.com/awwright> and I can just go back to arguing about it for JSON Hyper-Schema and related work. I had thought @dret <https://github.com/dret> had some interest in this as well, but if he doesn't see anything worth supporting here, and no one else has shown up with interest in the last three months, I'm not particularly interested in trying to sell the idea myself.

i am interested. to me, JSON home alone would be a good motivation to do it because by definition, the resources described by JSON home might be good candidates to simply reuse the link hints. or you could see it the other way around: any API using link hints could very well and easily described via JSON home if the link hint design would be shared across JSON home and those APIs described by it. JSON hyperschema is another good candidate, i think. i'd be interested to pursue this. we could look at hyperschema and home as two use cases. we could use home's model as inspiration. whether or not home actually "switched" then is something we'd see. we could even go as far as referring to home's hints as (some of) the hints defined in a separate link hints spec.

I would find it ideal to base anything in JSON Hyper-Schema off of a more generic approach to link hinting, but we certainly don't need such an approach to move forward and one project is not enough to motivate an independent spec. Link hints aren't even the only possible approach to solving Hyper-Schema's needs here.

agreed. but i think we have at least two already (hyperschema and home). and for the APIs exposed through home, maybe it would be convenient to have the hints being readily reusable through a link hints spec. in summary: i think we have some momentum and some use cases. if we drive it by using hyperschema and home as scenarios, we might have enough input to get a first version started. i don't know yet, but i could imagine link hints adopting a registry model, with initial hints being defined by the spec, but with an extension model that is based on a registry. in that case, this should be usable for both basic users, and those with more advanced link hinting needs.

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#211 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABw1JPowfvzOv9uD0fLccVgjwK0y_oTuks5r7ygKgaJpZM4MK2kP>.

-- erik wilde | mailto:erik.wilde@dret.net | | http://dret.net/netdret | | http://twitter.com/dret |

[mnot]

I'm not sure about a common registry. As per 5988bis, link attributes are defined by link relation types and/or link serialisations.

Presumably we'd define a registry for hints (attributes) that a link relation and/or serialisation could opt into -- but they'd be opting into all of the registered values, no matter what the context.

For example, if such a registry were defined, I suspect it'd quickly be populated by COAP folks for IoT-specific use cases.

What utility would that provide?

OTOH, I can see some value in having a common set of "HTTP Link Hints" for reuse by different relations / serialisations, so they don't have to be constantly re-invented. My inclination would be just to define the set in the spec, updating the spec as necessary to expand it, rather than using a registry (unless the scope of the registry was very tightly defined).

[dret]

On 2017-06-20 02:48, Mark Nottingham wrote: OTOH, I /can/ see some value in having a common set of "HTTP Link Hints" for reuse by different relations / serialisations, so they don't have to be constantly re-invented. My inclination would be just to define the set in the spec, updating the spec as necessary to expand it, rather than using a registry (unless the scope of the registry was very tightly defined).

https://tools.ietf.org/html/draft-wilde-registries is my personal attempt to think about registries a bit more principled. it's a design spectrum. wikis may be on the one end, specs with hard-coded value sets on the other. so to some extent, if the spec at least makes it clear that the set of values is expected to evolve and defines a processing model covering this, then it would be a registry of sorts to say that new values are added by spec revision. that's how w3c seems to approach this currently since they don't have a registry system. my main concern is to see many APIs where people reinvent the link hint wheel over and over again. i am glad to see them adding links to their APIs, which means they are thinking about hypermedia. it would be great if we could help them avoiding designing snowflakes and instead using "link patterns" they can build their APIs around. my thought always was that JSON home could be a perfect design example for how to design a hypermedia media type. that's my main goal. if that example for now has a fixed set of link hints that may evolve with future revisions, then i can live with that as well. as long as the model itself is open and extensible and thus can be used as a foundation for webby designs.

[mnot]

I've re-started work on link-hints; see
https://mnot.github.io/I-D/link-hint/
and
https://github.com/mnot/I-D/issues?q=is%3Aissue+is%3Aopen+label%3Alink-hint

[dret]

On 2018-03-27 04:49, Mark Nottingham wrote: I've re-started work on link-hints; see https://mnot.github.io/I-D/link-hint/ and https://github.com/mnot/I-D/issues?q=is%3Aissue+is%3Aopen+label%3Alink-hint

great news! and it's quite a coincidence, because in the context of looking at hypermedia annotations, just last week i went back to https://tools.ietf.org/html/draft-wilde-link-desc-01, thinking that something like that would (still) be nice to have. there's quite a bit of overlap, and as we discussed when we talked about these two drafts in the past, it would be useful to have a standalone way for these descriptions outside of the scope of home documents.

[mnot]

Talking to folks at IETF101 last week convinced me that it's worth a try.

[dret]

On 2018-03-27 08:07, Mark Nottingham wrote: Talking to folks at IETF101 last week convinced me that it's worth a try.

that's great to hear! i'd be interested to hear what changed your mind. i think last time (when i suggested to possibly join forces for the hints and descriptions drafts) you thought that maybe it was too early to take on as a separate thing, and that you'd rather see it in the home draft.