added links proposal

[fehguy]

To support HAL-style linking, as suggested in #688, I propose the following:

• A mechanism for design-time, static links between responses and operations to make operations on responses visible to consumers
• A reusable linking structure to allow link definitions to be DRY
• A mechasim for parameterized link traversal to avoid server-side computation of links through reuse of existing, OAS-defined operations

Additional considerations:

• This is not intended to be a run time linking solution. This solution does not require users to redesign their runtime API. It may be able to represent the intent of runtime HAL-style links in responses. It does not intend to represent dynamically generated, arbitrary links.
• A clear separation of concerns, between payloads and operations. Therefore no operation data shall be tied to payload schemas.

Spec changes

• Introduction of a Link Object to represent a single link
• Addition of a links object in the Response Object to define what links are available from a specific operation
• Addition of a links section in the Components Object to allow reuse across operations
• Introduction of a Link Parameters Object to represent parameters to build a link from
• A mechanism for reading values from response payloads and using them for client-side link computation

[fehguy]

Need to add an example using query parameters

[darrelmiller]

@fehguy This might be a useful point to consider using URI Template syntax for specifying query parameters. It would allow defining optional query parameters and all the logic of how to resolve those. It also would help with resolving your red, green, blue scenario.

[darrelmiller]

@fehguy Having considered this more, the only time URI Templates would be useful is for external links where there are optional query parameters. If I understand correctly, an external link that used query parameters would look something like,

components: 
   links:
     UserRepositories:
        # returns array of '#/components/definitions/repository'
        href: 'https://na2.gigantic-server.com/2.0/repositories?username={$response.username}&active={$response.activeOnly}

With this format, if response.activeOnly was null then the resulting url would contain &active= which may not be acceptable to the target server. Unfortunately, defining a URI Template for this scenario wouldn't work either because the syntax that allows optional parameters uses a technique that would prevent us from naming the query parameters correctly.

[fehguy]

@OAI/tdc this is per the last TDC call. Please add comments, note I have added several open issues that need your help with responses or documentation.

[ePaul]

I'm not sure that this special-case handling for arrays is really useful and generalizable for more complex objects.
What happens if you have an array of arrays? Or an object containing an array, which in turn contains an object with a property we want to access?

As I understand, the point is that if some information appears multiple times in the response, we get multiple links generated. So we need a way to extract that information in an array.
Why don't we use something like JSONPath to extract what we want from the response, instead of defining our own ad-hoc syntax?

With that, in your example the link definitions would look like this (the rest of the example remains same):

  links:
    UserRepositories:
      # returns array of '#/components/definitions/repository'
      operationId: getRepositoriesByOwner
      parameters:
        username: $.username
    UserRepository:
      # returns '#/components/definitions/repository'
      operationId: getRepository
      parameters:
        username: $[*].owner.username
        slug: $[*].slug
    RepositoryPullRequests:
      # returns '#/components/definitions/pullrequest'
      operationId: getPullRequestsByRepository
        params: 
          username: $.owner.username
          slug: $.slug
    PullRequestMerge:
      # executes /2.0/repositories/{username}/{slug}/pullrequests/{pid}/merge
      operationId: mergePullRequest
      parameters:
        username: $.user.username # Should be $response.author.username?
        slug: $.repository.slug
        pid: $.id

Apart from $response → $, the main difference is in the array access in the UserRepository link.
That also shows a possible problem: Does this create one link for each combination of username and slug (not wanted), or aligns the links from the user names and slugs with each other? We want the latter, but how to specify this if the array is not at top-level, and we even might be accessing different arrays in both parameters?

[fehguy]

@ePaul recall that link generation is a client-side activity, so the consumer is responsible for generating the links themselves.

As far as nested arrays, yes, the goal is to produce multiple links. Again, it is up to the consumer of the response to decide if it should follow them or not. And with nested arrays, yes there can be combinatorics. That is actually desirable.

As far as syntax, I'm not really partial to any syntax other than:

1. Making sure it's not trying to do too much. This applies for the hesitation for general JSON Schema support or path templating
2. It's supported and understood well enough

If we can use something else, then great. If the other syntaxes are far more than we need, we can choose to support a subset or not use them.

[ePaul]

@ePaul recall that link generation is a client-side activity, so the consumer is responsible for generating the links themselves.

Yes, the goal is to make it possible, given an OpenAPI definition and a response from one operation, to know exactly which link URIs are to be generated. (Whether an user of a potential client library actually retrieves this list, and/or follows them, is of course not decided by the API definition.)

My main point was that the proposed spec change was a bit nebulous enough about the array effect, letting an implementor guess what to do. This certainly needs rewording. (I can try to find a better wording, though not right now ... a bit too tired to think clearly.)

The second point is, that the proposed syntax doesn't indicate where there is an array expected, like how JSONPath's [*] does it.
(While this might be intended (i.e. to match HAL's links, which can either come as an object or an array of objects), this contradicts the whole point of being "deterministic" which Swagger/OpenAPI had until now.)

Inventing a new sub-syntax seemed odd, thus I proposed JSONPath as I found that recently and it allows doing this retrieval (I've done exactly this).
Of course, it also allows more things, which we might not actually need, so defining the actually used subset (like just . and [*]) might be a way to go. (Then implementors can choose to use an existing JSONPath implementation, or implement the used subset themselves.)

I understand that the $response keyword was selected to be able to later extend this to request properties/parameters.

[ePaul]

Another thing I noted while trying to write down my proposal:

• In the href field we have an URL template in which parts within {} will be interpreted as some kind of expression and then replaced.
• In the link parameters object, we have "static or dynamic values". The specification text doesn't really say what this is, but from the examples it looks like "dynamic values" are strings starting with $response (which are interpreted as the same kind of expressions as before), while static values are everything else (i.e. booleans, numbers, objects, arrays, or strings not starting with $response). I guess only primitives are sensible for simple parameters (header/query/path), but for body parameters having an object or array would make sense. When using the dynamic values, we could also extract objects/arrays from the response, but then it is not clear how this would interact with the array expansion logic.

[ePaul]

Here is a proposal trying to make it a bit clearer. I have omitted the parts which didn't change.

This stays with the $response prefix for the expressions, but adds a [*] whenever we are accessing an array element. It also shows how a property expression can be validated against a schema.

---

Link Object

The Link Object is responsible for defining a possible operation based on a single response.

Field Name	Type	Description
href	url	a relative or absolute URL template (see below) to a linked resource. This field is mutually exclusive with the operationId field.
operationId	string	the name of an existing, resolvable OAS operation, as defined with a unique operationId. This field is mutually exclusive with the href field. Relative href values may be used to locate an existing Operation Object in the OAS.
parameters	Link Parameters Object	an Object representing parameters to pass to an operation as specified with operationId or identified via href.
description	string	a description of the link, supports GFM.

Link URI templates

The href field is a string which can contain variable placeholders which will be filled from the response. Those placeholders are surrounded by braces ({}) and contain each an expression which follows the property expression grammar explained below. For use in templates, only expressions with a primitive result schema are allowed.

When evaluating a template against a response, each expression is evaluated and produces a list of values (possibly just one). The cross product of those lists is created, and for each element of the cross product we get one link by replacing the values into the URL template.

(TODO: this needs an example. Also, it doesn't really solve when we have properties in objects in an array, where we don't really want a cross product, but zipping their result lists.)

Property Expressions

A property expression (used in URL templates for links, as well as in links parameters inside a response) can have any of the following forms. Its validity depends on the response's schema definition, and with an actual response object (i.e. an object matching a schema) it can be evaluated to a result list (a list of values, each matching the schema).

Form	Validity condition	Schema	Evaluation (result list)
$response	The response has a schema.	The response's schema.	A list with the whole response value as the single element, or an empty list if there is no response value.
expression . property-name	The expression's schema has type object, and includes a property with name property-name	The schema of that property	For each element x of the result of expression, if x has a field named property-name, include that field's value in the result list, else nothing.
expression [*]	The expression's schema has type array, and has a defined items schema.	The expression's schema's items schema.	For each element x of the result of expression and for each element y of x, include y in the result list.

(TODO: add examples)

Link Parameters

Using the operationId to reference an operation in the definition has many benefits, including the ability to define media-type options, security requirements, response and error payloads. Many operations require parameters to be passed, and these may be dynamic depending on the response itself.

To specify parameters required by the operation, we can use a Link Parameters Object. This object contains parameter names along with static or dynamic values.

Fields

Field Pattern	Type	Description
{name}	Any	{name} must be the name of a parameter defined for the operation identified by operationId. The value must be either a literal value matching the type of that parameter's schema, or a string with a property expression with has (in the context of the current response) a schema which is compatible to the named parameter's schema.

(TODO: What happens if the operation has multiple parameters of the same name, like one header parameter and one query parameter of same name?)

---

This omits all the examples. If wanted, I can do a pull request with my changes relative to this branch, including the given examples (adapting them to the new syntax).

Here is a still open problem (with both my wording as the existing one):

Given the Bitbucket API in your example, with a link definition like this:

    UserRepository:
      # returns '#/components/definitions/repository'
      operationId: getRepository
      parameters:
        username: $response[*].owner.username
        slug: $response[*].slug

Assume an example response body like this:

[ { "slug": "OpenAPI-Specification", "owner": { "username" : "OAI" } },
  { "slug": "swagger-codegen", "owner": { "username": "swagger" } } ]

We want that this produces two links (/2.0/repositories/OAI/OpenAPI-Specification and /2.0/repositories/swagger/swagger-codegen), not four of them (i.e. neither /2.0/repositories/swagger/OpenAPI-Specification nor /2.0/repositories/OAI/swagger-codegen).

[fehguy]

@ePaul i suggest that we bring wording suggestions into the PR instead of comments.

[DavidBiesack]

@fehguy can you elaborate on what you mean by "I don't think that mime-type traversals should be supported when following links (perhaps explicitly forbidden)"? I don't understand what this implies.

For example, in our APIs we use content type negotiation and wish to use different links to expose different flavors of the operation. For example, for a report resource, we may want to describe multiple links to the same resource/operation such as

method: GET
uri: /reports/{reportId}
  link 1:  rel: self with response media type application/vnd.sas.report+json
  link 2:  rel: pdf with response media type application/pdf
  link 3:  rel: msword with a response media type application/vnd.openxmlformats-officedocument.wordprocessingml.document

(This ties back to our old friend, #146 )

I would not want this type of API specification to be forbidden.

[fehguy]

Need to consider further the content-type negotiation between links

[fehguy]

I've updated this PR. Some comments are now out-of-place and resolved, so I'm removing them.