Resource oriented

[josephgardner]

Designing an API usually starts by identifying resources. Operations usually indicate how to transition through the system. The paths/request/response are semantics for utilizing an operation as it pertains to a resource.

Therefore, it makes sense to structure the document as a hierarchy network of resources and transitions. However, the path is hoisted all the way to the top which isn't natural in this paradigm.

There is certainly a lot more to explore on this idea, but wanted to present it as an alternative solution to many issues this attempts to address.

[sazzer]

The importance of URI paths is always an interesting one because something like OpenAPI puts significant importance on them - to the extent that this issue is talking about them as "a hierarchy of resources", whereas HTTP doesn't care at all - it's just an opaque string - and Hypermedia APIs mean that the client doesn't ever care even if the server does.

Specifically, a Hypermedia API would mean that the URI path is just an opaque string to the client with no meaning at all, which runs counter to some of how things like OpenAPI work.

That's not to say that there's no merit to this proposal. For systems that make heavy use of OpenAPI I think this would probably be a significant improvement. (It's exactly how RAML works, for example) but it's something to bear in mind.

[josephgardner]

this issue is talking about [URI paths] as "a hierarchy of resources"

I'm using the term "resource" to mean,

The target of an HTTP request is called a "resource", whose nature isn't defined further (mozilla)

This proposal deemphasizes the path in favor of the resource. As you suggested, the path is incidental. In fact, there could be many.

Quoting an old SO answer,

"Every finger belongs to exactly one hand" does not imply "every hand has exactly one finger". "Every URI designates exactly one resource" does not imply "every resource is designated by exactly one URI".

My use of the term "hierarchy" was intended to refer to the document structure; not necessarily the resources which would remain flat. The operations are the conduit for navigating across resources.

For systems that make heavy use of OpenAPI I think this would probably be a significant improvement.

Considering we are discussing the design of OpenAPI, this sounds like an endorsement :)

[josephgardner]

Took a crack at uml.

[josephgardner]

Something I'm hand-waving here is that each entity (Resource, Representation, Operation) has some type of id.

[ralfhandl]

I like the idea:

• define a resource Foo
• define its representations for
  • create,
  • update,
  • read,
  • read-list (for example an array wrapper around the read representation)

Then define

• URI template foo{?options*}
  • operation create using the create representation as the request and the read representation as the response
  • operation readMany with the read-list representation as the response
• URI template foo/{id}{?options*}
  • operation readOne with the read representation as the response
  • operation update using the update representation as the request and the read representation as the response if request header Prefer:return=reprentation is present
  • operation updateSilent using the update representation as the request and a 204 No Content response if request header Prefer:return=minimal is present
  • operation delete with no request and a 204 No Content response

Now those two URI templates and their operations are no longer unrelated and show their connection to a resource, which would be an alternative to the nested paths proposed #23.

[josephgardner]

• define [a resource Foo's representation] forread-list (for example an array wrapper around the read representation)

The read-list is particularly interesting; I hadn't considered the collection as a representation of the item resource. Rather, I envision the collection as a separate resource (e.g FooCollection), with its own set of corresponding representations.

However, I also expected the collection's read response representation to consist of an array of list-item representations for the item type resource it contains.

That draws a distinction between an item resource and a collection resource but also loosely couples them by allowing the collection representation to reference a partial representation of the item. It also wouldn't prevent you from creating multiple collection resources for the same item.

[handrews]

JSON Hyper-Schema in its last iteration before being tabled while we sorted out the Core and Validation specs was highly resource-oriented.

A resource was defined by a schema for its representation, with a "rel": "self" link attached to that schema's root that defined its HTTP interface. For an individual item resource, a "rel": "collection" link could also be attached, defining the collection interface. For a collection resource, "rel": "item" links could be attached to each item in the collection defining the individual item interface. A "self" link's targetSchema would be a $ref back to schema to which it is attached. Typically, a "collection" or "item" link's targetSchema would be a $ref to the representation of the collection or item, respectively, which would have its own "self" link.

There was no overall equivalent to an OpenAPI document, although you could define a home document schema linking to all endpoints that would largely serve that purpose. That schema didn't necessarily even have to have a functional "self" link as long as it was clear how to get it. This idea was not unlike mnot's old HTTP JSON Home I-D, although I didn't encounter that until later.

Each link would define which HTTP operations are allowed (by mimicking the HTTP Allow header- schemas could describe request and response headers). GET, PUT, and DELETE payloads were determined by HTTP semantics for those methods + resource representations. PATCH was determined by the declared PATCH media types. Error responses were defined as their own resources (I forget the intended way of correlating error responses with endpoints).

For POST, a separate submissionSchema could be added to a link. For a collection resource's "self" link, the submissionSchema would typically $ref the item resource's schema, minus any server-assigned identifier field (there's a pretty straightforward pattern for handling that and other minor HTTP-method-specific variations).

For user-submitted query string variables or any other user-submitted URI Template data, there was an hrefSchema. URI Template variables could also be resolved from the representation instance (e.g. {id} in /something/{id} was usually handled this way, not as user input).

Hyper-Schema's goal is/was to describe hypermedia resources that comply with HTTP semantics. It can describe those in a very compact manner; much more so than OpenAPI. However, the farther you went from that, the more difficult or awkward it became to use HyperSchema, while OpenAPI's verbosity stays constant. It's definitely a trade-off.

[evert]

This is a bit of of a 'me too', but I would also love to see resources disconnected from the uri template. It feels like defining a list of resources at the top level, and a separate object that effectively routes uri templates to resources gives you everything you have today, and opens the door up to this being used in HATEOAS scenarios in the future (even if it doesn't do links/following/state transitions day 1).

[evert]

Given that your preferred approach actively prevents a more suitable standard for hypermedia-based systems, whereas my preferred approach would make it easier for multiple paradigms to co-exist, I'm curious what your reasoning against is. Is it just the cognitive overhead of having a new top-level concept (resource type), or is there more to this?

This is a very long standing issue with OpenAPI.

Another case where this is relevant, many standards for APIs (think: AtomPub, ActivityPub, CalDAV) don't specify a URI structure at all. Wouldn't it be nice if authors of these standards can ship an OpenAPI document for clients and servers? As of right now, it's only possible to do this for specific implementations.

The starting point of any modern protocol is often something like a well .well-known endpoint.

[darrelmiller]

I have no issue with a new top level concept, which is why I suggested adding a linkRelationTypes. My concern is introducing a new concept that has no clear definition. How is "resource Type" different than something in #/components/schemas ?

What I have seen in the past is people try and associate a set of operations to a reusable concept and then further down the road they realize this is a problem because not all operations are supported in all places the concept is used.

[evert]

Based on your reply, I would indeed define 'resourcetype' as a set of operations. If it was just about schemas, I would have agreed that you can just point to reusable schemas. But in the case of links, it's very much not the case.

If not all operations are supported in a specific context, arguably that's a different resource type, or should be handled via an error response like 405, or should be discovered in advance (absence of a link for hypermedia, link hints, OPTIONS, etc)

Somewhat related is the link hints draft: https://www.ietf.org/archive/id/draft-nottingham-link-hint-03.html . Quite a simple spec in comparison to Open API but it solves a similar problem specifically when discussing 'describing the set of operations at the target of a link'.

[darrelmiller]

Link hints is a great solution for runtime discovery. But that isn't really the problem most people are trying to address with OpenAPI. OpenAPI is all about design time.

As for the concept of resource type, OData is an interesting case study. They started by allowing you to apply constraints on entity types, and over time have discovered that constraints mostly need to be applied to a specific path.

As I stated earlier, I think this is going to require an in person conversation to really get on the same page.

[evert]

My intention with point out link hints was simply that it addresses a similar need. The reason I'm here is to make OpenAPI work better for a wider range of APIs and especially HTTP based standard protocols. At design time.

It sounds like you made up your mind, but my door is always open if you want to have a dialog about this.

As a final thought, consider that it's not even possible to use OpenAPI to generically document Oauth2, a relatively simple protocol. All because it insists on keying everything by URL

[rattrayalex]

I've built a few systems for docs and codegen on top of OpenAPI, and I've always had to build my own resource map on top of OpenAPI. For example, did this at Stripe to power the sidebar/structure of the API docs, and the organization of the client libraries. Having it baked-in would be great.

If it helps, something I do today is roughly like this:

resources:
  foos:
    models:
      Foo: #/components/schemas/foo
    methods:
      create: post /foos
      retrieve: get /foos/{id}
      list: get /foos
    resources:
      bars:
        methods:
          create: post /foos/{id}/bars

Some methods tend to live on the top level, of course, and some models tend to be "shared"/"common" (like an Address type).

Today, people often use tags for this, but it's a hack and they don't allow nesting, attaching to schemas, etc.

I don't currently have much opinion on how exactly this should look for OAI, but I'd be delighted to collaborate on concepts if the OpenAPI maintainers are interested in exploring this.

[rattrayalex]

(for the curious, related discussion in this thread: #30 (comment))

[handrews]

I've written similar preprocessing formats myself, both to organize around resources and reduce the amount of boilerplate that needs to be written by hand.

[toumorokoshi]

Hello! Wanted to bump this thread - we are yet another group who model our APIs as resource-oriented (http://aep.dev/), an in-development fork of https://google.aip.dev/ which is also resource-oriented.

It would be great if there was a way to model this relationship. I think this goes beyond the relationship between a request / response and it's schema - that is primarily about defining the structure of objects to build structured objects off of it.

How would we get some legs on this proposal? would it help if there was a specific PR or an issue filed?

[handrews]

@toumorokoshi we're not to the point of issues or PRs on Moonwalk. See the recent blog post for the state of things. I expect this will get taken up in the new year. It's part of point 3, which is intended to encourage more elegant ways of specifying both resource-oriented and RPC-ish APIs.

[darrelmiller]

The URI hierarchy is what defines the resource hierarchy. It makes no sense to define two separate hierarchies. It is true that the client application sees a URL as an opaque string, but the API designer absolutely does not and neither should the human consuming the API. Designing URLs is part of HTTP API design.

I have written about this in more detail here https://apievolution.tavis.ca/posts/2023/the-majesty-of-trees/ and I have talked about it more here https://www.youtube.com/watch?v=9Fvt3QzMI34&list=PL3hoUDjLa7eQfYOEmuQNG8he3AeOeWaz8&index=11

@rattrayalex Why not build the docs and the client libraries off the URL hierarchy instead of inventing a new hierarchy?

[darrelmiller]

Yeah, the reality is ugly. However, my goal is to encourage the building of tools that make life easier for people who do care about the design of their API. The burden of managing two resource hierarchies should only be on the people who didn't make the effort. I'm not suggesting we should go out of our way to prevent people from "papering over" ugliness.

The argument about not wanting to change a URL is an interesting one. For Microsoft Graph we will tell people to create the new correct URL and then mark the old one as deprecated and give customers 3 years to get off it. There's no harm in having two URLs that do the same thing and only document/generate SDKs for the good one.

[handrews]

I want to highlight a few moments from this thread:

(@darrelmiller) I know the docs people are trying to do the best thing for customers but in the end it makes life so much harder for the developer who now has to do a translation in their head between the way the API should have been designed and the way it actually was implemented.

This speaks to confusion on our part as to the OAS's primary audience. Is the goal of the OAS ultimately UX, with improving DX a means to that end? Or is the OAS really about DX, and whether and how that impacts UX is a separate concern (probably the concern of tooling vendors).

For me, if there's a choice between asking the users to map their needs to weird developer choices, or asking developers to map the idealized-as-presented-to-the-user design to those same weird choices, I'll always put the burden on the developers. They have more ability to mitigate the weirdness, and are paid to deal with this sort of thing.

But better than either option would be to structure Moonwalk so that if you set it up so that your docs are idealized, your SDKs will be, too. Doc gen and code gen are tools. If one can paper over a mess, the other ought to be able to as well. How can we support this? Which leads me to...

(@darrelmiller) URLs are UX for APIs and creating a different UX for docs and SDKs isn't helpful in my opinion.

I agree with this, but feel that the problem is that OAS, as it stands, encourages (or doesn't discourage) this outcome. Right now documentation tools are taking advantage of having human users to finesse problems that are more challenging for rigid, and typically very mechanical, SDK generation. I'd argue that this is further complicated by the mismatch between JSON Schema as a technology and generative use cases (particularly code generation).

With Moonwalk, we should also consider the question: Does the emergence of AI in the SDK generation space give us more ways to align documentation and SDK generation and experience? Particularly if Moonwalk is not as locked to the current limitations of JSON Schema?

We're used to thinking of code generation (SDKs) as the most challenging and limiting factor, particularly in strongly typed languages. What if emerging technology makes this less limited? What possibilities does that open up for Moonwalk?

(@darrelmiller) If instead we created client libraries and docs that did rely on the URL structure then developers would care about the structure of the URLs they designed, then we would have better designed URL hierarchies and we wouldn't have to spend time creating these secondary hierarchies that just lead to developer confusion.

(@rattrayalex) Based on my experience, I don't think restrictions in tooling will be enough to change the state of affairs.

I'm with @rattrayalex here. In my experience, you can encourage good behavior with tools by making it easy. But you cannot discourage bad behavior by making it hard. People will find a way to keep doing awful things. If you make it harder, they'll just complain more and eventually stop using the thing. Wanting people to design and stay within "good" URL hierarchies is, to me, only slightly more realistic than wanting people to design "proper" resources and links.

I also want to separate legacy support from design approaches. We can impact API design in new systems by making certain things easy. But legacy and compatibility issues are particularly intractable. People will do messy things unless they are outright prevented. And outright preventing will keep people on 3.x or motivate them to look for alternatives.

@darrelmiller's comment about "creating these secondary hierarchies" lands differently in the two cases. In the legacy case, the need for such hierarchies are often externally imposed (as I'm aware that Darrel knows 🙂 ). We need to support that reality. But in the new design case, I agree with Darrel that we want to avoid people thinking in weird dual hierarchies.

I'm still thinking on how to separate these things and address them in a way that does each justice. There is a real, elegant case where a resource-oriented view solves design problems. It may also solve some legacy compatibility problems, but is unrelated to others.

[rattrayalex]

I can't help but engage with @darrelmiller on the finer points of API maintenance (as it is what we are all passionate about here after all) but I think it's increasingly off-topic, so collapsing.

> For Microsoft Graph we will tell people to create the new correct URL and then mark the old one as deprecated and give customers 3 years to get off it.

3 years is a long time for the maintaining team! Many smaller teams would find this kind of thing onerous internally. Yet, still puts work on the consuming team, for no fault of their own. Accordingly, many API teams have a policy to avoid this sort of rename wherever possible, even if the old path was "wrong".

There's no harm in having two URLs that do the same thing and only document/generate SDKs for the good one.

This can make for poor docs experiences; if 99% of code out there in the wild (and associated third party blog posts etc) references the old version, but the docs only show the new version, substantial user confusion can ensue. To resolve this, you need the old URL findable in ~all places where you'd also find the new URL, but clearly-yet-unobtrusively documented as legacy - which presents a nontrivial writing and design challenge for docs teams.

I am generally quite +1 to @handrews 's thoughts here, with a few notes:

1. There are (rare) times when docs and SDKs do diverge in their desired hierarchies. I can't think of examples offhand but could likely dig some up from Stripe or others if useful. This presents challenges in a resource-oriented OAS design. I do agree the vast majority of the time they should be aligned (or alignable).
2. Relatedly, there are even times when endpoints should show up in multiple places in docs and/or SDKs (e.g., for legacy reasons). I'm not sure how this would be handled in a good resource-oriented OAS design, but I would like it to be and believe it to be possible (we have things for this at Stainless).
3. I'm very curious for more of @handrews 's thoughts on the shortcomings of json-schema (I suspect he knows them better than ~anyone else) and how they might relate to AI-powered codegen. I certainly have my gripes about json-schema for SDK codegen but most of them are more ergonomic rather than capability based (and I am overall a big fan).

[handrews]

@rattrayalex first some philosophizing regarding your points 1 and 2, I think this is a bit related to my comment on @lornajane 's proposal for better tags about non-hierarchical organization. Lorna quite reasonably wants to keep that very concrete discussion aligned with known use cases, so this is probably a better space for this topic.

Whether they involve hypermedia or not, most large APIs are more like graphs than trees. This can also be true of small APIs, but they're much less of a challenge to organize. While it's hard to map a graph into more linear representations like a table of contents, with resource-oriented APIs I think there's a lot to be done with capturing more general relationships among resources. Again, this need not be done as hyperlinks in the API itself (as much as I might prefer that). It could be a more generic organizational technique.

That could help with the "multiple places" legacy stuff, but also more of a "see also" connection to help people map complex APIs to their actual use cases (workflows can also help here).

---

As far as JSON Schema codegen shortcomings, there's a fundamental mismatch between JSON Schema as a constraint system, where the empty schema allows everything, and data definition systems (including but not limited to OO) where an empty object has no fields and does nothing. OO inheritance does not map well to any JSON Schema construct (and OAS's Discriminator is valiant effort but also breaks JSON Schema in some odd ways, particularly the "allOf" form of it).

Although I suppose these could be classified as "ergonomic" issues as long as your tooling works with/around them sufficiently. But you really do need well-designed tools for that, and the design problems are not all obvious. I've seen plenty of tools that do weird things like just always treat the first entry in an allOf as a "base class" and the rest as child classes. Which... is not how JSON Schema works.

And then some code gen tools try to get keywords that are fundamentally dynamic run-time constructs ("anyOf", "if"/"then"/"else", "not") to somehow work with code gen. Which, if you make enough assumptions or include enough restrictions can sometimes work... although I'm generally more a fan of not trying to make every schema keyword work in code gen.

But all of this is based on a fundamental truth: There is no specification whatsoever for generating code (or any data declaration) from JSON Schema. There are some keywords and subsets of possible schemas that are sufficiently "obvious" in some sense that most tools will handle them as close to the same way as each programming language allows. And then it gets very, very messy. Which can be great for vendor lock-in, but awful for interoperability.

The rise of companies that generate SDKs across multiple languages helps... until someone needs an unsupported language. Or until they need to work across multiple APIs which do not use the same tools to generate SDKs, and therefore the code-level interface is inconsistent relative to the schemas visible in the documentation.

Obviously, none of this make code gen impossible, or makes it impossible ot use APIs with a variety of SDKs (or lack thereof). Code gen is a huge part of the JSON Schema / OpenAPI ecosystem.

But it's an inconsistent mess as a whole, even when individual solutions are elegant. You can work around it in a limited scale, but if you need to interact with a large number of APIs that are all "compliant" because there is no definition of compliance, well... the cost of that lack of consistency is externalized to the user. With no motivation for any one provider to align with any others. That alignment only really happens if there are standards, which is the whole point of both OpenAPI and JSON Schema. So when it comes to code generation, that's a failure of standardization, because it's really not standardized at all. It just kinda piggy-backs on validation which works well enough, enough of the time, to be... OK.

[rafalkrupinski]

Whether they involve hypermedia or not, most large APIs are more like graphs than trees

That's true at the abstract level, but when designing paths, grouping functions (operations) on the server side, displaying UI or grouping functions on the client side, one tends to work with hierarchies, as they are only way known to us of organizing code. After all you can't have files in other form than directory/file hierarchy. Whatever large set you have, you just divide it into groups, subgroups and so on.

I'd like to use url templates as the resource hierarchy, as it feels natural. It's not trivial, as they contain variables and path components may be verbs, so they'd need some form of annotations.

paths:
  /v1/shop/product/${product}/sell/{quantity}:
    path-annotations: # index denotes path component after n-th forward slash character
    - type: text # (perhaps version, as per #82)
    - type: group
    - type: resource
    - type: var
    - type: verb

Code generator or UI can group operations by group and resource (and version), and name operations using the verb

As for ill-designed paths, I could see a couple of solutions, like using $refs for paths or operations (would also aid path migration, as server side function could be attached to more than one paths)

paths:
  /some/bad/path:
    $ref: #/paths/~1resourceGroup~1resource
  /some/otherBad/path:
    get:
      $ref: #/paths/~1resourceGroup~1resource # 'get' implied

or adding an (perhaps x-) attribute for the path that should have been.

paths:
  /some/bad/path:
    x-resource-path: /resourceGroup/resource # could use a better key name, it's a fake path, only used to denote resource and group paths

[asbjornu]

I think one source of contention here might be that OpenAPI serves both client and server implementations. For server implementations, I agree that the hierarchical design of URLs is essential. The hierarchical URL captures naming, state, and dependencies; enforcing architecture and design upon the resources in an API that are hard to achieve otherwise.

But when it comes to the client side, I strongly oppose the notion that clients need to know at compile time what every URL that may exist in the API will look like. The URL is not what identifies the content served from the URL. That's the role of the headers served with the URL, most importantly Content-Type. The client absolutely needs to know at compile-time which resource representations it will be served by an API, but the URL from which the resource representation is served shouldn't matter. Requiring that it does, is requiring an RPC design on every instance of an API described by OpenAPI.

I'm fine with paths being the only organizing principle of OpenAPI as of now, but I find it puzzling that the idea of moving towards a future where alternatives to paths exist, is met with so much resistance.

My concern is introducing a new concept that has no clear definition. How is "resource Type" different than something in #/components/schemas ?

To me, a resource is a combination of both data (schema) and behavior (operations). The definition of a resource may well be equal to the definition of a schema in many cases, but it exists at a higher level of abstraction, just like how close-account is a higher-level abstraction over DELETE /account/{account_id}.

I attempt to explain this and other resource-oriented design principles in my 2016 talk, The REST And Then Some.

[peteraritchie]

But when it comes to the client side, I strongly oppose the notion that clients need to know at compile time what every URL that may exist in the API will look like. The URL is not what identifies the content served from the URL. That's the role of the headers served with the URL, most importantly Content-Type. The client absolutely needs to know at compile-time which resource representations it will be served by an API, but the URL from which the resource representation is served shouldn't matter. Requiring that it does, is requiring an RPC design on every instance of an API described by OpenAPI.

Each endpoint also serves multiple content-types. Most notably, 200 content and 400 content. e.g., application/json for 200, and application/problem+json for 400. i.e., each endpoint needs to serve multiple resources.

[evert]

When we talk about resources/ resource types it's usually the full set of operations, requests and responses.

[peteraritchie]

I think "resource type" and "media type"/"content type" may be conflated (and maybe we're aligned @evert). An endpoint (and thus a URI) can define a resource, but that endpoint may (will?) respond with multiple types of content and accept multiple types of content (depending on command). In the 200 response case there's may be a conceptual one-to-one relationship to a "resource". In the non-200 cases (3xx, 4xx, 5xx) it could be viewed that there really isn't a relationship of a response to a "resource". e.g., a /api/loan-application URI may represent the concept of a loan application resource and the schema of a loan application may be represented by a media type like application/vnd.finance.loan.application+json but accessing URI /api/loan-application will result in responses of type application/vnd.finance.loan.application+json, application/problem+json, etc. at different times and in different circumstances. If an "endpoint" was a resource then resources would have multiple, disparate, schemas.

[handrews]

Past resource-oriented API description case study

I've often mentioned the generic hypermedia client I wrote over a decade ago, which is still in use. For some reason, it never occurred to me to talk about the ReSchema Service Definition Format that drove it until a recent discussion with @rattrayalex

ReSchema started as a documentation-only system, probably at almost exactly the same time Swagger started as an internal tool. Swagger 1.2 was published about the time we put ReSchema and sleepwalker into production.

The CTO office director who came up with it was skeptical that such a thing could drive runtime behavior, or that a generic hypermedia-ish client was possible. I took this as a challenge and proved it could be done, which led to us updating the format to the structure I'll describe here.

If you read the ReSchema docs that I'll link, do keep in mind that it started out as one thing and became another. Also keep in mind that it was originally internal, and then it got published as things were starting to go downhill so it was never proofread or edited from its original form. :-/

I'm not going to discuss the generic client there, but it worked - we did not use code generation, and we did not use higher-level SDKs internally. The generic client was more robust.

I'll go through some notable points about the format, but if you don't care about that, skip down to "Thoughts for Moonwalk."

Terminology

This comment uses standard / OAS terminology rather than ReSchema's terminology, which I'll explain at the very end when linking to its docs.

I'm using the terms link, relation type, context, and target as they are defined in RFC 8288 §2.

ReSchema's Terminology

Note that ReSchema documentation calls the "self" link plus all operations links, and calls all other links relations. It does not use the term "operations" at all.

The links field was really about operations, with the "self" link defining the default operation target. There were standard operations for CRUD (more or less), plus custom operations for POST and special CRUD cases. The relations were closer to proper links._

I'll stick with the standard terminology for the rest of this comment, but be aware of the difference if you read the ReSchema docs yourself.

Resources

Each resource is organized around:

• A representation schema
• A "self" link with a URI Template
• optional additional parameter/input information
• optional links to related resources
• operations, which by default operate on the "self" link, but can operate on a separate URI Template

This assumes a client that maintains a context resource URL, like the browser's address bar, but without any implied HTTP operation. Your current URL determines what operations and links are availabe.

A session started by "bind"-ing to a resource, possibly by filling out a URI Template with information from elsewhere. Or starting on a non-templated resource. Binding did not trigger an HTTP request.

Links

Like OAS Link Objects, ReSchema links specified how to fill out the target resource's URI Template based on the context resource's representation data and/or input.

Unlike OAS Link Objects, there was no associated HTTP request. They identified a target resource instead of a target operation, and following a link just changed your current resource context.

Link relation types

Links are assumed to have a relation type, most notably:

• "self" (as registered in RFC 5988) was a special case
• "full" and "instances" which had been proposed in draft-04 of JSON Hyper-Schema were used with collections

We should have used RFC 6573's "collections" and "items" instead of "instances" and "full", but we'd started by trying to use hyper-schema and those relation names stuck.

Links from individual fields

Links could be attached at the schema root, using the whole resource as the context, or they could be attached to individual fields.

For example, each item in a collection representation had its own "full" link that took you to the individual resource for that specific item. Each resource that could be an item in a collection had an "instances" link pointing to the collection.

A "full" link could be attached to any resource id field, e.g. an "author_id" field in the representation of a book. This use of "full" really could have been done with "self", but we did not understand that at the time (this is why there is no IANA-registered "full" link relation type).

Operations

ReSchema required that HTTP methods, response codes, headers, etc. be used in accordance with the relevant RFCs. POST, of course, is allowed to do anything, so it had additional support.

GET, PUT, and DELETE

Standard CRUD

The operation names "get", "set", and "delete" were special, and indicated standard GET, PUT, and DELETE usage, respectively.

If we'd revised the format, we would have added shorthand for these because they were always specified exactly the same way. There was no need to explicitly associate the representation schema with requests and responses, as it was predictable based on the HTTP method and (were relevant) the Content-Location header).

Special cases

Additional named operations using GET, PUT, or DELETE could be added as special cases. For example, a named "get" that applies a specific use case's filter to a collection. Updating the ReSchema file would then update the filter without any need to change the code.

It was also common for collections to have a "clear" operation that used PUT to empty the collection without deleting it.

PATCH

The reason "set" was a special operation name and "put" was not is that we had the usual debate about whether to use PUT or PATCH. Or both. Or maybe even POST.

We found that most devs thought of PATCH because they thought about changes in isolation: "I want to set field X to Y", so they just want to send that. But for actual use cases, they always already had, or needed to get, a current representation. It would have been more work to construct a patch than to just modify the representation and PUT it back.

We told devs we'd implement PATCH support if it was ever needed, and it never was. If we had implemented it, we would have required a patch media type for the request and used that media type's rules plus the representation schema to determine the valid patch syntax.

POST

POST operations could use the "self" link like the others, but were also the only operations that could specify their own URI Template. This was done when the POST target was only relevant from a specific context.

A POST operation of this sort was shorthand for declaring the POST target as a separate resource, linking to it in both directions, following the link, executing the POST, and following the reverse link back. Because of this, it had all of the data-to-URI Template mapping capabilities of links that did not perform an operation, plus the operation capabilities.

Creating via collection

The "create" operation was a special case that used POST from a collection context with a request body using the representation schema of the related item resource. If the response was a 201 with a Content-Location header matching the item's URI, then the response body was also expected to match the item representation schema.

As with the other special cases, this would have gotten a shorthand form if we'd revised the format.

Arbitrary verbs

Custom named POST operations were also used for anything that didn't fit the other methods. They could use arbitrary request / response schemas that did not need to match any regular resource.

Errors

Errors used the application/problem+json media type from the draft what what later became RFC 7807 (now updated as RFC 9457).

Error types were defined globally for the resource, and were defined flexibly enough that the usage for specific operations should be clear. I don't recall anyone running into any problems with this or asking for per-operation error response schemas.

Schema Re-Use

ReSchema had a types field for re-usable schemas, but only used it for schemas that were not full resource representations. Each resource's representation schema was defined inline.

Single representation schemas for resources

We could get away with a single schema per resource representation because we defined conventions around GET/PUT/POST-to-create variations. If we were to use a concept of a single resource representation in Moonwalk, we'd need to work through more of the details here- there are several discussions on this topic already IIRC.

Merges

ReSchema also supported $merge, which is why I have always been so vehemently against it. It made a lot of things more complex to implement and harder to think about, especially because you ended up with this two-layer lazy evaluation in the interaction of $ref and $merge.

My dislike for merging as anything other than a preprocessing step (e.g. Overlays, or just applying a patch of whatever sort) has been reinforced by having to implement OAS 3.1's keyword overrides for Reference Objects and the Path Item Object's fuzzy rules around adjacent keywords for oascomply.

---

Thoughts for Moonwalk

Things that I think are worth keeping in mind:

• Organizing schemas, links, and operations around the concept of a resource makes it a lot easier to see coherent behavior in the API
  • Some of the most common uses of tags would no longer be needed
  • It's much easier to eliminate boiler-plate when defining these things together, e.g. because the associated schema can be a default schema for specific requests/responses
  • Having shorthand syntax for RFC-compliant patterns encourages them without making other usage any more difficult
• This organization is in many ways more flexible than the OAS
  • You can add as many named operations per method as you like
  • Multiple POST operations were expected and well-supported
  • A named operation can be a semantically important special-case of a generic CRUD operation
  • While it's technically "bad HTTP usage", you could still specify arbitrary usages of any HTTP method, not just POST
• Separating links from operations by default seems much easier to reason about to me
  • OAS Link Objects conflate them, so you need to think about every possible related operations
  • ReSchema let resource A tell you how resource B is related (and therefore why you want to use resource B) but let resource B own how to use it
  • This separation was not mandatory - POST operations with their own URI Template addressed cases where you really do want a specific operation only (and other HTTP methods could be given that flexibility)
  • Links without operations (that use IANA-registered link relation types) make patterns like collections and items much more visible
• Global errors are at least as important as per-operation errors
  • It seemed to encourage more thoughtful error design
  • However, this is anecdotal and coming from fuzzy decade+ old memories
• ReSchema was purely a "shapes" system, with deployment information separate
  • This is because it was designed for management APIs on network appliances
  • Every box in the data center had an instance of the API
  • Boxes on different software versions could have different API versions
  • Named resources and operations made dynamically-typed clients a lot more robust across minor variations

There's something here relating to signatures that I can't quite put my finger on yet. I'll have to keep thinking on it.

But really the biggest surprise for me going back through the ReSchema docs after so many years is that it fits the "Big Tent" principle quite well. It's opinionated, and in some cases restrictive (the error media type, for example). But the restrictions did not come from the resource-ness of it. As I noted, I'm pretty sure RPC is actually easier in ReSchema than OAS. I was not expecting that.

[ihodgesibm]

PATCH support

When it comes to API description methods these are well understood CRUD methods until we get to PATCH where we require the ability to partially update a resource but how we do this has not been defined in any manner in OpenApi.

From a Java Application Server point of view PATCH has been described and supported in Java EE/Jakarta EE via the JsonPatch object in javax.json and Jakarta.json packages. JsonPatch is an RFC6902 implementation where JSON entities are changed via a directing JSON message containing operation, path, and value.

However, when we get to API description via OpenApi we do not want to describe PATCH request bodies in terms of RFC6902. A better solution would be to use the same schema definition entry for POST/PUT/PATCH, so the complete data element is understood in terms of the operation, however, when the media type is defined as “application/json-patch+json” (From RFC6902) then the runtime message format utilises RFC6902 message type formats. However, there must be caveats.

When describing an API allowing new fields to be added to an object should be discouraged unless additionalPropties:true is defined in schema. Removing a field that is defined as required should similarly be discouraged.

From an OpenApi specification point of view defining what PATCH is and how it should operate would make for better and more conformant API’s.

I would appreciate hearing your thoughts in this one.

Thanks and regards,
Ian Hodges.

[handrews]

I feel like the HTTP API would be a better place if more people knew about and understood how to use Accept-Patch (@rattrayalex amusing to see this come up after our discussion where I cited Accept-Patch as a standard that should be wildely known but is, in practice, almost completely unknown).

When I was working on JSON Hyper-Schema, which was unapologetically opinionated, we did not even have a specific schema for PATCH. It was assumed that the resource schema for GET/PUT) plus the PATCH media type (which JSON Hyper-Schema could advertise in its model of expected headers) told you enough to validate the request body. Either directly, or by applying it and validating the result. I suspect that never would have been sufficient in practice given the way people use PATCH (without first GET-ing the resource), but Hyper-Schema never got far enough to test that theory.

Actual Proposal for Accept-Patch and Accept-Post

One thing I think we can and should do is make Accept-Patch more prominent. And Accept-Post, although that comes from the Linked Data Platform spec and has never been as widely supported (an I-D from @dret expired in 2015).

These headers are fundamental representation data, and clients should not have to poke through (for example) the HEAD request's response headers to figure out that they are there.

[rafalkrupinski]

@handrews Do I get it right? Do you want to make accept-patch and accept-post more prominent in OpenAPI? Doesn't it duplicate the content dictionary in requestBody, where you already advertises accepted media types?

Exposing PATCH method that accepts a partial object results in the entire schema being duplicated, once for normal operations, with required, second time for PATCH, without it. I find that ...inelegant.
Interesting that JSON Schema has readOnly and writeOnly, which are specific for communication mechanisms, but doesn't have anything for updates.

[handrews]

@rafalkrupinski

Doesn't it duplicate the content dictionary in requestBody, where you already advertises accepted media types?

No, you're making a lot of assumptions about what "more prominent" means. There's a lot of work to be done in the area of headers, particularly in setting headers shared across many / most resources. A way to say "in this API, resources that can be PATCHed accept this patch media type(s)" would make it more clear that a.) this is a thing you can do, and b.) it's part of API design, not individual request design.

Exposing PATCH method that accepts a partial object results in the entire schema being duplicated, once for normal operations, with required, second time for PATCH, without it. I find that ...inelegant.

You're making a lot of assumptions here as well. JSON Merge-Patch is somewhat like that, but not entirely, while JSON Patch looks completely different. There are also JSON Schema design techniques that make it a lot easier to compute a PATCH schema from a basic schema. Also, I've been regularly advocating for using something other than JSON Schema for data modeling, pretty much in every Moonwalk call. It's not really a data modeling system.

See:

• Context property annotations #56

for an in-depth discussion of keywords like readOnly and writeOnly and additional possibilities.

[rafalkrupinski]

There's a lot of work to be done in the area of headers, particularly in setting headers shared across many / most resources. A way to say "in this API, resources that can be PATCHed accept this patch media type(s)" would make it more clear that a.) this is a thing you can do, and b.) it's part of API design, not individual request design.

Sure, but I still don't understand the role of accept-patch or accept-post header in the context or OpenAPI.

Also, I've been regularly advocating for using something other than JSON Schema for data modeling, pretty much in every Moonwalk call. It's not really a data modeling system.

God bless you, sir.

[ihodgesibm]

When it comes to most of the methods the API’s data schema, i.e. the set of data elements the API understands, and the wire format are very similar, on the wire we ship a bucket of bits that contains the complete API schema data. However, when we get to PATCH we have a patch document that is not described in API terms.

If we used a media type application/patch+json and RFC6902 then what the API describes is a generic mechanism to update a JSON entity, it does not describe the API and the APIs data model at all.

That is where I think the problem lies, when it comes to PATCH we have two sets of data, the API’s data schema and the wire format of the PATCH document. Two very different things.

The introduction to the OpenAPI specification (3.0.3) states:

The OpenAPI Specification (OAS) defines a standard, language-agnostic interface to RESTful APIs which allows both humans and computers to discover and understand the capabilities of the service without access to source code, documentation, or through network traffic inspection. When properly defined, a consumer can understand and interact with the remote service with a minimal amount of implementation logic.

An OpenAPI definition can then be used by documentation generation tools to display the API, code generation tools to generate servers and clients in various programming languages, testing tools, and many other use cases.

When it comes to PATCH I think we fall short against this declaration of intent because we do not describe the API’s data model only the format of a wire PATCH document.

From a tooling implementors point of view if an OpenAPI document describes a PATCH operation in terms of a PATCH document like RFC6902 then there is a generic description of a wire format that is the same for every single API using that media type. This does not allow any tooling the ability to show the capabilities of a service in the terms of the data model the API understands.

From an API provider and client point of view we have the same problem a description of a PATCH document format and no description of the data model the API understands, this must be inferred from the other API method descriptions.

This is something we could address in Moonwalk to give clarity around what is the PATCH API schema data model and what is the PATCH document wire format. With those two things the PATCH API operation can be understood by tooling, by provider, and client.

This is where I would like to go.

[handrews]

See also the following issues that were filed in the OAI/OpenAPI-Specification repo but are closed as they are better discussed here. Please feel free to copy over comments that are particularly relevant:

• Relationship properties OpenAPI-Specification#601
• Link to operation mapping OpenAPI-Specification#1315
• Proposal for incorporating state transition into Open API OpenAPI-Specification#1329
• Supporting typed links OpenAPI-Specification#2069
• openapi link vs hypermedia link to represent relationships in APIs OpenAPI-Specification#2122

[serpro69]

This one also might be relevant OAI/OpenAPI-Specification#577

[kephas]

They have been discuessed here, but there is no notion of URI templates in Roy Fielding's thesis defining REST.

In a API using REpresentational State Transfer (REST), you have representattions of resources containing links for each action. A link in REST works a lot like a path in OAS, with the only real difference being that instead of building the URL from a template, the client gets the URL from a link in a resource representation. In a REST API, you still have at least one path, the entry point of the API (you could have several entry points).

OAS could support describing REST APIs with an added links section, here is a short example:

info:
  title: Sample Banking REST API
paths:
  /:
    get:
      summary: API entry point
      security: BearerAuth: []
      responses:
        '200':
          description: A users's representation
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/User'
components:
  schemas:
    User:
      type: object
      properties:
         accounts:
           type: array
           items:
             type: Link
             linkTarget: Account
links:
  Account:
    get:
      responses:
        '200':
          content:
            application/json:
              schema:
                type: object
                properties:
                  balance:
                    type: number
                  transfer:
                    type: Link
                    linkTarget: Transfer
                  close:
                    type: Link
                    linkTarget: AccountClosing
  AccountClosing:
    post:
      requestBody:
        required: false
        content:
          application/json:
            schema:
              type: object
              properties:
                transferBalanceToIBAN:
                  type: string
      responses:
        '200':
          content:
            application/json:
              schema:
                type: object
                properties:
                  success:
                    type: boolean
        '409':
          content:
            application/json:
              schema:
                type: object
                properties:
                  error:
                    type: string
                  explanation:
                    type: string
  Transfer:
    post:
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              properties:
                amount:
                  type: number
                transferToIBAN:
                  type: string
      responses:
        '200':
          content:
            application/json:
              schema:
                type: object
                properties:
                  success:
                    type: boolean
        '409':
          content:
            application/json:
              schema:
                type: object
                properties:
                  error:
                    type: string
                  explanation:
                    type: string

[rafalkrupinski]

Why would I build my API with REST? (the real one, with hypermedia)

Yeah, why would anyone, right?

IDK, my programs don't behave like a human using a browser, they use Web API as ...API, i.e. they expect contracts, not a D&D world to explore

[kephas]

@rafalkrupinski have you read the article‽ That objection is specifically addressed in detail.

[evert]

@kephas i would strongly suggest just ignoring the bad faith, unconstructive responses! It's a fools errand

[kephas]

I was working with the assumption that this frequent objection was at least 40% in good faith…

[rafalkrupinski]

not bad faith, just lazy. sorry