Allow for partial updates to a study

[jimallman]

[MOVED here from OpenTreeOfLife/treenexus/issues]

Given the size of existing (and likely) study data, I'd love to see a mechanism for partial updates. Here are some thoughts on how we might handle updates to, eg, a single tree, or just study metadata, or new annotations.

A common REST practice is to expose child resources at their own URLs. So we might fetch study 23 like so:
GET http://api.opentreeoflife.org/1/study/23.json
... while just one of its trees (also contained in the response to above) has its own URL:
GET http://api.opentreeoflife.org/1/study/23/tree/987.json
We could also use this URL to PUT changes to just this tree.

Of course, lots of other people have encountered the problem of needing partial updates to large, composite resources. Another approach that seems to be gaining favor is to patch the target resource, either by sending a partial document (with rules for resolution / deletion / etc) or using a more robust patch format like this:
http://tools.ietf.org/html/draft-ietf-appsawg-json-patch-08

Here are some other interesting posts on partial updates in REST APIs:
http://www.mnot.net/blog/2012/09/05/patch
http://tools.ietf.org/html/draft-dusseault-http-patch-16
http://stackoverflow.com/questions/8132445/how-to-support-partial-updates-patch-in-rest
http://wisdomofganesh.blogspot.com/2012/08/best-practice-for-use-of-patch-in.html
http://stackoverflow.com/questions/232041/how-to-submit-restful-partial-updates

[jimallman]

[ORIGINAL author of this comment was @leto]

The URL structure you describe seems very reasonable.

What response/error codes should we use when tree N doesn't exist?

If somebody attempts to write to a non-existent tree M, does it get created or throw an error?

What other child resources, other than trees, would users of the API want access to?

[jimallman]

https://github.com/stefankoegl/python-json-patch seems to be the recommended way to do JSON PATCHing in python, per jsonpatch.com

[jimallman]

[ORIGINAL author was @leto]

What response/error codes should we use when tree N doesn't exist?

For a GET (view) operation, return a 404. For a PUT operation, hmm.. maybe 404 again (see below). The IETF proposal for the PATCH verb has some thoughts on sensible error codes:
http://tools.ietf.org/html/draft-dusseault-http-patch-16

If somebody attempts to write to a non-existent tree M, does it get created or throw an error?

I'm inclined to play it safe and throw a 404 error. If our trees use system-wide, incrementing IDs, it doesn't make sense to me that the client could propose a new ID. Perhaps a magic value like NEW?
PUT http://api.opentreeoflife.org/1/study/23/tree/NEW

What other child resources, other than trees, would users of the API want access to?

I suspect we'll discover useful sub-paths (and child resources) as we go. My initial guesses are general study metadata, single-tree metadata, annotations attached at various locations:
PUT http://api.opentreeoflife.org/1/study/23/about
PUT http://api.opentreeoflife.org/1/study/23/tree/5/about
PUT http://api.opentreeoflife.org/1/study/23/tree/5/note/3
PUT http://api.opentreeoflife.org/1/study/23/tree/5/note/NEW
PUT http://api.opentreeoflife.org/1/study/23/tree/5/node/789/note/3
PUT http://api.opentreeoflife.org/1/study/23/tree/5/node/789/note/NEW

[jimallman]

Re: jsonpath.com: This looks pretty good! I'm happy to see that this will support patching either as an ordered series of operations or as a calculated diff of "before" & "after" JSON documents. I'm not sure which would be easier to generate on the client side, which would be easier to validate, etc.

It's probably easiest to just return a modified JSON doc and let the server generate the patch. But I fear we might lose some sense of "history" (the chain of editing decisions) versus building a sequence of operations.

[jimallman]

re: proposed URLs with magic 'NEW' identifier: Scratch that idea. The POST verb should be sufficient to create a new resource in the designated collection or location:
POST http://api.opentreeoflife.org/1/study/23/tree
POST http://api.opentreeoflife.org/1/study/23/tree/5/note
POST http://api.opentreeoflife.org/1/study/23/tree/5/node/789/note/

The server should return (or redirect to?) the URL of the new resource.

As an aside, we should settle on a convention for singular vs. plural names for collections like study and note above, as well as to keep or discard the terminal slash in URLs like those above, redirecting via 301 to keep things consistent.

[jar398]

On Sep 28, 2013, at 1:54 PM, Jim Allman wrote:

What response/error codes should we use when tree N doesn't exist?

For a GET (view) operation, return a 404. For a PUT operation, hmm.. maybe 404 again (see below).

A failing PUT, when the resource might have been created or written, should yield a 403 (see RFC 2616). 404 would never be right for a PUT. 404 is fine for a POST.

The IETF proposal for the PATCH verb has some thoughts on sensible error codes:
http://tools.ietf.org/html/draft-dusseault-http-patch-16

If somebody attempts to write to a non-existent tree M, does it get created or throw an error?

I'm inclined to play it safe and throw a 404 error. If our trees use system-wide, incrementing IDs, it doesn't make sense to me that the client could propose a new ID. Perhaps a magic value like NEW?
PUT http://api.opentreeoflife.org/1/study/23/tree/NEW

hmm.

What other child resources, other than trees, would users of the API want access to?

I suspect we'll discover useful sub-paths (and child resources) as we go. My initial guesses are general study metadata, single-tree metadata, annotations attached at various locations:
PUT http://api.opentreeoflife.org/1/study/23/about
PUT http://api.opentreeoflife.org/1/study/23/tree/5/about
PUT http://api.opentreeoflife.org/1/study/23/tree/5/note/3
PUT http://api.opentreeoflife.org/1/study/23/tree/5/note/NEW
PUT http://api.opentreeoflife.org/1/study/23/tree/5/node/789/note/3
PUT http://api.opentreeoflife.org/1/study/23/tree/5/node/789/note/NEW

—
Reply to this email directly or view it on GitHub.

[jimallman]

PUT http://api.opentreeoflife.org/1/study/23/tree/NEW

hmm.

Ugh, that moment of madness was corrected in my later comments.