Round tripping nodes

[mjordan]

As an experiment I retrieved a node using curl -v -o node.json "http://admin:islandora@localhost:8000/node/399?_format=json" and then tried to POST it back using the contents of node.json as my payload. Drupal responded with a 403 indicating {"message":"Access denied on creating field \u0027nid\u0027."}. I then removed the nid field from my JSON and tried again, and was successively told the same thing for the following fields:

• revision_timestamp
• revision_uuid
• changed

After removing those fields from my JSON, I was able to create the node using curl -i -X POST -H"Content-type: application/json" --data @node.json 'http://admin:islandora@localhost:8000/node?_format=json'. Here is the JSON:

{"uuid":[{"value":"ec9d9261-bde6-420c-8a52-7a3159d565bd"}],"vid":[{"value":421}],"langcode":[{"value":"en"}],"type":[{"target_id":"islandora_object","target_type":"node_type","target_uuid":"16176925-cb33-4132-b799-6a7742249cf5"}],"revision_log":[],"status":[{"value":true}],"uid":[{"target_id":1,"target_type":"user","target_uuid":"23a91b94-33ba-4f5d-af50-9b43b1356fee","url":"\/user\/1"}],"title":[{"value":"Looking across Burrard Inlet"}],"created":[{"value":"2019-07-16T03:03:04+00:00","format":"Y-m-d\\TH:i:sP"}],"promote":[{"value":true}],"sticky":[{"value":false}],"default_langcode":[{"value":true}],"revision_translation_affected":[{"value":true}],"content_translation_source":[{"value":"und"}],"content_translation_outdated":[{"value":false}],"field_3_valued_typed_relation":[],"field_access_terms":[],"field_alternative_title":[],"field_description":[{"value":"View from Deep Cove to Burnaby Mountain. Simon Fraser University is visible on the top of the mountain in the distance."}],"field_display_hints":[],"field_edtf_date":[],"field_edtf_date_created":[],"field_edtf_date_issued":[],"field_extent":[{"value":"1 item"}],"field_identifier":[],"field_linked_agent":[],"field_member_of":[],"field_model":[{"target_id":24,"target_type":"taxonomy_term","target_uuid":"f7be905e-4ecd-42fb-9108-c52922319a49","url":"\/taxonomy\/term\/24"}],"field_pid":[],"field_resource_type":[],"field_rights":[],"field_single_typed_relation":[],"field_subject":[]}

And here is the JSON representation of the newly created node:

{"nid":[{"value":402}],"uuid":[{"value":"ec9d9261-bde6-420c-8a52-7a3159d565bd"}],"vid":[{"value":421}],"langcode":[{"value":"en"}],"type":[{"target_id":"islandora_object","target_type":"node_type","target_uuid":"16176925-cb33-4132-b799-6a7742249cf5"}],"revision_timestamp":[{"value":"2019-07-17T14:38:01+00:00","format":"Y-m-d\\TH:i:sP"}],"revision_uid":[{"target_id":1,"target_type":"user","target_uuid":"23a91b94-33ba-4f5d-af50-9b43b1356fee","url":"\/user\/1"}],"revision_log":[],"status":[{"value":true}],"uid":[{"target_id":1,"target_type":"user","target_uuid":"23a91b94-33ba-4f5d-af50-9b43b1356fee","url":"\/user\/1"}],"title":[{"value":"Looking across Burrard Inlet"}],"created":[{"value":"2019-07-16T03:03:04+00:00","format":"Y-m-d\\TH:i:sP"}],"changed":[{"value":"2019-07-17T14:38:01+00:00","format":"Y-m-d\\TH:i:sP"}],"promote":[{"value":true}],"sticky":[{"value":false}],"default_langcode":[{"value":true}],"revision_translation_affected":[{"value":true}],"content_translation_source":[{"value":"und"}],"content_translation_outdated":[{"value":false}],"field_3_valued_typed_relation":[],"field_access_terms":[],"field_alternative_title":[],"field_description":[{"value":"View from Deep Cove to Burnaby Mountain. Simon Fraser University is visible on the top of the mountain in the distance."}],"field_display_hints":[],"field_edtf_date":[],"field_edtf_date_created":[],"field_edtf_date_issued":[],"field_extent":[{"value":"1 item"}],"field_identifier":[],"field_linked_agent":[],"field_member_of":[],"field_model":[{"target_id":24,"target_type":"taxonomy_term","target_uuid":"f7be905e-4ecd-42fb-9108-c52922319a49","url":"\/taxonomy\/term\/24"}],"field_pid":[],"field_resource_type":[],"field_rights":[],"field_single_typed_relation":[],"field_subject":[]}

If viewed as a node in a web browser, it is identical to the original. (I did not test media, see below.)

So it appears that the immutable fields (nid, revision_timestamp, revision_uuid, changed) can't be round tripped, but the others (including uuid) can. Some observations on this result:

• node IDs cannot be reused (makes sense since they are autoincrement database IDs) but node UUIDs can. (note: we may want to always establish relationships between entities using their UUIDs rather than IDs)
• revision data isn't reusable
• I performed this experiment using a single Drupal instance, whose configuration was identical at the time of export and import. It would be interesting to reproduce an identically configured Drupal and see if importing a node from one instance to the other worked as well as it did in my experiment.
• not sure how media work, but that would be the next experiment I will try.

If it is possible to round trip content reliably, it would be possible to recreate a repository from exported data (in Bags, for example, or from some other source) or to transfer content between Drupals as long as the field structure, etc. was compatible. I'm not sure how this works in relation to Fedora, but I did confirm that the new node was not indexed in Fedora.

[DiegoPino]

Yes, we do that at archipelago. See discussion here. https://groups.google.com/forum/#!topic/archipelago-commons/tamyrbqsmnk

[dannylamb]

Is that code available on Github @DiegoPino? I took a cursory look and couldn't find anything. Any insight you could provide would be helpful if @mjordan pursues this feature.

@mjordan Would it help to try this out with jsonapi instead of the regular json format? It works off of UUID so things might be a little easier when POSTing the back in? Total speculation, but might be worth trying.

[DiegoPino]

We are close to a beta release. Will make sure to share our code publicly here once that happens so @mjordan can give it a look. jsonapi is the lowest hanging fruit but it requires a multi step ingest, first upload files or media? then ingest the the json. I had some mixed issues with having normal rest and jsonapi at the same type when using the most basic authentication but it can be just us so we only use jsonapi too seed our instances. But also, we are doing things differently, so not sure how much it matches your use case. We keep the entity id (numeric) on the deposited object in filesystem as json, but strip that out on reingest and do some pre and post processing to make sure all dependencies, referenced entities match (or are even there), not really the vanilla D8 approach i guess Mark is searching for.

[mjordan]

@DiegoPino we discussed this capability at today's Islandora 8 tech call and the consensus was that preserving a less Drupal-version-specific representation of nodes and media (which would then be converted into whatever the specific target representation of the current day is) is worth investigating before we spend any more effort on development. That approach would be more complicated than sluping in round trippable JSON nodes but also more applicable in the indeterminate preservation future. TLDR is we're not sure what we'll do with this at the moment. Thanks so much for your comments, it's great to see you've dug fairly deep into this.

[mjordan]

@dannylamb can we close this in light of #1528 ?