Migrate ALL the datastreams

[dannylamb]

We should pull over all the datastreams (except RELS-EXT and RELS-INT? 🤷‍♂️ ) to make sure we get it all. That way users will have copies of their audit datastreams and whatever other custom stuff they might have going.

This introduces some complexities with processing the foxml to make sure you can line things up by PID. Current thoughts are to attempt to use Tuque to load the object and then extract info that way, though some exploration definitely needs to be done.

[ajs6f]

There are plenty of folks who use RELS-* to record descriptive or other kinds of metadata... we need a path forward for that. Does it become properties on the Drupal entity / triples in the linked data backend?

[dannylamb]

@ajs6f Yes, that's why I was suggesting we don't pull them over, because in theory all of that gets applied to the Node as fields.

[ajs6f]

Ah, cool, thanks! 👍

[mjordan]

Related issue: #854.

[dannylamb]

Revisiting this now that I've got everything set up correctly, it occurs to me we've considered various ways of pulling over everything dynamically, but no one has spoken for just a .yml file per datastream? It'd get unwieldy eventually, but using what you have right now as a template we could whip them up pretty darn quick. For a basic image in 7.x I've got

• RELS-EXT
• MODS
• DC
• OBJ
• TECHMD
• TN
• MEDIUM_SIZE

Obviously there's more than just that, and I'll have to do a count of all the types of datastreams in types that both softwares support to get a sense of the full depth of work. But I don't think it would be too terrible in the end and it's a pretty easy pattern to replicate if someone has a custom DS they want to move over.

I'm sure this has already been considered by @whikloj and @seth-shaw-unlv. Am I being naive?

[mjordan]

Sorry to ask after this issue and PR has been closed, but it looks like https://github.com/Islandora-Devops/migrate_7x_claw/pull/6/files#diff-e35e3b6c3d7dd65ff995bb5e4231abbfR209 specifically excludes the AUDIT datastream. Is it brought over somewhere else?

[whikloj]

No, AUDIT is a special case in that it is not returned by the Fedora API's listDatastreams method because it is part of the FoXML file and no where else. We could quite easily bring it over into a text field on the object with just some changes to the existing migrations.

[mjordan]

Reason I'm asking is that I'd like to parse out children of AUDIT as part of #847 (specifically, how you'd migrate existing fixity event data). Having it as a text field would be fine for that, but is there any reason we can't add it as a binary resource?