Have the option of not storing master binary in Drupal

[mjordan]

Title (Goal)	Have the option of not storing master binary in Drupal
Primary Actor	Repository admin
Scope	Architecture
Level	High
Story	As a repository admin, I want to be able to configure my Islandora repo to not store a copy of the master (formerly known as OBJ) binary in Drupal. The binary would be stored only in Fedora. Two motivations for this: 1) to avoid doubling storage requirements and 2) to avoid doubling management requirements, e.g. validating checksums.

[jonathangreen]

+1 for me on this as well. this is how I envision our repositories working at LYRASIS.

I also would like derivatives not to live in Fedora at all, but perhaps that should be another ticket.

[DiegoPino]

+1

[bryjbrown]

+1, derivs in Drupal and masters in Fedora seems like a thing people are gonna want.

[ajs6f]

+1 to this backwards. WHAAAAAA?!

I want to be able to store masters only in Drupal. Not joking. I want an LDP backend for several reasons, but preservation isn't one of them, so putting masters in LDP doesn't help me at all.

[mjordan]

@ajs6f I was seeing this as configurable ("the option to"), although not necessarily something you'd want to toggle every few days.

[ajs6f]

@mjordan Right on, I'm just adding that we want a three-way switch; masters to one side, masters to the other side, and (default) masters to both.

[dannylamb]

👍, but is there a need for storing both, really? I'm fine with just a regular switch between Drupal and Fedora.

[ajs6f]

I'm fine with whatever meets my use case, which is just the one way! 😁 As long as I don't have to push any more into Fedora than I actually want to, all good.

[dannylamb]

https://www.drupal.org/project/flysystem seems to support multiple filesystems. Adapters are written as plugins and in theory there'd be a way to denote the destination when uploading a file. I'd have to do a deep dive to confirm all that, though.

[ajs6f]

Would that mean synchronous operation?

[dannylamb]

It depends. For your use case, everything can still be asynchronous. Ingest into Drupal -> Generate derivatives -> Derivatives go in Fedora.

For the use case of putting preservation masters in Fedora but everything else in Drupal, staying asynch will incur double storage temporarily. That is, Ingest in Drupal -> Flush to Fedora -> Delete from Drupal. Going synchronous just on certain uploads would prevent that if it doesn't unravel the design of the entire application 😢

[ajs6f]

My use case actually doesn't include derivatives -> backend. It's just metadata -> backend, masters and derivatives stored elsewhere (not in Drupal). But it may be (it's not at all unlikely) that our set up is "weird" (= unlike most other sites) and if so, we can deprioritize this use case (meaning my particular flavor of this use case).

[dannylamb]

Weird use cases are ok. Maybe you wanna make one yourself / your organization? It wouldn't hurt.

I suspect you could get by with what we have now if you don't plan on generating derivatives through Islandora.

Back on point: it'll take some thought / exploration to figure out the least insane approach. Looks like we're butting up on some Drupal opinions like "there's only one file system".

[ajs6f]

@dannylamb I made #771, but I can't label it (or figure out how to use the use case template, which is probably just me not understanding Github).

[whikloj]

@ajs6f I added the label, the permissions in Github are weird. We're still missing something I think.

[dannylamb]

@ajs6f @whikloj So... finally found this. Looks like you need write permissions to add a label or move an issue around... I'd be fine with @ajs6f having those permissions, but I can't say I feel that way about everybody.

[ajs6f]

Nice, Github. :slow-clap:

[dannylamb]

@Islandora-CLAW/committers Install PR for work that directly addresses this use case is up^^^ I have to circle through all the related PRs and update/add tests, etc... but you can still totally check it out and give the feature a whirl.

[dannylamb]

I should mention this also addresses @ajs6f's reversed use case, as well ;-0

[ajs6f]

Snazzy! I am finishing up work on a reliable test instance, and should be able to look at this soon, but maybe not in time to test. Def don't wait to hear from me to merge.

[rosiel]

Is this solved by the new configuration opportunities we have using Flysystem?

[dannylamb]

@rosiel It most certainly is.