Skip to content

March 22, 2017

Jump to bottom Edit New page
dannylamb edited this page Mar 22, 2017 · 11 revisions

Time/Place

This meeting is a hybrid teleconference and IRC chat. Anyone is welcome to join. Here is the info:

Attendees

  • Kirsta Staplefeldt
  • Pat Dunlavey
  • Ed Fugikawa
  • Mark Jordan
  • Aaron Coburn
  • Kim Pham
  • Bryan Brown
  • John Yobb
  • Andrew Woods
  • Dan Davis
  • Jon Green
  • Melissa Anez
  • Daniel Lamb
  • Natkeeran
  • Jordan Fields
  • Mark Bagget
  • Rosie LeFaive
  • Jared
  • Diego

Agenda

  1. Welcome new CLAW committer, Jonathan Green!
  2. Let's Talk Fedora! De-obsfucate the spec.
  3. Pick up an easy ticket!
  4. (please add your agenda items here...)

Minutes (Pending Danny's Review)

1. Welcome new CLAW committer, Jonathan Green!

Danny: Welcome to Jonathan. He has contributed tremendously, including a lot of grunt work.

Jonathan: Thank you. Excited to join. Thank you to LYRASIS for enabling me to contribute.

Let's Talk Fedora! De-obsfucate the spec

Overview

Danny: I can provide a condensed overview/summary of the Fedora Specs.

''Others:** Yes, will be helpful. We need to provide feedback.

Danny: We have to see how the specs are going to impact CLAW. We have to be able to work with it. Ones needs to be familiar with a lot of new terminology. Most of it is related to the Linked Data Platform (LDP) specifications.

Fedora specs specify basic CRUD Operations on resources ovr HTTP. It also specifies Versioning, Messaging, Atomic Operations, and ways of storing binary resources.

Messaging specification will be of interest from the CLAW side. Activitystream format will be used.

Storing binary resource is also of interest to people familiar with Fedora 3.

Storing binary resources is of interest to people familiar with Fedora 3. How do the specs deal datastreams pointing to external urls. This is the use case where Fedora acts as the middleware. There is a ticket about this issue: https://github.com/fcrepo/fcrepo-specification/issues/41.

Binary Resources: Managed, External (Externally Managed, Re-Directs)

Acobrun: There are two types/models of binary resources: managed by fedora, available externally (publicly!).

Dan Davis: The use case of my concern is when I don't want to keep binary resource in Drupal database. I would like to keep everything else. I would like to stream that from Fedora.

Danny: From the CLAW point of view. We have not considered the external url use case particularly. We are making use of the File API of the Drupal!. Drupal is storing the URIs to the the binary.

Dan Davis: We have a large set of (over 6 million) images. Interested in how we can store and retrieve them.

Danny: Drupal can handle externally reference files.

Diego: We can make use of what Drupal provides. There is a project to enable swapping the Drupal file storage with other file storage systems such as AWS.

Dan Davis: We have done this in Drupal. We are able to store binaries somewhere else and retrieve it to display via Drupal.

Andrew: Going back to external content. There are three types of binary resources.

  1. Managed
  2. Externally Managed
  3. Re-Direct - 303 is returned

We have some questions from institutions related to migrations. One major issue is moving large amount of content as part of the migration. Fedora 3 supports externally re direct resource. How many people use this? Is there a major requirement for Externally Managed resource, where Fedora plays the middleman. If the Externally Managed is useful, does this need to be reflected in the specs?

Diego: I have reviewed the issues. Locally, we are moving to S3 based storage. Can we see if we can approach the Externally Managed and Re Directed Resources in a consistent way. Another related issue is getting the Fixity of the external content. Authorization is also a major issue relating to external resources.

Jared: We don't have a use case for Fedora 3 Re-Direct being added to Fedora 4. Since we will be doing that in the Drupal layer, there is no need to do that in Fedora.

My concern is about large objects. I would need to get the preservation copy sometimes. Can Drupal/Claw handle externally reference content, and not fedora!

Dan Davis: Yes. When I want to display an image, we should be able to get it from the right location, example image server. Fedora store can store a reference copy. We have to be flexible about where the binary comes from for GET requests.

Danny: We are trying to flexible with respect to storage. Are people satisfied with what is possible? Is there a need to add analog to external (redirect) datastream?

Rosiel: There is interest in storing large objects in external storage places. We need some type of mediation. For example, we can lock down the external storage down to serve only fedora resources.

Danny: If just redirects. Then these resources are publicly available.

Dan Davis: I was making the argument that we can do the mediation in Drupal. Drupal can also provide authorization.

acoburn: Would like to add one more thing. With LDP, we can add a resolvable external link to the graph. Similar to redirect. It is not pretending to be a fedora resource. The question is how much we want Fedora to be involved in the mediation. If we don't need Fedora to manage the life cycle of a URL, then we can just link to it as a web resource or url.

Danny: Anyone here comfortable bringing this to the Fedora community. Consistency of behaviour is something of interest. We can explore authorization control from CLAW.

Jared: Are there people who need redirects? For example to say to make migration easier? Lets query the Islandora user community to get sense of the usage of redirects.

From IRC, it seems several users are using this, such as rosiel, jyobb, ddavis.

Is there something we can do on the Islandora side?

Danny: We internally (as an Islandora community) need to ensure that we need the redirect before pushing it to Fedora. We will need the input from the wider community before proceeding.

Versioning

Danny: Versioning and Atomic operations are other two components that needs to be reviewed.

Versioning will be based on Momento specifications. Conceptually, when you preform a GET request, you need to provide a time reference. Fedora will figure out proper version to return.

ajs6f: Yes. Fedora handles the version creation part.

Diego: Does the specification specify what part of the LDP graph will be versioned?

ajs6f: Good question. My understanding is that a resource alone is versioned. Have to clarify this.

Atomic Operations

Danny: Atomic operations will let you do multiple operations all at once, allow you to roll back if any one goes bad. It is being called Atomic operation rather than Transactions to avoid people associating them with https://en.wikipedia.org/wiki/ACID.

DDavis: My applications are written in Islandora. How to synch with Fedora. Does Fedora support rollback capabilities?

One of the use case of related to atomic operations is people editing a resource multiple times before publishing. The question is when does it get put into Fedora?

ajs6f: Might be better to handle in Drupal or other means than Fedora!

Danny: Thank you all. Fedora Specs discussion will continue.

Jared: There is a meeting to discuss the specs and other Fedora issues tomorrow: https://wiki.duraspace.org/display/FF/2017-03-23+-+Fedora+Tech+Meeting

You may be looking for the islandora-community wiki · new to islandora? · community calendar · interest groups · roadmap

This is an archive. For new Tech Call notes, click here

⚠️ ARCHIVED Islandora Tech Calls

⚠️ ARCHIVED Islandora User Calls

Clone this wiki locally