Investigate API-X

[dannylamb]

Basically, read this.

We have a lot of folks attending the calls, but have not made a clear decision whether or not to use API-X. I need to know if this is something we're all on board with before it becomes part of MVC.

TL;DR:

• Use RDF to describe services
• Use RDF to 'bind' services to rdf:types in Fedora
• Expose API for registering and binding services
• Use a proxy in front of Fedora to route to 'extensions' dynamically.

[dannylamb]

And if we are cool with it, then do we use the OSGi version that is being developed, or bite off a chunk and go for implementing this in PHP?

[acoburn]

If I understand API-X correctly, the core of it runs in an OSGi container, but individual services can be developed/deployed in any language/runtime. Meaning: the API-X framework simply routes requests to the appropriate service endpoint and determines which services apply to a given Fedora resource; and those services can be written in PHP, JavaScript, Rust, etc.

[ruebot]

As an API-X stakeholder;

Use RDF to describe services

👍

Use RDF to 'bind' services to rdf:types in Fedora

👍

Expose API for registering and binding services

👍

Use a proxy in front of Fedora to route to 'extensions' dynamically.

👍 We already do this in 1.x with Fedora, so don't see why it would be a problem in CLAW. I think this could be the only contentious issue.

And if we are cool with it, then do we use the OSGi version that is being developed, or bite off a chunk and go for implementing this in PHP?

What @acoburn said.

[acoburn]

I would also add that a number of API-X stakeholders want the core of the system to run in an OSGi container. But translating OSGi code into a more traditional Servlet container is easy. Going the other way is most certainly not. That is, if someone would want to run API-X in, say, Tomcat, one would need only to rewrite the various blueprint XML files into spring XML (which is an easy conversion).

[dannylamb]

@whikloj @DiegoPino last two committers not to chime in. thoughts?

[DiegoPino]

I agree on what @acoburn said in #366 (comment). Also, as the name suggests the important part here is the "API" part in the API-X. It's a spec, we can go for the demo - reference - implementation or, in a future build something that fits better our needs (and still being api-x compliant).

But, as titile of this issue also suggest, it's about investigation API-X, means giving it a spin, see how this/where this fits in our future islandora, what is missing, does it add much overhead to simple operations? Is this needed for all islandora installations? etc. Without that info i find it complicated to decide.

[dannylamb]

@DiegoPino I've spun up a copy of what's on github and given it a whirl. It's just a Go proxy at the moment. The OSGi stuff they're making (which would replace it) isn't finished and isn't part of the build machinery.

So then the question kind of turns into: are we willing to wait for the reference implementation? Or to take it a step further, are we willing to contribute to the reference implementation to help push it along?

[dannylamb]

Also, they're using maven to package up Docker files, which is pretty cool. We could probably do the same with Gradle.

[ajs6f]

You guys have an interesting advantage here: putting a proxy in front of Fedora is a key sticking point for a lot of people, and rightly so. @azaroth42 made some telling arguments against that; I agree with him. But as @ruebot says, y'all are already doing that.

[dannylamb]

If the proxy issue is super-contentious, we could dance around it. @acoburn's FITS extension implementation doesn't create an additional URI pointing to the exact same resource, which is the point of contention for the reverse proxy.

Or just roll with it?

[DiegoPino]

@dannylamb maybe i'm confused so i could need a bit of clarification: Our microservices (Crayfish and PDX) are already acting as a Proxy:

• We are exposing a different URL/Port for existing Fedora Resources wrapping around fedora REST API
• We are exposing a different URL/Port for LDP CRUD wrapping around fedora REST API
• We are rewriting headers/ hosts, etc
• We are trying to do this as transparent as possible, keeping responses close to the original.
• We are using UUID's to map to LDP Paths.

So, if so, what does it mean that the proxy issue is super-contentious or dance around it? Means abandoning our current approach (Silex Microservices)?

@ruebot and @whikloj you both where involved in the microservices dev so your opinion on this would be super helpful for me also. That said, maybe i don't understand the proxy issue?

Thanks!

[whikloj]

I think the concern around a proxy in front of Fedora, is that when our microservices try to access Fedora we might get additional actions not expected (or described) by the Fedora REST api. But perhaps I'm misunderstanding.

[ruebot]

@birkland @acoburn @ajs6f do you have that Google doc link handy that had @azaroth42's comment in it?

[DiegoPino]

@whikloj yeah. So i guess we could configure/decide/set which calls we do want to make through that second API-X proxy and which ones we do, via Chullo, directly to fedora right?
Also, can API-X also act on JMS messaging, not only direct endpoint invocation? I assume yes.

[whikloj]

@ruebot https://docs.google.com/document/d/1R9OymmIhsUv4yuWOMQO8MqBMW7PBezeiFWGbkHoXJP8/edit#heading=h.kpr10lc606ro

[DiegoPino]

I think this explains a bit the concerns. In the comments one of issues seems to be about URL "mugging" as @azaroth42 explains:

"if R* URIs are exposed on the web, then only R* URIs exist."

Since Islandora does not expose Fedora nor API-X URIs, but Drupal ones, we are already adapting stored Linked data to our published reality, so we should not have concerns related to that, at least not big ones. Still, interacting with various layers of proxy can be complicated if we need to play a lot with headers.

[birkland]

@dannylamb Yeah, so as mentioned on the last API-X call and will be followed up today we're aiming for a milestone of the core framework around the end-of-month timeframe. That will help along the CLAW analysis. Unfortunately, I've been the only one able to contribute to development of late; and have been cut down a bit over the past couple weeks due to various illnesses that have swept through the family.

[dannylamb]

@birkland Thanks for the update. I'll be attending today's call.

@DiegoPino Didn't mean to cause so much trouble. Just wanted to make sure everyone was cool with the implications of what we're doing, since I think @azaraoth42's comments are valid. It would be prohibitive for an organization that's already entrenched in certain URIs to have to start using the proxy's URIs if Islandora/API-X was adopted.

But yes... we are already doing it. Was expecting a chorus of 'roll with it'

[DiegoPino]

@dannylamb no trouble at all, lot of moving pieces, so just needed to be sure i was understanding things right. And 'roll with it'! 👍

[birkland]

@dannylamb @DiegoPino I think it's important for an organization adopting CLAW or API-X to have a grasp on which resources they intend to publish on the web (repository resources? Drupal resources? both?), and what URIs they're committing to. That high-level notion then drives the systems and infrastructure. So I see proxies as one of many tools one may deploy in an institution's black box to make it happen.

Placing a proxy over an infrastructure that didn't have one before yet still exposing the same URIs to the public is not categorically impossible; it's an engineering exercise that carries risk. Depending on the particulars of the software being proxied (substitute "API-X" for "proxy" in that paragraph), the results can range from "easy and cheap to implement" to "technically possible, but expensive and unpalatable".