Add Rdf Descriptions of Houdini and Hypercube to Fedora #612

dannylamb · 2017-04-26T12:37:47Z

Relevant reading: https://github.com/fcrepo4-labs/fcrepo-api-x/blob/master/src/site/markdown/extension-definition-and-binding.md#examples and https://github.com/fcrepo4-labs/fcrepo-api-x/blob/master/src/site/markdown/service-discovery-and-binding.md#example-ldp-service-instance-registry.

Those links skip to the example sections, but the entire documents are relevant if you're seeking more context.

We need to include a service description and an extension description in Fedora for both Houdini and Hypercube to be found by Api-X. Ideally, we'd make bundles for them in Drupal so we could have forms, but for now, let's just manually jam in some RDF and revisit after Islandoracon.

The scope of this work is to create RDF documents for service and extension for both Houdini and Hypercube (4 documents in total) and to cURL them into Fedora during provisioning in claw_vagrant.

ajs6f · 2017-04-27T15:28:40Z

This is blocked by #504 , right?

dannylamb · 2017-04-27T15:35:13Z

Loosely. You can do the work no problem, but verifying API-X picks up the service can't happen until #504

ajs6f · 2017-04-27T15:37:55Z

Okay, cool, unless anyone wants this particularly, I will push through #504 and then turn to this real quick (which should be quick once we have a nicely-integrated "Lion Force Voltron"-style CLAW/API-X Vagrant.

dannylamb · 2017-04-27T16:18:39Z

@ajs6f I prefer "Clawzord' from Power Rangers Samurai, though it may be a bit too literal.

ajs6f · 2017-04-27T16:52:51Z

I wasn't young enough to catch Power Rangers when they got big, but holy cow that is totally what the new CLAW conf swag should be like. @manez ^^^

whikloj · 2017-04-28T03:09:57Z

So as I was learning here I think this work needs to be done before #611. So we know what the service says it handles before we try to handle those requests.

ajs6f · 2017-04-28T13:32:58Z

So we have a kind of loose chain of #504, this, #611?

whikloj · 2017-05-01T14:40:07Z

I'd agree with that assessment, but in not fluent in API-X either.

ajs6f · 2017-05-01T14:41:17Z

I'm working #504, but it's going to be slow as I physically move my family and change jobs. As we speak. If this "chain" of tix is important for MVC progress, we should have a discussion about it.

whikloj · 2017-05-01T14:43:04Z

@ajs6f I can spend some time on #504 too and see if I can help, but I also think I can move on this and #611 without #504. Its just that for testing and validation of that work, #504 would be a huge help.

ajs6f · 2017-05-01T14:47:50Z

I am open for anything. I think the two moves right now from which we can choose (and we could do both) are:

Install a shim to make API-X work with Syn. We've talked about that elsewhere. That would work for the vagrant box, for now.
Get help from @birkland et al. to make API-X Syn-aware. That is the real long-term solution.

@dannylamb CLAW call agenda item? Too minor?

birkland · 2017-05-01T14:58:04Z

I'd be happy to help make API-X Syn-aware. Can somebody give a high-level description of what generally API-X would need to do? I'm assuming this is for the part where API-X authenticates itself to Fedora so that it can maintain its own various registries?

ajs6f · 2017-05-01T14:59:17Z

@birkland Exactly. Syn is a valve for Tomcat that deals in JWT.

birkland · 2017-05-01T15:12:25Z

@ajs6f Ah, right. I'm not familiar with JWT, but will take a look this afternoon to see what it would take.

dannylamb · 2017-06-07T14:45:51Z

Ok, circling back to this one. I've added an OPTIONS request to Hypercube here. Then I ran into the following error when trying to curl up the url to api-x's loader service:

org.apache.camel.CamelExchangeException: JettyClient failed cause by: HTTP protocol violation: Authentication challenge without WWW-Authenticate header. Exchange[ID-claw-38214-1496784667042-6-28]. Caused by: [org.eclipse.jetty.client.HttpResponseException - HTTP protocol violation: Authentication challenge without WWW-Authenticate header]
	at org.apache.camel.component.jetty9.JettyContentExchange9.doTaskCompleted(JettyContentExchange9.java:156)
	at org.apache.camel.component.jetty9.JettyContentExchange9$2.onComplete(JettyContentExchange9.java:222)
	at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:193)
	at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:185)
	at org.eclipse.jetty.client.ResponseNotifier.forwardFailureComplete(ResponseNotifier.java:242)
	at org.eclipse.jetty.client.AuthenticationProtocolHandler$AuthenticationListener.forwardFailureComplete(AuthenticationProtocolHandler.java:204)
	at org.eclipse.jetty.client.AuthenticationProtocolHandler$AuthenticationListener.onComplete(AuthenticationProtocolHandler.java:113)
	at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:193)
	at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:185)
	at org.eclipse.jetty.client.HttpReceiver.terminateResponse(HttpReceiver.java:453)
	at org.eclipse.jetty.client.HttpReceiver.responseSuccess(HttpReceiver.java:400)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.messageComplete(HttpReceiverOverHTTP.java:266)
	at org.eclipse.jetty.http.HttpParser.parseContent(HttpParser.java:1403)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1245)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.parse(HttpReceiverOverHTTP.java:156)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:117)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:69)
	at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:89)
	at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:123)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:748)

From the logs:

2017-06-07 14:12:18,226 | ERROR | t(0x993f52d)-949 | DefaultErrorHandler              | 57 - org.apache.camel.camel-core - 2.18.1 | Failed delivery for (MessageId: ID-claw-38214-1496784667042-6-29 on ExchangeId: ID-claw-38214-1496784667042-6-28). Exhausted after delivery attempt: 1 caught: org.apache.camel.CamelExchangeException: JettyClient failed cause by: HTTP protocol violation: Authentication challenge without WWW-Authenticate header. Exchange[ID-claw-38214-1496784667042-6-28]. Caused by: [org.eclipse.jetty.client.HttpResponseException - HTTP protocol violation: Authentication challenge without WWW-Authenticate header]

Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId              ProcessorId          Processor                                                                        Elapsed (ms)
[loader-http       ] [loader-http       ] [jetty:http://0.0.0.0:32080/load?httpMethodRestrict=GET%2COPTIONS%2CPOST&matchO] [       431]
[loader-http       ] [choice15          ] [when[{header{header(CamelHttpMethod)} == GET}]choice[when[{header{header(Camel] [       429]
[loader-http       ] [to118             ] [direct:prepare_load                                                           ] [       418]
[load-prepare      ] [choice16          ] [when[{header{header(Content-Type)} == text/plain}]choice[]                    ] [         6]
[load-prepare      ] [setHeader21       ] [setHeader[service.uri]                                                        ] [         0]
[load-prepare      ] [choice17          ] [when[{header{header(service.uri)} is null}]choice[]                           ] [       401]
[load-prepare      ] [to122             ] [direct:load                                                                   ] [       396]
[load-service-uri  ] [setHeader22       ] [setHeader[ApixLoaderRequestUri]                                               ] [         0]
[load-service-uri  ] [removeHeaders3    ] [removeHeaders[*]                                                              ] [         0]
[load-service-uri  ] [setBody4          ] [setBody[{null}]                                                               ] [         0]
[load-service-uri  ] [setHeader23       ] [setHeader[CamelHttpMethod]                                                    ] [         0]
[load-service-uri  ] [setHeader24       ] [setHeader[CamelHttpUri]                                                       ] [         0]
[load-service-uri  ] [setHeader25       ] [setHeader[Accept]                                                             ] [         0]
[load-service-uri  ] [process20         ] [Processor@0x575f16be                                                          ] [         9]
[load-service-uri  ] [to123             ] [jetty:http://localhost                                                        ] [       384]

I created another interceptor to manually jam in a dummy WWW-Authenticate header here to test the waters. But it looks like it never gets called. I'm still getting the same errors.

I tracked it down to here in the Api-X code and it looks like maybe it's because the jetty component is used to make the request?

@birkland Do you think this is worth filing an issue or am I just doing it wrong?

@ajs6f Your thoughts would be appreciated, too.

birkland · 2017-06-07T14:58:36Z

@dannylamb

File an issue. It's stripping off all headers from the incoming request. I wonder what the right behavior is:

Use the credentials or HttpClient given to API-X, which API-X currently uses when API-X is a client of the repository
Use the credentials provided by the POST to the loader (i.e don't strip those headers. The credentials may be different from the ones API-X uses to engage the repository on its own behalf).
Require/expect the OPTIONS request to the specified resource to respond without authentication
Something else.

ajs6f · 2017-06-07T14:58:45Z

@birkland is this maybe a point where API-X isn't yet using the "custom" injected client?

ajs6f · 2017-06-07T15:00:22Z

I think that this is API-X acting on behalf of the initial request, right? So it should use the HttpClient that it uses for other against-the-repo action... I think. :)

birkland · 2017-06-07T15:15:38Z

So if it's "on behalf of the initial request", the interaction is like:

Bootstrap to API-X: "I'm ___, please load the extension at this service URI"
API-X to service URI: "I'm API-X, please let me see OPTIONS"
Service: "OK, API-X, you authenticate, here you go"

Does that sound about right?

ajs6f · 2017-06-07T15:31:22Z

I think so. Here's my reasoning: we have always imagined that the registry for API-X might not be the repository (not sure how likely that is in practice, but anyway). Suppose that it is instead a SQL db somewhere. It makes no sense in that case for API-X to use the same credentials used against it to go against that db, right?

This is, from a certain angle, just about API-X storing its own config...

birkland · 2017-06-07T15:43:56Z

@ajs6f So you're saying that this particular interaction is analogous; where API-X itself (and not a client) is requesting access to service, for API-X's own purpose. API-X wants to perform OPTIONS so that it can populate its registry. Therefore, it makes sense to use API-X's own credentials for this interaction.

Makes sense to me.

There's a separate and unrelated issue of "secure the loader", i.e. protect the loader service, so that only authenticated clients with the appropriate authorization can instruct API-X load an extension. If you're securing the repository, and individual extensions, then I imagine that would be useful as well.

dannylamb · 2017-06-07T16:27:47Z

@ajs6f @birkland Yeah, that seems fair. API-X should be using its own credentials and not passing through auth headers from the client. I'll file an issue. Thanks.

dannylamb · 2017-06-07T16:47:12Z

Oh wait... I read that all wrong. We're issuing the OPTIONS request against the service itself, not the repository, which in our case is locked down the same way as the repository, but potentially not for others. So we can get away with using the same http client retrieved by the httpClientFetcher bean, but most people won't.

I'll update the issue I just created.

birkland · 2017-06-08T20:15:00Z

by the way, there's a PR for this now: fcrepo4-labs/fcrepo-api-x#122

I don't know if you want to try it, or if we can just merge it in. It's a fairly straightforward change. It might be easier to try it out from a -SNAPSHOT after it gets merged in

dannylamb added MVP labels Apr 26, 2017

dannylamb mentioned this issue Jun 2, 2017

Convert Hypercube and Houdini to respect Apix-Ldp-Resource header #611

Closed

dannylamb self-assigned this Jun 7, 2017

dannylamb mentioned this issue Jun 7, 2017

OPTIONS request against service to load extension can require credentials fcrepo4-labs/fcrepo-api-x#121

Closed

dannylamb mentioned this issue Jun 12, 2017

Issue 612 Islandora/Crayfish#38

Merged

dannylamb mentioned this issue Jun 27, 2017

Using LATEST API-X islandora-deprecated/claw_vagrant#49

Merged

DiegoPino closed this as completed in Islandora/Crayfish#38 Aug 15, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Rdf Descriptions of Houdini and Hypercube to Fedora #612

Add Rdf Descriptions of Houdini and Hypercube to Fedora #612

dannylamb commented Apr 26, 2017

ajs6f commented Apr 27, 2017

dannylamb commented Apr 27, 2017

ajs6f commented Apr 27, 2017

dannylamb commented Apr 27, 2017

ajs6f commented Apr 27, 2017

whikloj commented Apr 28, 2017

ajs6f commented Apr 28, 2017

whikloj commented May 1, 2017

ajs6f commented May 1, 2017

whikloj commented May 1, 2017

ajs6f commented May 1, 2017 •

edited

Loading

birkland commented May 1, 2017

ajs6f commented May 1, 2017

birkland commented May 1, 2017

dannylamb commented Jun 7, 2017

birkland commented Jun 7, 2017

ajs6f commented Jun 7, 2017

ajs6f commented Jun 7, 2017

birkland commented Jun 7, 2017

ajs6f commented Jun 7, 2017

birkland commented Jun 7, 2017

dannylamb commented Jun 7, 2017

dannylamb commented Jun 7, 2017

birkland commented Jun 8, 2017

Add Rdf Descriptions of Houdini and Hypercube to Fedora #612

Add Rdf Descriptions of Houdini and Hypercube to Fedora #612

Comments

dannylamb commented Apr 26, 2017

ajs6f commented Apr 27, 2017

dannylamb commented Apr 27, 2017

ajs6f commented Apr 27, 2017

dannylamb commented Apr 27, 2017

ajs6f commented Apr 27, 2017

whikloj commented Apr 28, 2017

ajs6f commented Apr 28, 2017

whikloj commented May 1, 2017

ajs6f commented May 1, 2017

whikloj commented May 1, 2017

ajs6f commented May 1, 2017 • edited Loading

birkland commented May 1, 2017

ajs6f commented May 1, 2017

birkland commented May 1, 2017

dannylamb commented Jun 7, 2017

birkland commented Jun 7, 2017

ajs6f commented Jun 7, 2017

ajs6f commented Jun 7, 2017

birkland commented Jun 7, 2017

ajs6f commented Jun 7, 2017

birkland commented Jun 7, 2017

dannylamb commented Jun 7, 2017

dannylamb commented Jun 7, 2017

birkland commented Jun 8, 2017

ajs6f commented May 1, 2017 •

edited

Loading