Test Fedora 5 RC #966

dannylamb · 2018-11-06T16:08:46Z

Slap the newest RC for Fedora onto a claw-playbook instance and put it through its paces. Put your results in the release testing results page on the fcrepo wiki.

mjordan · 2018-11-19T00:17:04Z

Initial test:

Replaced /var/lib/tomcat8/webapps/fcrepo.war on a fresh CLAW Vagrant with https://github.com/fcrepo4/fcrepo4/releases/download/fcrepo-5.0.0-RC-1/fcrepo-webapp-5.0.0-RC-1.war
Restarted tomcat
Tried to upload a TIFF file as a media on an Repository Item, I get a The file could not be uploaded. error.
1 http://localhost:8080/fcrepo/rest/ is asking for credentials (didn't do this with the stock claw-playbook)
fedoraAdmin/fedoraAdmin does not work.
Put back the original fcrepo.war, and things work fine.

Am I missing something obvious about 5.0.0 the password?

awoods · 2018-11-19T17:33:42Z

Can you try:
fedoraAdmin:secret3... or whatever you have your tomcat-users.xml set up to expect.

mjordan · 2018-11-19T17:41:33Z

@awoods, @whikloj is troubleshooting this now on #islandora IRC. But WRT your suggestion, here's the tomcat-users.xml file:

<?xml version='1.0' encoding='utf-8'?>
<tomcat-users xmlns="http://tomcat.apache.org/xml"
              xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xsi:schemaLocation="http://tomcat.apache.org/xml tomcat-users.xsd"
              version="1.0">
  <user username="islandora" password="islandora" roles="manager-gui"/>
</tomcat-users>

Neither curl -v -u islandora:islandora http://localhost:8080/fcrepo/rest nor the GUI works with those credentials.

mjordan · 2018-11-19T17:59:30Z

Also curl -v -H "Authorization: Bearer: islandora" http://localhost:8080/fcrepo/rest returns a 410.

whikloj · 2018-11-19T18:18:48Z

So first things first, @mjordan with Fedora 5 WebAC is enabled by default and now uses Shiro. It is looking for a fedoraUser or fedoraAdmin from tomcat. The only user defined there is the islandora:islandora user which has themanager-gui role (for looking at the Tomcat admin pages http://localhost:8080/manager/html).

First I discovered #979 and so to do step 2 I had to go into the claw-playbook/roles/external/Islandora.tomcat8/tasks/main.yml and add these lines

My initial test

Replaced /var/lib/tomcat8/webapps/fcrepo.war on a fresh CLAW Vagrant with https://github.com/fcrepo4/fcrepo4/releases/download/fcrepo-5.0.0-RC-1/fcrepo-webapp-5.0.0-RC-1.war

Re-ran playbook fcrepo-syn tasks to re-setup the SYN JWT authorizer.

ansible-playbook -i inventory/vagrant -e"islandora_distro=ubuntu/xenial64" -t fcrepo-syn,tomcat8 -u vagrant playbook.yml

The above didn't really work for me (sidenote: see sidenote below), so I had to go into the vagrant box and manually setup the Syn valve by
1. Copy the islandora-syn-0.1.0-all.jar from /opt/syn to /var/lib/tomcat8/lib
2. Add the below line to the end of /var/lib/tomcat8/conf/context.xml right before the closing </Context>
```
 <Valve className="ca.islandora.syn.valve.SynValve" pathname="/var/lib/tomcat8/conf/syn-settings.xml" />
```
1. Edit the /var/lib/tomcat8/webapps/fcrepo/WEB-INF/web.xml and change lines 95-98 from
```
<auth-constraint>
   <role-name>fedoraUser</role-name>
   <role-name>fedoraAdmin</role-name>
</auth-constraint>
```
to
```
<auth-constraint>
   <role-name>fedoraUser</role-name>
   <role-name>fedoraAdmin</role-name>
   <role-name>islandora</role-name>
</auth-constraint>
```
1. Restart tomcat with sudo service tomcat8 restart
2. Check the /var/log/tomcat/catalina.out to ensure everything started cleanly
Run a curl command and it explodes

vagrant@claw:/var/lib/tomcat8/conf$ curl -i http://localhost:8080/fcrepo/rest -H"Authorization: Bearer islandora"
HTTP/1.1 500 Internal Server Error
Server: Apache-Coyote/1.1
Cache-Control: private
Expires: Thu, 01 Jan 1970 00:00:00 UTC
Set-Cookie: rememberMe=deleteMe; Path=/fcrepo; Max-Age=0; Expires=Wed, 14-Nov-2018 20:22:06 GMT
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 3685
Date: Thu, 15 Nov 2018 20:22:06 GMT
Connection: close

<!DOCTYPE html><html><head><title>Apache Tomcat/8.0.32 (Ubuntu) - Error report</title><style type="text/css">H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}</style> </head><body><h1>HTTP Status 500 - Filtered request failed.</h1><div class="line"></div><p><b>type</b> Exception report</p><p><b>message</b> <u>Filtered request failed.</u></p><p><b>description</b> <u>The server encountered an internal error that prevented it from fulfilling this request.</u></p><p><b>exception</b></p><pre>javax.servlet.ServletException: Filtered request failed.
	org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:384)
	org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
	org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357)
	org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270)
</pre><p><b>root cause</b></p><pre>org.apache.shiro.authc.AuthenticationException: Authentication token of type [class org.fcrepo.auth.common.ContainerAuthToken] could not be authenticated by any configured realms.  Please ensure that at least one realm can authenticate these tokens.
	org.apache.shiro.authc.pam.AtLeastOneSuccessfulStrategy.afterAllAttempts(AtLeastOneSuccessfulStrategy.java:58)
	org.apache.shiro.authc.pam.ModularRealmAuthenticator.doMultiRealmAuthentication(ModularRealmAuthenticator.java:235)
	org.apache.shiro.authc.pam.ModularRealmAuthenticator.doAuthenticate(ModularRealmAuthenticator.java:269)
	org.apache.shiro.authc.AbstractAuthenticator.authenticate(AbstractAuthenticator.java:198)
	org.apache.shiro.mgt.AuthenticatingSecurityManager.authenticate(AuthenticatingSecurityManager.java:106)
	org.apache.shiro.mgt.DefaultSecurityManager.login(DefaultSecurityManager.java:274)
	org.apache.shiro.subject.support.DelegatingSubject.login(DelegatingSubject.java:260)
	org.fcrepo.auth.common.ServletContainerAuthFilter.doFilter(ServletContainerAuthFilter.java:83)
	org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
	org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
	org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
	org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
	org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
	org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387)
	org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
	org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
	org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357)
	org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270)
</pre><p><b>note</b> <u>The full stack trace of the root cause is available in the Apache Tomcat/8.0.32 (Ubuntu) logs.</u></p><hr class="line"><h3>Apache Tomcat/8.0.32 (Ubuntu)</h3></body></html>

Long story short, I don't think the Syn Tomcat Valve and the Fedora WebAC servlet filter are working together correctly.

sidenote: We haven't be enabling the Syn value in ansible at all Islandora-Devops/islandora-playbook#83

mjordan · 2018-11-19T18:27:47Z

@whikloj your curl query used Authorization: Bearer islandora - should that not be Authorization: Bearer: islandora?

acoburn · 2018-11-19T18:39:40Z

Authorization: Bearer <token> is the proper form. You should not have a semicolon after the Bearer part.

whikloj · 2018-11-19T19:11:28Z

@mjordan, as @acoburn points out the header is Authorization and everything else is the value so it should work (and has in past) but now we are conflicting with Fedora 5.

acoburn · 2018-11-19T19:18:33Z

@whikloj it sounds like this has to do with the order in which the servlet filters are applied. If the shiro filter is applied before the SYN filter, there'd be no java.security.Principal on which to operate. In your notes, I see that you add the SYN valve at the end of the configuration. Have you tried placing it earlier in the configuration? I'd mention that I'm no expert on servlet filter ordering: in the JAX-RS world, you can just use javax.annotation.Priority to define filter ordering.

whikloj · 2018-11-19T19:28:44Z

@acoburn Syn is a Tomcat Valve, so I'm not sure if I can control the ordering as this authentication mechanism is set across the entire JVM. Might be time to dust off the Syn servlet filter (Islandora/Syn#11)

acoburn · 2018-11-19T19:30:48Z

Oh, my mistake, I didn't realize that PR was never merged. 👍 to moving to a regular Servlet filter.

mjordan · 2018-11-19T22:01:26Z

After completely rebuilding my CLAW vagrant with the Syn version bump, replacing /var/lib/tomcat8/webapps/fcrepo.war with https://github.com/fcrepo4/fcrepo4/releases/download/fcrepo-5.0.0-RC-2/fcrepo-webapp-5.0.0-RC-2.war, and restarting tomcat, curl -v -H "Authorization: Bearer islandora" http://localhost:8080/fcrepo/rest (notice correct use of header 🎉 ) still yields a 403.

mjordan · 2018-11-19T22:03:36Z

@whikloj at least it didn't blow up real good like it did when you tried 😄

whikloj · 2018-11-20T15:50:11Z

Yeah, that is a little weird to me.

whikloj · 2018-11-27T21:33:56Z

Okay, so much to report.

Because Fedora is now set to expect BASIC auth and our requests don't have a Basic authorization header they get denied by Tomcat before anything else happens.
- This where a Tomcat Valve works and a Servlet Filter doesn't, alternative is to turn off the BASIC auth requirement and still leave Shiro in place. This is more of a security risk for non-locked down Fedora's. Speaking with @peichman-umd, the University of Maryland also has a desire to reduce their dependency on Tomcat Valves so perhaps we can work with them to find a solution for this.
There was a bug where our users which did not have a fedoraAdmin or fedoraUser role were making Shiro vomit and cause the 500 errors I was seeing.

I reverted to the Valve and added fedoraUser as a role to all allowed requests and was able to make a GET request fine. Still need to test PUT and DELETE.

My recommendation is to revert my SynFilter PR for now and add one of fedoraUser or fedoraAdmin. I'm not sure as one has all the power in the world and the other can't do anything without some help. Probably we should add fedoraUser and prime our Fedora in claw-playbook with a more relaxed WebAC policy. Like allowing acl:Write by "admin" or something.

dannylamb · 2018-12-18T20:19:52Z

Given that our installer now provisions with fcrepo 5 by default and we've sorted out all of our authentication issues, I think we're good to close this one.

Big thanks to @whikloj!

mjordan · 2018-12-18T23:39:22Z

Indeed, @whikloj ++

mjordan · 2018-12-19T02:25:08Z

Built a new vagrant from the master branch, and AFAICT everything works as intended. I was able to ingest a 'islandora_object' node, and attach an image media entity. Islandora Whole Object shows the following (all 👍 ):

For kicks, I thought I'd test fixity on the media image. Issuing the following:

curl -v -X HEAD -H "Want-Digest: SHA256" http://localhost:8080/fcrepo/rest/2018-12/abstract_group.png

returns a Digest header with the hash (so 👍 for compliance with the Fedora Spec API). However, hitting http://localhost:8080/fcrepo/rest/2018-12/abstract_group.png/fcr:fixity returns a 403. Not important, but I just wanted to point that out since I think it's a change in behaviour from FCREPO 4.7.5.

mjordan · 2018-12-19T02:31:13Z

And also successfully ran a migration from CSV using https://github.com/dannylamb/migrate_islandora_csv.

dannylamb added the fcrepo label Nov 6, 2018

dannylamb mentioned this issue Nov 19, 2018

Bumping fcrepo role version in requirements.yml Islandora-Devops/islandora-playbook#83

Closed

dannylamb mentioned this issue Nov 19, 2018

Change Tomcat Valve to servlet Filter Islandora/Syn#11

Merged

dannylamb mentioned this issue Nov 28, 2018

Servlet filter revert Islandora/Syn#13

Merged

dannylamb closed this as completed Dec 18, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test Fedora 5 RC #966

Test Fedora 5 RC #966

dannylamb commented Nov 6, 2018

mjordan commented Nov 19, 2018 •

edited

Loading

awoods commented Nov 19, 2018

mjordan commented Nov 19, 2018

mjordan commented Nov 19, 2018

whikloj commented Nov 19, 2018

mjordan commented Nov 19, 2018

acoburn commented Nov 19, 2018

whikloj commented Nov 19, 2018

acoburn commented Nov 19, 2018

whikloj commented Nov 19, 2018

acoburn commented Nov 19, 2018

mjordan commented Nov 19, 2018

mjordan commented Nov 19, 2018

whikloj commented Nov 20, 2018

whikloj commented Nov 27, 2018

dannylamb commented Dec 18, 2018

mjordan commented Dec 18, 2018

mjordan commented Dec 19, 2018

mjordan commented Dec 19, 2018

Test Fedora 5 RC #966

Test Fedora 5 RC #966

Comments

dannylamb commented Nov 6, 2018

mjordan commented Nov 19, 2018 • edited Loading

awoods commented Nov 19, 2018

mjordan commented Nov 19, 2018

mjordan commented Nov 19, 2018

whikloj commented Nov 19, 2018

mjordan commented Nov 19, 2018

acoburn commented Nov 19, 2018

whikloj commented Nov 19, 2018

acoburn commented Nov 19, 2018

whikloj commented Nov 19, 2018

acoburn commented Nov 19, 2018

mjordan commented Nov 19, 2018

mjordan commented Nov 19, 2018

whikloj commented Nov 20, 2018

whikloj commented Nov 27, 2018

dannylamb commented Dec 18, 2018

mjordan commented Dec 18, 2018

mjordan commented Dec 19, 2018

mjordan commented Dec 19, 2018

mjordan commented Nov 19, 2018 •

edited

Loading