Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Need help diagnosing issue: java.lang.AbstractMethodError on template #172

Closed
RussellMcOrmond opened this issue Nov 15, 2017 · 5 comments
Closed

Need help diagnosing issue: java.lang.AbstractMethodError on template #172

RussellMcOrmond opened this issue Nov 15, 2017 · 5 comments

Comments

@RussellMcOrmond
Copy link

Yesterday was the day we moved Cantaloupe into production, after having tested for many months (including providing a public demonstration at Canada's ACCESS2017 conference).

We deployed with 3.3.2, but after noticing many 500 errors relating to creating cache directories (and remembering something about that in the 3.3.3 release), we moved to 3.3.3.

While the locking works and we don't get any errors on creating caches, and derivative images work, we are getting an odd error when looking at pages that have a template.

Example:

russell@russell-desktop2:~$ curl  http://image.swiss.canadiana.ca/ 
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 500 Server Error</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /. Reason:
<pre>    Server Error</pre></p><h3>Caused by:</h3><pre>java.lang.AbstractMethodError: org.apache.velocity.runtime.resource.loader.ResourceLoader.getResourceStream(Ljava/lang/String;)Ljava/io/InputStream;
	at org.apache.velocity.Template.process(Template.java:108)
	at org.restlet.ext.velocity.TemplateRepresentation.&lt;init&gt;(TemplateRepresentation.java:176)
	at edu.illinois.library.cantaloupe.resource.AbstractResource.template(AbstractResource.java:675)
	at edu.illinois.library.cantaloupe.WebApplication$CustomStatusService.toRepresentation(WebApplication.java:82)
	at org.restlet.engine.application.StatusFilter.afterHandle(StatusFilter.java:115)
	at org.restlet.routing.Filter.handle(Filter.java:199)
	at org.restlet.routing.Filter.doHandle(Filter.java:150)
	at org.restlet.routing.Filter.handle(Filter.java:197)
	at org.restlet.engine.CompositeHelper.handle(CompositeHelper.java:202)
	at org.restlet.engine.application.ApplicationHelper.handle(ApplicationHelper.java:77)
	at org.restlet.Application.handle(Application.java:385)
	at org.restlet.routing.Filter.doHandle(Filter.java:150)
	at org.restlet.routing.Filter.handle(Filter.java:197)
	at org.restlet.routing.Router.doHandle(Router.java:422)
	at org.restlet.routing.Router.handle(Router.java:641)
	at org.restlet.routing.Filter.doHandle(Filter.java:150)
	at org.restlet.routing.Filter.handle(Filter.java:197)
	at org.restlet.routing.Router.doHandle(Router.java:422)
	at org.restlet.routing.Router.handle(Router.java:641)
	at org.restlet.routing.Filter.doHandle(Filter.java:150)
	at org.restlet.routing.Filter.handle(Filter.java:197)
	at org.restlet.engine.CompositeHelper.handle(CompositeHelper.java:202)
	at org.restlet.Component.handle(Component.java:408)
	at org.restlet.Server.handle(Server.java:507)
	at org.restlet.engine.connector.ServerHelper.handle(ServerHelper.java:63)
	at org.restlet.engine.adapter.HttpServerHelper.handle(HttpServerHelper.java:143)
	at org.restlet.ext.servlet.ServerServlet.service(ServerServlet.java:1117)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:837)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
	at org.eclipse.jetty.server.Server.handle(Server.java:524)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
	at java.lang.Thread.run(Thread.java:748)
</pre>
<hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.z-SNAPSHOT</a><hr/>

</body>
</html>
russell@russell-desktop2:~$

With 3.3.2 we get the splash page with the cantaloupe image indicating version, but a 500 error with 3.3.3.

There are further complications...

We are deploying Cantaloupe using Docker, with the configuration detailed at https://github.com/c7a/cihm-cantaloupe

If we deploy the identical 3.3.3 tagged image to 3 servers we have two that show the error, but for reasons we haven't been able to determine yet the third gives the expected message:

russell@russell-desktop2:~$ curl  http://image.queso.canadiana.ca/  ; echo
<!DOCTYPE html><html><head>            <base href="http://image.queso.canadiana.ca/">        <meta charset="utf-8">    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">    <meta http-equiv="X-UA-Compatible" content="IE=edge">    <meta name="viewport" content="width=device-width, initial-scale=1">    <link rel="stylesheet" href="static/styles/landing.css">    <title>Cantaloupe Image Server</title></head><body>    <div id="container">        <img src="static/images/cantaloupe-320.png" width="320" alt="Logo">        <h1>Cantaloupe Image Server <small>3.3.3</small></h1>    </div></body></html>
russell@russell-desktop2:~$

When I do a docker image inspect docker.c7a.ca/cihm-cantaloupe:3.3.3 , I can confirm I'm using the same image with the same RepoDigests and the same Layers. I'm using the same Docker version and build, and I even tried upgrading to the same kernel version.

We're using Puppet to deploy startup scripts for the containers, and the setup is the same other than the IP address that we are binding to which is different on each server.

I will continue to determine differences between these machines, but thought I would ask in case this looks familiar to anyone.

Note: We noticed these because we are getting 500 errors rather than 404 errors when browsers request /favicon.ico . http://image.ulloa.canadiana.ca/favicon.ico We'll be fixing this issue separately https://github.com/c7a/cihm-public-cos/issues/1
@adolski
Copy link
Contributor

adolski commented Nov 15, 2017

Hi @RussellMcOrmond,

While the locking works and we don't get any errors on creating caches, and derivative images work, we are getting an odd error when looking at pages that have a template.

This looks a lot like: #160 (comment) -- this was a bug introduced in 3.3.3 that is fixed now in the release/3.3 branch, and will appear in 3.3.4 (the fix I mean, not the bug).

So, I guess I should release that soon.

I don't know what's going on with the Docker inconsistencies, but sounds like it's related to the above.

@RussellMcOrmond
Copy link
Author

I'm still wondering if there are some external dependencies that I can look into. This morning I migrated a 4'th server to Cantaloupe, and this one isn't showing the error.

So far that is:

OK http://image.wellington.canadiana.ca/iiif/2
OK http://image.queso.canadiana.ca/iiif/2
Error: http://image.swiss.canadiana.ca/iiif/2
Error: http://image.ulloa.canadiana.ca/iiif/2

This deployment is also our first deployment to production using Docker. These types of inconsistencies are why I wanted to move to Docker, so I'm wanting to learn what type of external differences cause issues with our Canteloupe container so I can avoid them.

@adolski
Copy link
Contributor

adolski commented Nov 16, 2017

Just from looking it over briefly, I don't see any evidence that there is anything wrong with your setup. One way to test (this would also help me as I've only been able to reproduce the AbstractMethodError in Tomcat 7) would be to build your own Cantaloupe-3.3.4-SNAPSHOT.war (git checkout release/3.3; mvn clean package -DskipTests) and try that out.

In any case, I'll try to get a release out soon knowing that this a bigger problem than it appeared in #160.

@RussellMcOrmond
Copy link
Author

Thanks for the suggestion.

Our developer @SaschaAdler who I'd normally ask if he could build will be on vacation for a few days. I'm knee-deep in other deployment work (I'm mostly Ops in our currently 2-person DevOps team :-), and will hopefully have a chance to do a test build early next week.

@adolski
Copy link
Contributor

adolski commented Nov 17, 2017

That's OK. 3.3.4 has just been released and includes the fix for these AbstractMethodErrors. Please reopen if you notice any more of them.

@adolski adolski closed this as completed Nov 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@RussellMcOrmond @adolski