Issue with uploading files

[nwoodward]

Yesterday the upload files form on our Dataverse (v. 4.6) stopped working. We tested with different types of relatively small files, trying to upload them one at a time. In the logs (below) it looks like an error with the form's action.

[2017-02-22T11:22:04.842-0600] [glassfish 4.1] [WARNING] [] [javax.enterprise.web.core] [tid: _ThreadID=52 _ThreadName=jk-connector(5)] [timeMillis: 1487784124842] [levelValue: 900] [[
  Error invoking requestInitialized method on ServletRequestListener org.jboss.weld.servlet.WeldListener
java.lang.RuntimeException: Error in multipart initialization
        at org.apache.catalina.fileupload.Multipart.init(Multipart.java:99)
...
Caused by: java.io.FileNotFoundException: [dataverse_path]/generated/jsp/dataverse-4.6/upload_374bf8bb_15a4490fe1f__75d7_00000014.tmp (No such file or directory)

After we rebooted the VM, file uploads started working again. Any possible explanations for this behavior? One other thing, in the logs I see an occasional JHOVE config error.

[2017-02-22T14:36:09.834-0600] [glassfish 4.1] [SEVERE] [] [] [tid: _ThreadID=50 _ThreadName=Thread-9] [timeMillis: 1487795769834] [levelValue: 1000] [[
  [Error] jhove.conf:2:14: cvc-elt.1: Cannot find the declaration of element 'jhoveConfig'.]]

Any chance this is related? In config/jhove.conf, the directory specified in 'jhoveHome' doesn't exist. Should we have installed JHOVE as part of the Dataverse installation process?

I looked at issues #2619 and #951, but I'm not sure how they were eventually resolved.

[djbrooke]

@nwoodward thanks for the detailed info. Let me do some checking and I'll get you an answer ASAP.

[landreev]

No, I don't think your upload issues were related to the JHOVE errors.

These JHOVE errors are not fatal. This message means that there's something wrong with JHOVE configuration. And that in turn means that your Dataverse cannot use JHOVE for detecting content types of the files you are uploading - but it still tries to determine the type, it just ends up being less accurate.

How to fix:

No, you don't need to install JHOVE separately. It is just a library that the Dataverse application is built with. But it has its own configuration. I'm guessing this Dataverse installation is an older version that was upgraded to 4.6, correct?
Please copy the most recent jhove.conf and jhoveConfig.xsd from https://github.com/IQSS/dataverse/tree/master/conf/jhove into your <GLASSFISH DOMAIN>/config directory. Then you may need to edit the jhove.conf file. Don't worry about the jhoveHome; but please check the "xsi:schemaLocation= ... file:" line. And change it to correctly point to the jhoveConfig.xsd file you just installed, if your Glassfish domain is not in /usr/local/glassfish4/glassfish/domains/domain1.

(this config file was changed 2 or 3 releases back; for newer installations the above would be done by our installer script automatically. however, older installations needed to apply this fix manually. we published an upgrade instruction - but it may have ended up buried in the release notes...)

[landreev]

As for the main issue you reported - upload just stopped working - unfortunately, I can't immediately think of an explanation. The error message just says that the file failed to upload - but with no clues as to why...

[nwoodward]

Thank you both for the help. You are correct that this instance was an older version of Dataverse that has been upgraded a few times and is now at 4.6. We'll update our JHOVE configuration since it can't hurt, and as for the failing uploads we'll just keep an eye on things and speak up if it happens again and we see more information in the logs.

[pdurbin]

@nwoodward any news? Should we leave this issue open? You could always open a fresh one if the problem resurfaces.

[nwoodward]

We've been monitoring the issue, and we haven't seen uploads fail again. There's still no explanation for how it happened, but if we see it again I can open a new issue and hopefully include more information. Please feel free to close this one. Thanks!

[pdurbin]

@nwoodward thanks. @djbrooke and I discussed this issue this morning and went ahead and closed it. Please do open a new issue if it resurfaces.

[qqmyers]

FWIW - we just saw the same RuntimeException at QDR on v4.10.1 - although in our case the underlying cause was org.jboss.weld.context.ContextNotActiveException: WELD-001303: No active contexts for scope type javax.enterprise.context.SessionScoped. We had to kill/restart glassfish to recover. My guess is that this was somehow related to an old browser window, but so far we can't reproduce it. (I don't think it's worth a new issue but thought I'd add a note here since this issue came up when I searched on the exception...)

[pdurbin]

this issue came up when I searched on the exception

Me too. I just searched for "java.lang.RuntimeException: Error in multipart initialization at org.apache.catalina.fileupload.Multipart.init" and this issue was the first hit. The current problem we're seeing is captured in a couple places:

• 400: Bad Request when uploading file: https://groups.google.com/g/dataverse-community/c/ZEZ5tgEoQrk/m/Tbl8029qDAAJ
• https://help.hmdc.harvard.edu/Ticket/Display.html?id=292329 (internal ticket)

In the current problem the "caused by" is different. It's this:

Caused by: java.io.IOException: java.lang.InterruptedException at org.glassfish.grizzly.nio.transport.TCPNIOTransportFilter.handleRead(TCPNIOTransportFilter.java:92)