HDF5 files show type "unknown"

[pameyer]

Noticed on https://dataverse.harvard.edu/file.xhtml?fileId=3052745&version=RELEASED&version=.0.
HDF5 is a reasonably widely used file format (https://www.hdfgroup.org/HDF5/)

[pdurbin]

Good idea, @pameyer . Once we detecte the MIME Type, the change to the display is trivial and looks like 7410cc9 but how would we detect the MIME Type? If the file ends in .h5 like the bayestar2017.h5 example above? Does jhove know about HDF5? Dataverse uses a few tricks to determine file types.

[pameyer]

@pdurbin I don't know about jhove; my usual way of checking is to check the extension (.h5 or .hdf5) and see if h5ls produces useful information.

[mheppler]

We will look into this fix as part of the jhove library upgrade in #2202. Will report back here with results of that upgrade.

[mheppler]

@pameyer with #2202 having been merged, I went to the production server, and downloaded a handful of "Unknown" .h5 files, albeit they were all under 1 MB, and not the 4.3 GB behemoth you referenced in your original comment. They all have successfully been identified as "Hierarchical Data Format" files and categorized as "Data" in the facets.

The new /api/files/<FILEID>/redetect API will be available to run on production and correct these unknowns. With your blessing, we can close this issue once those files have been redected with the new API and correctly identified. Will have to coordinate with @kcondon and @landreev on the status of that curation effort after the release.

[djbrooke]

The redetection in prod is captured in #21, we'll run this after deployment.

[landreev]

The new type identification method caught 891 previously unrecognized files with the filename extension 'h5'. They have been re-typed as "application/x-h5". All future uploads of this file type will be properly identified automatically.