As someone kicking the tires on Dataverse, I'd like to play around with QDR's data file previewers

[pdurbin]

These data file previewers look awesome (thanks @qqmyers !!):

Installation instructions can be found at https://github.com/QualitativeDataRepository/dataverse-previewers

It would be great to be able to enable these from main.yml, especially when spinning up Dataverse instances on EC2 from the script at http://guides.dataverse.org/en/4.11/developers/deployment.html

[qqmyers]

I'm not familiar with the EC2 process, but if you can run the curl commands, or insert the equivalent sql statements into the externaltool table, that should be all that's required.

[pdurbin]

@qqmyers IQSS maintains a script called ec2-create-instance.sh that you're welcome to take a look at and hack on. The current version is at https://github.com/IQSS/dataverse/blob/v4.11/scripts/installer/ec2-create-instance.sh and it can be downloaded from that "deployment" page in the dev guide that I linked above.

The script leans very heavily on dataverse-ansible to install and configure Dataverse. @donsizemore has been an absolute prince when in comes to our many feature requests but especially useful was when we added Counter Processor support for Make Data Count in 50d445c. It was a huge help with out dev and QA effort. I don't know if you ever have a desire to spin up Dataverse with some sample users and data but we (ok, almost entirely Don) has been working away at IQSS/dataverse#5235 and you're welcome to weigh in with any thoughts or code on anything going on with this dataverse-ansible repo.

[pdurbin]

I just ran this curl command (from the dataverse-previewers README) on my "phoenix" server:

curl -X POST -H 'Content-type: application/json' http://localhost:8080/api/admin/externalTools -d \ "{ \"displayName\":\"Read Text\", \"description\":\"Read the text file.\", \"type\":\"explore\", \"toolUrl\":\"https://qualitativedatarepository.github.io/dataverse-previewers/previewers/TextPreview.html\", \"toolParameters\": { \"queryParameters\":[ {\"fileid\":\"{fileId}\"}, {\"siteUrl\":\"{siteUrl}\"}, {\"key\":\"{apiToken}\"}, {\"datasetid\":\"{datasetId}\"}, {\"datasetversion\":\"{datasetVersion}\"} ] }, \"contentType\":\"text/plain\" }"

The button showed up (screenshots below) but I think this issue will be blocked by the lack of valid SSL certs on these EC2 instances (#53).

[pdurbin]

Actually, if you just install the previewers locally instead of using the ones on https://qualitativedatarepository.github.io they seem to work fine on non-SSL servers.

I did this on the phoenix server:

cd /var/www/html

git clone https://github.com/QualitativeDataRepository/dataverse-previewers.git

vim /etc/httpd/conf.d/phoenix.dataverse.org.conf (add ProxyPassMatch ^/dataverse-previewers !)

service httpd restart

curl http://localhost:8080/api/admin/externalTools/1 -X DELETE

curl -X POST -H 'Content-type: application/json' http://localhost:8080/api/admin/externalTools -d \ "{ \"displayName\":\"Read Text\", \"description\":\"Read the text file.\", \"type\":\"explore\", \"toolUrl\":\"http://phoenix.dataverse.org/dataverse-previewers/previewers/TextPreview.html\", \"toolParameters\": { \"queryParameters\":[ {\"fileid\":\"{fileId}\"}, {\"siteUrl\":\"{siteUrl}\"}, {\"key\":\"{apiToken}\"}, {\"datasetid\":\"{datasetId}\"}, {\"datasetversion\":\"{datasetVersion}\"} ] }, \"contentType\":\"text/plain\" }" {"status":"OK","data":{"id":2,"displayName":"Read Text","description":"Read the text file.","type":"explore","toolUrl":"http://phoenix.dataverse.org/dataverse-previewers/previewers/TextPreview.html","toolParameters":"{\"queryParameters\":[{\"fileid\":\"{fileId}\"},{\"siteUrl\":\"{siteUrl}\"},{\"key\":\"{apiToken}\"},{\"datasetid\":\"{datasetId}\"},{\"datasetversion\":\"{datasetVersion}\"}]}","contentType":"text/plain"}}

And now the "text previewer" works fine:

[qqmyers]

@pdurbin - it looks like the error you showed above was for mixed content - trying to go to http Dataverse URLs from an https page. It looks like github.io always redirects to https these days (see https://help.github.com/en/articles/securing-your-github-pages-site-with-https) so I think you'd have to serve Dataverse over https to match that. Since the ajax and other calls from the previewers are all from the browser (no proxies/other servers), I think if your browser accepts your server's cert, allowing you to see your main site, the previewer calls to your dataverse would work too. (Actually I know this is true because that's how qdr's dev server works - I have to accept its self-signed cert to see the Dataverse, and then the previewers work fine since they are calling the now-trusted server.)

[pdurbin]

@qqmyers thanks. Yeah, https://dev1.dataverse.org has a valid cert but I need to upgrade from postgres 8.4 on it in order to run Dataverse 4.12 or newer. I was thinking about trying out the previewers there once I fix that server up. I could also put a valid cert on phoenix and configure it to load up the previewers on every build (I drop the database on each build, hence the name).

As I mentioned, I also opened #53 for some day get valid certs on the EC2 instances we spin up with dataverse-ansible. (dev1 and phoenix run downstairs on VMWare, not on AWS.)

[qqmyers]

@pdurbin - I don't think you need a valid cert though - just use https rather than http. Once you accept the self-signed cert in your browser to get to https://phoenix.dataverse.org, the previewers on github.io should work. If having http://phoenix.dataverse.org also be available is confusing, maybe it would be better to block/redirect that. (https everywhere!)

[pdurbin]

@qqmyers oh, sure. I just despite browser warnings about invalid certs (self signed or whatever) so I try not to use them.

[pdurbin]

@qqmyers is going to add the new previewers to the external tools list in IQSS/dataverse#5738

@donsizemore I've been meaning to tell you that they're working in ec2 but I believe I was seeing duplicates in the JSON output when I list tools. Two of the text previewer for example. I'd like us to double check this before closing this issue.

[donsizemore]

@pdurbin I think you're seeing duplicated description elements due to multiple contentTypes: "Play an audio file" goes for both audio/mp3 and audio/ogg. The externaltool entries in the database look correct to me.

[pdurbin]

Whoops! You're right. Thanks!

[qqmyers]

FWIW - I thought it might be a good update to allow tools to register for multiple mimetypes, versus having to have multiple registrations of the same tool but didn't get to implementing that. Previewers might be the only use case, but if there are others, perhaps this can help...

[pdurbin]

@qqmyers good idea but it would require a change in Dataverse itself, right?

[pdurbin]

@donsizemore I just wanted to thank you again for adding file previewers to dataverse-ansible. Now we're talking about making use of them as part of IQSS/dataverse#3758 and it's great that it's so easy to get them on EC2.

And thank you again to @qqmyers for creating the file previewers in the first place!