Configuration for Availability of "Download All Dataset"

[djbrooke]

The "Download All Dataset" feature introduced to the Dataverse Software in 5.0 has been working great on the Harvard Dataverse Repository for the last several months. :) However, we should consider that:

• On the Harvard Dataverse Repository we in parallel activated the zipper external service to move the zipping off the application server, which means we haven't run into system stability issues. Not every installation will enable this service.
• Because of improvements in large data uploads, we're now seeing some datasets that are pretty big. See Ryan Enos’ recent dataset at 100+ GB (https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/A40X5L). Providing a good user experience around 100+ GB Zip downloads through the UI may be difficult, and there may be cost considerations for some installations.

A proposal surfaced in #7557 is to use the :ZipDownloadLimit setting to control whether or not the Download All option is available. I think this could serve the two use cases of:

• As a Dataverse installation administrator I want to provide download options that are more suited for my infrastructure/cost
• As a Dataverse installation downloader I want to only be presented with a "Download All" option when I'm going to get a zip of all of the files

I think we should move forward with this.

Three things to note:

• The UI impact here would be to either not have the button, or provide the button and some messaging about why the zip cannot be downloaded. To discuss.
• after implementing this, we'll still have the issue of incomplete zips (with manifest.txt showing the missing files) when someone goes through and selects a number of files that are above the :ZipDownloadLimit, but it won't happen with the prominently featured "Download All" option at the top of the page. I don't want to solve this right now.
• that the :ZipDownloadLimit is ignored when the zipper service is available, so in those installations running the zipper service this would solely be a setting that controls the availability of Download All. I considered suggesting a new option to be more specific, but I'll let more technical people discuss the best approach.

[djbrooke]

• the option to show an informative message about why the option is not available is more desirable than not showing the button (we should figure this out first)
• we have the dataset size easily available, so from a code perspective this should be easy
• we should apply these changes to the API as well - an error code and a message. Note that the crawlable access can provide this functionality for API users.

[TaniaSchlatter]

Use cases and options for messaging in the Access dropdown and in a popup. See the Design doc