Download All Error Reporting - Hidden in Manifest if not all files are downloaded (4.9.4)

[CCMumma]

The TDL's Texas Data Repository is on v 4.9.4.

Jessica Trelogan, Research Data Services Coordinator from the TDL's member UT Austin, "noticed the other day that when you select the download all option to get all the files at once from a given dataset, you may not actually get ALL the files, but you won’t know unless you dig into the MANIFEST.txt file that gets bundled with the zipped up files you downloaded. No error. No notice that something didn’t get included, except for a line like this inside the manifest: “Interstates.dbf (application/octet-stream) skipped because the total size of the download bundle exceeded the limit of 104857600 bytes.”

Try it yourself with this dataset:
https://dataverse.tdl.org/dataset.xhtml?persistentId=doi:10.18738/T8/B62UHN (https://dataverse.tdl.org/dataset.xhtml?persistentId=doi:10.18738/T8/B62UHN)
I think this is not a feature (or if so, is not awesome), and should probably get reported to DV community."

Thanks in advance for any help, and I've attached the manifest she refers to.
MANIFEST.TXT

[pdurbin]

@CCMumma thanks for the bug report. Unfortunately, this is a long standing usability issue. Please take a look at #4576 for some ideas that have been suggested to address this. A picture is worth a thousand words so here's a mockup of a warning that could be displayed, perhaps:

[CCMumma]

That warning looks like it would be useful, for sure. I'll share with the member and get some of their feedback and add to this issue.

[djbrooke]

I closed #4576 as a duplicate of this but I wanted to move over two important comments from @landreev:

"See some discussion in #4000 on returning http code 207 ("incomplete"). @matthew-a-dunlap has a working implementation of it in that branch; but being able to return this code comes at a price of having to run a full permission check before we can produce any output."

"To make sure everybody is aware of/remembers this: we already have a UI implementation of warning the UI user, in a nice popup, if any of the files they have selected for a batch download have to be dropped because of permissions; so in order to implement this, we simply need to add the extra checks for size to this already existing framework of checks and notification. Meaning, this is an easier task, than creating this functionality from scratch."

@CCMumma - any feedback from the TDL member?

[pdurbin]

@TaniaSchlatter @mheppler I just noticed @LauraHuisintveld asking a related question at https://groups.google.com/d/msg/dataverse-community/IN-ap31FcDU/H9S4C6poBQAJ

"I received a request from our DataverseNL users if it would be possible to display the total size of a dataset in on the dataset page. I can imagine this would be useful information if you are planning to download all the files of a dataset at once.

I know there are plans to update the dataset page, but does that include this suggestion? I did not see it in these mock-ups: https://sketch.cloud/s/PWk71/a/REaedl . (Found this link on the dataverse.org Roadmap page)."

[pdurbin]

This bug was just reported here:

• DownloadAll in GUI drops files DownloadAll in GUI drops files #6454

In other news, there is a new plan to some day show the size in MB or GB or whatever on the dataset landing page:

• Display the total size of a dataset on the dataset page Display the total size of a dataset on the dataset page #6400

[djbrooke]

Hey @CCMumma and all. With the release of #7714/#7806 in 5.5 we'll handle this much better. The user will be directed to other download options and will not encounter a manifest.txt. I'll close this out.