failed to clone from demo.dataverse.org

[pdurbin]

Hi! I was just playing around with the example of cloning at https://docs.datalad.org/projects/dataverse/en/latest/tutorial.html#clone-a-dataset-from-dataverse which shows this:

$ datalad clone \
 'datalad-annex::?type=external&externaltype=dataverse&encryption=none&exporttree=no&url=https%3A//demo.dataverse.org&doi=doi:10.70122/FK2/NQPP6A' \
  my-clone
[INFO   ] Remote origin uses a protocol not supported by git-annex; setting annex-ignore
[INFO   ] access to 1 dataset sibling dataverse-storage not auto-enabled, enable with:
|            datalad siblings -d "/tmp/my-clone" enable -s dataverse-storage
install(ok): /tmp/tmp/my-clone-of-annex-mode (dataset)

Since https://demo.dataverse.org/dataset.xhtml?persistentId=doi:10.70122/FK2/NQPP6A is still around I was hoping I'd be able to run the command above as-is to clone the dataset but I'm getting this error:

$ datalad clone \
>  'datalad-annex::?type=external&externaltype=dataverse&encryption=none&exporttree=no&url=https%3A//demo.dataverse.org&doi=doi:10.70122/FK2/NQPP6A' \
>   my-clone
install(error): /Users/pdurbin/distribits/dl-dv/my-clone (dataset) [Failed to clone from any candidate source URL. Encountered errors per each url were:
- datalad-annex::?type=external&externaltype=dataverse&encryption=none&exporttree=no&url=https%3A//demo.dataverse.org&doi=doi:10.70122/FK2/NQPP6A
  CommandError: 'git -c diff.ignoreSubmodules=none -c core.quotepath=false clone --progress 'datalad-annex::?type=external&externaltype=dataverse&encryption=none&exporttree=no&url=https%3A//demo.dataverse.org&doi=doi:10.70122/FK2/NQPP6A' /Users/pdurbin/distribits/dl-dv/my-clone' failed with exitcode 128 [err: 'Cloning into '/Users/pdurbin/distribits/dl-dv/my-clone'...
fatal: TypeError('<' not supported between instances of 'NoneType' and 'str')']]

Am I doing something wrong? Am I misunderstanding the docs? I'll ask at Distribits! Thanks!

I'm using datalad 0.19.6 and the Dataverse demo site is running Dataverse 6.1.

[welcome]

Hi! 👋 We are happy that you opened your first issue here! 😄 If you haven't done so already, please make sure you check out our Code of Conduct.

[pdurbin]

Since https://demo.dataverse.org/dataset.xhtml?persistentId=doi:10.70122/FK2/NQPP6A is still around

As @bpoldrack pointed out is dataset is gone, a 404, so he created a new one ( https://demo.dataverse.org/dataset.xhtml?persistentId=doi:10.70122/FK2/2ZPHY7 ) and I was able to clone it BUT I had to make sure I had the following extensions configured (this should be added to the tutorial somwhere, please!):

(venv) HMDC-beamish:dataset.xhtml pdurbin$ tail -3 ~/.gitconfig 
[datalad "extensions"]
	load = next
	load = dataverse

Then I was able to follow the tutorial and clone like this:

(venv) HMDC-beamish:distribits pdurbin$ datalad clone https://demo.dataverse.org/dataset.xhtml?persistentId=doi:10.70122/FK2/2ZPHY7
[INFO   ] Remote origin uses a protocol not supported by git-annex; setting annex-ignore                   
[INFO   ] access to 1 dataset sibling dataverse-storage not auto-enabled, enable with:                     
| 		datalad siblings -d "/Users/pdurbin/distribits/dataset.xhtml" enable -s dataverse-storage 
install(ok): /Users/pdurbin/distribits/dataset.xhtml (dataset)
(venv) HMDC-beamish:distribits pdurbin$ 
(venv) HMDC-beamish:distribits pdurbin$ datalad siblings -d "/Users/pdurbin/distribits/dataset.xhtml" enable -s dataverse-storage
.: dataverse-storage(?) [git]
(venv) HMDC-beamish:distribits pdurbin$ 
(venv) HMDC-beamish:distribits pdurbin$ cd dataset.xhtml/
(venv) HMDC-beamish:dataset.xhtml pdurbin$ ls
some
(venv) HMDC-beamish:dataset.xhtml pdurbin$ ls -al
total 8
drwxr-xr-x   6 pdurbin  staff  192 Apr  6 11:17 .
drwxr-xr-x   6 pdurbin  staff  192 Apr  6 11:16 ..
drwxr-xr-x   4 pdurbin  staff  128 Apr  6 11:17 .datalad
drwxr-xr-x  14 pdurbin  staff  448 Apr  6 11:17 .git
-rw-r--r--   1 pdurbin  staff   55 Apr  6 11:17 .gitattributes
lrwxr-xr-x   1 pdurbin  staff  108 Apr  6 11:17 some -> .git/annex/objects/5X/Xg/MD5E-s4--3cd7a0db76ff9dca48979e24c39b408c/MD5E-s4--3cd7a0db76ff9dca48979e24c39b408c
(venv) HMDC-beamish:dataset.xhtml pdurbin$ file some 
some: broken symbolic link to .git/annex/objects/5X/Xg/MD5E-s4--3cd7a0db76ff9dca48979e24c39b408c/MD5E-s4--3cd7a0db76ff9dca48979e24c39b408c
(venv) HMDC-beamish:dataset.xhtml pdurbin$ 
(venv) HMDC-beamish:dataset.xhtml pdurbin$ datalad get some
get(ok): some (file) [from dataverse-storage...]                                                           
(venv) HMDC-beamish:dataset.xhtml pdurbin$ file some 
some: ASCII text
(venv) HMDC-beamish:dataset.xhtml pdurbin$ cat some 
bla

[yarikoptic]

I went through the same rabbit hole, just through a different path to get to darkness.

Could it be avoided to need for manual datalad.extensions setup whenever they should be auto-discovered through entrypoints? I "appreciate" that there might be need for an order of loading of extensions, but IMHO it should be solved then programmatically.