Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failure to clone on Windows - slow file system? #337

Open
adswa opened this issue Oct 31, 2024 · 8 comments
Open

Failure to clone on Windows - slow file system? #337

adswa opened this issue Oct 31, 2024 · 8 comments

Comments

@adswa
Copy link
Member

adswa commented Oct 31, 2024

I tried the JülichData tutorial on Windows, and found that the cloning of the example dataset was seemingly stuck. A possible hint was visible when I git clone'ed and interrupted the hanging process:

C:\Users\mih\TMP\my-dataset>git clone "datalad-annex::?type=external&externaltype=dataverse&encryption=none&exporttree=yes&url=https://demo.dataverse.org/&doi=doi:10.70122/FK2/KXGHCC" testingk
12:13:36.900291 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
12:13:36.900291 git.c:479               trace: built-in: git clone 'datalad-annex::?type=external&externaltype=dataverse&encryption=none&exporttree=yes&url=https://demo.dataverse.org/&doi=doi:10.70122/FK2/KXGHCC' testingk
Cloning into 'testingk'...
12:13:36.938082 run-command.c:667       trace: run_command: git remote-datalad-annex origin '?type=external&externaltype=dataverse&encryption=none&exporttree=yes&url=https://demo.dataverse.org/&doi=doi:10.70122/FK2/KXGHCC'
12:13:36.938082 run-command.c:928       trace: start_command: git remote-datalad-annex origin '?type=external&externaltype=dataverse&encryption=none&exporttree=yes&url=https://demo.dataverse.org/&doi=doi:10.70122/FK2/KXGHCC'
12:13:36.953730 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
12:13:36.969355 git.c:783               trace: exec: git-remote-datalad-annex origin '?type=external&externaltype=dataverse&encryption=none&exporttree=yes&url=https://demo.dataverse.org/&doi=doi:10.70122/FK2/KXGHCC'
12:13:36.969355 run-command.c:667       trace: run_command: git-remote-datalad-annex origin '?type=external&externaltype=dataverse&encryption=none&exporttree=yes&url=https://demo.dataverse.org/&doi=doi:10.70122/FK2/KXGHCC'
12:13:36.969355 run-command.c:928       trace: start_command: git-remote-datalad-annex origin '?type=external&externaltype=dataverse&encryption=none&exporttree=yes&url=https://demo.dataverse.org/&doi=doi:10.70122/FK2/KXGHCC'
[INFO] 12:13:39.210060 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.213056 git.c:783               trace: exec: git-annex init -c annex.dotfiles=true
[INFO] 12:13:39.213056 run-command.c:667       trace: run_command: git-annex init -c annex.dotfiles=true
| 12:13:39.213056 run-command.c:928       trace: start_command: git-annex init -c annex.dotfiles=true
[INFO] 12:13:39.278087 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.282087 git.c:479               trace: built-in: git config --null --list
[INFO] 12:13:39.333964 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.336968 git.c:479               trace: built-in: git show-ref --hash refs/heads/git-annex
[INFO] 12:13:39.350967 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.352966 git.c:479               trace: built-in: git show-ref --verify -q refs/remotes/origin/git-annex
[INFO] 12:13:39.367974 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.370977 git.c:479               trace: built-in: git update-index -z --index-info
[INFO] 12:13:39.384972 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.386970 git.c:479               trace: built-in: git write-tree
[INFO] 12:13:39.406918 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.408921 git.c:479               trace: built-in: git commit-tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904 --no-gpg-sign -m 'branch created'
[INFO] 12:13:39.424862 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.426862 git.c:479               trace: built-in: git update-ref refs/heads/git-annex 80c539149f3e30a05c3c163dd59b98e952ba5789
[INFO] 12:13:39.449864 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.451871 git.c:479               trace: built-in: git config annex.uuid 244e03d6-d211-4d83-81fe-263ed5f29570
[INFO] 12:13:39.467865 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.469866 git.c:479               trace: built-in: git config --null --list
[INFO] Detected a filesystem without fifo support.
[INFO] Disabling ssh connection caching.
[INFO] 12:13:39.484863 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.486864 git.c:479               trace: built-in: git config annex.sshcaching false
[INFO] 12:13:39.500867 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.503864 git.c:479               trace: built-in: git config --null --list
[INFO] Detected a crippled filesystem.
[INFO] 12:13:39.518867 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.520863 git.c:479               trace: built-in: git config annex.crippledfilesystem true
[INFO] 12:13:39.535869 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.539862 git.c:479               trace: built-in: git config --null --list
[INFO] 12:13:39.553861 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.555873 git.c:479               trace: built-in: git show-ref git-annex
[INFO] 12:13:39.568860 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.570861 git.c:479               trace: built-in: git show-ref --hash refs/heads/git-annex
[INFO] 12:13:39.584861 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.587868 git.c:479               trace: built-in: git log refs/heads/git-annex..80c539149f3e30a05c3c163dd59b98e952ba5789 '--pretty=%H' -n1
[INFO] 12:13:39.610876 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.612865 git.c:479               trace: built-in: git hash-object -w --no-filters --stdin-paths
[INFO] 12:13:39.619872 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.622868 git.c:479               trace: built-in: git cat-file --batch
[INFO] 12:13:39.623868 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.627863 git.c:479               trace: built-in: git update-index -z --index-info
[INFO] 12:13:39.632869 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.635862 git.c:479               trace: built-in: git diff-index --raw -z -r --no-renames -l0 --cached refs/heads/git-annex --
[INFO] 12:13:39.661867 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.663872 git.c:479               trace: built-in: git config annex.version 10
[INFO] 12:13:39.683872 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
[INFO] 12:13:39.688863 git.c:479               trace: built-in: git config --null --list
Deletion of directory 'testingk/.git/dl-repoannex/origin/repoannex' failed. Should I try again? (y/n) Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-remote-datalad-annex.exe\__main__.py", line 7, in <module>
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad_next\gitremotes\datalad_annex.py", line 1311, in main
    remote.communicate()
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad_next\gitremotes\datalad_annex.py", line 765, in communicate
    self.get_remote_refs(raise_on_error=True)
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad_next\gitremotes\datalad_annex.py", line 992, in get_remote_refs
    ra = self.repoannex
         ^^^^^^^^^^^^^^
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad_next\gitremotes\datalad_annex.py", line 522, in repoannex
    ra.call_annex(
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad\support\annexrepo.py", line 1318, in call_annex
    return self._call_annex(
           ^^^^^^^^^^^^^^^^^
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad\support\annexrepo.py", line 997, in _call_annex
    return runner.run(
           ^^^^^^^^^^^
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad\runner\runner.py", line 225, in run
    results_or_iterator = threaded_runner.run()
                          ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad\runner\nonasyncrunner.py", line 347, in run
    return self._locked_run()
           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad\runner\nonasyncrunner.py", line 520, in _locked_run
    return self.process_loop()
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad\runner\nonasyncrunner.py", line 527, in process_loop
    self.process_queue()
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad\runner\nonasyncrunner.py", line 641, in process_queue
    file_number, state, data = self.output_queue.get(
                               ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\queue.py", line 180, in get
    self.not_empty.wait(remaining)
  File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 338, in wait
    gotit = waiter.acquire(True, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt
Deletion of directory 'testingk/.git/dl-repoannex/origin/repoannex' failed. Should I try again? (y/n)

I'm investigating why the deletion failed. It could be that some process still has the directory or that Windows is so slow that it thinks thats the case, or maybe a permission issue...
@adswa
Copy link
Member Author

adswa commented Oct 31, 2024 •
edited
Loading

It does not look like a permission issue on first sight (I tried equipping the rmtree call in datalad-next's datalad-annex remote with "by-default" chmod'ding and nothing changed - plus there is no difference between a non-admin and admin CMD). It may be a process holding the directory hostage. If I kill the git-annex-remote-dataverse process in the task manager deletion succeeds. The processes holding this directory are

image

@adswa
Copy link
Member Author

adswa commented Oct 31, 2024

ah, sorry, no. The failure to remove is a fluke. It happens somewhere inside, the error is simply a vial attempt to clean up after the CTRL-C crash.

@adswa
Copy link
Member Author

adswa commented Oct 31, 2024

installing 7zip did also not help (though I suspect it is required eventually)

@adswa
Copy link
Member Author

adswa commented Oct 31, 2024

chatter at initremote stalls at credential retrieval


C:\Users\mih\TMP\iopo\.git\dl-repoannex\origin\repoannex>git annex -d -v initremote origin type=external externaltype=dataverse encryption=none url==https://data-beta.fz-juelich.de doi=doi:10.0346/JUELICH-DATA-BETA/IHHXNN
initremote origin [2024-10-31 14:46:19.8282501] (Utility.Process) process [36756] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"]
[2024-10-31 14:46:19.8522554] (Utility.Process) process [36756] done ExitSuccess
[2024-10-31 14:46:19.8602477] (Utility.Process) process [36620] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"]
[2024-10-31 14:46:19.8762605] (Utility.Process) process [36620] done ExitSuccess
[2024-10-31 14:46:19.8922564] (Utility.Process) process [36056] chat: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","hash-object","-w","--no-filters","--stdin-paths"]
[2024-10-31 14:46:19.9002488] (Utility.Process) process [36000] chat: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"]
[2024-10-31 14:46:19.9082473] (Utility.Process) process [2880] feed: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index-info"]
[2024-10-31 14:46:19.9162582] (Utility.Process) process [36492] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","diff-index","--raw","-z","-r","--no-renames","-l0","--cached","refs/heads/git-annex","--"]
[2024-10-31 14:46:19.9403] (Utility.Process) process [36492] done ExitSuccess
[2024-10-31 14:46:19.9403] (Utility.Process) process [2880] done ExitSuccess
[2024-10-31 14:46:19.9482561] (Annex.Branch) read remote.log
[2024-10-31 14:46:19.9562495] (Annex.Branch) read proxy.log
[2024-10-31 14:46:19.9562495] (Annex.Branch) read trust.log
[2024-10-31 14:46:19.9802516] (Utility.Process) process [29900] chat: C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe []
[2024-10-31 14:46:22.2534089] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> VERSION 1
[2024-10-31 14:46:22.2546005] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- EXTENSIONS INFO GETGITREMOTENAME UNAVAILABLERESPONSE ASYNC
[2024-10-31 14:46:22.2574092] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> EXTENSIONS[2024-10-31 14:46:22.2594092] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- LISTCONFIGS
[2024-10-31 14:46:22.2614087] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG credential name of a DataLad credential with a Dataverse API token to use
[2024-10-31 14:46:22.262409] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG doi DOI-style persistent identifier of the Dataverse dataset
[2024-10-31 14:46:22.2634089] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG rootpath optional alternative root path to use in the Dataverse dataset
[2024-10-31 14:46:22.2634089] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG url URL of the Dataverse site
[2024-10-31 14:46:22.2644091] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIGEND
[2024-10-31 14:46:22.2724089] (Utility.Process) process [34340] chat: C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe []
[2024-10-31 14:46:24.486602] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> VERSION 1
[2024-10-31 14:46:24.486602] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- EXTENSIONS INFO GETGITREMOTENAME UNAVAILABLERESPONSE ASYNC
[2024-10-31 14:46:24.486602] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> EXTENSIONS
[2024-10-31 14:46:24.486602] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- EXPORTSUPPORTED
[2024-10-31 14:46:24.5194871] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> EXPORTSUPPORTED-SUCCESS
[2024-10-31 14:46:24.5275202] (Utility.Process) process [37324] chat: C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe []
[2024-10-31 14:46:26.6214733] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> VERSION 1
[2024-10-31 14:46:26.6224663] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- EXTENSIONS INFO GETGITREMOTENAME UNAVAILABLERESPONSE ASYNC
[2024-10-31 14:46:26.6244626] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> EXTENSIONS[2024-10-31 14:46:26.6549717] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- LISTCONFIGS
[2024-10-31 14:46:26.6579495] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG credential name of a DataLad credential with a Dataverse API token to use
[2024-10-31 14:46:26.6589577] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG doi DOI-style persistent identifier of the Dataverse dataset
[2024-10-31 14:46:26.6599579] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG rootpath optional alternative root path to use in the Dataverse dataset
[2024-10-31 14:46:26.6599579] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG url URL of the Dataverse site
[2024-10-31 14:46:26.6609464] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIGEND
[2024-10-31 14:46:26.6619461] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- INITREMOTE[2024-10-31 14:46:26.6619461] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETCONFIG url
[2024-10-31 14:46:26.662946] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE =https://data-beta.fz-juelich.de
[2024-10-31 14:46:26.6639475] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETCONFIG doi
[2024-10-31 14:46:26.6639475] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE doi:10.0346/JUELICH-DATA-BETA/IHHXNN
[2024-10-31 14:46:26.6649458] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETCONFIG rootpath
[2024-10-31 14:46:26.6689483] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE
[2024-10-31 14:46:26.6699547] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETGITDIR
[2024-10-31 14:46:26.6699547] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE .
[2024-10-31 14:46:26.7489599] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETCONFIG credential
[2024-10-31 14:46:26.7489599] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE

@adswa
Copy link
Member Author

adswa commented Oct 31, 2024

ok, giving up for now. My current suspicion is the credential handling. I get a step further if I add ```credential=data-beta-fzj`` to init remote or the datalad-annex url. Whether its a fix I don't yet know - despite providing the correct token to datalad credentials, init remote throws an authentication failure.



C:\Users\mih\TMP\iopoo\.git\dl-repoannex\origin\repoannex>git annex -d -v initremote origin type=external externaltype=dataverse encryption=none url=https://data-beta.fz-juelich.de doi=doi:10.0346/JUELICH-DATA-BETA/IHHXNN credential=data-beta-fzj
initremote origin [2024-10-31 15:08:17.2859862] (Utility.Process) process [30152] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"]
[2024-10-31 15:08:17.3049788] (Utility.Process) process [30152] done ExitSuccess
[2024-10-31 15:08:17.311979] (Utility.Process) process [28204] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"]
[2024-10-31 15:08:17.3359915] (Utility.Process) process [28204] done ExitSuccess
[2024-10-31 15:08:17.3499798] (Utility.Process) process [31820] chat: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","hash-object","-w","--no-filters","--stdin-paths"]
[2024-10-31 15:08:17.3569955] (Utility.Process) process [32448] chat: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"]
[2024-10-31 15:08:17.3659852] (Utility.Process) process [31240] feed: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index-info"]
[2024-10-31 15:08:17.3770019] (Utility.Process) process [33748] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","diff-index","--raw","-z","-r","--no-renames","-l0","--cached","refs/heads/git-annex","--"]
[2024-10-31 15:08:17.402991] (Utility.Process) process [33748] done ExitSuccess
[2024-10-31 15:08:17.4079762] (Utility.Process) process [31240] done ExitSuccess
[2024-10-31 15:08:17.4169903] (Annex.Branch) read remote.log
[2024-10-31 15:08:17.419992] (Annex.Branch) read proxy.log
[2024-10-31 15:08:17.421994] (Annex.Branch) read trust.log
[2024-10-31 15:08:17.4479829] (Utility.Process) process [31464] chat: C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe []
[2024-10-31 15:08:19.7525621] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> VERSION 1
[2024-10-31 15:08:19.7525621] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- EXTENSIONS INFO GETGITREMOTENAME UNAVAILABLERESPONSE ASYNC
[2024-10-31 15:08:19.7545822] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> EXTENSIONS
[2024-10-31 15:08:19.7555402] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- LISTCONFIGS
[2024-10-31 15:08:19.7915205] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG credential name of a DataLad credential with a Dataverse API token to use
[2024-10-31 15:08:19.7955321] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG doi DOI-style persistent identifier of the Dataverse dataset
[2024-10-31 15:08:19.7975327] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG rootpath optional alternative root path to use in the Dataverse dataset
[2024-10-31 15:08:19.7985184] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG url URL of the Dataverse site
[2024-10-31 15:08:19.7995342] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIGEND
[2024-10-31 15:08:19.8065335] (Utility.Process) process [32988] chat: C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe []
[2024-10-31 15:08:22.3345151] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> VERSION 1
[2024-10-31 15:08:22.3345151] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- EXTENSIONS INFO GETGITREMOTENAME UNAVAILABLERESPONSE ASYNC
[2024-10-31 15:08:22.3377687] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> EXTENSIONS
[2024-10-31 15:08:22.3742578] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- EXPORTSUPPORTED
[2024-10-31 15:08:22.3742578] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> EXPORTSUPPORTED-SUCCESS
[2024-10-31 15:08:22.3957719] (Utility.Process) process [34112] chat: C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe []
[2024-10-31 15:08:24.3739557] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> VERSION 1
[2024-10-31 15:08:24.3739557] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- EXTENSIONS INFO GETGITREMOTENAME UNAVAILABLERESPONSE ASYNC
[2024-10-31 15:08:24.3773166] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> EXTENSIONS
[2024-10-31 15:08:24.3773166] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- LISTCONFIGS
[2024-10-31 15:08:24.3813546] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG credential name of a DataLad credential with a Dataverse API token to use
[2024-10-31 15:08:24.3813546] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG doi DOI-style persistent identifier of the Dataverse dataset
[2024-10-31 15:08:24.3823616] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG rootpath optional alternative root path to use in the Dataverse dataset
[2024-10-31 15:08:24.3833621] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG url URL of the Dataverse site
[2024-10-31 15:08:24.384364] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIGEND
[2024-10-31 15:08:24.3853616] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- INITREMOTE
[2024-10-31 15:08:24.3873563] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETCONFIG url
[2024-10-31 15:08:24.3883595] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE https://data-beta.fz-juelich.de
[2024-10-31 15:08:24.3893712] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETCONFIG doi
[2024-10-31 15:08:24.4054932] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE doi:10.0346/JUELICH-DATA-BETA/IHHXNN
[2024-10-31 15:08:24.4092051] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETCONFIG rootpath
[2024-10-31 15:08:24.410248] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE
[2024-10-31 15:08:24.4122259] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETGITDIR
[2024-10-31 15:08:24.4132282] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE .
[2024-10-31 15:08:24.5220632] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETCONFIG credential
[2024-10-31 15:08:24.5220632] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE data-beta-fzj
[2024-10-31 15:08:25.375985] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG Traceback (most recent call last):
[2024-10-31 15:08:25.3949969] (Remote.External) Traceback (most recent call last):
[2024-10-31 15:08:25.3949969] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\annexremote\annexremote.py", line 873, in Listen
[2024-10-31 15:08:25.3970026] (Remote.External)   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\annexremote\annexremote.py", line 873, in Listen
[2024-10-31 15:08:25.3979952] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG     reply = self.protocol.command(line)
[2024-10-31 15:08:25.3990043] (Remote.External)     reply = self.protocol.command(line)
[2024-10-31 15:08:25.3999954] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-31 15:08:25.4019866] (Remote.External)             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-31 15:08:25.40299] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\annexremote\annexremote.py", line 552, in command
[2024-10-31 15:08:25.4049903] (Remote.External)   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\annexremote\annexremote.py", line 552, in command
[2024-10-31 15:08:25.4059909] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG     reply = method()
[2024-10-31 15:08:25.4069891] (Remote.External)     reply = method()
[2024-10-31 15:08:25.4080112] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG             ^^^^^^^^
[2024-10-31 15:08:25.4089892] (Remote.External)             ^^^^^^^^
[2024-10-31 15:08:25.4109927] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\annexremote\annexremote.py", line 574, in do_INITREMOTE
[2024-10-31 15:08:25.4199873] (Remote.External)   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\annexremote\annexremote.py", line 574, in do_INITREMOTE
[2024-10-31 15:08:25.4209919] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG     self.remote.initremote()
[2024-10-31 15:08:25.4219903] (Remote.External)     self.remote.initremote()
[2024-10-31 15:08:25.4239963] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad_dataverse\baseremote.py", line 130, in initremote
[2024-10-31 15:08:25.4260058] (Remote.External)   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad_dataverse\baseremote.py", line 130, in initremote
[2024-10-31 15:08:25.4269998] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG     self.prepare()
[2024-10-31 15:08:25.4279981] (Remote.External)     self.prepare()
[2024-10-31 15:08:25.4289974] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad_dataverse\baseremote.py", line 119, in prepare
[2024-10-31 15:08:25.4299983] (Remote.External)   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad_dataverse\baseremote.py", line 119, in prepare
[2024-10-31 15:08:25.4319897] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG     self._dvds = OnlineDataverseDataset(api, doi, root_path=dv_root_path)
[2024-10-31 15:08:25.432991] (Remote.External)     self._dvds = OnlineDataverseDataset(api, doi, root_path=dv_root_path)
[2024-10-31 15:08:25.4339901] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-31 15:08:25.4349898] (Remote.External)                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-31 15:08:25.4369869] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad_dataverse\dataset.py", line 90, in __init__
[2024-10-31 15:08:25.4379902] (Remote.External)   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\datalad_dataverse\dataset.py", line 90, in __init__
[2024-10-31 15:08:25.4389887] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG     resp = api.get_info_version()
[2024-10-31 15:08:25.440987] (Remote.External)     resp = api.get_info_version()
[2024-10-31 15:08:25.4419937] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG            ^^^^^^^^^^^^^^^^^^^^^^
[2024-10-31 15:08:25.442989] (Remote.External)            ^^^^^^^^^^^^^^^^^^^^^^
[2024-10-31 15:08:25.4539868] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\pyDataverse\api.py", line 2073, in get_info_version
[2024-10-31 15:08:25.4559883] (Remote.External)   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\pyDataverse\api.py", line 2073, in get_info_version
[2024-10-31 15:08:25.4579902] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG     return self.get_request(url, auth=auth)
[2024-10-31 15:08:25.4589877] (Remote.External)     return self.get_request(url, auth=auth)
[2024-10-31 15:08:25.4599876] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-31 15:08:25.4609885] (Remote.External)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-10-31 15:08:25.4619913] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\pyDataverse\api.py", line 192, in get_request
[2024-10-31 15:08:25.4639864] (Remote.External)   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\pyDataverse\api.py", line 192, in get_request
[2024-10-31 15:08:25.4649893] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG     return self._sync_request(
[2024-10-31 15:08:25.4659908] (Remote.External)     return self._sync_request(
[2024-10-31 15:08:25.4669885] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG            ^^^^^^^^^^^^^^^^^^^
[2024-10-31 15:08:25.4679887] (Remote.External)            ^^^^^^^^^^^^^^^^^^^
[2024-10-31 15:08:25.46999] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\pyDataverse\api.py", line 457, in _sync_request
[2024-10-31 15:08:25.4709894] (Remote.External)   File "C:\Users\mih\AppData\Local\Programs\Python\Python312\Lib\site-packages\pyDataverse\api.py", line 457, in _sync_request
[2024-10-31 15:08:25.4719965] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG     raise ApiAuthorizationError(
[2024-10-31 15:08:25.4739878] (Remote.External)     raise ApiAuthorizationError(
[2024-10-31 15:08:25.4749916] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> DEBUG pyDataverse.exceptions.ApiAuthorizationError: ERROR: HTTP 401 - Authorization error https://data-beta.fz-juelich.de/api/v1/info/version. MSG: Bad api key
[2024-10-31 15:08:25.4860021] (Remote.External) pyDataverse.exceptions.ApiAuthorizationError: ERROR: HTTP 401 - Authorization error https://data-beta.fz-juelich.de/api/v1/info/version. MSG: Bad api key
[2024-10-31 15:08:25.4869878] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> ERROR ERROR: HTTP 401 - Authorization error https://data-beta.fz-juelich.de/api/v1/info/version. MSG: Bad api key

git-annex: external special remote error: ERROR: HTTP 401 - Authorization error https://data-beta.fz-juelich.de/api/v1/info/version. MSG: Bad api key
failed
[2024-10-31 15:08:25.5159847] (Utility.Process) process [32448] done ExitSuccess
[2024-10-31 15:08:25.5219884] (Utility.Process) process [31820] done ExitSuccess
initremote: 1 failed

datalad clone probably swallows the auth failure?


C:\Users\mih\TMP>datalad clone "datalad-annex::?type=external&externaltype=dataverse&encryption=none&exporttree=yes&url=https://demo.dataverse.org/&doi=doi:10.70122/FK2/KXGHCC&credential=data-beta-fzj2" thestingkjj
15:15:06.370026 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
15:15:06.377044 git.c:479               trace: built-in: git describe --tags --dirty --always --long --match '[[:digit:]]*'
15:15:06.502079 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
15:15:06.505053 git.c:479               trace: built-in: git rev-parse HEAD
15:15:06.588037 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
15:15:06.591039 git.c:479               trace: built-in: git rev-parse --abbrev-ref HEAD
15:15:06.683349 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
15:15:06.689350 git.c:479               trace: built-in: git show -s '--format=%ci' HEAD
CommandError: '"git" "-c" "diff.ignoreSubmodules=none" "-c" "core.quotepath=false" "symbolic-ref" "HEAD"' failed with exitcode 128
15:15:07.063623 exec-cmd.c:266          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
15:15:07.068681 git.c:479               trace: built-in: git symbolic-ref HEAD
fatal: No such ref: HEAD

@adswa
Copy link
Member Author

adswa commented Oct 31, 2024

well, still not the end of the story... setting the token by hand works!!! -.- Which makes git annex initremote succeed:

C:\Users\mih\TMP\iopoo\.git\dl-repoannex\origin\repoannex>datalad credentials remove data-beta-fzj type=token
data-beta-fzj(secret ✗):

C:\Users\mih\TMP\iopoo\.git\dl-repoannex\origin\repoannex>datalad credentials set data-beta-fzj type=token
secret:
secret (repeat):
data-beta-fzj(token ✓):

C:\Users\mih\TMP\iopoo\.git\dl-repoannex\origin\repoannex>git annex -d -v initremote origin type=external externaltype=dataverse encryption=none url=https://data-beta.fz-juelich.de doi=doi:10.0346/JUELICH-DATA-BETA/IHHXNN credential=data-beta-fzj
[2024-10-31 15:37:07.2434052] (Utility.Process) process [35320] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"]
[2024-10-31 15:37:07.263424] (Utility.Process) process [35320] done ExitSuccess
[2024-10-31 15:37:07.271399] (Utility.Process) process [31648] read: git ["config","--local","--list"] in "."
[2024-10-31 15:37:07.292397] (Utility.Process) process [31648] done ExitSuccess
[2024-10-31 15:37:07.3124002] (Utility.Process) process [32640] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"]
[2024-10-31 15:37:07.3374092] (Utility.Process) process [32640] done ExitSuccess
[2024-10-31 15:37:07.347413] (Utility.Process) process [30856] call: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","config","annex.uuid","9dc68e98-4038-4525-9b55-10937f814072"]
[2024-10-31 15:37:07.3674105] (Utility.Process) process [30856] done ExitSuccess
[2024-10-31 15:37:07.372403] (Utility.Process) process [31320] read: git ["config","--null","--list"] in "."
[2024-10-31 15:37:07.3904102] (Git.Config) git config read: [("",[""]),("annex.uuid",["9dc68e98-4038-4525-9b55-10937f814072"]),("core.autocrlf",["true"]),("core.fscache",["true"]),("core.symlinks",["false"]),("credential.helper",["manager"]),("credential.https://dev.azure.com.usehttppath",["true"]),("credential.https://gin.g-node.org.provider",["generic"]),("datalad.credential.data-beta-fzj.type",["token"]),("datalad.credential.data-betafzj.type",["token"]),("datalad.credential.gin.g-node.org.last-used",["2022-09-27T17:10:06.894992"]),("datalad.credential.gin.g-node.org.realm",["https://gin.g-node.org"]),("datalad.credential.gin.g-node.org.type",["token"]),("datalad.extensions.load",["dataverse","next"]),("datalad.gooey.active-suite",["gooey-simplified"]),("diff.astextplain.textconv",["astextplain"]),("filter.annex.process",["git-annex filter-process"]),("filter.lfs.clean",["git-lfs clean -- %f"]),("filter.lfs.process",["git-lfs filter-process"]),("filter.lfs.required",["true"]),("filter.lfs.smudge",["git-lfs smudge -- %f"]),("http.sslbackend",["openssl"]),("http.sslcainfo",["C:/Program Files/Git/mingw64/etc/ssl/certs/ca-bundle.crt"]),("init.defaultbranch",["main"]),("pull.rebase",["false"]),("user.email",["michael.hanke@gmail.com"]),("user.name",["Michael Hanke"])]
[2024-10-31 15:37:07.3923983] (Utility.Process) process [31320] done ExitSuccess

  Detected a filesystem without fifo support.

  Disabling ssh connection caching.
[2024-10-31 15:37:07.4114115] (Utility.Process) process [32408] call: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","config","annex.sshcaching","false"]
[2024-10-31 15:37:07.4344111] (Utility.Process) process [32408] done ExitSuccess
[2024-10-31 15:37:07.4393983] (Utility.Process) process [31172] read: git ["config","--null","--list"] in "."
[2024-10-31 15:37:07.4643959] (Git.Config) git config read: [("",[""]),("annex.sshcaching",["false"]),("annex.uuid",["9dc68e98-4038-4525-9b55-10937f814072"]),("core.autocrlf",["true"]),("core.fscache",["true"]),("core.symlinks",["false"]),("credential.helper",["manager"]),("credential.https://dev.azure.com.usehttppath",["true"]),("credential.https://gin.g-node.org.provider",["generic"]),("datalad.credential.data-beta-fzj.type",["token"]),("datalad.credential.data-betafzj.type",["token"]),("datalad.credential.gin.g-node.org.last-used",["2022-09-27T17:10:06.894992"]),("datalad.credential.gin.g-node.org.realm",["https://gin.g-node.org"]),("datalad.credential.gin.g-node.org.type",["token"]),("datalad.extensions.load",["dataverse","next"]),("datalad.gooey.active-suite",["gooey-simplified"]),("diff.astextplain.textconv",["astextplain"]),("filter.annex.process",["git-annex filter-process"]),("filter.lfs.clean",["git-lfs clean -- %f"]),("filter.lfs.process",["git-lfs filter-process"]),("filter.lfs.required",["true"]),("filter.lfs.smudge",["git-lfs smudge -- %f"]),("http.sslbackend",["openssl"]),("http.sslcainfo",["C:/Program Files/Git/mingw64/etc/ssl/certs/ca-bundle.crt"]),("init.defaultbranch",["main"]),("pull.rebase",["false"]),("user.email",["michael.hanke@gmail.com"]),("user.name",["Michael Hanke"])]
[2024-10-31 15:37:07.4664216] (Utility.Process) process [31172] done ExitSuccess

  Detected a crippled filesystem.
[2024-10-31 15:37:07.4811834] (Utility.Process) process [7196] call: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","config","annex.crippledfilesystem","true"]
[2024-10-31 15:37:07.5021774] (Utility.Process) process [7196] done ExitSuccess
[2024-10-31 15:37:07.5071934] (Utility.Process) process [23384] read: git ["config","--null","--list"] in "."
[2024-10-31 15:37:07.5261899] (Git.Config) git config read: [("",[""]),("annex.crippledfilesystem",["true"]),("annex.sshcaching",["false"]),("annex.uuid",["9dc68e98-4038-4525-9b55-10937f814072"]),("core.autocrlf",["true"]),("core.fscache",["true"]),("core.symlinks",["false"]),("credential.helper",["manager"]),("credential.https://dev.azure.com.usehttppath",["true"]),("credential.https://gin.g-node.org.provider",["generic"]),("datalad.credential.data-beta-fzj.type",["token"]),("datalad.credential.data-betafzj.type",["token"]),("datalad.credential.gin.g-node.org.last-used",["2022-09-27T17:10:06.894992"]),("datalad.credential.gin.g-node.org.realm",["https://gin.g-node.org"]),("datalad.credential.gin.g-node.org.type",["token"]),("datalad.extensions.load",["dataverse","next"]),("datalad.gooey.active-suite",["gooey-simplified"]),("diff.astextplain.textconv",["astextplain"]),("filter.annex.process",["git-annex filter-process"]),("filter.lfs.clean",["git-lfs clean -- %f"]),("filter.lfs.process",["git-lfs filter-process"]),("filter.lfs.required",["true"]),("filter.lfs.smudge",["git-lfs smudge -- %f"]),("http.sslbackend",["openssl"]),("http.sslcainfo",["C:/Program Files/Git/mingw64/etc/ssl/certs/ca-bundle.crt"]),("init.defaultbranch",["main"]),("pull.rebase",["false"]),("user.email",["michael.hanke@gmail.com"]),("user.name",["Michael Hanke"])]
[2024-10-31 15:37:07.5281889] (Utility.Process) process [23384] done ExitSuccess
[2024-10-31 15:37:07.5330104] (Utility.Process) process [31112] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"]
[2024-10-31 15:37:07.5570195] (Utility.Process) process [31112] done ExitSuccess
[2024-10-31 15:37:07.5621564] (Utility.Process) process [31400] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"]
[2024-10-31 15:37:07.5821542] (Utility.Process) process [31400] done ExitSuccess
[2024-10-31 15:37:07.5951475] (Utility.Process) process [7464] chat: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","hash-object","-w","--no-filters","--stdin-paths"]
[2024-10-31 15:37:07.6011577] (Utility.Process) process [8364] chat: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"]
[2024-10-31 15:37:07.6081472] (Utility.Process) process [40700] feed: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index-info"]
[2024-10-31 15:37:07.6151568] (Utility.Process) process [31420] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","diff-index","--raw","-z","-r","--no-renames","-l0","--cached","refs/heads/git-annex","--"]
[2024-10-31 15:37:07.6411688] (Utility.Process) process [31420] done ExitSuccess
[2024-10-31 15:37:07.6451598] (Utility.Process) process [40700] done ExitSuccess
[2024-10-31 15:37:07.6731481] (Utility.Process) process [15484] call: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","config","annex.version","10"]
[2024-10-31 15:37:07.6961544] (Utility.Process) process [15484] done ExitSuccess
[2024-10-31 15:37:07.7011579] (Utility.Process) process [20956] read: git ["config","--null","--list"] in "."
[2024-10-31 15:37:07.7211595] (Git.Config) git config read: [("",[""]),("annex.crippledfilesystem",["true"]),("annex.sshcaching",["false"]),("annex.uuid",["9dc68e98-4038-4525-9b55-10937f814072"]),("annex.version",["10"]),("core.autocrlf",["true"]),("core.fscache",["true"]),("core.symlinks",["false"]),("credential.helper",["manager"]),("credential.https://dev.azure.com.usehttppath",["true"]),("credential.https://gin.g-node.org.provider",["generic"]),("datalad.credential.data-beta-fzj.type",["token"]),("datalad.credential.data-betafzj.type",["token"]),("datalad.credential.gin.g-node.org.last-used",["2022-09-27T17:10:06.894992"]),("datalad.credential.gin.g-node.org.realm",["https://gin.g-node.org"]),("datalad.credential.gin.g-node.org.type",["token"]),("datalad.extensions.load",["dataverse","next"]),("datalad.gooey.active-suite",["gooey-simplified"]),("diff.astextplain.textconv",["astextplain"]),("filter.annex.process",["git-annex filter-process"]),("filter.lfs.clean",["git-lfs clean -- %f"]),("filter.lfs.process",["git-lfs filter-process"]),("filter.lfs.required",["true"]),("filter.lfs.smudge",["git-lfs smudge -- %f"]),("http.sslbackend",["openssl"]),("http.sslcainfo",["C:/Program Files/Git/mingw64/etc/ssl/certs/ca-bundle.crt"]),("init.defaultbranch",["main"]),("pull.rebase",["false"]),("user.email",["michael.hanke@gmail.com"]),("user.name",["Michael Hanke"])]
[2024-10-31 15:37:07.7241412] (Utility.Process) process [20956] done ExitSuccess
[2024-10-31 15:37:07.7361403] (Utility.Process) process [18900] read: uname ["-n"]
[2024-10-31 15:37:07.848716] (Utility.Process) process [18900] done ExitSuccess
[2024-10-31 15:37:07.8507161] (Annex.Branch) read uuid.log
[2024-10-31 15:37:07.8597163] (Annex.Branch) set uuid.log
[2024-10-31 15:37:07.8597163] (Annex.Branch) read remote.log
[2024-10-31 15:37:07.863716] (Annex.Branch) read proxy.log
[2024-10-31 15:37:07.8787158] (Utility.Process) process [30036] chat: C:\Program Files\Git\usr\bin\git-annex.exe ["init","-c","annex.debug=true","--autoenable"] in "."
[2024-10-31 15:37:08.3710283] (Utility.Process) process [30036] done ExitSuccess
[2024-10-31 15:37:08.3760282] (Utility.Process) process [184] read: git ["config","--null","--list"] in "."
[2024-10-31 15:37:08.4283137] (Git.Config) git config read: [("",[""]),("annex.crippledfilesystem",["true"]),("annex.sshcaching",["false"]),("annex.uuid",["9dc68e98-4038-4525-9b55-10937f814072"]),("annex.version",["10"]),("core.autocrlf",["true"]),("core.fscache",["true"]),("core.symlinks",["false"]),("credential.helper",["manager"]),("credential.https://dev.azure.com.usehttppath",["true"]),("credential.https://gin.g-node.org.provider",["generic"]),("datalad.credential.data-beta-fzj.type",["token"]),("datalad.credential.data-betafzj.type",["token"]),("datalad.credential.gin.g-node.org.last-used",["2022-09-27T17:10:06.894992"]),("datalad.credential.gin.g-node.org.realm",["https://gin.g-node.org"]),("datalad.credential.gin.g-node.org.type",["token"]),("datalad.extensions.load",["dataverse","next"]),("datalad.gooey.active-suite",["gooey-simplified"]),("diff.astextplain.textconv",["astextplain"]),("filter.annex.process",["git-annex filter-process"]),("filter.lfs.clean",["git-lfs clean -- %f"]),("filter.lfs.process",["git-lfs filter-process"]),("filter.lfs.required",["true"]),("filter.lfs.smudge",["git-lfs smudge -- %f"]),("http.sslbackend",["openssl"]),("http.sslcainfo",["C:/Program Files/Git/mingw64/etc/ssl/certs/ca-bundle.crt"]),("init.defaultbranch",["main"]),("pull.rebase",["false"]),("user.email",["michael.hanke@gmail.com"]),("user.name",["Michael Hanke"])]
[2024-10-31 15:37:08.4303138] (Utility.Process) process [184] done ExitSuccess
initremote origin [2024-10-31 15:37:08.4403139] (Annex.Branch) read trust.log
[2024-10-31 15:37:08.4563136] (Utility.Process) process [14204] chat: C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe []
[2024-10-31 15:37:10.9102492] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> VERSION 1
[2024-10-31 15:37:10.9102492] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- EXTENSIONS INFO GETGITREMOTENAME UNAVAILABLERESPONSE ASYNC
[2024-10-31 15:37:10.9415447] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> EXTENSIONS
[2024-10-31 15:37:10.9415447] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- LISTCONFIGS
[2024-10-31 15:37:10.9575452] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG credential name of a DataLad credential with a Dataverse API token to use
[2024-10-31 15:37:10.9706737] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG doi DOI-style persistent identifier of the Dataverse dataset
[2024-10-31 15:37:10.9706737] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG rootpath optional alternative root path to use in the Dataverse dataset
[2024-10-31 15:37:10.9706737] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG url URL of the Dataverse site
[2024-10-31 15:37:10.9757673] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIGEND
[2024-10-31 15:37:10.9757673] (Utility.Process) process [28708] chat: C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe []
[2024-10-31 15:37:13.1441912] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> VERSION 1
[2024-10-31 15:37:13.1451866] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- EXTENSIONS INFO GETGITREMOTENAME UNAVAILABLERESPONSE ASYNC
[2024-10-31 15:37:13.1481889] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> EXTENSIONS
[2024-10-31 15:37:13.149098] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- EXPORTSUPPORTED
[2024-10-31 15:37:13.1715948] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> EXPORTSUPPORTED-SUCCESS
[2024-10-31 15:37:13.181608] (Utility.Process) process [20348] chat: C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe []
[2024-10-31 15:37:15.4985984] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> VERSION 1
[2024-10-31 15:37:15.4991274] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- EXTENSIONS INFO GETGITREMOTENAME UNAVAILABLERESPONSE ASYNC
[2024-10-31 15:37:15.4991274] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> EXTENSIONS
[2024-10-31 15:37:15.4991274] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- LISTCONFIGS
[2024-10-31 15:37:15.4991274] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG credential name of a DataLad credential with a Dataverse API token to use
[2024-10-31 15:37:15.4991274] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG doi DOI-style persistent identifier of the Dataverse dataset
[2024-10-31 15:37:15.4991274] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG rootpath optional alternative root path to use in the Dataverse dataset
[2024-10-31 15:37:15.5071242] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIG url URL of the Dataverse site
[2024-10-31 15:37:15.5071242] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> CONFIGEND
[2024-10-31 15:37:15.5071242] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- INITREMOTE
[2024-10-31 15:37:15.5071242] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETCONFIG url
[2024-10-31 15:37:15.5071242] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE https://data-beta.fz-juelich.de
[2024-10-31 15:37:15.5071242] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETCONFIG doi
[2024-10-31 15:37:15.5071242] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE doi:10.0346/JUELICH-DATA-BETA/IHHXNN
[2024-10-31 15:37:15.5071242] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETCONFIG rootpath
[2024-10-31 15:37:15.5071242] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE
[2024-10-31 15:37:15.5071242] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETGITDIR
[2024-10-31 15:37:15.5071242] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE .
[2024-10-31 15:37:15.5844644] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> GETCONFIG credential
[2024-10-31 15:37:15.5844644] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] <-- VALUE data-beta-fzj
[2024-10-31 15:37:17.5030666] (Annex.ExternalAddonProcess) C:\Users\mih\AppData\Local\Programs\Python\Python312\Scripts\git-annex-remote-dataverse.exe[1] --> INITREMOTE-SUCCESS
[2024-10-31 15:37:17.5100812] (Utility.Process) process [25028] call: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","config","remote.origin.annex-externaltype","dataverse"]
[2024-10-31 15:37:17.5300667] (Utility.Process) process [25028] done ExitSuccess
[2024-10-31 15:37:17.5370559] (Utility.Process) process [33332] read: git ["config","--null","--list"] in "."
[2024-10-31 15:37:17.5570515] (Git.Config) git config read: [("",[""]),("annex.crippledfilesystem",["true"]),("annex.sshcaching",["false"]),("annex.uuid",["9dc68e98-4038-4525-9b55-10937f814072"]),("annex.version",["10"]),("core.autocrlf",["true"]),("core.fscache",["true"]),("core.symlinks",["false"]),("credential.helper",["manager"]),("credential.https://dev.azure.com.usehttppath",["true"]),("credential.https://gin.g-node.org.provider",["generic"]),("datalad.credential.data-beta-fzj.last-used",["2024-10-31T15:37:17.307989"]),("datalad.credential.data-beta-fzj.realm",["https://data-beta.fz-juelich.de/dataverse"]),("datalad.credential.data-beta-fzj.type",["token"]),("datalad.credential.data-betafzj.type",["token"]),("datalad.credential.gin.g-node.org.last-used",["2022-09-27T17:10:06.894992"]),("datalad.credential.gin.g-node.org.realm",["https://gin.g-node.org"]),("datalad.credential.gin.g-node.org.type",["token"]),("datalad.extensions.load",["dataverse","next"]),("datalad.gooey.active-suite",["gooey-simplified"]),("diff.astextplain.textconv",["astextplain"]),("filter.annex.process",["git-annex filter-process"]),("filter.lfs.clean",["git-lfs clean -- %f"]),("filter.lfs.process",["git-lfs filter-process"]),("filter.lfs.required",["true"]),("filter.lfs.smudge",["git-lfs smudge -- %f"]),("http.sslbackend",["openssl"]),("http.sslcainfo",["C:/Program Files/Git/mingw64/etc/ssl/certs/ca-bundle.crt"]),("init.defaultbranch",["main"]),("pull.rebase",["false"]),("remote.origin.annex-externaltype",["dataverse"]),("user.email",["michael.hanke@gmail.com"]),("user.name",["Michael Hanke"])]
[2024-10-31 15:37:17.5580559] (Utility.Process) process [33332] done ExitSuccess
[2024-10-31 15:37:17.5650524] (Utility.Process) process [34216] call: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","config","remote.origin.annex-uuid","f62ddde3-ce31-47c5-adfe-b4a2f63558b2"]
[2024-10-31 15:37:17.5928702] (Utility.Process) process [34216] done ExitSuccess
[2024-10-31 15:37:17.5968682] (Utility.Process) process [34092] read: git ["config","--null","--list"] in "."
[2024-10-31 15:37:17.6218678] (Git.Config) git config read: [("",[""]),("annex.crippledfilesystem",["true"]),("annex.sshcaching",["false"]),("annex.uuid",["9dc68e98-4038-4525-9b55-10937f814072"]),("annex.version",["10"]),("core.autocrlf",["true"]),("core.fscache",["true"]),("core.symlinks",["false"]),("credential.helper",["manager"]),("credential.https://dev.azure.com.usehttppath",["true"]),("credential.https://gin.g-node.org.provider",["generic"]),("datalad.credential.data-beta-fzj.last-used",["2024-10-31T15:37:17.307989"]),("datalad.credential.data-beta-fzj.realm",["https://data-beta.fz-juelich.de/dataverse"]),("datalad.credential.data-beta-fzj.type",["token"]),("datalad.credential.data-betafzj.type",["token"]),("datalad.credential.gin.g-node.org.last-used",["2022-09-27T17:10:06.894992"]),("datalad.credential.gin.g-node.org.realm",["https://gin.g-node.org"]),("datalad.credential.gin.g-node.org.type",["token"]),("datalad.extensions.load",["dataverse","next"]),("datalad.gooey.active-suite",["gooey-simplified"]),("diff.astextplain.textconv",["astextplain"]),("filter.annex.process",["git-annex filter-process"]),("filter.lfs.clean",["git-lfs clean -- %f"]),("filter.lfs.process",["git-lfs filter-process"]),("filter.lfs.required",["true"]),("filter.lfs.smudge",["git-lfs smudge -- %f"]),("http.sslbackend",["openssl"]),("http.sslcainfo",["C:/Program Files/Git/mingw64/etc/ssl/certs/ca-bundle.crt"]),("init.defaultbranch",["main"]),("pull.rebase",["false"]),("remote.origin.annex-externaltype",["dataverse"]),("remote.origin.annex-uuid",["f62ddde3-ce31-47c5-adfe-b4a2f63558b2"]),("user.email",["michael.hanke@gmail.com"]),("user.name",["Michael Hanke"])]
[2024-10-31 15:37:17.6238695] (Utility.Process) process [34092] done ExitSuccess
[2024-10-31 15:37:17.6248757] (Annex.Branch) read uuid.log
[2024-10-31 15:37:17.6438663] (Annex.Branch) set uuid.log
[2024-10-31 15:37:17.6458825] (Annex.Branch) read remote.log
[2024-10-31 15:37:17.6548686] (Annex.Branch) set remote.log
[2024-10-31 15:37:17.6618751] (Utility.Process) process [8288] call: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","config","remote.origin.skipFetchAll","true"]
[2024-10-31 15:37:17.6878612] (Utility.Process) process [8288] done ExitSuccess
[2024-10-31 15:37:17.6908765] (Utility.Process) process [34560] read: git ["config","--null","--list"] in "."
[2024-10-31 15:37:17.7113213] (Git.Config) git config read: [("",[""]),("annex.crippledfilesystem",["true"]),("annex.sshcaching",["false"]),("annex.uuid",["9dc68e98-4038-4525-9b55-10937f814072"]),("annex.version",["10"]),("core.autocrlf",["true"]),("core.fscache",["true"]),("core.symlinks",["false"]),("credential.helper",["manager"]),("credential.https://dev.azure.com.usehttppath",["true"]),("credential.https://gin.g-node.org.provider",["generic"]),("datalad.credential.data-beta-fzj.last-used",["2024-10-31T15:37:17.307989"]),("datalad.credential.data-beta-fzj.realm",["https://data-beta.fz-juelich.de/dataverse"]),("datalad.credential.data-beta-fzj.type",["token"]),("datalad.credential.data-betafzj.type",["token"]),("datalad.credential.gin.g-node.org.last-used",["2022-09-27T17:10:06.894992"]),("datalad.credential.gin.g-node.org.realm",["https://gin.g-node.org"]),("datalad.credential.gin.g-node.org.type",["token"]),("datalad.extensions.load",["dataverse","next"]),("datalad.gooey.active-suite",["gooey-simplified"]),("diff.astextplain.textconv",["astextplain"]),("filter.annex.process",["git-annex filter-process"]),("filter.lfs.clean",["git-lfs clean -- %f"]),("filter.lfs.process",["git-lfs filter-process"]),("filter.lfs.required",["true"]),("filter.lfs.smudge",["git-lfs smudge -- %f"]),("http.sslbackend",["openssl"]),("http.sslcainfo",["C:/Program Files/Git/mingw64/etc/ssl/certs/ca-bundle.crt"]),("init.defaultbranch",["main"]),("pull.rebase",["false"]),("remote.origin.annex-externaltype",["dataverse"]),("remote.origin.annex-uuid",["f62ddde3-ce31-47c5-adfe-b4a2f63558b2"]),("remote.origin.skipfetchall",["true"]),("user.email",["michael.hanke@gmail.com"]),("user.name",["Michael Hanke"])]
[2024-10-31 15:37:17.7123226] (Utility.Process) process [34560] done ExitSuccess
ok
[2024-10-31 15:37:17.7550005] (Utility.Process) process [34728] feed: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index-info"]
[2024-10-31 15:37:17.7843409] (Utility.Process) process [34728] done ExitSuccess
[2024-10-31 15:37:17.7951962] (Utility.Process) process [34668] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"]
[2024-10-31 15:37:17.8112148] (Utility.Process) process [34668] done ExitSuccess
(recording state in git...)
[2024-10-31 15:37:17.8192506] (Utility.Process) process [29852] feed: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","update-index","-z","--index-info"]
[2024-10-31 15:37:17.8272105] (Utility.Process) process [34316] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","diff-index","--raw","-z","-r","--no-renames","-l0","--cached","refs/heads/git-annex","--"]
[2024-10-31 15:37:17.8544852] (Utility.Process) process [34316] done ExitSuccess
[2024-10-31 15:37:17.8625099] (Utility.Process) process [29852] done ExitSuccess
[2024-10-31 15:37:17.870528] (Utility.Process) process [33524] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","write-tree"]
[2024-10-31 15:37:17.8982573] (Utility.Process) process [33524] done ExitSuccess
[2024-10-31 15:37:17.9195339] (Utility.Process) process [34224] read: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","commit-tree","8ce941162348ee8bfc7360cd44b8000b03d0a38f","--no-gpg-sign","-p","refs/heads/git-annex","-m","update"]
[2024-10-31 15:37:17.936628] (Utility.Process) process [34224] done ExitSuccess
[2024-10-31 15:37:17.936628] (Utility.Process) process [33912] call: git ["--git-dir=.","--literal-pathspecs","-c","annex.debug=true","update-ref","refs/heads/git-annex","0c37deeada29899f4b1c19f1582e2c6d0da492b5"]
[2024-10-31 15:37:17.9687261] (Utility.Process) process [33912] done ExitSuccess
[2024-10-31 15:37:17.9766185] (Utility.Process) process [8364] done ExitSuccess
[2024-10-31 15:37:17.9846326] (Utility.Process) process [7464] done ExitSuccess

and also datalad clone!

C:\Users\mih\TMP>datalad clone "datalad-annex::?type=external&externaltype=dataverse&encryption=none&exporttree=no&url=https://data-beta.fz-juelich.de&doi=doi:10.0346/JUELICH-DATA-BETA/IHHXNN&credential=data-beta-fzj"
[INFO] Detected a filesystem without fifo support.
[INFO] Disabling ssh connection caching.
[INFO] Detected a crippled filesystem.
[INFO] Entering an adjusted branch where files are unlocked as this filesystem does not support locked files.
[INFO] scanning for annexed files (this may take some time)
[INFO] Updating files:   2% (69/2389)
...
[INFO] Updating files: 100% (2389/2389), done.
[INFO] Switched to branch 'adjusted/master(unlocked)'
[INFO] Remote origin uses a protocol not supported by git-annex; setting annex-ignore
[INFO] RIA store unavailable. -caused by- Failed to access http://studyforrest.ds.inm7.de/ria-layout-version -caused by- Failed to access http://studyforrest.ds.inm7.de/ria-layout-version -caused by- Failed to establish a new session 1 times.  -caused by- HTTPConnectionPool(host='studyforrest.ds.inm7.de', port=80): Max retries exceeded with url: /ria-layout-version (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x00000179ED0B48C0>: Failed to resolve 'studyforrest.ds.inm7.de' ([Errno 11001] getaddrinfo failed)"))
install(ok): C:\Users\mih\TMP\IHHXNN&credential=data-beta-fzj (dataset)

However (!) the clone call succeeds afterwards even without the credential in the URL ?

C:\Users\mih\TMP>datalad clone "datalad-annex::?type=external&externaltype=dataverse&encryption=none&exporttree=no&url=https://data-beta.fz-juelich.de&doi=doi:10.0346/JUELICH-DATA-BETA/IHHXNN"  bla
[INFO] Detected a filesystem without fifo support.
[INFO] Disabling ssh connection caching.
[INFO] Detected a crippled filesystem.
[INFO] Entering an adjusted branch where files are unlocked as this filesystem does not support locked files.
[INFO] scanning for annexed files (this may take some time)
[INFO] Updating files:   2% (69/2389)
...
[INFO] Updating files: 100% (2389/2389), done.
[INFO] Switched to branch 'adjusted/master(unlocked)'
[INFO] Remote origin uses a protocol not supported by git-annex; setting annex-ignore
[INFO] RIA store unavailable. -caused by- Failed to access http://studyforrest.ds.inm7.de/ria-layout-version -caused by- Failed to access http://studyforrest.ds.inm7.de/ria-layout-version -caused by- Failed to establish a new session 1 times.  -caused by- HTTPConnectionPool(host='studyforrest.ds.inm7.de', port=80): Max retries exceeded with url: /ria-layout-version (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x000002B808377D10>: Failed to resolve 'studyforrest.ds.inm7.de' ([Errno 11001] getaddrinfo failed)"))
install(ok): C:\Users\mih\TMP\bla (dataset)

@adswa
Copy link
Member Author

adswa commented Oct 31, 2024

This is not the end of the issue, I still haven't understood what exactly is going on. What I know is that the init remote call needs authentication, and that on a (Windows?) system that hasn't interacted with Dataverse yet this does not work. There is an interaction with the credential handling - maybe its an issue of interactive prompts that fail, a faulty/missing credential manager underneath, ...?. For added fun, copy-pasting into a password prompt of datalad credentials in Windows CMD inserts some crap that makes the password invalid. Typing the token is not fun.

The workaround to the issue is to explicitly set a credential before hand (and by hand) , and then append it to the URL that is cloned from.

@mslw
Copy link
Contributor

mslw commented Oct 31, 2024

However (!) the clone call succeeds afterwards even without the credential in the URL ?

I don't know the involved special remote, but it could be that, on one of the previous attempts, git-annex caches the credentials in its own way, i.e. in .git/annex/creds (if I remember correctly). It does for some remotes (and on linux, the file is user-read-only), you can check if that's the case here,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mslw @adswa