Towards more user-friendliness for Windows Users #588

adswa · 2020-10-08T14:05:05Z

I'm working my way through the handbook on Windows and add "Windows-Workaround" Sections where ever necessary. So far, its frequently necessary.

wget isn't pre-installed on most windows systems, and installing it involves adjusting the PATH. May be easier to just go with curl

Unlike on Unix, Windows can linebreak a command with backslashes. One either needs to copy the command as one line, or use the ^ character instead of the slash

The escape sequences in the here doc make it fail when copy-pasted into a script

At least on my Windows machine, the git log -p output is very long, confusing, and fails to only display the most recent commits.

Given the adjusted branch handling, all files will show up as being changed, which is not useful at all for a user

This returns the same history as the one reported by the handbook, without the adjusted branch commits

Windows has a native convert command that would be used instead and throw 'invalid option' errors

…on windows anyway

…ectories, and also hidden ones

adswa · 2020-10-15T14:51:31Z

@mih it would be nice if we could chat about installation recommendations.

adswa · 2020-10-19T11:35:20Z

This is quite substantial, and yet far from done. I'm sure that much more can and will be done to support Windows users, but this chunk is ready to go in from my point of view. @mih, I've written the "So ... Windows... eh?" page you asked for, and it is here: https://datalad-handbook--588.org.readthedocs.build/en/588/intro/windows.html#ohnowindows ;-) Not sure whether it is what you had in mind, please check.

adswa · 2020-10-21T11:29:57Z

In this PR (among many other things) I am switching recommendations about where to install git-annex under Windows -- instead of installing Git via conda-forge (which we recommended to get cd, but installing m2-base via conda is an alternative), I have switched to a Git-for-Windows installation and recommend installing git-annex inside of it, since its more likely that users already have Git installed: https://datalad-handbook--588.org.readthedocs.build/en/588/intro/installation.html#windows-10.
If anyone has opinions on how to do the Windows installation I would appreciate hearing them. There is one issue in datalad where someone hasn't yet succeeded in installing datalad with git-annex in the miniconda library and rebooting their computer without error, and I think it would be cool if we could point them to an installation method (that we have thought about, or even better, experience with). Given more people have a Windows computer for testing than me (@bpoldrack, @yarikoptic, @mih?), have you made any experiences about what works and what doesn't?

I also wrote a section on overthinking installation on Windows, but I would be reluctant to merge this without no-one else having had a chance to look at it: https://datalad-handbook--588.org.readthedocs.build/en/588/intro/windows.html#ohnowindows. Feedback would be really appreciated!

yarikoptic · 2020-10-21T11:41:32Z

someone hasn't yet succeeded in installing datalad with git-annex in the miniconda library and rebooting their computer without error,

What error exactly? About assistant?

adswa · 2020-10-21T11:51:09Z

datalad/datalad#5000

yarikoptic · 2020-10-21T13:40:43Z

oh well, a proper long term resolution to windows installation problems would be to build git-annex in conda-forge for windows (conda-forge/git-annex-feedstock#15), and IMHO would be the most effective investment of time (even though might take longer than writing documentation outlining how things do not work, no pun intended whatsoever). The dances with forcing annex into anaconda environment and then changing installed files sure thing cannot be relied on just prolong the pains IMHO, even if @joeyh agrees to make git-annex become more robust in such heterogeneous installations to use system-wide (not conda's) git: https://git-annex.branchable.com/bugs/old__40__er__41___cp_from_coreutils_lacks_--reflink/#comment-9819622af7c173000af5c8337dee4b10 .

mih · 2020-10-23T06:20:32Z

Trying this out right now. Impulse response:

Switching away from the conda install for git is an immediate monster-win for a windows noob like me. The Git installer leaves a really nicely configured system behind (TT fonts, symlinks, performance tuning) -- all stuff that I would have had to discover otherwise, and probably would have given up on long before reaching this state.

mih

First set of comments after replicating the new installation instructions in win10. Much better!!

docs/intro/installation.rst

mih · 2020-10-23T06:27:32Z

docs/intro/installation.rst

+    Please make sure to download and install it.
+    Please also make sure that it is available in your ``PATH`` environment variable: ``Environment Variables`` -> ``User Variables`` -> ``path`` -> ``edit`` -> ``new`` -> add the path: ``C:\ProgramFiles\7-zip``) -> ``OK``. Lastly, within ``C:\ProgramFiles\7-zip``, copy the ``7z.exe`` file into ``C:\Users\<user>\Miniconda3\Library\usr\bin\``.
+  - Many Unix command-line tools such as ``cp`` or ``tree`` are not available by default.
+    You can get a good set of tools by installing :term:`conda`\s ``m2-base`` package via ``conda install m2-base`` in an Anaconda prompt.


In the git installer I chose to use an msys terminal by default. The resulting environment had tree, but did not have cp. Maybe remove tree in the text to avoid "I can get away with this alternative approach" scenarios.

mih · 2020-10-23T06:30:19Z

docs/intro/installation.rst

-
-    Note: Is has to be from ``conda-forge``, the anaconda version does not
-    provide the ``cp`` command.
+  - If you haven't installed :term:`Git` yet, please download and install the latest release from `git-scm.com/ <https://git-scm.com/>`_.


Just FTR: Many options are advertised by the git installer. We should leave recommendations, if we have any. Candidates are:

true type fonts (OMG my eyes can see for the first time on windows)

msys terminal (stellar C&P performance in comparison)

credential manager

FS tuning options (do they have an effect?)

enable symlinks?

We need to automate, https://github.com/datalad/datalad/issues/5078

mih · 2020-10-23T06:42:22Z

After following the installation instructions, and running the git-annex tests I get:

8 out of 704 tests failed (511.91s)
(Failures above could be due to a bug in git-annex, or an incompatibility
   with utilities, such as git, installed on this system.)

Output is pretty garbled. OK initially (with colors), but then riddled with ASCI control sequences.

adswa · 2020-10-23T07:04:01Z

ah shit :/ is that on the computer under your desk? I haven't actually run the git-annex test suite on my Windows computer. But this may be a very useful recommendation to give people a way to judge whether everything is okay

…nh-windows

yarikoptic · 2020-10-24T11:39:37Z

After following the installation instructions, and running the git-annex tests I get:
8 out of 704 tests failed (511.91s)
(Failures above could be due to a bug in git-annex, or an incompatibility
   with utilities, such as git, installed on this system.)
Output is pretty garbled. OK initially (with colors), but then riddled with ASCI control sequences.

Those?
https://git-annex.branchable.com/bugs/storeKey_when_already_present_failures_on_Windows/

yarikoptic · 2020-10-24T11:48:24Z

docs/intro/installation.rst


  - During installation, you will be prompted to "Choose Install Location".
-    **Install it into the miniconda Library directory**, e.g.
-    ``C:\Users\me\Miniconda3\Library``.
+    **Install it into the directory in which Git is installed**, e.g., ``C:\Program Files (x86)\Git``.


I went with 64bit version of git and annex so no x86

yarikoptic · 2020-10-24T11:49:23Z

docs/intro/installation.rst


 - **Step 4**: Install DataLad via pip

  - ``pip`` was installed by ``miniconda``. In the ``Anaconda prompt``, run::

-       pip install datalad~=0.12
+       pip install datalad


What not conda install since already using conda

yarikoptic · 2020-10-24T11:53:50Z

docs/intro/installation.rst

-
-    Note: Is has to be from ``conda-forge``, the anaconda version does not
-    provide the ``cp`` command.
+  - If you haven't installed :term:`Git` yet, please download and install the latest release from `git-scm.com/ <https://git-scm.com/>`_.


We need to automate, https://github.com/datalad/datalad/issues/5078

adswa · 2020-11-02T10:33:44Z

@yarikoptic I have updated the installation instructions to "native Git (with TrueType Fonts) + miniconda + conda-forge datalad" for now. I very much support automation, but until then, shall we merge this as a temporary improvement?

yarikoptic · 2020-11-02T23:11:16Z

sounds good for now, but you might like to add (unless you already did) a note that

they should use "native Git Bash" terminal to interact with DataLad, for that they should have conda be available within that shell as well (I have not backtraced my steps with starting from a clean install to see how my git bash ended up seeing conda)
have not investigated in full yet on either there would or not be side-effect: there should be no m2-bash (or any other package which posix conda package depends on) installed ATM, due to m2-bash: looses PATHs, prevents any use as a shell to run other commands from conda ContinuumIO/anaconda-issues#12124 . For that effect also, if someone enters bash from m2-bash in their conda environment, they might end up with datalad which would not see git-annex since it is installed under ...\Library\bin and that one is dropped by m2-bash. If not a note -- something FYI

adswa · 2020-11-03T13:04:55Z

they should use "native Git Bash" terminal to interact with DataLad, for that they should have conda be available within that shell as well (I have not backtraced my steps with starting from a clean install to see how my git bash ended up seeing conda)

Off, I didn't have that solution (using the Git Bash) on my radar at all (I hat Windows so much... its an endless rabbit hole of non-optimal options)! One way I found to make it work was to tick the box to add Miniconda to the PATH variable during Miniconda installation. This terminal does look nicer, ~~but I can't get it to copy-paste~~, and it lacks commands like tree or wget.

have not investigated in full yet on either there would or not be side-effect: there should be no m2-bash (or any other package which posix conda package depends on) installed ATM, due to ContinuumIO/anaconda-issues#12124 . For that effect also, if someone enters bash from m2-bash in their conda environment, they might end up with datalad which would not see git-annex since it is installed under ...\Library\bin and that one is dropped by m2-bash. If not a note -- something FYI

Ahhhhhh crap 🤦

adswa · 2020-11-03T14:23:57Z

Ok, I will merge this now. Its definitely not done, but a huge improvement to the current status.

adswa added 28 commits October 8, 2020 08:16

Mention installation method for tree on Windows

99a4ff3

Win: provide curl alternative to wget call

857d338

wget isn't pre-installed on most windows systems, and installing it involves adjusting the PATH. May be easier to just go with curl

Win: Demonstrate line-breaking under windows

0a6aa9d

Unlike on Unix, Windows can linebreak a command with backslashes. One either needs to copy the command as one line, or use the ^ character instead of the slash

Win: no heredocs under windows

11e9c18

Win: warn of missing tree options

0da80d0

Win: provide fixed script to copy paste with editor

d413a96

The escape sequences in the here doc make it fail when copy-pasted into a script

Win: Note on git log in comment

6659e9b

At least on my Windows machine, the git log -p output is very long, confusing, and fails to only display the most recent commits.

Win: Provide copy-paste version of changed list_titles.sh script

b9ec299

Win: discourage usage of datalad diff

4b4a608

Given the adjusted branch handling, all files will show up as being changed, which is not useful at all for a user

Win: Recommend git logging the master branch

bf21aea

This returns the same history as the one reported by the handbook, without the adjusted branch commits

Win: Provide installation help for ImageMagick.

242f7e4

Windows has a native convert command that would be used instead and throw 'invalid option' errors

Win: Adjust the run section for the fact that everything is unlocked …

d6ac17f

…on windows anyway

Win: don't discourage WSL2, add info on windowsworkarounds

997d024

BF: fix link to ebook

ece75ef

Gloss: add crippled file system and adjusted branches

e650584

Gloss: unify git-annex spelling

7b3ce59

Win: add note on windows tree command - does not show files, only dir…

73f66d7

…ectories, and also hidden ones

Win: provide command help for viewing master branch version of one file

6995a07

Win: start work towards explaining adjusted branches

e7a97d3

Win: overhaul installation instructions

20a55db

Win: explain data handling without symlinks on Windows

ac9bcf3

Win: Fix datalad diff calls

516172d

Win: fix gitattributes explanation

e4b5a76

Win: there is no python3 (maybe)

e30f163

Win: assure users that their keyboard works when tying in passwords

be6f18a

Win: start an honest assessment of windows functions

58a053c

update submodule

b289408

Tweaks to install page

0978305

Win: Write up a section on assessing the state of DataLad on Windows

f9ec5d9

Win: accessing symlinked files under WSL and Windows

8fb97b4

adswa added the ready-for-review label Oct 19, 2020

Merge branch 'master' into enh-windows

643e081

mih reviewed Oct 23, 2020

View reviewed changes

adswa added 3 commits October 23, 2020 09:10

Win: encourage git-annex tests after installation

b842bb9

Merge branch 'enh-windows' of github.com:datalad-handbook/book into e…

7d1d785

…nh-windows

Win: suggestions by mih

f77da82

This was referenced Oct 23, 2020

More windows testing datalad/datalad#5073

Merged

Windows revelation not so obvious to me - may be for someone else? datalad/datalad#5082

Closed

adswa mentioned this pull request Oct 24, 2020

windows installation instructions are "suboptimal" #598

Closed

yarikoptic reviewed Oct 24, 2020

View reviewed changes

adswa added 2 commits November 2, 2020 11:19

Install: remove 0.12 spec from pip calls

484820f

Win: adjust installation instructions to native Git + conda forge

f7e0bd8

adswa added 2 commits November 3, 2020 15:12

Win: point to official microsoft docs

27f511b

Win: apply Yariks suggestions

08cd115

adswa merged commit 4007cba into master Nov 3, 2020

adswa deleted the enh-windows branch November 3, 2020 14:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Towards more user-friendliness for Windows Users #588

Towards more user-friendliness for Windows Users #588

adswa commented Oct 8, 2020

adswa commented Oct 15, 2020

adswa commented Oct 19, 2020

adswa commented Oct 21, 2020

yarikoptic commented Oct 21, 2020

adswa commented Oct 21, 2020

yarikoptic commented Oct 21, 2020

mih commented Oct 23, 2020

mih left a comment

mih Oct 23, 2020

mih Oct 23, 2020

yarikoptic Oct 24, 2020

mih commented Oct 23, 2020

adswa commented Oct 23, 2020

yarikoptic commented Oct 24, 2020

yarikoptic Oct 24, 2020

yarikoptic Oct 24, 2020

yarikoptic Oct 24, 2020

adswa commented Nov 2, 2020

yarikoptic commented Nov 2, 2020

adswa commented Nov 3, 2020 •

edited

Loading

adswa commented Nov 3, 2020

Towards more user-friendliness for Windows Users #588

Towards more user-friendliness for Windows Users #588

Conversation

adswa commented Oct 8, 2020

adswa commented Oct 15, 2020

adswa commented Oct 19, 2020

adswa commented Oct 21, 2020

yarikoptic commented Oct 21, 2020

adswa commented Oct 21, 2020

yarikoptic commented Oct 21, 2020

mih commented Oct 23, 2020

mih left a comment

Choose a reason for hiding this comment

mih Oct 23, 2020

Choose a reason for hiding this comment

mih Oct 23, 2020

Choose a reason for hiding this comment

yarikoptic Oct 24, 2020

Choose a reason for hiding this comment

mih commented Oct 23, 2020

adswa commented Oct 23, 2020

yarikoptic commented Oct 24, 2020

yarikoptic Oct 24, 2020

Choose a reason for hiding this comment

yarikoptic Oct 24, 2020

Choose a reason for hiding this comment

yarikoptic Oct 24, 2020

Choose a reason for hiding this comment

adswa commented Nov 2, 2020

yarikoptic commented Nov 2, 2020

adswa commented Nov 3, 2020 • edited Loading

adswa commented Nov 3, 2020

adswa commented Nov 3, 2020 •

edited

Loading