Skip to content

Commit

Permalink
Merge branch 'master' of github.com:datalad-handbook/book
Browse files Browse the repository at this point in the history
@adswa
adswa committed Mar 22, 2021
2 parents 42ed489 + 9707568 commit 66a1da0
Showing 5 changed files with 17 additions and 119 deletions.
2 changes: 1 addition & 1 deletion docs/basics/101-109-rerun.rst
View file
Original file line number Diff line number Diff line change
@@ -255,7 +255,7 @@ give the file path to :command:`git log` (prefixed by ``--``):

.. windows-wit:: use "git log master -- recordings/podcasts.tsv"

A previous Windows-Workaround already advised to append ``main`` or ``master``, the common "default :term:`branch`", to any command that starts with ``git log``.
A previous Windows Wit already advised to append ``main`` or ``master``, the common "default :term:`branch`", to any command that starts with ``git log``.
Here, the last part of the command specifies a file (``-- recordings/podcasts.tsv``).
Please append ``main`` or ``master`` to ``git log``, prior to the file specification.

2 changes: 1 addition & 1 deletion docs/basics/101-110-run2.rst
View file
Original file line number Diff line number Diff line change
@@ -170,7 +170,7 @@ If outputs already exist...

Please don't skip the next section -- it is useful to know how datasets behave on other systems.
Just be mindful that you will not encounter the errors that the handbook displays next.
And while this all sounds quite cryptic and vague, an upcoming Windows-Workaround will provide more information.
And while this all sounds quite cryptic and vague, an upcoming Windows Wit will provide more information.

Looking at the resulting image, you wonder whether 400x400 might be a tiny bit to small.
Maybe we should try to resize it to 450x450, and see whether that looks better?
28 changes: 15 additions & 13 deletions docs/basics/101-115-symlinks.rst
View file
Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@ We'll take a look together, using the ``books/`` directory as an example:
First of all, the ``tree`` equivalent provided by :term:`conda`\s ``m2-base`` package doesn't list individual files, only directories.
And, secondly, even if you list the individual files (e.g., with ``ls -l``), you would not see the :term:`symlink`\s shown below.
Due to insufficient support of symlinks on Windows, git-annex does not use them.
Please read on for a basic understanding of how git-annex usually works -- a Windows-Workaround at the end of this section will then highlight the difference in functionality on Windows.
Please read on for a basic understanding of how git-annex usually works -- a Windows Wit at the end of this section will then highlight the difference in functionality on Windows.

.. runrecord:: _examples/DL-101-115-101
:language: console
@@ -299,13 +299,26 @@ Alternatively, use the :command:`ls` command in a terminal instead of a file man
Other tools may be more more specialized, smaller, or domain-specific, and may fail to correctly work with broken symlinks, or display unhelpful error messages when handling them, or require additional flags to modify their behavior (such as the :ref:`BIDS Validator <bidsvalidator>`, used in the neuroimaging community).
When encountering unexpected behavior or failures, try to keep in mind that a dataset without retrieved content appears to be a pile of broken symlinks to a range of tools, consult a tools documentation with regard to symlinks, and check whether data retrieval fixes persisting problems.


Finally, if you are still in the ``books/`` directory, go back into the root of
the superdataset.

.. runrecord:: _examples/DL-101-115-106
:workdir: dl-101/DataLad-101/books
:language: console
:notes: understanding how symlinks work will help you with everyday file management operations.
:cast: 03_git_annex_basics

$ cd ../


.. _wslfiles:

Cross-OS filesharing with symlinks (WSL2 only)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Are you using DataLad on the Windows Subsystem for Linux?
If so, please take a look into the Windows-Workaround below.
If so, please take a look into the Windows Wit below.

.. windows-wit:: Accessing symlinked files from your Windows system

@@ -325,17 +338,6 @@ If so, please take a look into the Windows-Workaround below.
The branch is called ``adjusted/<branchname>(unlocked)`` (e.g., if the original branch name was ``main``, the new, adjusted branch will be called ``adjusted/main(unlocked)``).
You can switch back to your original branch using ``git checkout <branchname>``.

Finally, if you are still in the ``books/`` directory, go back into the root of
the superdataset.

.. runrecord:: _examples/DL-101-115-106
:workdir: dl-101/DataLad-101/books
:language: console
:notes: understanding how symlinks work will help you with everyday file management operations.
:cast: 03_git_annex_basics

$ cd ../

.. rubric:: Footnotes

.. [#f1] Note, though, that the information below applies to everything that is not an
52 changes: 0 additions & 52 deletions docs/code_from_chapters/DLBasicsMPI.rst
View file
Original file line number Diff line number Diff line change
@@ -439,58 +439,6 @@ Next, we set up and configure a dataset for the analysis.
Given that code is frequently modified and should be easily editable, and would be useful to share right away if desired, it makes sense to keep it stored in Git.
Thus, we preconfigure and prestructure the dataset with a few configurations:

.. windows-wit:: Cave! The text2git configuration does not work!

If you are using **Windows 10** with a **native** (i.e., not `Windows Subsystem for Linux (WSL) <https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux>`_-based) installation of DataLad and its underlying tools, you need to do a work-around here.

We're really sorry about that -- as foreshadowed in section :ref:`install`, Windows comes with a range of file system issues, and one of them concerns the ``text2git`` configuration.
This configuration detects file types (such as text files, or image files) based on their `MIME type <https://www.howtogeek.com/192628/mime-types-explained-why-linux-and-mac-os-x-dont-need-file-extensions/>`_, but Windows lacks MIME type support, unfortunately.

Instead of running ``datalad create -c text2git -c yoda ml-project``, please remove the configuration ``-c text2git`` from the command and run only ``datalad create -c yoda ml-project``::

$ datalad create -c yoda ml-project
[INFO] Creating a new annex repo at C:\Users\mih\ml-project
[INFO] Detected a filesystem without fifo support.
[INFO] Disabling ssh connection caching.
[INFO] Detected a crippled filesystem.
[INFO] Scanning for unlocked files (this may take some time)
[INFO] Entering an adjusted branch where files are unlocked as this filesystem does not support locked files.
[INFO] Switched to branch 'adjusted/master(unlocked)'
[INFO] Running procedure cfg_yoda
[INFO] == Command start (output follows) =====
[INFO] == Command exit (modification check follows) =====
create(ok): C:\Users\mih\ml-project (dataset)

Instead of the ``text2git`` configuration, you need to create a configuration by hand by pasting the following lines of text into the (hidden) ``.gitattributes`` file in your newly created dataset.
:ref:`chapter_config` can explain the details of this procedure.

Here are lines that need to be appended to the existing lines in ``.gitattributes`` and will mimic the configuration ``-c text2git`` would apply::

*.json annex.largefiles=nothing

You can achieve this by copy-pasting the following code snippets into your terminal (but you can also add them using a text editor of your choice):

.. code-block::
$ echo\ >> .gitattributes && echo *.json annex.largefiles=nothing >> .gitattributes
Afterwards, these should be the contents of ``.gitattributes``:

.. code-block::
$ cat .gitattributes
* annex.backend=MD5E
**/.git* annex.largefiles=nothing
CHANGELOG.md annex.largefiles=nothing
README.md annex.largefiles=nothing
*.json annex.largefiles=nothing
Lastly, run this piece of code to save your changes:

.. code-block:: bash
$ datalad save -m "Windows-workaround: custom config to place text into Git" .gitattributes
.. code-block:: bash
cd ../
52 changes: 0 additions & 52 deletions docs/code_from_chapters/usecase_ml_code.rst
View file
Original file line number Diff line number Diff line change
@@ -425,58 +425,6 @@ Next, we set up and configure a dataset for the analysis.
Given that code is frequently modified and should be easily editable, and would be useful to share right away if desired, it makes sense to keep it stored in Git.
Thus, we preconfigure and prestructure the dataset with a few configurations:

.. windows-wit:: Cave! The text2git configuration does not work!

If you are using **Windows 10** with a **native** (i.e., not `Windows Subsystem for Linux (WSL) <https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux>`_-based) installation of DataLad and its underlying tools, you need to do a work-around here.

We're really sorry about that -- as foreshadowed in section :ref:`install`, Windows comes with a range of file system issues, and one of them concerns the ``text2git`` configuration.
This configuration detects file types (such as text files, or image files) based on their `MIME type <https://www.howtogeek.com/192628/mime-types-explained-why-linux-and-mac-os-x-dont-need-file-extensions/>`_, but Windows lacks MIME type support, unfortunately.

Instead of running ``datalad create -c text2git -c yoda ml-project``, please remove the configuration ``-c text2git`` from the command and run only ``datalad create -c yoda ml-project``::

$ datalad create -c yoda ml-project
[INFO] Creating a new annex repo at C:\Users\mih\ml-project
[INFO] Detected a filesystem without fifo support.
[INFO] Disabling ssh connection caching.
[INFO] Detected a crippled filesystem.
[INFO] Scanning for unlocked files (this may take some time)
[INFO] Entering an adjusted branch where files are unlocked as this filesystem does not support locked files.
[INFO] Switched to branch 'adjusted/master(unlocked)'
[INFO] Running procedure cfg_yoda
[INFO] == Command start (output follows) =====
[INFO] == Command exit (modification check follows) =====
create(ok): C:\Users\mih\ml-project (dataset)

Instead of the ``text2git`` configuration, you need to create a configuration by hand by pasting the following lines of text into the (hidden) ``.gitattributes`` file in your newly created dataset.
:ref:`chapter_config` can explain the details of this procedure.

Here are lines that need to be appended to the existing lines in ``.gitattributes`` and will mimic the configuration ``-c text2git`` would apply::

*.json annex.largefiles=nothing

You can achieve this by copy-pasting the following code snippets into your terminal (but you can also add them using a text editor of your choice):

.. code-block::
$ echo\ >> .gitattributes && echo *.json annex.largefiles=nothing >> .gitattributes
Afterwards, these should be the contents of ``.gitattributes``:

.. code-block::
$ cat .gitattributes
* annex.backend=MD5E
**/.git* annex.largefiles=nothing
CHANGELOG.md annex.largefiles=nothing
README.md annex.largefiles=nothing
*.json annex.largefiles=nothing
Lastly, run this piece of code to save your changes:

.. code-block:: bash
$ datalad save -m "Windows-workaround: custom config to place text into Git" .gitattributes
.. code-block:: bash
cd ../

0 comments on commit 66a1da0

Please sign in to comment.