Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the index more comprehensive #694

Merged
merged 2 commits into from
Apr 2, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/basics/101-105-install.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
.. index:: ! datalad command; clone
.. _installds:

Install datasets
Expand Down
2 changes: 2 additions & 0 deletions docs/basics/101-107-summary.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ and making simple modifications *locally*.
Furthermore, we have discovered the basics of installing a published DataLad dataset,
and experienced the concept of modular nesting datasets.

.. index:: ! datalad command; clone

* A published dataset can be installed with the :command:`datalad clone` command::

$ datalad clone [--dataset PATH] SOURCE-PATH/URL [DESTINATION PATH]
Expand Down
6 changes: 6 additions & 0 deletions docs/basics/101-116-sharelocal.rst
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,8 @@ home directory. Furthermore, let's for now disregard anything about
:term:`permissions`. In a real-world example you likely would not be able to read and write
to a different user's directories, but we will talk about permissions later.

.. index:: ! datalad command; clone

After creation, navigate into ``mock_user`` and install the dataset ``DataLad-101``.
To do this, use :command:`datalad clone`, and provide a path to your original
dataset. Here is how it looks like:
Expand Down Expand Up @@ -132,6 +134,8 @@ object tree. To reassure your room mate that everything is fine you
quickly explain to him the concept of a symlink and the :term:`object-tree`
of :term:`git-annex`.

.. index:: ! datalad command; clone

"But why does the PDF not open when I try to open it?" he repeats.
True, these files cannot be opened. This mimics our experience when
installing the ``longnow`` subdataset: Right after installation,
Expand Down Expand Up @@ -350,6 +354,8 @@ with the first subdataset::
.. find-out-more:: datalad clone versus datalad install
:name: fom_clonevsinstall

.. index:: ! datalad command; clone

You may remember from section :ref:`installds` that DataLad has two commands to obtain datasets,
:command:`datalad clone` and :command:`datalad install`.
The command structure of :command:`install` and :command:`datalad clone` are
Expand Down
5 changes: 3 additions & 2 deletions docs/basics/101-130-yodaproject.rst
Original file line number Diff line number Diff line change
Expand Up @@ -603,6 +603,7 @@ syllabus, this should be done via :term:`GitHub`.
.. image:: ../artwork/src/screenshot_submodule.png
:alt: The input dataset is linked

.. index:: ! datalad command; create-sibling-github
.. _publishtogithub:

Publishing the dataset to GitHub
Expand All @@ -626,13 +627,13 @@ For this, you need to
- configure this GitHub repository to be a :term:`sibling` of the ``midterm_project`` dataset,
- and *publish* your dataset to GitHub.

.. index:: ! datalad command; create-sibling-gitlab

Luckily, DataLad can make all of this very easy with the
:command:`datalad create-sibling-github` (:manpage:`datalad-create-sibling-github` manual)
command (or, for `GitLab <https://about.gitlab.com/>`_, :command:`datalad create-sibling-gitlab`,
:manpage:`datalad-create-sibling-gitlab` manual).

.. index:: ! datalad command; create-sibling-github, ! datalad command; create-sibling-gitlab

The two commands have different arguments and options.
Here, we look at :command:`datalad create-sibling-github`.
The command takes a repository name and GitHub authentication credentials
Expand Down
5 changes: 4 additions & 1 deletion docs/basics/101-138-sharethirdparty.rst
Original file line number Diff line number Diff line change
Expand Up @@ -319,8 +319,10 @@ be managed and accessed via DataLad/git-annex.
To actually share your dataset with someone, you need to *publish* it to Github,
Gitlab, or a similar hosting service.

.. index:: ! datalad command; create-sibling-github

You could, for example, create a sibling of the ``DataLad-101`` dataset
on GitHub with the command :command:`datalad-sibling-github`.
on GitHub with the command :command:`create-sibling-github`.
This will create a new GitHub repository called "DataLad-101" under your account,
and configure this repository as a :term:`sibling` of your dataset
called ``github`` (exactly like you have done in :ref:`yoda_project`
Expand Down Expand Up @@ -490,6 +492,7 @@ books, or the cropped logos from chapter :ref:`chapter_run`::
$ datalad get books/TLCL.pdf
get(ok): /home/some/other/user/DataLad-101/books/TLCL.pdf (file) [from dropbox-for-friends]

.. index:: ! datalad command; create-sibling-github
.. _gitlfs:

Use GitHub for sharing content
Expand Down
4 changes: 4 additions & 0 deletions docs/basics/101-139-gin.rst
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,8 @@ repository's ``https`` url. This does not require a user account on Gin.

.. importantnote:: Take the URL in the browser, not the copy-paste URL

.. index:: ! datalad command; clone

Please note that you need to use the browser URL of the repository, not the copy-paste URL on the upper right hand side of the repository if you want to get anonymous HTTPS access!
The two URLs differ only by a ``.git`` extension:

Expand All @@ -171,6 +173,8 @@ repository's ``https`` url. This does not require a user account on Gin.
Subsequently, :command:`datalad get` calls will be able to retrieve all annexed
file contents that have been published to the repository.

.. index:: ! datalad command; clone

If it is a **private** dataset, cloning the dataset from Gin requires a user
name and password for anyone you want to share your dataset with.
The "Collaboration" tab under Settings lets you set fine-grained access rights,
Expand Down
4 changes: 4 additions & 0 deletions docs/basics/101-141-push.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ or storage of datasets [#f1]_, or a regular clone.

.. find-out-more:: all of the ways to configure siblings

.. index:: ! datalad command; create-sibling-github
.. index:: ! datalad command; create-sibling-gitlab
.. index:: ! datalad command; create-sibling-ria

- Add an existing repository as a sibling with the :command:`datalad siblings`
command. Here are common examples::

Expand Down
4 changes: 3 additions & 1 deletion docs/basics/101-146-gists.rst
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,8 @@ Here is a one-liner to get this info::
Backing-up datasets
^^^^^^^^^^^^^^^^^^^

.. index:: ! datalad command; create-sibling

In order to back-up datasets you can publish them to a
:term:`Remote Indexed Archive (RIA) store` or to a sibling dataset. The former
solution does not require Git, git-annex, or DataLad to be installed on the
Expand Down Expand Up @@ -263,4 +265,4 @@ Afterwards, if your dataset has a sibling, the branch needs to be
history, also checkout
`this thread <https://git-annex.branchable.com/forum/safely_dropping_git-annex_history/>`_
for more information on shrinking git-annex's history and helpful safeguards and
potential caveats.
potential caveats.
8 changes: 6 additions & 2 deletions docs/beyond_basics/101-147-riastores.rst
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,8 @@ Other applications may require *only* the special remote, such as cases where Gi
For most storage or back-up scenarios, special remote capabilities are useful, though,
and thus the default.

.. index:: ! datalad command; create-sibling-ria

By default, the :command:`datalad create-sibling-ria` command will automatically create a
dataset representation in a RIA store (and set up the RIA store, if it does not
exist), and configure a sibling to allow publishing to the RIA store and updating
Expand Down Expand Up @@ -270,11 +272,11 @@ from standard DataLad workflows. The paragraphs below detail how to create and
populate a RIA store, how to clone datasets and retrieve data from it, and also
how to handle permissions or hide technicalities.

.. index:: ! datalad command; create-sibling-ria

Creating or publishing to RIA stores
""""""""""""""""""""""""""""""""""""

.. index:: ! datalad command; create-sibling-ria

A dataset can be added into an existing or not yet existing RIA store by
running the :command:`datalad create-sibling-ria` command
(:manpage:`datalad-create-sibling-ria` manual), and subsequently published into
Expand Down Expand Up @@ -447,6 +449,8 @@ As a demonstration, we'll do it for the ``midterm_project`` subdataset:
Thus, in order to create and populate RIA stores, only the commands
:command:`datalad create-sibling-ria` and :command:`datalad push` are required.

.. index:: ! datalad command; clone

Cloning and updating from RIA stores
""""""""""""""""""""""""""""""""""""

Expand Down
1 change: 1 addition & 0 deletions docs/beyond_basics/101-148-clonepriority.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
.. index:: ! datalad command; clone
.. _cloneprio:

Prioritizing subdataset clone locations
Expand Down
4 changes: 4 additions & 0 deletions docs/beyond_basics/101-168-dvc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -487,6 +487,8 @@ Currently, the dataset can thus be shared via :term:`GitHub` or similar hosting

.. find-out-more:: Really?

.. index:: ! datalad command; create-sibling-github

Sure.
Let's demonstrate this.
First, we create a sibling on GitHub for this dataset and push its contents to the sibling:
Expand Down Expand Up @@ -532,6 +534,8 @@ Here's an example of pushing a dataset to a local sibling nevertheless:

**Step 1: Set up the sibling**

.. index:: ! datalad command; create-sibling

The easiest way to share data is via a local sibling [#f7]_.
This won't share only annexed data, but it instead will push everything, including the Git aspect of the dataset.
First, we need to create a local sibling:
Expand Down
4 changes: 4 additions & 0 deletions docs/usecases/collaborative_data_management.rst
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ thanks to the yoda procedure:
$ cd myanalysis
$ tree

.. index:: ! datalad command; clone

Bob knows that a DataLad dataset can contain other datasets. He also knows that
as any content of a dataset is tracked and its precise state is recorded,
this is a powerful method to specify and later resolve data dependencies,
Expand Down Expand Up @@ -188,6 +190,8 @@ command:
$ datalad update -s alice --merge


.. index:: ! datalad command; create-sibling

Finally, when Bob is ready to share his results with the world or a remote
collaborator, he makes his dataset available by uploading them to a webserver
via SSH. Bob does so by creating a sibling for the dataset on the server, to
Expand Down
2 changes: 2 additions & 0 deletions docs/usecases/datastorage_for_institutions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,8 @@ requiring expert or domain knowledge about the data. At its core, it is a flat,
file-system based repository representation of any number of datasets, limited
only by disk-space constrains of the machine it lies on.

.. index:: ! datalad command; create-sibling-ria

Put simply, a RIA store is a dataset storage location that allows for access to
and collaboration on DataLad datasets.
The high-level workflow overview is as follows: Create a dataset,
Expand Down