Merge pull request #210 from juaml/enh/docs_flowchart

[ENH]: Add flowchart for users and improve docs

docs/_static/css/custom.css

@@ -20,4 +20,27 @@
 #what-s-new li,
 #what-s-new a {
   font-size: medium;
+}
+
+#first-steps-with-junifer div.mermaid {
+  overflow: scroll;
+}
+
+#first-steps-with-junifer svg {
+  overflow: scroll;
+  width: 200%;
+}
+
+body:not([data-theme="light"]) .mermaid svg .flowchart-link {
+  stroke: white !important;
+}
+
+body:not([data-theme="light"]) .mermaid svg .edgeLabel {
+  color: white !important;
+  background-color: #202020 !important;
+}
+
+body:not([data-theme="light"]) .mermaid svg .marker{
+  stroke: white !important;
+  fill: white !important;
 }

docs/changes/newsfragments/210.doc

@@ -0,0 +1 @@
+Add sections *starting* and *help* in the documentation by `Fede Raimondo`_

docs/conf.py

@@ -51,6 +51,7 @@
     "numpydoc",  # support for NumPy style docstrings
     "gh_substitutions",  # custom GitHub substitutions
     "sphinx_copybutton",  # copy button for code blocks
+    "sphinxcontrib.mermaid",  # mermaid support
 ]
 
 if use_multiversion:

docs/help.rst

@@ -0,0 +1,108 @@
+.. include:: links.inc
+
+.. _help:
+
+Getting Help
+============
+
+    | Help! I need somebody
+    | Help! not just anybody
+    | Help! you know I need someone
+    | Help!
+
+    -- The Beatles, Help!
+
+This fragment of the song originally appeared in the 1965 film Help! and was written by Lennon and McCartney, just a
+few years after the development of Mark 1, the first commercial computer. Maybe just a random coincidence, or maybe
+Lennon was just trying to write an email to McCartney.
+
+While the song might have been written with another meaning in mind, it is a good way to describe the situation of many
+researchers who are presented with a new toolbox. Indeed, the situation of many researchers is that the projects they
+are working on are becoming more and more complex in terms of methods and data. Thus, we *open up the doors* to new
+possibilites:
+
+    | When I was younger, so much younger than today
+    | I never needed anybody's help in any way
+    | But now these days are gone, I'm not so self assured
+    | Now I find I've changed my mind and opened up the doors
+
+    -- The Beatles, Help!
+
+The setback with modern research is that current methods are often more complex and require more computing, which
+means that we need to learn concepts from computer science, mathematics, statistics, etc. This is a good thing, but it
+also means that we need to learn new tools and new ways of thinking, which can be a bit overwhelming, to the point that
+we start relying more and more on other researchers. In the end, we might feel like we lose our independence:
+
+    | And now my life has changed in oh so many ways
+    | My independence seems to vanish in the haze
+    | But every now and then I feel so insecure
+    | I know that I just need you like I've never done before
+
+    -- The Beatles, Help!
+
+We can continue with the song, but we think you get the point. The point is that we will need help, and we will need
+to ask for it. We will need to ask for help from our colleagues, from our supervisors, from our friends.
+
+We are a small team of researchers and developers, and we are not experts in *everything at once*. Each one of us has
+a specific expertise, and we are trying to use this expertise to create Junifer. When we conceived Junifer, we thought
+of researchers' problems and tried to come up with the best way to help them, by building a tool that is easy to
+understand, learn and use. Most importantly, we made it to help. We are here to help you and your research.
+
+If you have any questions, problems and / or suggestions, please do not hesitate to contact us. We will be happy to help you
+and we will be happy to hear from you.
+
+Seems nice, no? But we have one condition: **help us help you**.
+
+Communication is the key for you to help us and in turn help you solve your problems. We cannot know what you are
+trying to do, unless you tell us. **The more detailed explanation you give us, the faster we can help you**.
+We have opened several communication channels so that you can contact us in the way that is most convenient for you.
+
+Some people prefer to **write, in detail, with code and figures**. If you are one of those, use the
+`junifer Discussions`_. site on GitHub. This is a place where you can ask questions, and where you can discuss topics
+such as potential new features, or potential new methods.
+
+Some people do **not have a clear idea of what they want**, but they know that they need help. This is not a
+problem, but it is a bit more involved. Since it will require more frequent interactions to try to understand
+what you are trying to do, we have the `junifer matrix channel`_ in which you can chat with us and other junifer users.
+
+Finally, some people **prefer to communicate verbally**. If you are one of those, you might want to join our
+*office hours*. Given that our agenda might vary, office hours will be announced on the
+`junifer matrix channel`_ chat. Feel free to join and just *listen in* if you are too shy to write.
+
+In short, these are the 3 communication channels to get help:
+
+#. Discussions on Github (`junifer Discussions`_):
+
+   Pros:
+
+   * You can write your question in detail.
+   * Your question will not get mixed with other conversations.
+   * Likely to be the fastest way to solve the issue in the long-run.
+
+   Cons:
+
+   * You need to write your question in detail.
+
+#. Chat (`junifer matrix channel`_):
+
+   Pros:
+
+   * You can ask questions in real-time.
+   * You can including code and figures in your question.
+
+   Cons:
+
+   * Real-time depends on the availability of the other users.
+   * It might be difficult to follow if several conversation happens at the same time.
+
+#. Video Calls (*office hours*)
+
+   Pros:
+
+   * You can get real-time feedback.
+   * You will get our undivided attention (no overlapping discussions).
+
+   Cons:
+
+   * We can't read or write code, or even propose solutions for you to test.
+   * You will need to wait for the next *office hour*.

docs/index.rst

@@ -23,12 +23,14 @@ enabling others to extend it easily.
    :caption: Contents:
 
    installation
+   starting
    understanding/index.rst
    using/index.rst
    builtin
    extending/index.rst
    auto_examples/index.rst
    api/index.rst
+   help
    contribution
    maintaining
    faq

docs/links.inc

@@ -30,6 +30,7 @@
 .. _`Github`: https://github.com/
 .. _`junifer Github`: https://github.com/juaml/junifer
 .. _`junifer Discussions`: https://github.com/juaml/junifer/discussions
+.. _`junifer matrix channel`: https://matrix.to/#/#junifer-gen:inm7.de
 
 .. _`Wikipedia`: https://wikipedia.org
 .. _`YAML`: https://yaml.org

docs/starting.rst

@@ -0,0 +1,158 @@
+.. include:: links.inc
+
+.. _starting:
+
+First steps with Junifer
+========================
+
+
+.. note:: 
+  To scroll the graph left and right, click on the graph and use the arrows on your keyboard.
+
+.. mermaid::
+
+   flowchart TD
+     start((( Start )))
+     read_understanding(Read Understanding Junifer)
+     start --> read_understanding
+     question_features{Can I compute\nthe features I want\nusing junifer?}
+     read_understanding --> question_features
+     question_features -->|Yes| read_using_final
+     question_features -->|I'm not sure| read_using
+     question_features -->|No| question_feature_type
+     read_using --> question_features
+
+     read_using(Read Using Junifer)
+
+     question_feature_type{"What am I missing\nfrom junifer"}
+     question_feature_type --> missing_datagrabber
+     question_feature_type --> missing_preprocessing
+     question_feature_type --> missing_marker
+     question_feature_type --> missing_other
+     missing_datagrabber("A Dataset/DataGrabber")
+     missing_preprocessing("A Preprocessing")
+     missing_marker("A Marker")
+     missing_other("Something else")
+
+     missing_datagrabber --> question_datagrabber_junifarm
+     question_datagrabber_junifarm{Is the\ndataset/datagrabber\nin juni-farm?}
+     question_datagrabber_junifarm -->|Yes| read_using_final
+     question_datagrabber_junifarm -->|No| read_extending_datagrabber_start
+     read_extending_datagrabber_start(Read Creating a Junifer extension)
+     read_extending_datagrabber_start --> read_extending_datagrabber
+     read_extending_datagrabber(Read Creating Data Grabbers)
+     read_extending_datagrabber --> question_datagrabber_kind
+     question_datagrabber_kind{Is your dataset\na datalad one?}
+     question_datagrabber_kind -->|Yes| extend_datagrabber_datalad
+     question_datagrabber_kind -->|No| extend_datagrabber_pattern
+     extend_datagrabber_pattern(Use\nPatternDataGrabber)
+     extend_datagrabber_datalad(Use\nPatternDataladDataGrabber)
+     extend_datagrabber_pattern --> question_datagrabber_solved
+     extend_datagrabber_datalad --> question_datagrabber_solved
+     question_datagrabber_solved{Can you use\nyour data now?}
+     question_datagrabber_solved -->|Yes| question_contribute_datagrabber
+     question_datagrabber_solved -->|No| contact_help
+     question_contribute_datagrabber{Do you think\nyour DataGrabber\nis useful for other users?}
+     question_contribute_datagrabber -->|Yes| contribute_datagrabber
+     question_contribute_datagrabber -->|No| final_run
+     contribute_datagrabber(Create a\nDATASET REQUEST\nissue on Github)
+     contribute_datagrabber --> final_run
+
+     missing_marker --> question_marker_junifarm
+     question_marker_junifarm{Is the marker\nin juni-farm?}
+     question_marker_junifarm -->|Yes| read_using_final
+     question_marker_junifarm -->|No| read_extending_marker_start
+     read_extending_marker_start(Read Creating a Junifer extension)
+     read_extending_marker_start --> read_extending_marker
+     read_extending_marker(Read Creating Markers)
+     read_extending_marker --> question_marker_solved
+     question_marker_solved{Can you use\nyour marker now?}
+     question_marker_solved -->|Yes| question_contribute_marker
+     question_marker_solved -->|No| contact_help
+     question_contribute_marker{Do you think\nyour Marker\nis useful for other users?}
+     question_contribute_marker -->|Yes| contribute_marker
+     question_contribute_marker -->|No| final_run
+     contribute_marker(Create a\nMARKER REQUEST\nissue on Github)
+     contribute_marker --> final_run
+
+
+     missing_preprocessing --> contact_help
+
+     missing_mask{"A Mask?"}
+     missing_parcellation{"A Parcellation?"}
+     missing_coordinates{"Coordinates?"}
+     missing_other_other{"Something else?"}
+     missing_other --> missing_mask
+     missing_other --> missing_parcellation
+     missing_other --> missing_coordinates
+     missing_other --> missing_other_other
+     missing_other_other --> contact_help
+
+     contact_help(((Contact the\nJunifer team)))
+
+     missing_mask --> read_adding_mask_start
+     read_adding_mask_start("Read Creating a Junifer extension")
+     read_adding_mask_start --> read_adding_mask
+     read_adding_mask("Read Adding Masks")
+     read_adding_mask --> missing_other_solved
+
+     missing_parcellation --> read_adding_parcellation_start
+     read_adding_parcellation_start("Read Creating a Junifer extension")
+     read_adding_parcellation_start --> read_adding_parcellation
+     read_adding_parcellation("Read Adding Parcellations")
+     read_adding_parcellation --> missing_other_solved
+
+     missing_coordinates --> read_adding_coordinates_start
+     read_adding_coordinates_start("Read Creating a Junifer extension")
+     read_adding_coordinates_start --> read_adding_coordinates
+     read_adding_coordinates("Read Adding Coordinates")
+     read_adding_coordinates --> missing_other_solved
+
+     missing_other_solved{Did you solve your issue?}
+     missing_other_solved -->|Yes| read_using_final
+     missing_other_solved -->|No| missing_other_contact
+     missing_other_contact(Contact the\nJunifer team)
+     missing_other_contact --> missing_other_issue
+     missing_other_issue(((Submit a\nFEATURE REQUEST\nissue in Github)))
+
+     read_using_final(Read Using Junifer)
+     read_using_final --> final_yaml
+     final_yaml(Create/edit the YAML file)
+     final_yaml --> final_run
+     final_run(Use junifer run to test your YAML configuration)
+     final_run --> question_final_run_worked
+     question_final_run_worked{"Did it work?"}
+     question_final_run_worked -->|No| question_error_run
+     question_error_run{"Is it an issue\nwith my YAML file?"}
+     question_error_run -->|Yes| final_yaml
+     question_error_run -->|No| error_contact
+     error_contact(Contact the\nJunifer team)
+     error_contact --> error_issue
+     error_issue(((Submit a\nBUG REPORT issue\nin Github)))
+     question_final_run_worked -->|Yes| final_queue
+     final_queue(Use junifer queue to compute your features)
+     final_queue --> final_magic
+     final_magic(((Let junifer do its magic!)))
+
+     click read_understanding href "https://juaml.github.io/junifer/main/understanding/index.html"
+     click read_using href "https://juaml.github.io/junifer/main/using/index.html"
+     click read_using_final href "https://juaml.github.io/junifer/main/using/index.html"
+     click read_extending_datagrabber_start href "https://juaml.github.io/junifer/main/extending/extension.html"
+     click read_extending_datagrabber href "https://juaml.github.io/junifer/main/extending/datagrabber.html"
+     click read_extending_marker_start href "https://juaml.github.io/junifer/main/extending/extension.html"
+     click read_extending_marker href "https://juaml.github.io/junifer/main/extending/marker.html"
+     click read_adding_mask_start href "https://juaml.github.io/junifer/main/extending/extension.html"
+     click read_adding_mask href "https://juaml.github.io/junifer/main/extending/masks.html"
+     click read_adding_parcellation_start href "https://juaml.github.io/junifer/main/extending/extension.html"
+     click read_adding_parcellation href "https://juaml.github.io/junifer/main/extending/parcellations.html"
+     click read_adding_coordinates_start href "https://juaml.github.io/junifer/main/extending/extension.html"
+     click read_adding_coordinates href "https://juaml.github.io/junifer/main/extending/coordinates.html"
+
+     click error_issue href "https://github.com/juaml/junifer/issues/new/choose" _blank
+     click missing_other_issue href "https://github.com/juaml/junifer/issues/new/choose" _blank
+     click contribute_marker href "https://github.com/juaml/junifer/issues/new/choose" _blank
+     click contribute_datagrabber href "https://github.com/juaml/junifer/issues/new/choose" _blank
+
+     click error_contact href "https://juaml.github.io/junifer/main/help.html"
+     click contact_help href "https://juaml.github.io/junifer/main/help.html"
+     click missing_other_contact href "https://juaml.github.io/junifer/main/help.html"

docs/understanding/pipeline.rst

@@ -17,14 +17,52 @@ The element that is passed accross the pipeline is called the :ref:`Data Object<
 
 The following is a graphical representation of the pipeline:
 
-.. image:: ../images/pipeline/pipeline.001.png
+.. mermaid:: 
+
+   flowchart LR
+     dg[Data Grabber]
+     dr[Data Reader]
+     pp[Pre-processing]
+     mc[Marker Computation]
+     st[Storage]
+     dg --> dr
+     dr --> pp
+     pp --> mc
+     mc --> st
+
 
 However, it is usually the case that several markers are computed for the same data. Thus, the *markers* step 
 of the pipeline is defined as a list of markers. The following is a graphical representation of the pipeline execution
 on multiple markers:
 
-.. image:: ../images/pipeline/pipeline.002.png
+.. mermaid:: 
 
+   flowchart LR
+     dg[Data Grabber]
+     dr[Data Reader]
+     pp[Pre-processing]
+     mc1[Marker Computation]
+     mc2[Marker Computation]
+     mc3[Marker Computation]
+     mc4[Marker Computation]
+     mc5[Marker Computation]
+     st1[Storage]
+     st2[Storage]
+     st3[Storage]
+     st4[Storage]
+     st5[Storage]
+     dg --> dr
+     dr --> pp
+     pp --> mc1
+     pp --> mc2
+     pp --> mc3
+     pp --> mc4
+     pp --> mc5
+     mc1 --> st1
+     mc2 --> st2
+     mc3 --> st3
+     mc4 --> st4
+     mc5 --> st5
 
 .. note:: To avoid keeping in memory all of the computed marker, the storage step is called after each marker
     computation, releasing the memory used to compute each marker.