diff --git a/docs/dictionaries.md b/docs/data_models/dictionaries.md similarity index 99% rename from docs/dictionaries.md rename to docs/data_models/dictionaries.md index a0b3b10a..cf0f745f 100644 --- a/docs/dictionaries.md +++ b/docs/data_models/dictionaries.md @@ -2,7 +2,7 @@ ## Overview When you annotate a phenotypic TSV using the Neurobagel annotation tool -(see also the [section on the annotation tool](annotation_tool.md)), +(see also the [section on the annotation tool](../user_guide/annotation_tool.md)), your annotations are automatically stored in a JSON data dictionary. A Neurobagel data dictionary essentially describes the meaning and properties of columns and column values using standardized vocabularies. diff --git a/docs/graph_data.md b/docs/data_models/graph_data.md similarity index 96% rename from docs/graph_data.md rename to docs/data_models/graph_data.md index b0d1946b..992149b4 100644 --- a/docs/graph_data.md +++ b/docs/data_models/graph_data.md @@ -1,7 +1,7 @@ # Neurobagel graph data files ## Overview -Using the Neurobagel CLI (see also the [section on the CLI](cli.md)), +Using the Neurobagel CLI (see also the [section on the CLI](../user_guide/cli.md)), a Neurobagel data dictionary (`.json`) for a dataset can be processed together with the corresponding tabular data (`.tsv`) and BIDS dataset (if available) to generate subject-level [linked data](https://www.w3.org/wiki/LinkedData) that can be encoded in a knowledge graph. The Neurobagel graph-ready data are stored in [JSON-LD](https://json-ld.org/) format (`.jsonld`), and include a representation of each subject's harmonized phenotypic properties and imaging metadata. diff --git a/docs/term_naming_standards.md b/docs/data_models/term_naming_standards.md similarity index 100% rename from docs/term_naming_standards.md rename to docs/data_models/term_naming_standards.md diff --git a/docs/variables.md b/docs/data_models/variables.md similarity index 100% rename from docs/variables.md rename to docs/data_models/variables.md diff --git a/docs/getting_help.md b/docs/getting_help.md index c1e616aa..6546d8e7 100644 --- a/docs/getting_help.md +++ b/docs/getting_help.md @@ -25,5 +25,5 @@ We are always looking to improve our user documentation. To request a change or addition to existing documentation, please [open an issue in the `neurobagel/documentation` repo](https://github.com/neurobagel/documentation/issues). ## Propose a new variable to be added to the Neurobagel data model -If there is a variable you'd like to be able to harmonize and query using Neurobagel that is not [currently supported](variables.md), +If there is a variable you'd like to be able to harmonize and query using Neurobagel that is not [currently supported](data_models/variables.md), please suggest it using a discussion in our dedicated [GitHub Discussion category](https://github.com/orgs/neurobagel/discussions/categories/subject-data-model-requests). diff --git a/docs/glossary.md b/docs/glossary.md index 74301281..02b8853b 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -102,7 +102,7 @@ This glossary compiles some key terms used in the Neurobagel documentation and d of describing tabular demographic, cognitive, and/or clinical (phenotypic) data for a dataset with terms from controlled vocabularies to create machine understandable data dictionaries for the data. You can learn - more about this process in our [documentation](annotation_tool.md). + more about this process in our [documentation](user_guide/annotation_tool.md). ### Aggregated results : If the owner of a Neurobagel node decides that query responses diff --git a/docs/annotation_tool.md b/docs/user_guide/annotation_tool.md similarity index 91% rename from docs/annotation_tool.md rename to docs/user_guide/annotation_tool.md index efb0cfa2..132b49f3 100644 --- a/docs/annotation_tool.md +++ b/docs/user_guide/annotation_tool.md @@ -76,7 +76,7 @@ However, special criteria for each page need to be be met in order to move forwa The Home page is where you can upload data tables and dictionaries either for a brand new annotation or to continue a previous annotation session. -![Home page with uploaded table and dictionary files](./imgs/annotate/readme_home_page.png) +![Home page with uploaded table and dictionary files](../imgs/annotate/readme_home_page.png) #### Selecting a data table (participants.tsv) @@ -97,7 +97,7 @@ _Next page criteria:_ A `participants.tsv` file must be uploaded to proceed to c The Categorization page is where you link the columns in your data table to the categories found in Neurobagel's metadata schema. Current categories include 'Subject ID', 'Age', 'Sex' and 'Diagnosis'. Aside from the 'Subject ID' which is a special case, categories represent different data types, 'Sex' and 'Diagnosis' are categorical while 'Age' is continuous. -![Home page with uploaded table and dictionary files](./imgs/annotate/readme_categorization_page.png) +![Home page with uploaded table and dictionary files](../imgs/annotate/readme_categorization_page.png) #### Categorizing data table columns @@ -114,11 +114,11 @@ The Annotation page is where you can annotate the values in your uploaded `parti * When multiple columns have been linked with a category, each column will have its own column in the annotation components on the page. * Any value in the annotation interface can be marked as a 'missing value' (i.e. `'N/A'`, empty string, etc.) -![Home page with uploaded table and dictionary files](./imgs/annotate/readme_annotation_page.png) +![Home page with uploaded table and dictionary files](../imgs/annotate/readme_annotation_page.png) #### Age rows (continuous values) -![Home page with uploaded table and dictionary files](./imgs/annotate/readme_annotation_page_continuous_value.png) +![Home page with uploaded table and dictionary files](../imgs/annotate/readme_annotation_page_continuous_value.png) All values for columns categorized as `Age` can be annotated here with a set of continuous value transformations. Clicking on the dropdown will allow you to select a transformation. Current transformations available include `bounded`, `euro`, `float`, `int`, and `iso8601`. The raw values from the column are shown and when a transformation type is selected a preview of how that transformation would alter each raw value is shown. @@ -126,19 +126,19 @@ All values for columns categorized as `Age` can be annotated here with a set of All values for columns categorized as either `Sex` or `Diagnosis` can be annotated here with labels that come from Neurobagel and the controlled vocabularies it utilizes to help form the Neurobagel metadata model. Clicking on the drop down on each raw value row will allow you to select an appropriate Neurobagel label for this value. -![Home page with uploaded table and dictionary files](./imgs/annotate/readme_annotation_page_categorical_value.png) +![Home page with uploaded table and dictionary files](../imgs/annotate/readme_annotation_page_categorical_value.png) #### Missing values Any continuous or categorical value can be marked as 'missing' via the adjacent `Mark as missing` button. This will remove the value from the annotatable values and place it in the `Missing values` section on the page. This can be undone for any value by clicking the `Not Missing` button in this section of the page. -![Home page with uploaded table and dictionary files](./imgs/annotate/readme_annotation_page_missing_values.png) +![Home page with uploaded table and dictionary files](../imgs/annotate/readme_annotation_page_missing_values.png) #### Unlinking columns Data table columns can also be unlinked (e.g. un-categorized) on the Annotation page in this section of the page labeled `Review the annotated columns`. This will stop the column from being annotated and have any annotations made for its values removed. The change is also reflected on the previous Categorization page. -![Home page with uploaded table and dictionary files](./imgs/annotate/readme_annotation_page_unlinking.png) +![Home page with uploaded table and dictionary files](../imgs/annotate/readme_annotation_page_unlinking.png) _Next page criteria:_ At least one annotation must be made from any of the categorized columns @@ -146,4 +146,4 @@ _Next page criteria:_ At least one annotation must be made from any of the categ Click the download annotated data button to download what we refer to an 'annotated' data dictionary that is a Neurobagel-enhanced BIDS-style data dictionary. This file will include any entries in the original data dictionary that you uploaded on the home page. -![Home page with uploaded table and dictionary files](./imgs/annotate/readme_download_button.png) +![Home page with uploaded table and dictionary files](../imgs/annotate/readme_download_button.png) diff --git a/docs/api.md b/docs/user_guide/api.md similarity index 100% rename from docs/api.md rename to docs/user_guide/api.md diff --git a/docs/cli.md b/docs/user_guide/cli.md similarity index 99% rename from docs/cli.md rename to docs/user_guide/cli.md index 2c523c74..e2335a84 100644 --- a/docs/cli.md +++ b/docs/user_guide/cli.md @@ -39,7 +39,7 @@ The Neurobagel CLI can compile information from several different data sources t
- [A phenotypic TSV](./data_prep.md) -- [A Neurobagel JSON data dictionary](./dictionaries.md) for the TSV +- [A Neurobagel JSON data dictionary](../data_models/dictionaries.md) for the TSV - (Optional) The imaging dataset in [BIDS](https://bids-specification.readthedocs.io/en/stable/) format, if subjects have imaging data available (1) - (Optional) A TSV containing subject statuses for any image processing pipelines that have been run, following the [Nipoppy processing status file schema](https://nipoppy.readthedocs.io/en/latest/schemas/index.html#bagel-file) (2) diff --git a/docs/config.md b/docs/user_guide/config.md similarity index 93% rename from docs/config.md rename to docs/user_guide/config.md index d18abd59..51dac703 100644 --- a/docs/config.md +++ b/docs/user_guide/config.md @@ -113,24 +113,36 @@ To change the location of your password files, simply edit the variable `NB_GRAP Make sure to not share your password files with others. -## `local_nb_nodes.json` - -This file is used by the federation API, and so is only necessary for deployment profiles that include the f-API. -`local_nb_nodes.json` contains the URLs and (arbitrary) names of the **node APIs** of local nodes you wish to federate over. -Each node must be denoted by a dictionary `{}` with two key-value pairs: +## Configuring local node names and URLs for federation + +When using a deployment profile that provides federation (i.e., includes the federation API), you can define +the URLs and display names of the **node APIs** of any local nodes you wish to federate over in a file called `local_nb_nodes.json`. +This file is used by the f-API. + +Each node to be federated over is defined using a dictionary with two key-value pairs: +```json +{ + "NodeName": "", + "ApiURL": "" +} +``` -- `"NodeName"`: name of the node -- `"ApiURL"`: URL of the **node API** exposed for that node +Values of `NodeName` are arbitrary. Multiple nodes must be wrapped in a list `[]`. -Multiple nodes must be wrapped in a list `[]`. +!!! Info "Nodes that do not need to be manually configured" + We maintain a list of publicly accessible Neurobagel nodes + [here](https://github.com/neurobagel/menu/blob/main/node_directory/neurobagel_public_nodes.json). + By default, every new f-API will look up this list + on startup and include it in its internal list of nodes to + federate over (this can be disabled using the environment variable [`NB_FEDERATE_REMOTE_PUBLIC_NODES`](#environment-variables)). + This also means that **you do not have to explicitly add these public nodes** to your `local_nb_nodes.json` file. -Example: -Let's assume there are two local nodes already running on different servers of your institutional network, and you want to set up federation across both nodes: +Example: Assume there are two local nodes already running on different servers of your institutional network, and you want to set up federation across both nodes: - a node named `"My Institute"` running on your local computer (localhost), on port `8000` and - a node named `"Node Recruitment"` running on a different computer with the local IP `192.168.0.1`, listening on the default http port `80`. -In your `local_nb_nodes.json` file you would configure this as follows: +You would configure your `local_nb_nodes.json` as follows: ``` {.json title="local_nb_nodes.json"} [ { @@ -143,7 +155,6 @@ In your `local_nb_nodes.json` file you would configure this as follows: } ] ``` -Ensure that you not accidentally provide the address of your actual federation API for `"ApiURL"` - this will cause an infinite request loop that will likely overload your service (as an f-API will be repeatedly making requests to itself). !!! warning "Do not use `localhost`/`127.0.0.1` in `local_nb_nodes.json`" @@ -152,17 +163,9 @@ Ensure that you not accidentally provide the address of your actual federation A you cannot use `localhost` for the `"ApiURL"` and must instead provide a network-accessible URL, IP address, or container name. For an example, see the configuration for the node called `"My Institute"` above. +**Ensure that you not accidentally provide the address of your actual federation API for `"ApiURL"`!** This will cause an infinite request loop that will likely overload your service (as an f-API will be repeatedly making requests to itself). -!!! Info "Nodes that do not need to be manually configured" - We maintain a list of public Neurobagel nodes - [here](https://github.com/neurobagel/menu/blob/main/node_directory/neurobagel_public_nodes.json). - By default every new f-API will lookup this list - on startup and include it in the list of nodes to - federate over. - This also means that you do not have to manually - configure public nodes, i.e. you **do not have to explicitly add them** to your `local_nb_nodes.json` file. - -To add one or more local nodes to the list of nodes known to your f-API, simply add more dictionaries to this file. +To add one or more local nodes to the list of nodes known to your f-API, simply add more dictionaries to `local_nb_nodes.json`. ## Behind a reverse proxy @@ -478,7 +481,7 @@ you can use the hierarchical relationships between concepts themselves can also be represented. Including these relationships in a graph is important to be able to answer questions such as how many different diagnoses are represented in a graph database, to query for higher-order concepts for a given variable, and more. -The participant variables modeled by Neurobagel are named using Neurobagel's own vocabulary (for more information, see this page on [controlled terms](./term_naming_standards.md)). +The participant variables modeled by Neurobagel are named using Neurobagel's own vocabulary (for more information, see this page on [controlled terms](../data_models/term_naming_standards.md)). This vocabulary, which defines internal relationships between vocabulary terms, is serialized in the file [`nb_vocab.ttl`](https://github.com/neurobagel/recipes/blob/main/vocab/nb_vocab.ttl) available from the `neurobagel/recipes` repository. If you have cloned this repository, you will already have downloaded the vocabulary file. diff --git a/docs/data_prep.md b/docs/user_guide/data_prep.md similarity index 99% rename from docs/data_prep.md rename to docs/user_guide/data_prep.md index a776eb7a..fd2a9d45 100644 --- a/docs/data_prep.md +++ b/docs/user_guide/data_prep.md @@ -43,7 +43,7 @@ The TSV MUST **NOT** contain: - Missing values in the columns you plan to annotate as containing the primary subject IDs and session IDs (if available) -For all phenotypic variables currently modeled by Neurobagel, see the [data dictionary section](dictionaries.md). +For all phenotypic variables currently modeled by Neurobagel, see the [data dictionary section](../data_models/dictionaries.md). ### Datasets with imaging (BIDS) data diff --git a/docs/getting_started.md b/docs/user_guide/getting_started.md similarity index 98% rename from docs/getting_started.md rename to docs/user_guide/getting_started.md index 6f21369c..c58366e6 100644 --- a/docs/getting_started.md +++ b/docs/user_guide/getting_started.md @@ -5,7 +5,7 @@ and a local federation API (everything in blue in the picture below) that lets you search across the data in your node and in public Neurobagel nodes. -![Neurobagel node](imgs/neurobagel_local_node.jpg) +![Neurobagel node](../imgs/neurobagel_local_node.jpg) To prepare your Neurobagel node for production use (i.e., for local or other users), and to configure your deployment according to your specific needs, @@ -162,4 +162,4 @@ our [service profile documentation](config.md#available-profiles) for details. - [Prepare your own dataset](./data_prep.md) for annotation with Neurobagel - [Add your own data to your Neurobagel graph](maintaining.md#updating-the-data-in-your-graph) to search - Learn about the different [configuration options](config.md) for your Neurobagel node -- Hopefully all went well, but if you are experiencing issues, see how to [get help](./getting_help.md) \ No newline at end of file +- Hopefully all went well, but if you are experiencing issues, see how to [get help](../getting_help.md) \ No newline at end of file diff --git a/docs/overview.md b/docs/user_guide/index.md similarity index 100% rename from docs/overview.md rename to docs/user_guide/index.md diff --git a/docs/maintaining.md b/docs/user_guide/maintaining.md similarity index 100% rename from docs/maintaining.md rename to docs/user_guide/maintaining.md diff --git a/docs/public_nodes.md b/docs/user_guide/public_nodes.md similarity index 100% rename from docs/public_nodes.md rename to docs/user_guide/public_nodes.md diff --git a/docs/query_tool.md b/docs/user_guide/query_tool.md similarity index 100% rename from docs/query_tool.md rename to docs/user_guide/query_tool.md diff --git a/md_link_check_config.json b/md_link_check_config.json index ca232f39..f8367aad 100644 --- a/md_link_check_config.json +++ b/md_link_check_config.json @@ -8,7 +8,8 @@ {"pattern": "https://api.neurobagel.org/.*"}, {"pattern": "http://purl.org/nidash/nidm#.*"}, {"pattern": "http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl#"}, - {"pattern": "https://www.cognitiveatlas.org/task/id/"} + {"pattern": "https://www.cognitiveatlas.org/task/id/"}, + {"pattern": "http://purl.bioontology.org/ontology/SNOMEDCT/"} ], "timeout": "60s" } \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 95310658..e19f609c 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -24,24 +24,24 @@ repo_url: https://github.com/neurobagel/documentation nav: - Welcome: "index.md" - User guide: - - Overview: "overview.md" + - "user_guide/index.md" - Running cohort queries: - - Search the public nodes: "public_nodes.md" - - How to use the query tool: "query_tool.md" - - Querying the API directly: "api.md" + - Search the public nodes: "user_guide/public_nodes.md" + - How to use the query tool: "user_guide/query_tool.md" + - Querying the API directly: "user_guide/api.md" - Setting up your own Neurobagel node: - - Getting started: "getting_started.md" - - Configuring a node: "config.md" - - Maintaining a node: "maintaining.md" + - Getting started: "user_guide/getting_started.md" + - Configuring a node: "user_guide/config.md" + - Maintaining a node: "user_guide/maintaining.md" - Annotating your data: - - Preparing data for annotation: "data_prep.md" - - Annotation tool guide: "annotation_tool.md" - - Generating harmonized subject-level metadata: "cli.md" + - Preparing data for annotation: "user_guide/data_prep.md" + - Annotation tool guide: "user_guide/annotation_tool.md" + - Generating harmonized subject-level metadata: "user_guide/cli.md" - Data models: - - Harmonized variables: "variables.md" - - Augmented BIDS data dictionaries: "dictionaries.md" - - Data files for Neurobagel graphs: "graph_data.md" - - Naming conventions for terms: "term_naming_standards.md" + - Harmonized variables: "data_models/variables.md" + - Augmented BIDS data dictionaries: "data_models/dictionaries.md" + - Data files for Neurobagel graphs: "data_models/graph_data.md" + - Naming conventions for terms: "data_models/term_naming_standards.md" - Contributing: - How to contribute: "contributing/CONTRIBUTING.md" - Our team: "contributing/team.md"