From 5ab337313ddce6ce0aef87a3a2ee3496056b8d21 Mon Sep 17 00:00:00 2001 From: David Gardner <96306125+dagardner-nv@users.noreply.github.com> Date: Wed, 24 Jul 2024 13:53:05 -0700 Subject: [PATCH] Fix documentation links to work in both source repo and documentation builds (#1814) * Replace symlinks to markdown files under `examples/` with stub files using `include` directives. * Replace existing `https` to relative urls * Add stub file for the devcontainer instructions Closes #1783 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: https://github.com/nv-morpheus/Morpheus/pull/1814 --- .devcontainer/README.md | 2 +- docs/source/cloud_deployment_guide.md | 6 ++--- docs/source/conf.py | 2 +- docs/source/devcontainer.md | 25 +++++++++++++++++++ docs/source/examples.md | 2 +- .../examples/abp_nvsmi_detection/README.md | 22 +++++++++++++++- .../examples/abp_nvsmi_detection/pipeline.png | 1 - .../examples/abp_pcap_detection/README.md | 22 +++++++++++++++- docs/source/examples/doca/README.md | 22 +++++++++++++++- .../gnn_fraud_detection_pipeline/README.md | 22 +++++++++++++++- docs/source/examples/llm/agents/README.md | 22 +++++++++++++++- docs/source/examples/llm/completion/README.md | 22 +++++++++++++++- docs/source/examples/llm/rag/README.md | 22 +++++++++++++++- .../llm/rag/images/persistent_pipeline.png | 1 - docs/source/examples/llm/vdb_upload/README.md | 22 +++++++++++++++- docs/source/examples/log_parsing/README.md | 22 +++++++++++++++- .../examples/nlp_si_detection/README.md | 22 +++++++++++++++- .../examples/nlp_si_detection/pipeline.png | 1 - .../examples/ransomware_detection/README.md | 22 +++++++++++++++- .../examples/root_cause_analysis/README.md | 22 +++++++++++++++- .../examples/sid_visualization/README.md | 22 +++++++++++++++- .../digital_fingerprinting/starter/README.md | 2 +- examples/llm/agents/README.md | 2 +- 23 files changed, 306 insertions(+), 24 deletions(-) create mode 100644 docs/source/devcontainer.md mode change 120000 => 100644 docs/source/examples/abp_nvsmi_detection/README.md delete mode 120000 docs/source/examples/abp_nvsmi_detection/pipeline.png mode change 120000 => 100644 docs/source/examples/abp_pcap_detection/README.md mode change 120000 => 100644 docs/source/examples/doca/README.md mode change 120000 => 100644 docs/source/examples/gnn_fraud_detection_pipeline/README.md mode change 120000 => 100644 docs/source/examples/llm/agents/README.md mode change 120000 => 100644 docs/source/examples/llm/completion/README.md mode change 120000 => 100644 docs/source/examples/llm/rag/README.md delete mode 120000 docs/source/examples/llm/rag/images/persistent_pipeline.png mode change 120000 => 100644 docs/source/examples/llm/vdb_upload/README.md mode change 120000 => 100644 docs/source/examples/log_parsing/README.md mode change 120000 => 100644 docs/source/examples/nlp_si_detection/README.md delete mode 120000 docs/source/examples/nlp_si_detection/pipeline.png mode change 120000 => 100644 docs/source/examples/ransomware_detection/README.md mode change 120000 => 100644 docs/source/examples/root_cause_analysis/README.md mode change 120000 => 100644 docs/source/examples/sid_visualization/README.md diff --git a/.devcontainer/README.md b/.devcontainer/README.md index 8a91347e49..8a28296fe9 100644 --- a/.devcontainer/README.md +++ b/.devcontainer/README.md @@ -17,7 +17,7 @@ limitations under the License. # Morpheus Devcontainer -The Morpheus devcontainer is provided as a quick-to-set-up development and exploration environment for use with [Visual Studio Code](https://code.visualstudio.com) (Code). The devcontainer is a lightweight container which mounts-in a conda environment with cached packages, alleviating long conda download times on subsequent launches. It provides a simple framework for adding developer-centric [scripts](#scripts), and incorperates some helpful Code plugins, such as clangd and cmake support. +The Morpheus devcontainer is provided as a quick-to-set-up development and exploration environment for use with [Visual Studio Code](https://code.visualstudio.com) (Code). The devcontainer is a lightweight container which mounts-in a conda environment with cached packages, alleviating long conda download times on subsequent launches. It provides a simple framework for adding developer-centric [scripts](#development-scripts), and incorperates some helpful Code plugins, such as clangd and cmake support. More information about devcontainers can be found at [containers.dev](https://containers.dev/). diff --git a/docs/source/cloud_deployment_guide.md b/docs/source/cloud_deployment_guide.md index 31c3b9441a..0f6adc7345 100644 --- a/docs/source/cloud_deployment_guide.md +++ b/docs/source/cloud_deployment_guide.md @@ -402,7 +402,7 @@ To publish messages to a Kafka topic, we need to copy datasets to locations wher kubectl -n $NAMESPACE exec sdk-cli-helper -- cp -R /workspace/examples/data /common ``` -Refer to the [Morpheus CLI Overview](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/docs/source/basics/overview.rst) and [Building a Pipeline](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/docs/source/basics/building_a_pipeline.md) documentation for more information regarding the commands. +Refer to the [Morpheus CLI Overview](./basics/overview.rst) and [Building a Pipeline](./basics/building_a_pipeline.md) documentation for more information regarding the commands. > **Note**: Before running the example pipelines, ensure the criteria below are met: - Ensure models specific to the pipeline are deployed. @@ -782,8 +782,8 @@ kubectl -n $NAMESPACE exec deploy/broker -c broker -- kafka-topics.sh \ ## Additional Documentation For more information on how to use the Morpheus Python API to customize and run your own optimized AI pipelines, Refer to below documentation. -- [Morpheus Developer Guides](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/docs/source/developer_guide/guides.md) -- [Morpheus Pipeline Examples](https://github.com/nv-morpheus/Morpheus/tree/branch-24.10/examples) +- [Morpheus Developer Guides](./developer_guide/guides.md) +- [Morpheus Pipeline Examples](./examples.md) ## Troubleshooting diff --git a/docs/source/conf.py b/docs/source/conf.py index 709e8230e0..902d7e6210 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -50,7 +50,7 @@ from github_link import make_linkcode_resolve # noqa -# Set an environment variable we can use to determine ifuncf we are building docs +# Set an environment variable we can use to determine if we are building docs os.environ["MORPHEUS_IN_SPHINX_BUILD"] = "1" # -- Project information ----------------------------------------------------- diff --git a/docs/source/devcontainer.md b/docs/source/devcontainer.md new file mode 100644 index 0000000000..d6e7bd58a3 --- /dev/null +++ b/docs/source/devcontainer.md @@ -0,0 +1,25 @@ + + + +```{eval-rst} +:orphan: +``` +```{include} ../../.devcontainer/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/docs/source/examples.md b/docs/source/examples.md index 922bf796cb..b1f7d6b9d5 100644 --- a/docs/source/examples.md +++ b/docs/source/examples.md @@ -44,4 +44,4 @@ The following are the supported environments: | [Conda](./developer_guide/contributing.md#build-in-a-conda-environment) | Morpheus is built from source by the end user, and dependencies are installed via the Conda package manager. | | [Morpheus Docker Container](./developer_guide/contributing.md#build-in-docker-container) | A Docker container that is built from source by the end user, Morpheus is then built from source from within the container. | | [Morpheus Release Container](./getting_started.md#building-the-morpheus-container) | Pre-built Docker container that is built from source by the Morpheus team, and is available for download from the [NGC container registry](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/morpheus/containers/morpheus/tags), or can be built locally from source. | -| [Dev Container](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/.devcontainer/README.md) | A [Dev Container](https://containers.dev/) that is built from source by the end user, Morpheus is then built from source from within the container. | +| [Dev Container](./devcontainer.md) | A [Dev Container](https://containers.dev/) that is built from source by the end user, Morpheus is then built from source from within the container. | diff --git a/docs/source/examples/abp_nvsmi_detection/README.md b/docs/source/examples/abp_nvsmi_detection/README.md deleted file mode 120000 index 4a60061ddd..0000000000 --- a/docs/source/examples/abp_nvsmi_detection/README.md +++ /dev/null @@ -1 +0,0 @@ -../../../../examples/abp_nvsmi_detection/README.md \ No newline at end of file diff --git a/docs/source/examples/abp_nvsmi_detection/README.md b/docs/source/examples/abp_nvsmi_detection/README.md new file mode 100644 index 0000000000..3d1ddbab37 --- /dev/null +++ b/docs/source/examples/abp_nvsmi_detection/README.md @@ -0,0 +1,21 @@ + + +```{include} ../../../../examples/abp_nvsmi_detection/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/docs/source/examples/abp_nvsmi_detection/pipeline.png b/docs/source/examples/abp_nvsmi_detection/pipeline.png deleted file mode 120000 index 0af176f32c..0000000000 --- a/docs/source/examples/abp_nvsmi_detection/pipeline.png +++ /dev/null @@ -1 +0,0 @@ -../../../../examples/abp_nvsmi_detection/pipeline.png \ No newline at end of file diff --git a/docs/source/examples/abp_pcap_detection/README.md b/docs/source/examples/abp_pcap_detection/README.md deleted file mode 120000 index 3f20525785..0000000000 --- a/docs/source/examples/abp_pcap_detection/README.md +++ /dev/null @@ -1 +0,0 @@ -../../../../examples/abp_pcap_detection/README.md \ No newline at end of file diff --git a/docs/source/examples/abp_pcap_detection/README.md b/docs/source/examples/abp_pcap_detection/README.md new file mode 100644 index 0000000000..f6d91f484d --- /dev/null +++ b/docs/source/examples/abp_pcap_detection/README.md @@ -0,0 +1,21 @@ + + +```{include} ../../../../examples/abp_pcap_detection/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/docs/source/examples/doca/README.md b/docs/source/examples/doca/README.md deleted file mode 120000 index c3e43df7d9..0000000000 --- a/docs/source/examples/doca/README.md +++ /dev/null @@ -1 +0,0 @@ -../../../../examples/doca/README.md \ No newline at end of file diff --git a/docs/source/examples/doca/README.md b/docs/source/examples/doca/README.md new file mode 100644 index 0000000000..47573d0595 --- /dev/null +++ b/docs/source/examples/doca/README.md @@ -0,0 +1,21 @@ + + +```{include} ../../../../examples/doca/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/docs/source/examples/gnn_fraud_detection_pipeline/README.md b/docs/source/examples/gnn_fraud_detection_pipeline/README.md deleted file mode 120000 index 74d89a609c..0000000000 --- a/docs/source/examples/gnn_fraud_detection_pipeline/README.md +++ /dev/null @@ -1 +0,0 @@ -../../../../examples/gnn_fraud_detection_pipeline/README.md \ No newline at end of file diff --git a/docs/source/examples/gnn_fraud_detection_pipeline/README.md b/docs/source/examples/gnn_fraud_detection_pipeline/README.md new file mode 100644 index 0000000000..fdbb543a53 --- /dev/null +++ b/docs/source/examples/gnn_fraud_detection_pipeline/README.md @@ -0,0 +1,21 @@ + + +```{include} ../../../../examples/gnn_fraud_detection_pipeline/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/docs/source/examples/llm/agents/README.md b/docs/source/examples/llm/agents/README.md deleted file mode 120000 index 25df8ebb21..0000000000 --- a/docs/source/examples/llm/agents/README.md +++ /dev/null @@ -1 +0,0 @@ -../../../../../examples/llm/agents/README.md \ No newline at end of file diff --git a/docs/source/examples/llm/agents/README.md b/docs/source/examples/llm/agents/README.md new file mode 100644 index 0000000000..6b0a454800 --- /dev/null +++ b/docs/source/examples/llm/agents/README.md @@ -0,0 +1,21 @@ + + +```{include} ../../../../../examples/llm/agents/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/docs/source/examples/llm/completion/README.md b/docs/source/examples/llm/completion/README.md deleted file mode 120000 index acf33313c1..0000000000 --- a/docs/source/examples/llm/completion/README.md +++ /dev/null @@ -1 +0,0 @@ -../../../../../examples/llm/completion/README.md \ No newline at end of file diff --git a/docs/source/examples/llm/completion/README.md b/docs/source/examples/llm/completion/README.md new file mode 100644 index 0000000000..215564a75c --- /dev/null +++ b/docs/source/examples/llm/completion/README.md @@ -0,0 +1,21 @@ + + +```{include} ../../../../../examples/llm/completion/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/docs/source/examples/llm/rag/README.md b/docs/source/examples/llm/rag/README.md deleted file mode 120000 index 4d253e05dd..0000000000 --- a/docs/source/examples/llm/rag/README.md +++ /dev/null @@ -1 +0,0 @@ -../../../../../examples/llm/rag/README.md \ No newline at end of file diff --git a/docs/source/examples/llm/rag/README.md b/docs/source/examples/llm/rag/README.md new file mode 100644 index 0000000000..50cca84d70 --- /dev/null +++ b/docs/source/examples/llm/rag/README.md @@ -0,0 +1,21 @@ + + +```{include} ../../../../../examples/llm/rag/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/docs/source/examples/llm/rag/images/persistent_pipeline.png b/docs/source/examples/llm/rag/images/persistent_pipeline.png deleted file mode 120000 index f74a9ea3f9..0000000000 --- a/docs/source/examples/llm/rag/images/persistent_pipeline.png +++ /dev/null @@ -1 +0,0 @@ -../../../../../../examples/llm/rag/images/persistent_pipeline.png \ No newline at end of file diff --git a/docs/source/examples/llm/vdb_upload/README.md b/docs/source/examples/llm/vdb_upload/README.md deleted file mode 120000 index 8e4daece9f..0000000000 --- a/docs/source/examples/llm/vdb_upload/README.md +++ /dev/null @@ -1 +0,0 @@ -../../../../../examples/llm/vdb_upload/README.md \ No newline at end of file diff --git a/docs/source/examples/llm/vdb_upload/README.md b/docs/source/examples/llm/vdb_upload/README.md new file mode 100644 index 0000000000..8acf34e387 --- /dev/null +++ b/docs/source/examples/llm/vdb_upload/README.md @@ -0,0 +1,21 @@ + + +```{include} ../../../../../examples/llm/vdb_upload/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/docs/source/examples/log_parsing/README.md b/docs/source/examples/log_parsing/README.md deleted file mode 120000 index ee8517820f..0000000000 --- a/docs/source/examples/log_parsing/README.md +++ /dev/null @@ -1 +0,0 @@ -../../../../examples/log_parsing/README.md \ No newline at end of file diff --git a/docs/source/examples/log_parsing/README.md b/docs/source/examples/log_parsing/README.md new file mode 100644 index 0000000000..be0450c9f7 --- /dev/null +++ b/docs/source/examples/log_parsing/README.md @@ -0,0 +1,21 @@ + + +```{include} ../../../../examples/log_parsing/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/docs/source/examples/nlp_si_detection/README.md b/docs/source/examples/nlp_si_detection/README.md deleted file mode 120000 index 8dbdbc0f30..0000000000 --- a/docs/source/examples/nlp_si_detection/README.md +++ /dev/null @@ -1 +0,0 @@ -../../../../examples/nlp_si_detection/README.md \ No newline at end of file diff --git a/docs/source/examples/nlp_si_detection/README.md b/docs/source/examples/nlp_si_detection/README.md new file mode 100644 index 0000000000..72cb1d280e --- /dev/null +++ b/docs/source/examples/nlp_si_detection/README.md @@ -0,0 +1,21 @@ + + +```{include} ../../../../examples/nlp_si_detection/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/docs/source/examples/nlp_si_detection/pipeline.png b/docs/source/examples/nlp_si_detection/pipeline.png deleted file mode 120000 index 393ae7e8fd..0000000000 --- a/docs/source/examples/nlp_si_detection/pipeline.png +++ /dev/null @@ -1 +0,0 @@ -../../../../examples/nlp_si_detection/pipeline.png \ No newline at end of file diff --git a/docs/source/examples/ransomware_detection/README.md b/docs/source/examples/ransomware_detection/README.md deleted file mode 120000 index ef96606ec5..0000000000 --- a/docs/source/examples/ransomware_detection/README.md +++ /dev/null @@ -1 +0,0 @@ -../../../../examples/ransomware_detection/README.md \ No newline at end of file diff --git a/docs/source/examples/ransomware_detection/README.md b/docs/source/examples/ransomware_detection/README.md new file mode 100644 index 0000000000..8a2583d846 --- /dev/null +++ b/docs/source/examples/ransomware_detection/README.md @@ -0,0 +1,21 @@ + + +```{include} ../../../../examples/ransomware_detection/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/docs/source/examples/root_cause_analysis/README.md b/docs/source/examples/root_cause_analysis/README.md deleted file mode 120000 index af7e9e0e4b..0000000000 --- a/docs/source/examples/root_cause_analysis/README.md +++ /dev/null @@ -1 +0,0 @@ -../../../../examples/root_cause_analysis/README.md \ No newline at end of file diff --git a/docs/source/examples/root_cause_analysis/README.md b/docs/source/examples/root_cause_analysis/README.md new file mode 100644 index 0000000000..3111f15a74 --- /dev/null +++ b/docs/source/examples/root_cause_analysis/README.md @@ -0,0 +1,21 @@ + + +```{include} ../../../../examples/root_cause_analysis/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/docs/source/examples/sid_visualization/README.md b/docs/source/examples/sid_visualization/README.md deleted file mode 120000 index a1042736b8..0000000000 --- a/docs/source/examples/sid_visualization/README.md +++ /dev/null @@ -1 +0,0 @@ -../../../../examples/sid_visualization/README.md \ No newline at end of file diff --git a/docs/source/examples/sid_visualization/README.md b/docs/source/examples/sid_visualization/README.md new file mode 100644 index 0000000000..f429e3123b --- /dev/null +++ b/docs/source/examples/sid_visualization/README.md @@ -0,0 +1,21 @@ + + +```{include} ../../../../examples/sid_visualization/README.md +:relative-docs: .. +:relative-images: +``` diff --git a/examples/digital_fingerprinting/starter/README.md b/examples/digital_fingerprinting/starter/README.md index 5360ad2255..75c6040707 100644 --- a/examples/digital_fingerprinting/starter/README.md +++ b/examples/digital_fingerprinting/starter/README.md @@ -22,7 +22,7 @@ We show here how to set up and run the DFP pipeline for three log types: CloudTr ## Environment Setup -Follow the instructions [here](https://github.com/nv-morpheus/Morpheus/blob/branch-24.10/docs/source/developer_guide/contributing.md) to set up your development environment in either a Docker container or conda environment. +Follow the instructions [here](../../../docs/source/developer_guide/contributing.md) to set up your development environment in either a Docker container or conda environment. ## Morpheus CLI diff --git a/examples/llm/agents/README.md b/examples/llm/agents/README.md index 7f51397c14..19b2b33cd8 100644 --- a/examples/llm/agents/README.md +++ b/examples/llm/agents/README.md @@ -161,7 +161,7 @@ The Kafka Example in the Morpheus LLM Agents demonstrates an streaming implement facilitate the near real-time processing of LLM queries. This example is similar to the Simple example but makes use of a KafkaSourceStage to stream and retrieve messages from the Kafka topic -First, to run the Kafka example, you need to create a Kafka cluster that enables the persistent pipeline to accept queries for the LLM agents. You can create the Kafka cluster using the following guide: [Quick Launch Kafka Cluster Guide](https://github.com/nv-morpheus/Morpheus/blob/branch-23.11/docs/source/developer_guide/contributing.md#quick-launch-kafka-cluster) +First, to run the Kafka example, you need to create a Kafka cluster that enables the persistent pipeline to accept queries for the LLM agents. You can create the Kafka cluster using the following guide: [Quick Launch Kafka Cluster Guide](../../../docs/source/developer_guide/contributing.md#quick-launch-kafka-cluster) Once the Kafka cluster is running, create Kafka topic to produce input to the pipeline.