Merge pull request #140 from quixio/dev

Docs Release 2023-04-003

README.md

@@ -3,63 +3,42 @@
 [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 [![Build](https://github.com/quixio/quix-docs/actions/workflows/sync-build-deploy.yaml/badge.svg)](https://github.com/quixio/quix-docs)
 
-This repository is the source content for the Quix documentation that is published on the web at https://quix.io/docs. The source files for the documentation are in GitHub-flavoured Markdown, with additions supported by our tooling, [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/).
+This repository is the source content for the Quix documentation that is published on the web at https://quix.io/docs. The source files for the documentation are in GitHub-flavoured Markdown, with the additions supported by our tooling, [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/).
 
 To get a free Quix account, [sign up](https://portal.platform.quix.ai/self-sign-up).
 
 ## Contributing
 
-If you would like to contribute to these docs, see the [Contribution Guide](./CONTRIBUTING.md). 
+If you would like to contribute to the docs, there are two main ways:
 
-If you create a PR on the docs `dev` branch, a review app consisting of a complete rendered docs build is created automatically for you, so you can preview your changes there. Currently, to rebuild the docs after pushing up any further changes, you need to close and reopen your PR.
+1. **Create a GitHub issue** - Create a GitHub issue and describe the bug or change you would like to see in the docs.
+2. **Create a GitHub Pull Request (PR)** - This would contain the patch you would like to apply to the docs.
 
-**IMPORTANT:** All docs PRs should be raised against the `dev` branch.
+> **IMPORTANT:** All docs PRs should be raised against the `dev` branch.
 
-If you plan to make more than basic changes to the documentation, you should also become familiar with the Quix [Writing Style Guide](./WRITING-STYLE.md) and the [Best Practice Guide](./BEST-PRACTICE.md).
+Your PR will be reviewed by a member of the Quix Docs team, and from that point on the standard GitHub workflow is applied. A member of the team will possibly suggest changes, but if everything is OK the PR will be approved and merged for you.
 
-## Running docs locally
+### Docs preview site
 
-The following is only required if you are part of the Quix technical writing team, or you contribute frequently, and want to check your docs PR locally before pushing up. Otherwise, the simplest approach is to refer to the review app that is created automatically for you, when you push up a PR on the `dev` branch.
+If you create a PR on the docs `dev` branch, a docs preview site consisting of a complete rendered docs build is created automatically for you, so you can see your changes there. 
 
-### Prerequisites
+> **NOTE:** Currently, to rebuild the docs after pushing up any further changes, you need to close and reopen your PR.
 
-To run these docs locally you'll need:
+To access the preview site follow the link generated for you on the PR, as shown in the following screenshot:
 
-* [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/)
-* Sign up to the [Insiders Programme](https://squidfunk.github.io/mkdocs-material/insiders/), if you want to see all features rendered locally.
-* A Git client (the command line is fine).
+![Docs preview site](./docs-preview-site.png)
 
-### Plugins used
+### Contribution guides
 
-If you want to fully render all documentation locally you will need to install the following plugins with `pip install`:
+If you plan to make more than basic changes to the documentation, you should become familiar with our contribution guides. There are three main guides in addition to this README:
 
-* [glightbox](https://pypi.org/project/mkdocs-glightbox/0.1.0/)
-* [multirepo](https://pypi.org/project/mkdocs-multirepo/)
-* [redirects](https://pypi.org/project/mkdocs-redirects/)
+1. [Contribution guide](./CONTRIBUTING.md) - more detailed notes on how to contribute through a GitHub issue or PR.
+2. [Writing Style Guide](./WRITING-STYLE.md) - this covers our documentation writing style in detail. Generally you should write in the second person, present tense, active voice, using International English. Try to use simple, directive language. Further details are contained in the guide.
+3. [Best Practice Guide](./BEST-PRACTICE.md) - this covers our general aproach to technical writing, which is to use topic-based writing, with effective incremental search, and structured navigation. You don't need to be familiar with this to contribute, but it's a useful resource if you're interested in making more in-depth contributions.
 
-You will also need to sign up to the [Insiders Programme](https://squidfunk.github.io/mkdocs-material/insiders/).
+### Running docs locally
 
-The [social plugin](https://squidfunk.github.io/mkdocs-material/setup/setting-up-social-cards/) is also used to automatically provide metadata. See the [social plugin documentation](https://squidfunk.github.io/mkdocs-material/setup/setting-up-social-cards/) for details on how to install the dependencies of the plugin. You might also need to first install `cffi` with `pip install cffi`, if not present on your system.
-
-### Linked repositories
-
-This repo uses the `multirepo` plugin to pull in client library content from the [Quix Streams repo](https://github.com/quixio/quix-streams). 
-
-You can read more about Quix Streams [here](https://github.com/quixio/quix-streams/blob/main/README.md).
-
-### Viewing the docs
-
-To view the docs locally:
-
-1. Follow the install guide for Material [here](https://squidfunk.github.io/mkdocs-material/getting-started/).
-2. Clone the repo as follows:
-
-   ```
-   git clone https://github.com/quixio/quix-docs.git
-   ```
-3. Change into the docs directory you cloned (there will be a `mkdocs.yml` file there). 
-4. Run `mkdocs serve`.
-5. Navigate your browser to `localhost:8000` to view the docs.
+If you are part of the Quix technical writing team, or you contribute frequently, and want to check your docs PR locally before pushing up, please read our guide on [running the docs locally](./RUNNING-DOCS-LOCALLY.md). Otherwise, the simplest approach is to refer to the documentation preview site that is created automatically for you, when you push up a PR on the `dev` branch.
 
 ## Getting in touch
 

RUNNING-DOCS-LOCALLY.md

@@ -0,0 +1,43 @@
+# Running docs locally
+
+The following is only required if you are part of the Quix technical writing team, or you contribute frequently, and want to check your docs PR locally before pushing up to the docs repo. Otherwise, the simplest approach is to refer to the [docs preview site](./README.md#docs-preview-site) that is created automatically for you, when you push up a PR on the `dev` branch.
+
+## Prerequisites
+
+To run these docs locally you'll need:
+
+* [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/)
+* Sign up to the [Insiders Programme](https://squidfunk.github.io/mkdocs-material/insiders/), if you want to see all features rendered locally.
+* A Git client (the command line is fine).
+
+## Plugins used
+
+If you want to fully render all documentation locally you need to install the following plugins with `pip install`:
+
+* [glightbox](https://pypi.org/project/mkdocs-glightbox/0.1.0/)
+* [multirepo](https://pypi.org/project/mkdocs-multirepo/)
+* [redirects](https://pypi.org/project/mkdocs-redirects/)
+
+You also need to sign up to the [Insiders Programme](https://squidfunk.github.io/mkdocs-material/insiders/).
+
+The [social plugin](https://squidfunk.github.io/mkdocs-material/setup/setting-up-social-cards/) is also used to automatically provide metadata. See the [social plugin documentation](https://squidfunk.github.io/mkdocs-material/setup/setting-up-social-cards/) for details on how to install the dependencies of the plugin. You might also need to first install `cffi` with `pip install cffi`, if not present on your system.
+
+## Linked repositories
+
+This repo uses the `multirepo` plugin to pull in client library content from the [Quix Streams repo](https://github.com/quixio/quix-streams). 
+
+You can read more about Quix Streams [here](https://github.com/quixio/quix-streams/blob/main/README.md).
+
+## Viewing the docs
+
+To view the docs locally:
+
+1. Follow the install guide for Material [here](https://squidfunk.github.io/mkdocs-material/getting-started/).
+2. Clone the repo as follows:
+
+   ```
+   git clone https://github.com/quixio/quix-docs.git
+   ```
+3. Change into the docs directory you cloned (there will be a `mkdocs.yml` file there). 
+4. Run `mkdocs serve`.
+5. Navigate your browser to `localhost:8000` to view the docs.

docs/apis/data-catalogue-api/aggregate-tags.md

@@ -6,7 +6,7 @@ you’ll want to group results by that tag. You can do so via the
 
 ## Before you begin
 
-  - If you don’t already have any Stream data in your workspace, you can use any Source of our [Quix Samples](../../platform/samples/samples.md) to set some up.
+  - If you don’t already have any Stream data in your workspace, you can use any Source from our [Code Samples](../../platform/samples/samples.md) to set some up.
 
   - [Get a Personal Access Token](authenticate.md)
     to authenticate each request.

docs/apis/data-catalogue-api/aggregate-time.md

@@ -5,7 +5,7 @@ You can downsample and upsample data from the catalogue using the
 
 ## Before you begin
 
-  - If you don’t already have any Stream data in your workspace, you can use any Source of our [Quix Samples](../../platform/samples/samples.md) to set some up.
+  - If you don’t already have any Stream data in your workspace, you can use any Source from our [Code Samples](../../platform/samples/samples.md) to set some up.
 
   - [Get a Personal Access Token](authenticate.md)
     to authenticate each request.

docs/apis/data-catalogue-api/filter-tags.md

@@ -5,7 +5,7 @@ so they can be used to efficiently filter data.
 
 ## Before you begin
 
-  - If you don’t already have any Stream data in your workspace, you can use any Source of our [Quix Samples](../../platform/samples/samples.md) to set some up.
+  - If you don’t already have any Stream data in your workspace, you can use any Source from our [Code Samples](../../platform/samples/samples.md) to set some up.
 
   - [Get a Personal Access Token](authenticate.md)
     to authenticate each request.

docs/apis/data-catalogue-api/intro.md

@@ -18,7 +18,7 @@ how to [form a typical request to the
 API](request.md).
 
 You’ll also need to have some data stored in the Quix platform for API
-use to be meaningful. You can use any Source of our [Quix Samples](../../platform/samples/samples.md) to do this using the Quix
+use to be meaningful. You can use any Source from our [Code Samples](../../platform/samples/samples.md) to do this using the Quix
 portal.
 
 ## Further documentation

docs/apis/data-catalogue-api/raw-data.md

@@ -6,7 +6,7 @@ results.
 
 ## Before you begin
 
-  - If you don’t already have any Stream data in your workspace, you can use any Source of our [Quix Samples](../../platform/samples/samples.md) to set some up.
+  - If you don’t already have any Stream data in your workspace, you can use any Source from our [Code Samples](../../platform/samples/samples.md) to set some up.
 
   - [Get a Personal Access Token](authenticate.md)
     to authenticate each request.

docs/apis/data-catalogue-api/streams-filtered.md

@@ -5,7 +5,7 @@ request to the `/streams` endpoint.
 
 ## Before you begin
 
-  - If you don’t already have any Stream data in your workspace, you can use any Source of our [Quix Samples](../../platform/samples/samples.md) to set some up.
+  - If you don’t already have any Stream data in your workspace, you can use any Source from our [Code Samples](../../platform/samples/samples.md) to set some up.
 
   - [Get a Personal Access Token](authenticate.md)
     to authenticate each request.

docs/apis/data-catalogue-api/streams-models.md

@@ -6,7 +6,7 @@ endpoint.
 
 ## Before you begin
 
-  - If you don’t already have any Stream data in your workspace, you can use any Source of our [Quix Samples](../../platform/samples/samples.md) to set some up.
+  - If you don’t already have any Stream data in your workspace, you can use any Source from our [Code Samples](../../platform/samples/samples.md) to set some up.
 
   - [Get a Personal Access Token](authenticate.md)
     to authenticate each request.

docs/apis/data-catalogue-api/streams-paged.md

@@ -8,7 +8,7 @@ use pagination parameters to group the results into smaller pages.
 
 ## Before you begin
 
-  - If you don’t already have any Stream data in your workspace, you can use any Source of our [Quix Samples](../../platform/samples/samples.md) to set some up.
+  - If you don’t already have any Stream data in your workspace, you can use any Source from our [Code Samples](../../platform/samples/samples.md) to set some up.
 
   - [Get a Personal Access Token](authenticate.md)
     to authenticate each request.

docs/platform/connectors/index.md

@@ -4,6 +4,6 @@ Connectors are part of our [open source](https://github.com/quixio/quix-samples)
 
 Connectors help our users connect with other vendors such as AWS and Kafka.
 
-You can explore the connector README files here in Quix Docs. When you are ready to start using them, head over to the Quix Samples [GitHub](https://github.com/quixio/quix-samples){target="_blank"} repository, or [sign up](https://quix.io/signup){target="_blank"} and [login to the platform](https://portal.platform.quix.ai/){target="_blank"}.
+You can explore the connector README files here in Quix Docs. When you are ready to start using them, head over to the Quix Code Samples [GitHub](https://github.com/quixio/quix-samples){target="_blank"} repository, or [sign up](https://quix.io/signup){target="_blank"} and [login to the platform](https://portal.platform.quix.ai/){target="_blank"}.
 
 [//]: <> (#connectors_tile_replacement)

docs/platform/how-to/deploy-public-page.md

@@ -4,13 +4,13 @@ The Quix SaaS platform allows you to deploy public-facing web pages and APIs.
 
 This how-to will help to explain the features and options and ensure projects containing public facing web pages and APIs are successful.
 
-## Samples
+## Code Samples
 
-In our samples you can find our `Web API Template` which demonstrates how to create a API using `Node`.
+In our Code Samples you can find our `Web API Template` which demonstrates how to create a API using `Node`.
 
 There are also examples dashboard and web/UI examples using `Dash`, `Streamlit` and `Angular`.
 
-![Samples](../images/library.png){width=500px}
+![Code Samples](../images/library.png){width=500px}
 
 ## The code
 
@@ -42,7 +42,7 @@ To access your public facing service or web site you must enable `Public Access`
 
 ## Security
 
-Please note that the basic examples, included in our Quix Samples, do not include any security features and come with no warranty.
+Please note that the basic examples, included in our Code Samples, do not include any security features and come with no warranty.
 
 Quix advises you to build in a security layer to ensure your data is secure and only the intended recipients have access to it.
 

docs/platform/how-to/jupyter-nb.md

@@ -12,8 +12,8 @@ analysis so much easier.
 
 ## Preparation
 
-You’ll need some data stored in the Quix platform. You can use any of
-our Data Sources available in the Quix Samples, or just follow the
+You`ll need some data stored in the Quix platform. You can use any of
+our Data Sources available in the Code Samples, or just follow the
 onboarding process when you [sign-up to
 Quix](https://portal.platform.quix.ai/self-sign-up?xlink=docs){target=_blank}.
 

docs/platform/how-to/replay.md

@@ -0,0 +1,45 @@
+# How to replay data
+
+Quix Platform features a **replay service**. This service allows you to replay persisted data into a topic, as if it were live data. This is very useful for the following use cases:
+
+* Testing and debugging connectors and transforms
+* [Stream reprocessing](https://quix.io/blog/intro-stream-reprocessing-python/?x-craft-preview=VDVjwJTquq&token=7oGSdC9yxYk0zECNUz2RtzJLtGqG-aZB){target=_blank}
+* Testing and retraining ML models
+
+Once you have persisted data, it is possible to set up a replay service in your pipeline. Rather than using the live source of data, you can replay the persisted data into a topic in your pipeline, and monitor the processing. 
+
+It is possible to configure the replay in various ways, for example:
+
+* You can edit the destination topic. This means you can play back the same persisted data into different topics for testing and debugging.
+* You can replay the data at the original speed or at maximum speed.
+* You can replay the data with either the original timestamps, or simulated timestamps. The simulated timestamps are based on present time, rather than the original timestamps.
+
+Once created, a replay service looks like any other service in your pipeline, and you can play or pause the streaming of data as required.
+
+## To replay persisted data into a topic
+
+You can only replay persisted data, so you need to persist a topic first. In your Quix workspace, select `Topics` in the left-hand sidebar to list all your topics. Ensure the topic you want to persist has persistence switched on, as shown in the following screenshot:
+
+![Enable persistence](../images/how-to/replay/replay-add-persist-topic.png)
+
+Click `Pipeline` in your Quix workspace, and then to create a new replay, click `Add new` in the top right corner:
+
+![Add replay](../images/how-to/replay/replay-add-new.png)
+
+Then select the `Replay` menu item. This displays the `Add replay` wizard. 
+
+Select the topic with persisted data you want to replay from (only topics with persisted data will be shown), then select the stream whose data you want to replay:
+
+![Replay select stream](../images/how-to/replay/replay-select-stream.png)
+
+Then click `Next`.
+
+Now configure your replay according to your use case. For example, you can retain the data’s original timestamps, or select simulated timestamps for timestamps based on the present time. You also need to select the output topic (the topic the data is played into):
+
+![Configure replay](../images/how-to/replay/replay-configuration.png)
+
+Once you have completed your configuration, click `Add replay`.
+
+Switch to `Pipeline` view. The replay is created as a service in your pipeline. The service may take some time to load. You can then replay data into your pipeline using the play/pause control, as if it were live data:
+
+![Replay persisted data](../images/how-to/replay/replay-play-data.png)

docs/platform/how-to/webapps.md

@@ -13,9 +13,9 @@ data plane.
   - Social networking applications that require broadcasting updates to
     many users at high frequency like live sharing of Strava data.
 
-## NodeJs
+## Node.js
 
-NodeJs applications can update parameter and event definitions and write
+Node.js applications can update parameter and event definitions and write
 data to streams using RESTful APIs. Quix supports WebSockets for clients
 that want to receive telemetry data and parameters/events updates in
-real-time. NodeJs clients must authenticate with Quix using [personal access tokens](../../apis/streaming-reader-api/authenticate.md#get-a-personal-access-token).
+real-time. Node.js clients must authenticate with Quix using [personal access tokens](../../apis/streaming-reader-api/authenticate.md#get-a-personal-access-token).

docs/platform/how-to/webapps/read.md

@@ -1,4 +1,4 @@
-# Read from Quix with NodeJs
+# Read from Quix with Node.js
 
 Quix supports real-time data streaming over WebSockets. JavaScript
 clients can receive updates on parameter and event definition updates,

docs/platform/how-to/webapps/write.md

@@ -1,4 +1,4 @@
-# Write to Quix with NodeJs
+# Write to Quix with Node.js
 
 Clients write data to Quix using streams opened on existing
 [topics](../../../platform/definitions.md#topics). Therefore, you need to