Skip to content

Commit

Permalink
adding docker image to reference (#74)
Browse files Browse the repository at this point in the history
  • Loading branch information
emanuel-quix authored Nov 22, 2024
1 parent 12cb617 commit 67394d8
Showing 1 changed file with 35 additions and 30 deletions.
65 changes: 35 additions & 30 deletions docs/yaml-reference/pipeline-descriptor.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Pipeline YAML descriptor (quix.yaml)
# Pipeline YAML Descriptor (quix.yaml)

The `quix.yaml` file serves as the Infrastructure as Code (IaC) descriptor for a Quix project. It defines the data pipeline, including the applications (deployments) and topics that compose the pipeline. This file allows you to configure and manage your data pipeline in a declarative manner, ensuring consistency across environments.

Expand Down Expand Up @@ -57,40 +57,47 @@ deployments:
description: This is the input topic to read data from
required: true
value: csv-data
- name: custom-service
image: custom-repo/my-service:1.2.3
deploymentType: Service
resources:
cpu: 300
memory: 600
replicas: 2
desiredStatus: Running
```

**Explanation:**
#### Explanation

- **name:** The identifier for the deployment within your pipeline. It's typically a descriptive name of what the deployment does.

- **application:** Specifies the name of the application or microservice to deploy. This should match the application folder where the code resides.
- **image:** The full Docker image reference (including repository and tag) to deploy. Used instead of `application` and `version` when deploying a custom Docker image.
- **deploymentType:** Defines the type of deployment. Common types include `Service` for long-running services, `Job` for tasks that run to completion, and others as required by your architecture.

- **version:** Indicates the version of the application to deploy. Using `latest` ensures that the most recent version is deployed, but this can be set to a specific version to ensure consistency. You can also use a commit reference in order to have a specific version.

- **version:** Indicates the version of the application to deploy. Using `latest` ensures that the most recent version is deployed, but this can be set to a specific version to ensure consistency. You can also use a commit reference to specify a specific version.
- **resources:** Defines the compute resources allocated to the deployment. This includes:
- **cpu:** The amount of CPU resources allocated (in millicores).
- **memory:** The amount of memory allocated (in megabytes).
- **replicas:** The number of instances of the application to run for scalability and fault tolerance.

- **cpu:** The amount of CPU resources allocated (in millicores).
- **memory:** The amount of memory allocated (in megabytes).
- **replicas:** The number of instances of the application to run for scalability and fault tolerance.
- **desiredStatus:** The state in which you want the application to be. Common statuses include `Running` and `Stopped`.

- **publicAccess:** Configures the public accessibility of the deployment. If enabled, the service will be accessible via a public URL.
- **enabled:** A boolean value that determines if public access is enabled (`true` or `false`).
- **urlPrefix:** The URL prefix under which the service will be accessible.

- **enabled:** A boolean value that determines if public access is enabled (`true` or `false`).
- **urlPrefix:** The URL prefix under which the service will be accessible.
- **network:** Defines the network settings for the deployment, including the exposed ports.
- **ports:** A list of ports to expose.
- **port:** The external port number that will be exposed.
- **targetPort:** The internal port number the traffic will be directed to.

- **ports:** A list of ports to expose.
- **port:** The external port number that will be exposed.
- **targetPort:** The internal port number the traffic will be directed to.
- **variables:** Defines any configurable parameters for the deployment. These are typically inputs or outputs such as:
- **name:** The name of the variable.
- **inputType:** The type of input, usually `InputTopic`, `OutputTopic`, `Secret`, or `FreeText`.
- **description:** A brief explanation of what this variable does.
- **required:** Indicates whether this variable must be provided (`true` or `false`).
- **value:** The default or specified value for this variable.
- **name:** The name of the variable.
- **inputType:** The type of input, usually `InputTopic`, `OutputTopic`, `Secret`, or `FreeText`.
- **description:** A brief explanation of what this variable does.
- **required:** Indicates whether this variable must be provided (`true` or `false`).
- **value:** The default or specified value for this variable.

### Notes on Docker Image Deployments

- **Mutual Exclusivity:** You cannot specify both `application`/`version` and `image` in the same deployment definition. Choose one approach depending on your use case.
- **Image References:** Ensure your Docker image reference includes the repository and tag (e.g., `custom-repo/my-service:1.2.3`).
- **Compatibility:** Direct Docker image deployments provide greater flexibility, especially for non-standardized applications or third-party containers.

### 3. Topics

Expand All @@ -112,14 +119,12 @@ topics:
**Explanation:**

- **name:** The unique identifier for the topic within your pipeline.

- **persisted:** A boolean value that indicates whether the data in this topic should be stored persistently. Setting this to `false` means data will not be stored permanently and might be deleted after processing.

- **configuration:** (Optional) This section allows you to fine-tune how the topic behaves. Common configurations include:
- **partitions:** The number of partitions within the topic. More partitions allow for greater parallelism but may complicate data ordering.
- **retentionInMinutes:** How long the data should be retained in the topic before being deleted or compacted.
- **retentionInBytes:** The total size of data that the topic can retain. When this limit is reached, older data is deleted.
- **cleanupPolicy:** Defines how the topic handles old data. `Delete` means old data is removed, whereas other policies like `Compact` could be used for different use cases.
- **partitions:** The number of partitions within the topic. More partitions allow for greater parallelism but may complicate data ordering.
- **retentionInMinutes:** How long the data should be retained in the topic before being deleted or compacted.
- **retentionInBytes:** The total size of data that the topic can retain. When this limit is reached, older data is deleted.
- **cleanupPolicy:** Defines how the topic handles old data. `Delete` means old data is removed, whereas other policies like `Compact` could be used for different use cases.

## How to Use the `quix.yaml` File

Expand Down

0 comments on commit 67394d8

Please sign in to comment.