Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Updated SS and microservices deployment docs #13083

Merged
merged 16 commits into from
Jun 5, 2024
Merged

feat: Updated SS and microservices deployment docs #13083

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
5ec06bf
feat: Updated SS and microservices deployment docs
Jayclifford345 May 30, 2024
c8c209c
fix: fixed title
Jayclifford345 May 30, 2024
45cb7ce
fix: fixed title
Jayclifford345 May 30, 2024
cc1d441
Merge branch 'main' into fix-helm-docs
Jayclifford345 May 30, 2024
248171f
fix: added endpoint
Jayclifford345 May 30, 2024
229d953
feat: fixed based on review
Jayclifford345 May 31, 2024
31debcd
feat: removed old comment
Jayclifford345 May 31, 2024
ec182cf
fix: fixed monitoring URL
Jayclifford345 May 31, 2024
a99fcca
fix: revert deleted file
Jayclifford345 May 31, 2024
7194db6
fix: naming
Jayclifford345 May 31, 2024
7a63d3a
fix: fixed aliases and removed old file
Jayclifford345 May 31, 2024
3ec4db1
Merge branch 'main' into fix-helm-docs
Jayclifford345 May 31, 2024
981360a
Added azure and s3 examples
Jayclifford345 Jun 3, 2024
f4f2031
Merge branch 'main' into fix-helm-docs
Jayclifford345 Jun 3, 2024
ff2f925
fix: removed kind install page
Jayclifford345 Jun 5, 2024
b8ab16a
Merge branch 'main' into fix-helm-docs
Jayclifford345 Jun 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
249 changes: 249 additions & 0 deletions docs/sources/setup/install/helm/install-microservices/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,249 @@
---

Check failure on line 1 in docs/sources/setup/install/helm/install-microservices/_index.md

View workflow job for this annotation

GitHub Actions / doc-validator 

[doc-validator] reported by reviewdog 🐶
The 'aliases' parameter in the front matter must be a YAML sequence of strings.

Raw Output:
{"message":"The 'aliases' parameter in the front matter must be a YAML sequence of strings.","location":{"path":"docs/sources/setup/install/helm/install-microservices/_index.md","range":{"start":{},"end":{}}},"severity":1,"source":{"name":"doc-validator"},"code":{"value":"parameter-must-be-string-sequence"}}
title: Install the microservice Helm chart
menuTitle: Install microservice Loki
description: Installing Loki in microservice (distributed) mode using the Helm chart.
aliases:
weight: 300
keywords:
---

# Install the microservice Helm charts
Jayclifford345 marked this conversation as resolved.
Show resolved Hide resolved

This Helm Chart deploys Grafana Loki on Kubernetes.

This chart configures Loki to run Loki in [microservice / distributed mode]({{< relref "../../../../get-started/deployment-modes#microservices-mode" >}}). The microservices deployment mode runs components of Loki as distinct processes.

The default Helm chart deploys the following components:
Jayclifford345 marked this conversation as resolved.
Show resolved Hide resolved
- **Compactor component** (1 replica): Compacts and processes stored data.
- **Distributor component** (3 replicas, maxUnavailable: 2): Distributes incoming requests. Up to 2 replicas can be unavailable during updates.
- **IndexGateway component** (2 replicas, maxUnavailable: 1): Handles indexing. Up to 1 replica can be unavailable during updates.
- **Ingester component** (3 replicas): Handles ingestion of data.
- **Querier component** (3 replicas, maxUnavailable: 2): Processes queries. Up to 2 replicas can be unavailable during updates.
- **QueryFrontend component** (2 replicas, maxUnavailable: 1): Manages frontend queries. Up to 1 replica can be unavailable during updates.
- **QueryScheduler component** (2 replicas): Schedules queries.

It is not recommended to run scalable mode with `filesystem` storage. For the purpose of this guide, we will use MinIO as the object storage to provide a complete example.

**Prerequisites**

- Helm 3 or above. See [Installing Helm](https://helm.sh/docs/intro/install/).
- A running Kubernetes cluster.
- (Optional) A Memcached deployment for better query performance. For information on configuring Memcached, refer to the [caching section](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/caching/).


**To deploy Loki in microservice mode (with MinIO):**


1. Add [Grafana's chart repository](https://github.com/grafana/helm-charts) to Helm:

```bash
helm repo add grafana https://grafana.github.io/helm-charts
```

2. Update the chart repository:

```bash
helm repo update
```

3. Create the configuration file `values.yaml`. The example below illustrates how to deploy Loki in test mode using MinIO as storage:

```yaml
loki:
schemaConfig:
configs:
- from: 2024-04-01
store: tsdb
object_store: s3
schema: v13
index:
prefix: loki_index_
period: 24h
ingester:
chunk_encoding: snappy
tracing:
enabled: true
querier:
# Default is 4, if you have enough memory and CPU you can increase, reduce if OOMing
max_concurrent: 4

#gateway:
# ingress:
# enabled: true
# hosts:
# - host: FIXME
# paths:
# - path: /
# pathType: Prefix

deploymentMode: Distributed

ingester:
replicas: 3
querier:
replicas: 3
maxUnavailable: 2
queryFrontend:
replicas: 2
maxUnavailable: 1
queryScheduler:
replicas: 2
distributor:
replicas: 3
maxUnavailable: 2
compactor:
replicas: 1
indexGateway:
replicas: 2
maxUnavailable: 1

bloomCompactor:
replicas: 0
bloomGateway:
replicas: 0

# Enable minio for storage
minio:
enabled: true

# Zero out replica counts of other deployment modes
backend:
replicas: 0
read:
replicas: 0
write:
replicas: 0

singleBinary:
replicas: 0
```

4. Install or upgrade the Loki deployment.
- To install:
```bash
helm install --values values.yaml loki grafana/loki
```
- To upgrade:
```bash
helm upgrade --values values.yaml loki grafana/loki
```


1. Verify that Loki is running:
```bash
kubectl get pods -n loki
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's helpful to add example output from this command so that users know what pods they should expect to see.

The output should an output similar to the following:

```bash
loki-canary-8thrx 1/1 Running 0 167m
loki-canary-h965l 1/1 Running 0 167m
loki-canary-th8kb 1/1 Running 0 167m
loki-chunks-cache-0 0/2 Pending 0 167m
loki-compactor-0 1/1 Running 0 167m
loki-compactor-1 1/1 Running 0 167m
loki-distributor-7c9bb8f4dd-bcwc5 1/1 Running 0 167m
loki-distributor-7c9bb8f4dd-jh9h8 1/1 Running 0 167m
loki-distributor-7c9bb8f4dd-np5dw 1/1 Running 0 167m
loki-gateway-77bc447887-qgc56 1/1 Running 0 167m
loki-index-gateway-0 1/1 Running 0 167m
loki-index-gateway-1 1/1 Running 0 166m
loki-ingester-zone-a-0 1/1 Running 0 167m
loki-ingester-zone-b-0 1/1 Running 0 167m
loki-ingester-zone-c-0 1/1 Running 0 167m
loki-minio-0 1/1 Running 0 167m
loki-querier-bb8695c6d-bv9x2 1/1 Running 0 167m
loki-querier-bb8695c6d-bz2rw 1/1 Running 0 167m
loki-querier-bb8695c6d-z9qf8 1/1 Running 0 167m
loki-query-frontend-6659566b49-528j5 1/1 Running 0 167m
loki-query-frontend-6659566b49-84jtx 1/1 Running 0 167m
loki-query-frontend-6659566b49-9wfr7 1/1 Running 0 167m
loki-query-scheduler-f6dc4b949-fknfk 1/1 Running 0 167m
loki-query-scheduler-f6dc4b949-h4nwh 1/1 Running 0 167m
loki-query-scheduler-f6dc4b949-scfwp 1/1 Running 0 167m
loki-results-cache-0 2/2 Running 0 167m
```

## AWS S3 Configuration

To configure Loki to use AWS S3 as the object storage rather than MinIO, you need to provide the following values in the `values.yaml` file:

```yaml
loki:
schemaConfig:
configs:
- from: 2024-04-01
store: tsdb
object_store: s3
schema: v13
index:
prefix: loki_index_
period: 24h
ingester:
chunk_encoding: snappy
tracing:
enabled: true
querier:
max_concurrent: 4

storage:
type: s3
bucketNames:
chunks: "chunks"
ruler: "ruler"
admin: "admin"
s3:
s3: <endpoint>
endpoint: <endpoint>
accessKeyId: <accessKeyID>
secretAccessKey: <secretAccessKey>
s3ForcePathStyle: true
insecure: true

deploymentMode: Distributed

# Disable minio storage
minio:
enabled: false

ingester:
replicas: 3
querier:
replicas: 3
maxUnavailable: 2
queryFrontend:
replicas: 2
maxUnavailable: 1
queryScheduler:
replicas: 2
distributor:
replicas: 3
maxUnavailable: 2
compactor:
replicas: 1
indexGateway:
replicas: 2
maxUnavailable: 1

bloomCompactor:
replicas: 0
bloomGateway:
replicas: 0

backend:
replicas: 0
read:
replicas: 0
write:
replicas: 0

singleBinary:
replicas: 0

```

To configure other storage providers, refer to the [Helm Chart Reference]({{< relref "../reference" >}}).

## Next Steps
* Configure an agent to [send log data to Loki](/docs/loki/<LOKI_VERSION>/send-data/).
* Monitor the Loki deployment using the [Meta Monitoring Healm chart](/docs/loki/<LOKI_VERSION>/monitor-and-alert/)
Loading
Loading