Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs for Redis scalers with cluster support #335

Merged
merged 1 commit into from
Jan 6, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 102 additions & 0 deletions content/docs/2.1/scalers/redis-cluster-lists.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
+++
title = "Redis Lists (supports Redis Cluster)"
layout = "scaler"
availability = "v2.1+"
maintainer = "Community"
description = "Redis Lists scaler with support for Redis Cluster topology"
go_file = "redis_scaler"
+++

### Trigger Specification

This specification describes the `redis` trigger that scales based on the length of a list in a Redis Cluster.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This specification describes the `redis` trigger that scales based on the length of a list in a Redis Cluster.
This specification describes the `redis-cluster` trigger that scales based on the length of a list in a Redis Cluster.


```yaml
triggers:
- type: redis-cluster
metadata:
addresses: localhost:6379 # Comma separated list of the format host:port
passwordFromEnv: REDIS_PASSWORD
listName: mylist # Required
listLength: "5" # Required
enableTLS: "false" # optional
# Alternatively, you can use existing environment variables to read configuration from:
# See details in "Parameter list" section
addressesFromEnv: REDIS_ADDRESSES # Optional. You can use this instead of `addresses` parameter
```

**Parameter list:**

- `addresses` - Comma separated list of hosts and ports of the Redis Cluster nodes.
- `hosts` - Comma separated list of hosts of the Redis Cluster nodes. Alternative to `addresses` and requires `ports` to be configured as well.
- `ports` - Comma separated list of corresponding ports for the hosts of the Redis Cluster nodes. Alternative to `addresses` and requires `hosts` to be configured as well.
- `passwordFromEnv` - Environment variable to read the authentication password from to authenticate with the Redis server.
- Both the hostname and password fields need to be set to the names of the environment variables in the target deployment that contain the host name and password respectively.
- `listName` - Name of the Redis List that you want to monitor
- `listLength` - Average target value to trigger scaling actions
- `enableTLS` - If set to `true` allow a connection to a redis queue using tls, the default value for this parameter is false.

Some parameters could be provided using environmental variables, instead of setting them directly in metadata. Here is a list of parameters you can use to retrieve values from environment variables:

- `addressesFromEnv` - The hosts and their respective ports of the Redis Cluster nodes, similar to `addresses`, but reads it from an environment variable on the scale target.
- `hostsFromEnv` - The hosts of the Redis Cluster nodes, similar to `hosts`, but reads it from an environment variable on the scale target.
- `portsFromEnv` - The corresponding ports for the hosts of the Redis Cluster nodes, similar to `ports`, but reads it from an environment variable on the scale target.

### Authentication Parameters

You can authenticate by using a password.

**Connection Authentication:**

- `addresses` - Comma separated list of host:port format.
- `hosts` - Comma separated list of hostname of the Redis Cluster nodes. If specified, the `ports` should also be specified
- `ports` - Comma separated list of ports of the Redis Cluster nodes. If specified, the `hosts` should also be specified

**Password Authentication:**

- `password` - Redis password to authenticate with

### Example

Here is an example of how to deploy a scaled object with the `redis-cluster` scale trigger which uses `TriggerAuthentication`.

You can also provide the `passwordFromEnv` on the `ScaledObject` directly.

```yaml
apiVersion: v1
kind: Secret
metadata:
name: votes-db-secret
namespace: my-project
type: Opaque
data:
redis_password: YWRtaW4=
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: keda-trigger-auth-redis-secret
namespace: my-project
spec:
secretTargetRef:
- parameter: password
name: votes-db-secret
key: redis_password
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: redis-scaledobject
namespace: my-project
spec:
scaleTargetRef:
name: votes
triggers:
- type: redis-cluster
metadata:
addresses: node1:6379, node2:6379, node3:6379
listName: mylist
listLength: "10"
authenticationRef:
name: keda-trigger-auth-redis-secret
```
145 changes: 145 additions & 0 deletions content/docs/2.1/scalers/redis-cluster-streams.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
+++
title = "Redis Streams (supports Redis Cluster)"
layout = "scaler"
availability = "v2.1+"
maintainer = "Community"
description = "Redis Streams scaler with support for Redis Cluster topology"
go_file = "redis_streams_scaler"
+++

### Trigger Specification

Redis 5.0 introduced [Redis Streams](https://redis.io/topics/streams-intro) which is an append-only log data structure.

One of its features includes [`Consumer Groups`](https://redis.io/topics/streams-intro#consumer-groups), that allows a group of clients to co-operate consuming a different portion of the same stream of messages.

This specification describes the `redis-cluster-streams` trigger that scales based on the *Pending Entries List* (see [`XPENDING`](https://redis.io/commands/xpending)) for a specific Consumer Group of a Redis Stream and supports Redis Cluster topology.


```yaml
triggers:
- type: redis-cluster-streams
metadata:
addresses: localhost:6379 # Required if hosts and ports are not provided. Format - comma separated list of host:port
hosts: localhost # Comma separated lists of hosts. Required if address is not provided
ports: "6379" # Comma separated lists of ports. Required if addresses are not provided and hosts has been provided.
passwordFromEnv: REDIS_PASSWORD # optional (can also use authenticationRef)
stream: my-stream # Required - name of the Redis Stream
consumerGroup: my-consumer-group # Required - name of consumer group associated with Redis Stream
pendingEntriesCount: "10" # Required - number of entries in the Pending Entries List for the specified consumer group in the Redis Stream
enableTLS: "false" # optional
# Alternatively, you can use existing environment variables to read configuration from:
# See details in "Parameter list" section
addressesFromEnv: REDIS_ADDRESSES # Optional. You can use this instead of `addresses` parameter
hostsFromEnv: REDIS_HOSTS # Optional. You can use this instead of `hosts` parameter
portsFromEnv: REDIS_PORTS # Optional. You can use this instead of `ports` parameter
```

**Parameter list:**

- `addresses`: Comma separated list of hosts and ports of Redis Cluster nodes in the format `host:port` for example `node1:6379, node2:6379, node3:6379`

> As an alternative to the `addresses` field, the user can specify `hosts` and `ports` parameters.

- `hosts`: Comma separated list of hosts of Redis Cluster nodes.

> It is not required if `addresses` has been provided

- `ports`: Comma separated list of ports for corresponding hosts of Redis Cluster nodes.

> It is only to be used along with the `hosts`/`hostsFromEnv` attribute and not required if `addresses` has been provided

- `passwordFromEnv` (optional): Name of the environment variable your deployment uses to get the Redis password.

- `stream`: Name of the Redis Stream
- `consumerGroup`: Name of the Consumer group associated with Redis Stream
- `pendingEntriesCount`: Threshold for the number of `Pending Entries List`. This is the average target value to scale the workload. Defaults to `5`
- `enableTLS`: Set this to `true` if TLS connection to Redis is required. Defaults to `false`

Some parameters could be provided using environmental variables, instead of setting them directly in metadata. Here is a list of parameters you can use to retrieve values from environment variables:

- `addressesFromEnv`: The hosts and corresponding ports of Redis Cluster nodes, similar to `addresses`, but reads it from an environment variable on the scale target.
Name of the environment variable your deployment uses to get the URLs of Redis Cluster nodes. The resolved hosts should follow a format like `node1:6379, node2:6379, node3:6379 ...`.
- `hostsFromEnv`: The hosts of the Redis Cluster nodes, similar to `hosts`, but reads it from an environment variable on the scale target.
- `portsFromEnv`: The corresponding ports for the hosts of Redis Cluster nodes, similar to `ports`, but reads it from an environment variable on the scale target.

### Authentication Parameters

The scaler supports two modes of authentication:

#### Using password authentication

Use the `password` field in the `metadata` to specify the name of an environment variable that your deployment uses to get the Redis password.

This is usually resolved from a `Secret V1` or a `ConfigMap V1` collections. `env` and `envFrom` are both supported.

Here is an example:

```yaml
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: redis-streams-scaledobject
namespace: default
spec:
scaleTargetRef:
name: redis-streams-consumer
pollingInterval: 15
cooldownPeriod: 200
maxReplicaCount: 25
minReplicaCount: 1
triggers:
- type: redis-cluster-streams
metadata:
addressesFromEnv: REDIS_ADDRESSES
passwordFromEnv: REDIS_PASSWORD # name of the environment variable in the Deployment
stream: my-stream
consumerGroup: consumer-group-1
pendingEntriesCount: "10"
```

#### Using `TriggerAuthentication`

You can use `TriggerAuthentication` CRD to configure the authentication. For example:

```yaml
apiVersion: v1
kind: Secret
metadata:
name: redis-streams-password
type: Opaque
data:
redis_password: <encoded redis password>
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: keda-redis-stream-triggerauth
spec:
secretTargetRef:
- parameter: password
name: redis-streams-password # name of the Secret
key: redis_password # name of the key in the Secret
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: redis-streams-scaledobject
namespace: default
spec:
scaleTargetRef:
name: redis-streams-consumer
pollingInterval: 15
cooldownPeriod: 200
maxReplicaCount: 25
minReplicaCount: 1
triggers:
- type: redis-cluster-streams
metadata:
address: node1:6379, node2:6379, node3:6379
stream: my-stream
consumerGroup: consumer-group-1
pendingEntriesCount: "10"
authenticationRef:
name: keda-redis-stream-triggerauth # name of the TriggerAuthentication resource
```