Skip to content

Commit

Permalink
add redis cache doc
Browse files Browse the repository at this point in the history
Signed-off-by: Jimmie Han <hanjinming@outlook.com>
  • Loading branch information
hanjm committed Nov 28, 2021
1 parent ab4111f commit 158a3c3
Show file tree
Hide file tree
Showing 3 changed files with 76 additions and 2 deletions.
28 changes: 28 additions & 0 deletions docs/components/query-frontend.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,34 @@ config:
expiration: 24h
```
#### Redis
The default redis config is:
```yaml mdox-exec="go run scripts/cfggen/main.go --name=queryfrontend.RedisResponseCacheConfig"
type: REDIS
config:
addr: ""
username: ""
password: ""
dial_timeout: 5s
read_timeout: 3s
write_timeout: 3s
pool_size: 100
min_idle_conns: 10
idle_timeout: 5m0s
max_conn_age: 0s
max_get_multi_concurrency: 100
get_multi_batch_size: 100
max_set_multi_concurrency: 100
set_multi_batch_size: 100
expiration: 24h0m0s
```
`expiration` specifies redis cache valid time. If set to 0s, so using a default of 24 hours expiration time.

Other cache configuration parameters, you can refer to [redis-index-cache](store.md#redis-index-cache).

### Slow Query Log

Query Frontend supports `--query-frontend.log-queries-longer-than` flag to log queries running longer than some duration.
Expand Down
48 changes: 46 additions & 2 deletions docs/components/store.md
Original file line number Diff line number Diff line change
Expand Up @@ -304,11 +304,54 @@ While the remaining settings are **optional**:
- `dns_provider_update_interval`: the DNS discovery update interval.
- `auto_discovery`: whether to use the auto-discovery mechanism for memcached.

### Redis index cache

The `memcached` index cache allows to use [Redis](https://redis.io) as cache backend. This cache type is configured using `--index-cache.config-file` to reference the configuration file or `--index-cache.config` to put yaml config directly:

```yaml mdox-exec="go run scripts/cfggen/main.go --name=cacheutil.RedisClientConfig"
type: REDIS
config:
addr: ""
username: ""
password: ""
dial_timeout: 5s
read_timeout: 3s
write_timeout: 3s
pool_size: 100
min_idle_conns: 10
idle_timeout: 5m0s
max_conn_age: 0s
max_get_multi_concurrency: 100
get_multi_batch_size: 100
max_set_multi_concurrency: 100
set_multi_batch_size: 100
```

The **required** settings are:

- `addr`: redis server address.

While the remaining settings are **optional**:

- `username`: the username to connect redis, only redis 6.0 and grater need this field.
- `password`: the password to connect redis.
- `dial_timeout`: the redis dial timeout.
- `read_timeout`: the redis read timeout.
- `write_timeout`: the redis write timeout.
- `pool_size`: maximum number of socket connections.
- `min_idle_conns`: specifies the minimum number of idle connections which is useful when establishing new connection is slow.
- `idle_timeout`: amount of time after which client closes idle connections. Should be less than server's timeout.
- `max_conn_age`: connection age at which client retires (closes) the connection.
- `max_get_multi_concurrency`: specifies the maximum number of concurrent GetMulti() operations.
- `get_multi_batch_size`: specifies the maximum size per batch for mget.
- `max_set_multi_concurrency`: specifies the maximum number of concurrent SetMulti() operations.
- `set_multi_batch_size`: specifies the maximum size per batch for pipeline set.

## Caching Bucket

Thanos Store Gateway supports a "caching bucket" with [chunks](../design.md#chunk) and metadata caching to speed up loading of [chunks](../design.md#chunk) from TSDB blocks. To configure caching, one needs to use `--store.caching-bucket.config=<yaml content>` or `--store.caching-bucket.config-file=<file.yaml>`.

Both memcached and in-memory cache "backend"s are supported:
memcached/in-memory/redis cache "backend"s are supported:

```yaml
type: MEMCACHED # Case-insensitive
Expand All @@ -333,7 +376,8 @@ metafile_content_ttl: 24h
metafile_max_size: 1MiB
```
`config` field for memcached supports all the same configuration as memcached for [index cache](#memcached-index-cache). `addresses` in the config field is a **required** setting
- `config` field for memcached supports all the same configuration as memcached for [index cache](#memcached-index-cache). `addresses` in the config field is a **required** setting
- `config` field for redis supports all the same configuration as redis for [index cache](#redis-index-cache).

Additional options to configure various aspects of [chunks](../design.md#chunk) cache are available:

Expand Down
2 changes: 2 additions & 0 deletions scripts/cfggen/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -65,11 +65,13 @@ var (
indexCacheConfigs = map[storecache.IndexCacheProvider]interface{}{
storecache.INMEMORY: storecache.InMemoryIndexCacheConfig{},
storecache.MEMCACHED: cacheutil.MemcachedClientConfig{},
storecache.REDIS: cacheutil.DefaultRedisClientConfig,
}

queryfrontendCacheConfigs = map[queryfrontend.ResponseCacheProvider]interface{}{
queryfrontend.INMEMORY: queryfrontend.InMemoryResponseCacheConfig{},
queryfrontend.MEMCACHED: queryfrontend.MemcachedResponseCacheConfig{},
queryfrontend.REDIS: queryfrontend.DefaultRedisConfig,
}
)

Expand Down

0 comments on commit 158a3c3

Please sign in to comment.