Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: Document Promtail global rate limiting #5737

Merged
merged 4 commits into from
Apr 5, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
* [5163](https://github.com/grafana/loki/pull/5163) **chaudum** Fix regression in fluentd plugin introduced with #5107 that caused `NoMethodError` when parsing non-string values of log lines.
* [5144](https://github.com/grafana/loki/pull/5144) **dannykopping** Ruler: fix remote write basic auth credentials.
* [5091](https://github.com/grafana/loki/pull/5091) **owen-d**: Changes `ingester.concurrent-flushes` default to 32
* [5031](https://github.com/grafana/loki/pull/5031) **liguozhong**: Promtail: Add global read rate limiting.
* [4879](https://github.com/grafana/loki/pull/4879) **cyriltovena**: LogQL: add __line__ function to | line_format template.
* [5081](https://github.com/grafana/loki/pull/5081) **SasSwart**: Add the option to configure memory ballast for Loki
* [5085](https://github.com/grafana/loki/pull/5085) **aknuds1**: Upgrade Cortex to [e0807c4eb487](https://github.com/cortexproject/cortex/compare/4e9fc3a2b5ab..e0807c4eb487) and Prometheus to [692a54649ed7](https://github.com/prometheus/prometheus/compare/2a3d62ac8456..692a54649ed7)
Expand Down
8 changes: 4 additions & 4 deletions clients/pkg/promtail/limit/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ type Config struct {
}

func (cfg *Config) RegisterFlagsWithPrefix(prefix string, f *flag.FlagSet) {
f.Float64Var(&cfg.ReadlineRate, prefix+"limit.readline-rate", 10000, "promtail readline Rate.")
f.IntVar(&cfg.ReadlineBurst, prefix+"limit.readline-burst", 10000, "promtail readline Burst.")
f.BoolVar(&cfg.ReadlineRateEnabled, prefix+"limit.readline-rate-enabled", false, "Set to false to disable readline rate limit.")
f.BoolVar(&cfg.ReadlineRateDrop, prefix+"limit.readline-rate-drop", true, "Set to true to drop log when rate limit.")
f.Float64Var(&cfg.ReadlineRate, prefix+"limit.readline-rate", 10000, "The rate limit in log lines per second that this instance of Promtail may push to Loki.")
f.IntVar(&cfg.ReadlineBurst, prefix+"limit.readline-burst", 10000, "The cap in the quantity of burst lines that this instance of Promtail may push to Loki.")
f.BoolVar(&cfg.ReadlineRateEnabled, prefix+"limit.readline-rate-enabled", false, "When true, enforces rate limiting on this instance of Promtail.")
f.BoolVar(&cfg.ReadlineRateDrop, prefix+"limit.readline-rate-drop", true, "When true, exceeding the rate limit causes this instance of Promtail to discard log lines, rather than sending them to Loki.")
}
23 changes: 23 additions & 0 deletions docs/sources/clients/promtail/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,9 @@ clients:
scrape_configs:
- [<scrape_config>]

# Configures global limits for this instance of Promtail
[limits_config: <limits_config>]

# Configures how tailed targets will be watched.
[target_config: <target_config>]

Expand Down Expand Up @@ -1756,6 +1759,26 @@ scrape_configs:
target_label: 'container'
```

## limits_config

The optional `limits_config` block configures global limits for this instance of Promtail.

```yaml
# When true, enforces rate limiting on this instance of Promtail.
[readline_rate_enabled: <bool> | default = false]

# The rate limit in log lines per second that this instance of Promtail may push to Loki.
[readline_rate: <int> | default = 10000]

# The cap in the quantity of burst lines that this instance of Promtail may push
# to Loki.
[readline_burst: <int> | default = 10000]

# When true, exceeding the rate limit causes this instance of Promtail to discard
# log lines, rather than sending them to Loki. When false, exceeding the rate limit
# causes this instance of Promtail to temporarily hold off on sending the log lines and retry later.
[readline_rate_drop: <bool> | default = true]
```

## target_config

Expand Down