Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

operator: Expose per_stream_rate_limit & burst #9630

Merged
merged 4 commits into from
Jun 7, 2023
Merged

operator: Expose per_stream_rate_limit & burst #9630

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8437458
per_stream_rate_limit & burst limits configurable
jpinsonneau Jun 5, 2023
6afa06f
update changelog
jpinsonneau Jun 5, 2023
d6971ac
updated doc reason
jpinsonneau Jun 6, 2023
cefe341
Merge branch 'main' into netobserv-717
periklis Jun 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions operator/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
## Main

- [9630](https://github.com/grafana/loki/pull/9630) **jpinsonneau**: Expose per_stream_rate_limit & burst
- [9623](https://github.com/grafana/loki/pull/9623) **periklis**: Fix timeout config constructor when only tenants limits
- [9457](https://github.com/grafana/loki/pull/9457) **Red-GV**: Set seccomp profile to runtime default
- [9448](https://github.com/grafana/loki/pull/9448) **btaani**: Include runtime-config in compiling the SHA1 checksum
Expand Down
14 changes: 14 additions & 0 deletions operator/apis/loki/v1/lokistack_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -622,6 +622,20 @@ type IngestionLimitSpec struct {
// +kubebuilder:validation:Optional
// +operator-sdk:csv:customresourcedefinitions:type=spec,xDescriptors="urn:alm:descriptor:com.tectonic.ui:number",displayName="Max Line Size"
MaxLineSize int32 `json:"maxLineSize,omitempty"`

// PerStreamRateLimit defines the maximum byte rate per second per stream. Units MB.
//
// +optional
// +kubebuilder:validation:Optional
// +operator-sdk:csv:customresourcedefinitions:type=spec,xDescriptors="urn:alm:descriptor:com.tectonic.ui:number",displayName="Maximum byte rate per second per stream (in MB)"
PerStreamRateLimit int32 `json:"perStreamRateLimit,omitempty"`

// PerStreamRateLimitBurst defines the maximum burst bytes per stream. Units MB.
//
// +optional
// +kubebuilder:validation:Optional
// +operator-sdk:csv:customresourcedefinitions:type=spec,xDescriptors="urn:alm:descriptor:com.tectonic.ui:number",displayName="Maximum burst bytes per stream (in MB)"
PerStreamRateLimitBurst int32 `json:"perStreamRateLimitBurst,omitempty"`
}

// RetentionStreamSpec defines a log stream with separate retention time.
Expand Down
18 changes: 18 additions & 0 deletions operator/apis/loki/v1beta1/lokistack_types_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -379,6 +379,8 @@ func TestConvertToV1_LokiStack(t *testing.T) {
MaxLabelNamesPerSeries: 1000,
MaxGlobalStreamsPerTenant: 10000,
MaxLineSize: 512,
PerStreamRateLimit: 10,
PerStreamRateLimitBurst: 20,
},
QueryLimits: &v1.QueryLimitSpec{
MaxEntriesLimitPerQuery: 1000,
Expand All @@ -396,6 +398,8 @@ func TestConvertToV1_LokiStack(t *testing.T) {
MaxLabelNamesPerSeries: 1000,
MaxGlobalStreamsPerTenant: 10000,
MaxLineSize: 512,
PerStreamRateLimit: 10,
PerStreamRateLimitBurst: 20,
},
QueryLimits: &v1.QueryLimitSpec{
MaxEntriesLimitPerQuery: 1000,
Expand All @@ -412,6 +416,8 @@ func TestConvertToV1_LokiStack(t *testing.T) {
MaxLabelNamesPerSeries: 1000,
MaxGlobalStreamsPerTenant: 10000,
MaxLineSize: 512,
PerStreamRateLimit: 10,
PerStreamRateLimitBurst: 20,
},
QueryLimits: &v1.QueryLimitSpec{
MaxEntriesLimitPerQuery: 1000,
Expand Down Expand Up @@ -704,6 +710,8 @@ func TestConvertFromV1_LokiStack(t *testing.T) {
MaxLabelNamesPerSeries: 1000,
MaxGlobalStreamsPerTenant: 10000,
MaxLineSize: 512,
PerStreamRateLimit: 10,
PerStreamRateLimitBurst: 20,
},
QueryLimits: &v1.QueryLimitSpec{
MaxEntriesLimitPerQuery: 1000,
Expand All @@ -721,6 +729,8 @@ func TestConvertFromV1_LokiStack(t *testing.T) {
MaxLabelNamesPerSeries: 1000,
MaxGlobalStreamsPerTenant: 10000,
MaxLineSize: 512,
PerStreamRateLimit: 10,
PerStreamRateLimitBurst: 20,
},
QueryLimits: &v1.QueryLimitSpec{
MaxEntriesLimitPerQuery: 1000,
Expand All @@ -737,6 +747,8 @@ func TestConvertFromV1_LokiStack(t *testing.T) {
MaxLabelNamesPerSeries: 1000,
MaxGlobalStreamsPerTenant: 10000,
MaxLineSize: 512,
PerStreamRateLimit: 10,
PerStreamRateLimitBurst: 20,
},
QueryLimits: &v1.QueryLimitSpec{
MaxEntriesLimitPerQuery: 1000,
Expand Down Expand Up @@ -1001,6 +1013,8 @@ func TestConvertFromV1_LokiStack(t *testing.T) {
MaxLabelNamesPerSeries: 1000,
MaxGlobalStreamsPerTenant: 10000,
MaxLineSize: 512,
PerStreamRateLimit: 10,
PerStreamRateLimitBurst: 20,
},
QueryLimits: &v1beta1.QueryLimitSpec{
MaxEntriesLimitPerQuery: 1000,
Expand All @@ -1018,6 +1032,8 @@ func TestConvertFromV1_LokiStack(t *testing.T) {
MaxLabelNamesPerSeries: 1000,
MaxGlobalStreamsPerTenant: 10000,
MaxLineSize: 512,
PerStreamRateLimit: 10,
PerStreamRateLimitBurst: 20,
},
QueryLimits: &v1beta1.QueryLimitSpec{
MaxEntriesLimitPerQuery: 1000,
Expand All @@ -1034,6 +1050,8 @@ func TestConvertFromV1_LokiStack(t *testing.T) {
MaxLabelNamesPerSeries: 1000,
MaxGlobalStreamsPerTenant: 10000,
MaxLineSize: 512,
PerStreamRateLimit: 10,
PerStreamRateLimitBurst: 20,
},
QueryLimits: &v1beta1.QueryLimitSpec{
MaxEntriesLimitPerQuery: 1000,
Expand Down
26 changes: 25 additions & 1 deletion operator/bundle/community-openshift/manifests/loki-operator.clusterserviceversion.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ metadata:
categories: OpenShift Optional, Logging & Tracing
certified: "false"
containerImage: docker.io/grafana/loki-operator:main-ac1c1fd
createdAt: "2023-05-24T15:10:18Z"
createdAt: "2023-06-05T12:57:30Z"
description: The Community Loki Operator provides Kubernetes native deployment
and management of Loki and related logging components.
operators.operatorframework.io/builder: operator-sdk-unknown
Expand Down Expand Up @@ -329,6 +329,18 @@ spec:
path: limits.global.ingestion.maxLineSize
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: PerStreamRateLimit defines the maximum byte rate per second per
stream. Units MB.
displayName: Maximum byte rate per second per stream (in MB)
path: limits.global.ingestion.perStreamRateLimit
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: PerStreamRateLimitBurst defines the maximum burst bytes per stream.
Units MB.
displayName: Maximum burst bytes per stream (in MB)
path: limits.global.ingestion.perStreamRateLimitBurst
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: MaxChunksPerQuery defines the maximum number of chunks that can
be fetched by a single query.
displayName: Max Chunk per Query
Expand Down Expand Up @@ -396,6 +408,18 @@ spec:
path: limits.tenants.ingestion.maxLineSize
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: PerStreamRateLimit defines the maximum byte rate per second per
stream. Units MB.
displayName: Maximum byte rate per second per stream (in MB)
path: limits.tenants.ingestion.perStreamRateLimit
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: PerStreamRateLimitBurst defines the maximum burst bytes per stream.
Units MB.
displayName: Maximum burst bytes per stream (in MB)
path: limits.tenants.ingestion.perStreamRateLimitBurst
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: MaxChunksPerQuery defines the maximum number of chunks that can
be fetched by a single query.
displayName: Max Chunk per Query
Expand Down
20 changes: 20 additions & 0 deletions operator/bundle/community-openshift/manifests/loki.grafana.com_lokistacks.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,16 @@ spec:
on ingestion path. Units in Bytes.
format: int32
type: integer
perStreamRateLimit:
description: PerStreamRateLimit defines the maximum byte
rate per second per stream. Units MB.
format: int32
type: integer
perStreamRateLimitBurst:
description: PerStreamRateLimitBurst defines the maximum
burst bytes per stream. Units MB.
format: int32
type: integer
type: object
queries:
description: QueryLimits defines the limit applied on querying
Expand Down Expand Up @@ -242,6 +252,16 @@ spec:
on ingestion path. Units in Bytes.
format: int32
type: integer
perStreamRateLimit:
description: PerStreamRateLimit defines the maximum
byte rate per second per stream. Units MB.
format: int32
type: integer
perStreamRateLimitBurst:
description: PerStreamRateLimitBurst defines the maximum
burst bytes per stream. Units MB.
format: int32
type: integer
type: object
queries:
description: QueryLimits defines the limit applied on querying
Expand Down
26 changes: 25 additions & 1 deletion operator/bundle/community/manifests/loki-operator.clusterserviceversion.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ metadata:
categories: OpenShift Optional, Logging & Tracing
certified: "false"
containerImage: docker.io/grafana/loki-operator:main-ac1c1fd
createdAt: "2023-05-24T15:10:16Z"
createdAt: "2023-06-05T12:57:28Z"
description: The Community Loki Operator provides Kubernetes native deployment
and management of Loki and related logging components.
operators.operatorframework.io/builder: operator-sdk-unknown
Expand Down Expand Up @@ -329,6 +329,18 @@ spec:
path: limits.global.ingestion.maxLineSize
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: PerStreamRateLimit defines the maximum byte rate per second per
stream. Units MB.
displayName: Maximum byte rate per second per stream (in MB)
path: limits.global.ingestion.perStreamRateLimit
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: PerStreamRateLimitBurst defines the maximum burst bytes per stream.
Units MB.
displayName: Maximum burst bytes per stream (in MB)
path: limits.global.ingestion.perStreamRateLimitBurst
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: MaxChunksPerQuery defines the maximum number of chunks that can
be fetched by a single query.
displayName: Max Chunk per Query
Expand Down Expand Up @@ -396,6 +408,18 @@ spec:
path: limits.tenants.ingestion.maxLineSize
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: PerStreamRateLimit defines the maximum byte rate per second per
stream. Units MB.
displayName: Maximum byte rate per second per stream (in MB)
path: limits.tenants.ingestion.perStreamRateLimit
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: PerStreamRateLimitBurst defines the maximum burst bytes per stream.
Units MB.
displayName: Maximum burst bytes per stream (in MB)
path: limits.tenants.ingestion.perStreamRateLimitBurst
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: MaxChunksPerQuery defines the maximum number of chunks that can
be fetched by a single query.
displayName: Max Chunk per Query
Expand Down
20 changes: 20 additions & 0 deletions operator/bundle/community/manifests/loki.grafana.com_lokistacks.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,16 @@ spec:
on ingestion path. Units in Bytes.
format: int32
type: integer
perStreamRateLimit:
description: PerStreamRateLimit defines the maximum byte
rate per second per stream. Units MB.
format: int32
type: integer
perStreamRateLimitBurst:
description: PerStreamRateLimitBurst defines the maximum
burst bytes per stream. Units MB.
format: int32
type: integer
type: object
queries:
description: QueryLimits defines the limit applied on querying
Expand Down Expand Up @@ -242,6 +252,16 @@ spec:
on ingestion path. Units in Bytes.
format: int32
type: integer
perStreamRateLimit:
description: PerStreamRateLimit defines the maximum
byte rate per second per stream. Units MB.
format: int32
type: integer
perStreamRateLimitBurst:
description: PerStreamRateLimitBurst defines the maximum
burst bytes per stream. Units MB.
format: int32
type: integer
type: object
queries:
description: QueryLimits defines the limit applied on querying
Expand Down
26 changes: 25 additions & 1 deletion operator/bundle/openshift/manifests/loki-operator.clusterserviceversion.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ metadata:
categories: OpenShift Optional, Logging & Tracing
certified: "false"
containerImage: quay.io/openshift-logging/loki-operator:v0.1.0
createdAt: "2023-05-24T15:10:20Z"
createdAt: "2023-06-05T12:57:32Z"
description: |
The Loki Operator for OCP provides a means for configuring and managing a Loki stack for cluster logging.
## Prerequisites and Requirements
Expand Down Expand Up @@ -342,6 +342,18 @@ spec:
path: limits.global.ingestion.maxLineSize
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: PerStreamRateLimit defines the maximum byte rate per second per
stream. Units MB.
displayName: Maximum byte rate per second per stream (in MB)
path: limits.global.ingestion.perStreamRateLimit
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: PerStreamRateLimitBurst defines the maximum burst bytes per stream.
Units MB.
displayName: Maximum burst bytes per stream (in MB)
path: limits.global.ingestion.perStreamRateLimitBurst
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: MaxChunksPerQuery defines the maximum number of chunks that can
be fetched by a single query.
displayName: Max Chunk per Query
Expand Down Expand Up @@ -409,6 +421,18 @@ spec:
path: limits.tenants.ingestion.maxLineSize
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: PerStreamRateLimit defines the maximum byte rate per second per
stream. Units MB.
displayName: Maximum byte rate per second per stream (in MB)
path: limits.tenants.ingestion.perStreamRateLimit
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: PerStreamRateLimitBurst defines the maximum burst bytes per stream.
Units MB.
displayName: Maximum burst bytes per stream (in MB)
path: limits.tenants.ingestion.perStreamRateLimitBurst
x-descriptors:
- urn:alm:descriptor:com.tectonic.ui:number
- description: MaxChunksPerQuery defines the maximum number of chunks that can
be fetched by a single query.
displayName: Max Chunk per Query
Expand Down
20 changes: 20 additions & 0 deletions operator/bundle/openshift/manifests/loki.grafana.com_lokistacks.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,16 @@ spec:
on ingestion path. Units in Bytes.
format: int32
type: integer
perStreamRateLimit:
description: PerStreamRateLimit defines the maximum byte
rate per second per stream. Units MB.
format: int32
type: integer
perStreamRateLimitBurst:
description: PerStreamRateLimitBurst defines the maximum
burst bytes per stream. Units MB.
format: int32
type: integer
type: object
queries:
description: QueryLimits defines the limit applied on querying
Expand Down Expand Up @@ -242,6 +252,16 @@ spec:
on ingestion path. Units in Bytes.
format: int32
type: integer
perStreamRateLimit:
description: PerStreamRateLimit defines the maximum
byte rate per second per stream. Units MB.
format: int32
type: integer
perStreamRateLimitBurst:
description: PerStreamRateLimitBurst defines the maximum
burst bytes per stream. Units MB.
format: int32
type: integer
type: object
queries:
description: QueryLimits defines the limit applied on querying
Expand Down
20 changes: 20 additions & 0 deletions operator/config/crd/bases/loki.grafana.com_lokistacks.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,16 @@ spec:
on ingestion path. Units in Bytes.
format: int32
type: integer
perStreamRateLimit:
description: PerStreamRateLimit defines the maximum byte
rate per second per stream. Units MB.
format: int32
type: integer
perStreamRateLimitBurst:
description: PerStreamRateLimitBurst defines the maximum
burst bytes per stream. Units MB.
format: int32
type: integer
type: object
queries:
description: QueryLimits defines the limit applied on querying
Expand Down Expand Up @@ -225,6 +235,16 @@ spec:
on ingestion path. Units in Bytes.
format: int32
type: integer
perStreamRateLimit:
description: PerStreamRateLimit defines the maximum
byte rate per second per stream. Units MB.
format: int32
type: integer
perStreamRateLimitBurst:
description: PerStreamRateLimitBurst defines the maximum
burst bytes per stream. Units MB.
format: int32
type: integer
type: object
queries:
description: QueryLimits defines the limit applied on querying
Expand Down
Loading