Skip to content

chore: add GCS HTTP config #4005

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 14, 2025
Merged

chore: add GCS HTTP config #4005

merged 3 commits into from
Mar 14, 2025

Conversation

kolesnikovae
Copy link
Collaborator

@kolesnikovae kolesnikovae commented Mar 13, 2025

The change will allow tuning the HTTP conn pool: the default parameters may cause excessive TLS handshakes:

Defaults:
image

1K idle connections, 10m idle timeout:
image

N.B.: it's worth exploring the GCS gRPC interface as well.

@kolesnikovae kolesnikovae changed the title chore: add GCP HTTP config chore: add GCS HTTP config Mar 13, 2025
@kolesnikovae kolesnikovae force-pushed the chore/gcs-http-config branch 2 times, most recently from cb0d7f4 to 3d0ae4e Compare March 13, 2025 10:47
@kolesnikovae kolesnikovae force-pushed the chore/gcs-http-config branch from 3d0ae4e to b007f51 Compare March 13, 2025 10:49
@kolesnikovae kolesnikovae marked this pull request as ready for review March 13, 2025 11:18
@kolesnikovae kolesnikovae requested review from a team as code owners March 13, 2025 11:18
@kolesnikovae kolesnikovae enabled auto-merge (squash) March 13, 2025 11:28
Copy link
Contributor

@simonswine simonswine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM: Great find!

Maybe worth capturing the grpc interface as an issue

Comment on lines +34 to +43
type HTTPConfig struct {
IdleConnTimeout time.Duration `yaml:"idle_conn_timeout"`
ResponseHeaderTimeout time.Duration `yaml:"response_header_timeout"`
InsecureSkipVerify bool `yaml:"insecure_skip_verify"`
TLSHandshakeTimeout time.Duration `yaml:"tls_handshake_timeout"`
ExpectContinueTimeout time.Duration `yaml:"expect_continue_timeout"`
MaxIdleConns int `yaml:"max_idle_conns"`
MaxIdleConnsPerHost int `yaml:"max_idle_conns_per_host"`
MaxConnsPerHost int `yaml:"max_conns_per_host"`
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably better to mark them all as advanced:

Suggested change
type HTTPConfig struct {
IdleConnTimeout time.Duration `yaml:"idle_conn_timeout"`
ResponseHeaderTimeout time.Duration `yaml:"response_header_timeout"`
InsecureSkipVerify bool `yaml:"insecure_skip_verify"`
TLSHandshakeTimeout time.Duration `yaml:"tls_handshake_timeout"`
ExpectContinueTimeout time.Duration `yaml:"expect_continue_timeout"`
MaxIdleConns int `yaml:"max_idle_conns"`
MaxIdleConnsPerHost int `yaml:"max_idle_conns_per_host"`
MaxConnsPerHost int `yaml:"max_conns_per_host"`
}
type HTTPConfig struct {
IdleConnTimeout time.Duration `yaml:"idle_conn_timeout"` `category:"advanced"`
ResponseHeaderTimeout time.Duration `yaml:"response_header_timeout"` `category:"advanced"`
InsecureSkipVerify bool `yaml:"insecure_skip_verify"` `category:"advanced"`
TLSHandshakeTimeout time.Duration `yaml:"tls_handshake_timeout"` `category:"advanced"`
ExpectContinueTimeout time.Duration `yaml:"expect_continue_timeout"` `category:"advanced"`
MaxIdleConns int `yaml:"max_idle_conns"` `category:"advanced"`
MaxIdleConnsPerHost int `yaml:"max_idle_conns_per_host"` `category:"advanced"`
MaxConnsPerHost int `yaml:"max_conns_per_host"` `category:"advanced"`
}

Copy link
Collaborator Author

@kolesnikovae kolesnikovae Mar 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been thinking about it and decided to make it symmetric with the S3 config.

Actually, now that I look at it, it should be advanced – I'll add it in the follow up PR

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding it in #4007

@kolesnikovae kolesnikovae merged commit 1420455 into main Mar 14, 2025
20 checks passed
@kolesnikovae kolesnikovae deleted the chore/gcs-http-config branch March 14, 2025 09:31
shelldandy pushed a commit to shelldandy/pyroscope that referenced this pull request Mar 14, 2025
* chore: add GCS HTTP config

* chore: update help message
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants