Make Write message type more flexble, address some feedback #1710
Conversation
Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
| // Under any marshaling scheme, v2 requests have a `Symbols` field of type []string. | ||
| // So would always have a `GetSymbols()` method which doesn't rely on any other types. | ||
| type v2Request interface { | ||
| GetSymbols() []string |
There was a problem hiding this comment.
This seems a bit hacky, but need it for the contentType detection. I wonder if there is a better way here
There was a problem hiding this comment.
Interesting. I think this is acceptable for now.
| // Under any marshaling scheme, v2 requests have a `Symbols` field of type []string. | ||
| // So would always have a `GetSymbols()` method which doesn't rely on any other types. | ||
| type v2Request interface { | ||
| GetSymbols() []string |
There was a problem hiding this comment.
Interesting. I think this is acceptable for now.
| @@ -74,6 +75,22 @@ func WithAPILogger(logger *slog.Logger) APIOption { | |||
| } | |||
| } | |||
|
|
|||
| // WithAPIEndpoint returns APIOption that allows providing endpoint. | |||
| func WithAPIEndpoint(endpoint string) APIOption { | |||
There was a problem hiding this comment.
You mean path? Using it in URL will mean path. Either we rename to path or not use client.URL or not use client structs at all.
I wanted to move this lib to separate client_golang/exp/ module for now, so we might want to get rid of client (improve it). Can do in my PR, but let's just switch to path here for now?
There was a problem hiding this comment.
Yeah it hit me after, let me update
There was a problem hiding this comment.
Switched to path, I can do exp module in next PR
| } | ||
|
|
||
| // NewRemoteWriteHandler returns HTTP handler that receives Remote Write 2.0 | ||
| // protocol https://prometheus.io/docs/specs/remote_write_spec_2_0/. | ||
| func NewRemoteWriteHandler(logger *slog.Logger, store writeStorage) http.Handler { | ||
| return &handler{logger: logger, store: store} | ||
| func NewRemoteWriteHandler(logger *slog.Logger, store writeStorage, decompressor remoteWriteDecompressor) http.Handler { |
There was a problem hiding this comment.
We might need opts here as well, separate ones 🤔
There was a problem hiding this comment.
Added for logger and decompressor
| @@ -279,18 +327,32 @@ type writeStorage interface { | |||
| Store(ctx context.Context, proto WriteProtoFullName, serializedRequest []byte) (_ WriteResponseStats, code int, _ error) | |||
| } | |||
|
|
|||
| // remoteWriteDecompressor is an interface that allows decompressing the body of the request. | |||
| type remoteWriteDecompressor interface { | |||
There was a problem hiding this comment.
can you remind me why we need this?
Another option is to use server middleware interface for this. There are common middlewares for this work ppl use e.g. Otel
There was a problem hiding this comment.
Some downstream users have their own request body size limiter+instrumentaion impls, that they would probably want to use. I know Thanos and Cortex have them #1658 (comment)
I don't think compression-wise things would be too different (maybe using s2).
There was a problem hiding this comment.
Cool, I will merge for now, but we need to switch to middleware injection technique
Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
…t and handler. (#1658) * api: Add remote API with write client; add remote handler. Signed-off-by: bwplotka <bwplotka@gmail.com> * Make Write message type more flexble, address some feedback (#1710) * Address remaining feedback Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Make Write message type more flexible Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> --------- Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Move remote write API to client_golang/exp (#1711) * Move remote write API to client_golang/exp Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Don't use api.Client structs, add options for middleware Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Fix reqBuf usage Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Fix url path Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Add separate mod file (and workspace file) Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Hook exp tests fmt; Test handler error case; Configure backoff Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> --------- Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * exp: Add README, address feedback, use sync.Pool (#1747) * Implement suggestion for Store interface and contentType Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Add README Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Use sync.Pool Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Implement review suggestions Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Release bufs right after compressPayload Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> --------- Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Bump exp to go 1.22 Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Saswata Mukherjee <saswataminsta@yahoo.com>
Addresses a few comments from PR
Also imports v1, as a convenience package as well (we can keep until we deprecate fully?)
Also makes Write marshaling more flexible with any type and gogo marshaling. Generics don't really seem to fit as you can't have multiple interface constraints on them.
Might be worth, merging with merge commit to keep original PR commits clear
cc: @bwplotka