Go: support context.Context

[marcind]

Describe the feature

All methods that cause network resources to be consumed should accept a context.Context to align with best practices for developing go applications.

Use Case

context.Context enables cooperative cancellation scoped metadata passing, which enables tracing integrations etc. Practically all go APIs and libraries that consume non-trivial resources (such as causing networking against a remote server) take a context as a parameter.

Proposed Solution

Add a context.Context as the first parameter to all *Command methods as well as others such as Transaction.Exec. Even if not fully used now, doing it now will avoid having to add a separate set of methods that accept it or releasing a v2 with a breaking change in the interface.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

Client version used

pre-release

Environment details (OS name and version, etc.)

All

[avifenesh]

@yipin-chen @asafpamzn @meitalkra @jbrinkman @niharikabhavaraju @vikaskpandey

[avifenesh]

Wrong observation of mine, the decision been made is to support context

@marcind hi, thanks for opening the issue and putting time into following the development, and wanting to help. I think that in the paradigm of glide, context is not feasible. To control a command through all of its lifetime, it requires moving context between go and rust layers. That will require a lot of overhead, and fake context for most of the life span of the commands, as context is built in go, but doesn't exist in Rust. Finding a way to do so without an overkill solution requires many thoughts and deep design and can't be introduced in the current timeline of the release, and the brute force solution will hurt performance badly. If context was for the client itself it was manageable and the overhead could be neglected, but adding another object to pass between layers and to control around the whole life span for each one of the action, I can't see how it is possible without doubling the work and complexity for every action.

I think, for every dev, deciding between x performance vs. x/2 performance but with context, the decision will be without.
Do you see it differently?
Do you have a solution that I missed?

[marcind]

@avifenesh I can appreciate the complexities involved in having the wrapper<->Rust bridge. Unfortunately I don't know Rust so I can't offer any suggestions on how to resolve whatever gaps might exist between it and Go, sorry.

If I may offer an observation, while I see (arguably from only a small amount of lurking on a handful of Go-related PRs) a lot of focus on the Rust components, I can't tell who is taking on the role of an advocate for the Go application developer experience. I was hoping that this project would be a compelling replacement for go-redis but it feels like it's really meant to be a Rust driver that happens to have Go bindings added on. Now I need to choose between a library that integrates into the Go ecosystem or one that will track any new Valkey features better but sacrifices those integrations.

For example, to your points about developer performance preferences, I think that's an artificial choice. I work on a service that gets sustained traffic of a few thousand RPS and while I care about the performance I also care very much about error handling, metrics, observability, and everything else that goes into operating a service. If you present the choice as "You can get a response to this command at a p50 of 5ms but with no insights into what's going on if things go wrong, or you can get a p50 of 8ms but you get to plug into OpenTelemetry and other observability practices so that you can more easily debug those occasional spikes the service is exhibiting" then my choice would be for the latter. I just noticed that #3531 exists and I imagine the question of Context will come up again.

[avifenesh]

@marcind actually, most of the effort on Go was done by Go developers,
while for myself, I'm not Go oriented, so I review the rust part, most of the effort, until lately when go is getting ready, was almost only go.

I genuinely understand your point, and we tried to give the best ergo we can for each language, no matter which.

But to your point, we are releasing open telemetry for this version, it will be in go as well, in the go style. For logging, while it will take a bit of time, the end goal is to integrate with the language, I can share another issue where I explain why glide has a special logger.
We created stats, so you can get loacl info about the client which is not part of the valkey api.
Integration with credential provider will come soon, which will be a very comfortable solution, and there's many more ergo that we offer to the end user.
We offer sharded pubsub, and pubsub state restoration on disconnection.
We offer outstanding topology management and bullet prof fault tolerance under extreme cases.
We offer a cluster wide scan, mget, mset, del, and soon batches to all commands,
And so on.
It happens to be written in rust under the table, but the client you'll use is go.

Wrong comment of mine, will be supported, and under rework

I love Node.js, there is nothing different for me when using glide, just when developing glide. You brought up a point, and there's already a previous comment about it, so we for sure going to consider the options to deliver it, and that's true that performance is not everything, but without careful design, that much more than a little hit in performance, context doesn't exist in other languages, it's a go thing. If ill go back to the Node.js example, its like me asking for a package that run on the browser only. It's a JavaScript thing, providing it just like that will create a problem. Providing it later after careful design? If possible and users want, it will happen. But this is a very specific feature which is challenging to emulate, other features do exist and provided.

[avifenesh]

And the lot of rust focus you see is because the connection layer is the dangerous one, and we are going to release a new version, so all the rust oriented folks are jumping in.