Port KafkaSinkSingle to Connection #1544
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This ended up a bit involved since I needed to add dummy response generation to the Connection type in order to get kafka working.
The connection logic used by cluster_connection_pool.rs already has this implemented https://github.com/shotover/shotover-proxy/blob/main/shotover/src/transforms/util/cluster_connection_pool.rs#L290 and many transforms rely on this.
The new implementation in Connection is more robust though, the one in cluster_connection_pool.rs has issues with cancel safety and a possible race condition.
The actual porting of KafkaSinkSingle to Connection is pretty straightforward, is very similar to RedisSinkSingle
windsock benchmarks show no regressions.