Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#1649 Reuse query counters #1671

Conversation

justinweng-instaclustr
Copy link
Collaborator

@justinweng-instaclustr justinweng-instaclustr commented Jun 19, 2024

This PR stores the query counters in a HashMap and reuses them when processing messages instead of registering query counters in every transform. According to the benchmark report, query_counter_pre_used gains 14% improvement thanks to the faster transform.
image

query_counter_fresh becomes a bit slower but it's expected because the first transform runs create the counters and insert them into the hashmap.
image

Progress towards #1649

Copy link

codspeed-hq bot commented Jun 19, 2024

CodSpeed Performance Report

Merging #1671 will improve performances by 17.72%

Comparing justinweng-instaclustr:1649-reuse-query-counter (c0ba0a1) with main (b9329ae)

Summary

⚡ 1 improvements
✅ 38 untouched benchmarks

Benchmarks breakdown

Benchmark main justinweng-instaclustr:1649-reuse-query-counter Change
query_counter_pre_used 14 µs 11.9 µs +17.72%

@justinweng-instaclustr justinweng-instaclustr changed the title 1649 Reuse query counters #1649 Reuse query counters Jun 19, 2024
@justinweng-instaclustr justinweng-instaclustr marked this pull request as ready for review June 19, 2024 03:50
Copy link
Member

@rukai rukai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work!
lets land this.

I think that if we ever want to further improve performance here, the best way would be to make redis_query_name return a new enum named RedisCommandType and then have a to_str() method on that enum.
That way we can make this work completely with &'static str and other usages of redis_query_name across the codebase would also be able to avoid the allocation.
But again, I think we've hit the sweetspot of how much time to spend on this problem since this will resolve the performance impacts that are being warned about in the new metrics release.

@justinweng-instaclustr
Copy link
Collaborator Author

nice work! lets land this.

I think that if we ever want to further improve performance here, the best way would be to make redis_query_name return a new enum named RedisCommandType and then have a to_str() method on that enum. That way we can make this work completely with &'static str and other usages of redis_query_name across the codebase would also be able to avoid the allocation. But again, I think we've hit the sweetspot of how much time to spend on this problem since this will resolve the performance impacts that are being warned about in the new metrics release.

Yeah sounds good. We can probably change redis_query_name to improve performance opportunistically in the future. Do we want to raise an issue for it just for tracking purpose?

@rukai
Copy link
Member

rukai commented Jun 19, 2024

I think just leave it, better in general to let performance work be driven by observed issues in the system rather then guessing which parts of the system are going to end up as the bottleneck.
I consider this work an exception since we had a warning pointing out a potential issue, better to resolve it than just ignore it.

@justinweng-instaclustr justinweng-instaclustr enabled auto-merge (squash) June 20, 2024 00:20
@justinweng-instaclustr justinweng-instaclustr merged commit 6607381 into shotover:main Jun 20, 2024
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants