-
-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(replays): Improve index page query performance #45098
Conversation
@cmanallen yeah i definitely think we can optimize project_id and user too. for for user_id, we can run a quick query to verify but it should be the same on all segments, so the strategy w/ |
@JoshFerge Wow, yes. Very astute observation. 33% speed-up on large datasets. |
Added. 25% speed-up. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚀
* master: (37 commits) ref(ppf): Don't use --commit-batch-size for futures queue length (#45182) feat(codecov-v2): Add more logging (#45225) fix(alerts): Center table items on alert history page (#45226) feat(CapMan): Pass `tenant_ids` to Snuba (#44788) ref(db): Drop `project_id` from Environment (model state) (#45207) chore(profiling): Rename context in profiles task (#45208) feat(replays): Improve index page query performance (#45098) chore(issue assignment): Add logging for`GroupOwner` auto assignment (#45142) fix(hybrid-cloud): Uncache organization when queueing it for deletion (#45213) fix(perf): Navigating to Transaction Summary from Trends widget should persist custom date selection (#45190) fix(pageFilter): Fix overflow (#45169) ref(git hooks): Only suggest autoupdate variable when pulling if not already set (#45179) fix(dashboard): Include dashboard filters in widget viewer (#45106) fix(alerts): Remove null projects from alerts list (#45202) feat(replay): Update Inline replay onboarding img to support dark mode (#45084) __iexact reduce call has default value now. (#45206) feat(replay): Use SDK value for LCP (#44868) chore(hybrid-cloud): breaking foreign keys (#45203) Revert "ref(db): Drop `project_id` from Environment (model state) (#45094)" ref(db): Drop `project_id` from Environment (model state) (#45094) ...
The /replays index page slows down when presented with large datasets. Given a large enough dataset some queries will OOM. By keeping fewer values in memory and making various performance optimizations to the query's structure we improve the performance of the query by 5x for our largest customers.
count_errors
field.urls_sorted
dependency forcount_urls
and replaces with simple count.activity
performance.groupUniqArray
withgroupArray(1)
. This is faster (no unique requirements) and more memory efficient (we only ever have one value in memory).project_id
is no longer a grouped scalar value. It is included in the GROUP BY clause.