Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Handle tag columns in low cardinality processor #5875

Merged
merged 3 commits into from
May 9, 2024

Conversation

evanh
Copy link
Member

@evanh evanh commented May 7, 2024

Discover will sometimes send filters on columns as tags instead of actual
columns, e.g. tags[environment]. This was causing the allocator bug to be hit
again on some queries. This change handles checking for this types of queries
and correctly casting them as well to avoid the allocator bugs.

Discover will sometimes send filters on columns as tags instead of actual
columns, e.g. tags[environment]. This was causing the allocator bug to be hit
again on some queries. This change handles checking for this types of queries
and correctly casting them as well to avoid the allocator bugs.
@evanh evanh requested a review from a team as a code owner May 7, 2024 20:50
@volokluev
Copy link
Member

Is this happening because the tag keys/values are being promoted to columns somewhere down the line? Can we fix this by putting the processor further down?

Also thank you for writing tests for the processor 🫶

@evanh
Copy link
Member Author

evanh commented May 8, 2024

Is this happening because the tag keys/values are being promoted to columns somewhere down the line

Yes, in the SQL query it uses the environment column:

(environment AS `_snuba_tags[environment]`), (count() AS _snuba_count), _snuba_count FROM discover_dist PREWHERE in(`_snuba_tags[environment]`, ['cron', 'worker'])

However I believe that happens in the storage mapping, so I would have to rewrite the processor as a storage processor. I think this is a cleaner workaround for now.

@evanh evanh enabled auto-merge (squash) May 9, 2024 15:00
@evanh evanh merged commit 03f73c8 into master May 9, 2024
30 checks passed
@evanh evanh deleted the evanh/fix/aliasing-subquery-bug branch May 9, 2024 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants