Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: rebuild missing bloom index while pruning #15738

Conversation

dantengsky
Copy link
Member

@dantengsky dantengsky commented Jun 5, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Fix missing bloom index during bloom pruning.

introduce new setting enable_auto_fix_missing_bloom_index which is false (disabled) by default.

when it was enabled, missing bloom indexes detected during pruning will be created automatically. (only those bloom index that used by the query, will be rebuilt, not all of them).

this feature is intended to repair a small amount of missing bloom indexes. if a table's bloom indexes are largely lost, such as being mostly deleted by mistake, and subsequent queries are expected to encounter most of these missing bloom indexes, please be aware that rebuilding bloom indexes takes time and may significantly impact query performance.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-bugfix this PR patches a bug in codebase label Jun 5, 2024
@dantengsky dantengsky marked this pull request as ready for review June 7, 2024 05:24
@dantengsky dantengsky requested a review from zhyass June 7, 2024 05:24
@dantengsky dantengsky added this pull request to the merge queue Jun 7, 2024
@BohuTANG BohuTANG removed this pull request from the merge queue due to a manual request Jun 7, 2024
@BohuTANG BohuTANG merged commit 7e44c94 into databendlabs:main Jun 7, 2024
74 checks passed
@dantengsky dantengsky mentioned this pull request Jun 21, 2024
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-bugfix this PR patches a bug in codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature: need a way to amend bloom index (rebuild missing indexes)
3 participants