Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cleanup boltdb files in queriers during startup/shutdown #2558

Conversation

sandeepsukhani
Copy link
Contributor

What this PR does / why we need it:
Queriers do not use existing boltdb files when they receive a query for a new table(never served queries for it since startup) and we do not remove boltdb files when queriers stop which means unwanted files would be lying around.

This PR removes files from the cache directory during graceful shutdown or during startup if it didn't get to remove them for some reason.

Ideally, it would be good to reuse those files but the files could be stale and we would have to sync them first with the store which would add up to startup delay. We can handle this in future incremental changes.

@codecov-commenter
Copy link

Codecov Report

Merging #2558 into master will increase coverage by 0.03%.
The diff coverage is 12.50%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2558      +/-   ##
==========================================
+ Coverage   62.98%   63.01%   +0.03%     
==========================================
  Files         168      168              
  Lines       14879    14886       +7     
==========================================
+ Hits         9371     9380       +9     
  Misses       4762     4762              
+ Partials      746      744       -2     
Impacted Files Coverage Δ
.../storage/stores/shipper/downloads/table_manager.go 67.74% <0.00%> (-3.05%) ⬇️
pkg/storage/stores/shipper/downloads/table.go 68.12% <25.00%> (-0.47%) ⬇️
pkg/querier/queryrange/downstreamer.go 95.87% <0.00%> (-2.07%) ⬇️
pkg/logql/evaluator.go 92.88% <0.00%> (+0.40%) ⬆️
pkg/promtail/targets/file/filetarget.go 69.14% <0.00%> (+2.28%) ⬆️
pkg/promtail/targets/file/tailer.go 75.00% <0.00%> (+4.16%) ⬆️

Copy link
Collaborator

@slim-bean slim-bean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@sandeepsukhani sandeepsukhani merged commit 4f6a4c2 into grafana:master Aug 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants