Store: Support disable cache index header file. #5773

wanjunlei · 2022-10-08T09:42:38Z

Signed-off-by: wanjunlei wanjunlei@kubesphere.io

I added CHANGELOG entry for this change.
Change is not relevant to the end user.

Changes

The block index header files can speed up store startup, but it will also take up the disk.

Suppose there is a sizeable k8s cluster, and there are many member clusters, there will be many blocks in the object storage so that the index header files will occupy more disks, and the replicas of the store need to be increased to improve the performance of the query, which will be doubled of increased disk consumption.

With this pr, the user can disable caching the index header files by --disable-caching-index-header-file, the store will not load index header files from the disk, also will not write them to the disk.

Verification

yeya24 · 2022-10-08T14:28:13Z

Umm, so we buffer the index header in memory instead of writing them all to disk?

But isn't disk cheaper than memory? If you want to consume less disk space you could set the posting offset in memory sampling.

yeya24 · 2022-10-08T14:18:08Z

cmd/thanos/store.go

@@ -83,7 +83,7 @@ func (sc *storeConfig) registerFlag(cmd extkingpin.FlagClause) {
 sc.grpcConfig = *sc.grpcConfig.registerFlag(cmd)

 cmd.Flag("data-dir", "Local data directory used for caching purposes (index-header, in-mem cache items and meta.jsons). If removed, no data will be lost, just store will have to rebuild the cache. NOTE: Putting raw blocks here will not cause the store to read them. For such use cases use Prometheus + sidecar.").
- Default("./data").StringVar(&sc.dataDir)
+ Default("").StringVar(&sc.dataDir)


We shouldn't change the default.

We shouldn't change the default.

If the default value is not "", users can not set the data-dir to "".

I don't think it is reasonable since it changes the default behavior. In this case please add a separate flag.

A flag --disable-caching-index-header-file was added to disable caching index header file.

wanjunlei · 2022-10-09T00:39:12Z

Umm, so we buffer the index header in memory instead of writing them all to disk?

But isn't disk cheaper than memory? If you want to consume less disk space you could set the posting offset in memory sampling.

Not a replacement, for the previous logic, the index headers exist both in memory and on disk, this pr just allows users not to write them all to disk. The data in memory has not changed.

fpetkovski · 2022-10-11T05:46:02Z

cmd/thanos/store.go

 Default("./data").StringVar(&sc.dataDir)

+ cmd.Flag("disable-caching-index-header-file", "Disable caching purposes (index-header, in-mem cache items and meta.jsons).").


I think this needs to be explained better, and I also wonder how it will interact with lazyIndexReaderEnabled=false

What about this:

Disable caching index-header, and meta.jsons file on disk. If set, the store will ignore the --data-dir, and will not load the index header and meta.jsons files from disk at startup, and will not create the index header and meta.jsons files when synchronizing blocks.

This parameter just disables loading and creating index header and meta.jsons files, all data stored in memory is still the same as before, so it can interact with the --store.enable-index-header-lazy-reader parameter.

Interesting, so the lazy reader does not require the index header to be on disk?

The current logic is, the lazy reader will load the index header from the disk if the index header file exists, or build it from the index in the object store and write it on the disk when the first calling load, then the files on disk will never be used unless the block is deleted or store restarted. so the lazy reader actually does not require the index header to be on the disk.

Understood, thanks for the explanation 👍

matej-g · 2022-10-17T07:18:11Z

cmd/thanos/store.go

 Default("./data").StringVar(&sc.dataDir)

+ cmd.Flag("disable-caching-index-header-file", "Disable caching index-header, and meta.jsons file on disk. If set, the store will ignore the --data-dir, and will not load the index header and meta.jsons files from disk on startup, and will not create the index header and meta.jsons files when synchronizing blocks.").


Should we describe here what this effect this flag has positively, i.e. what will it do instead of what it will not do? It is not clear what will Thanos use instead if I disable caching from this description.

pkg/block/indexheader/header_test.go

matej-g · 2022-10-17T07:23:39Z

pkg/block/indexheader/lazy_binary_reader.go

@@ -99,30 +98,34 @@ func NewLazyBinaryReader(
 metrics *LazyBinaryReaderMetrics,
 onClosed func(*LazyBinaryReader),
 ) (*LazyBinaryReader, error) {
- filepath := filepath.Join(dir, id.String(), block.IndexHeaderFilename)
+ indexHeaderFile := ""


Nit: I would prefer var statement if we're initializing new empty variable, it makes it more obvious to me.

matej-g · 2022-10-17T07:32:23Z

pkg/block/indexheader/lazy_binary_reader.go

- if !os.IsNotExist(err) {
- return nil, errors.Wrap(err, "read index header")
- }
+ if indexHeaderFile != "" {


Alternative: We could check directly for empty dir here and build the filepath inside the loop instead of doing the declarations and checks above.

matej-g · 2022-10-17T07:38:31Z

pkg/store/bucket.go

@@ -442,8 +442,10 @@ func NewBucketStore(
 return nil, errors.Wrap(err, "validate config")
 }

- if err := os.MkdirAll(dir, 0750); err != nil {
- return nil, errors.Wrap(err, "create dir")
+ if dir != "" {


We could check for dir == "" directly here and return if so, I think it's more readable than nested ifs.

matej-g · 2022-10-17T07:38:54Z

pkg/store/bucket.go

- id, ok := block.IsBlockDir(n)
- if !ok {
- continue
+ if s.dir != "" {


Same as comment above

matej-g · 2022-10-17T07:39:57Z

pkg/store/bucket.go

@@ -571,15 +575,20 @@ func (s *BucketStore) getBlock(id ulid.ULID) *bucketBlock {
 }

 func (s *BucketStore) addBlock(ctx context.Context, meta *metadata.Meta) (err error) {
- dir := filepath.Join(s.dir, meta.ULID.String())
+ dir := ""


Same comment as above (prefer var statement).

yeya24

Could you please add an E2E test case to verify it is working correctly? @wanjunlei

wanjunlei · 2022-10-21T07:46:44Z

Done

fpetkovski · 2022-12-18T13:51:32Z

@wanjunlei would you mind resolving conflicts to help unblock this PR?

wanjunlei · 2022-12-19T03:59:32Z

@wanjunlei would you mind resolving conflicts to help unblock this PR?

Done

fpetkovski

I really like this feature, it should help tremendously with large Thanos deployments!

Now that we have an implementation for memory reader/writer, can we change the file reader and writer to be wrappers around the memory one? This should help remove a lot of the if statements that are now necessary.

cmd/thanos/store.go

wanjunlei · 2022-12-26T06:24:44Z

I really like this feature, it should help tremendously with large Thanos deployments!

Now that we have an implementation for memory reader/writer, can we change the file reader and writer to be wrappers around the memory one? This should help remove a lot of the if statements that are now necessary.

I know your meaning, but I do not have a good idea. Do you have any suggestions?

yeya24 · 2023-01-19T07:19:55Z

Looks like the CI failure is related to #6055. Let's merge that one first and test again.

wanjunlei · 2023-01-19T08:26:47Z

Looks like the CI failure is related to #6055. Let's merge that one first and test again.OK

OK

matej-g

I updated the branch to resolve CI issue, pending CI pass this is good to go! Thanks for the solid work @wanjunlei!

yeya24 · 2023-01-19T17:06:09Z

E2E tests still need to be fixed

Signed-off-by: wanjunlei <wanjunlei@kubesphere.io>

fpetkovski

Looks ready to be merged now, thanks!

yeya24 · 2023-01-28T07:34:36Z

Thanks!

* Update Thanos engine to latest version (thanos-io#6069) This commit updates the Thanos PromQL engine to the latest version. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> * Receive: Tenants' external labels proposal (thanos-io#5720) * Receive external labels proposal Signed-off-by: haanhvu <haanh6594@gmail.com> * Restructure and edit proposal's content Signed-off-by: haanhvu <haanh6594@gmail.com> * Update proposal Signed-off-by: haanhvu <haanh6594@gmail.com> * Fix doc error Signed-off-by: haanhvu <haanh6594@gmail.com> Signed-off-by: haanhvu <haanh6594@gmail.com> * fixing doc CI (thanos-io#6072) Signed-off-by: Ben Ye <benye@amazon.com> Signed-off-by: Ben Ye <benye@amazon.com> * Fix stores filtering resets on reload (thanos-io#6063) * Fix stores filtering resets on reload `g0.store_matches` parameter appears in the url but doesn't applies in the frontend. Looks like it has been done on purpose and by removing a small piece of code fixes this issue. variable named `debugMode` is used for the store filtering checkbox which is an unappropriate name. Using `enableStoreFiltering` variable to represent the state of checkbox. Signed-off-by: Pradyumna Krishna <git@onpy.in> * Regenerate bindata.go Signed-off-by: Pradyumna Krishna <git@onpy.in> Signed-off-by: Pradyumna Krishna <git@onpy.in> * Store: Make initial sync more robust Added re-try mechanism for store inital sync, where if the initial sync fails, it tries to do the initial sync again for given timeout duration. Signed-off-by: Kartik-Garg <kartik.garg@infracloud.io> * Recover from panics in Series calls (thanos-io#6077) * Recover from panics in Series calls This commit adds panic recovery for Series calls in all Store servers. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> * Apply error suggestion Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> --------- Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> * query: reuse our own gate (thanos-io#6079) Do not call promgate directly but rather use our own wrapper that does everything we want - duration histogram, current in-flight calls, total calls. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> * Store: Support disable cache index header file. (thanos-io#5773) * Store: Support disable cache index header file. Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * Store: add a seprate flag to disable caching index header file Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * Tools: add cleanup API for bucket web Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * resolve conversation Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * resolve confilcts Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * change the flag to `--cache-index-header` Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * Wrap mem writer in file writer Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * update CHANGELOG Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * update CHANGELOG Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * fix bug Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> --------- Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> Co-authored-by: wanjunlei <wanjunlei@yujnify.com> * CVE: Fix Receiver malicious tenant (thanos-io#5969) If running as root or with enough privileges, receiver can create a directory outside of the configured TenantHeader. This commit fixes it up by sanitizing the user input and explicity not allowing such behavior. Signed-off-by: Daniel Mellado <dmellado@redhat.com> * Add adopter Grupo MasMovil (thanos-io#6084) Signed-off-by: Pablo Moncada Isla <pablo.moncada@masmovil.com> * fix typo (thanos-io#6087) Signed-off-by: cyip <cyip@jackhenry.com> Co-authored-by: cyip <cyip@jackhenry.com> * optimize selector to string (thanos-io#6076) Signed-off-by: Kama Huang <kamatogo13@gmail.com> * Fix: Failure to close BlockSeriesClient cause store-gateway deadlock (thanos-io#6086) * Fix: Failure to close BlockSeriesClient cause store-gateway deadlock Signed-off-by: Alan Protasio <alanprot@gmail.com> * Adding tests Signed-off-by: Alan Protasio <alanprot@gmail.com> * reverting the change on get series Signed-off-by: Alan Protasio <alanprot@gmail.com> * fix lint Signed-off-by: Alan Protasio <alanprot@gmail.com> --------- Signed-off-by: Alan Protasio <alanprot@gmail.com> * Cut 0.30.2 (thanos-io#6081) * tracing: fixed panic because of nil sampler (thanos-io#6066) * fixed panic because of nil sampler Signed-off-by: Vasiliy Rumyantsev <4119114+xBazilio@users.noreply.github.com> * added CHANGELOG entry Signed-off-by: Vasiliy Rumyantsev <4119114+xBazilio@users.noreply.github.com> Signed-off-by: Vasiliy Rumyantsev <4119114+xBazilio@users.noreply.github.com> * bump version to 0.30.2 Signed-off-by: Ben Ye <benye@amazon.com> * Updates busybox SHA (thanos-io#6046) Signed-off-by: GitHub <noreply@github.com> Signed-off-by: GitHub <noreply@github.com> Co-authored-by: yeya24 <yeya24@users.noreply.github.com> * Use `e2edb.NewMinio` to disable SSE-S3 in e2e tests (thanos-io#6055) * Use e2edb.NewMinio to disable SSE Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Use temp fork for TLS Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Fix broken rules api fanout test Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Fix broken query compatibility test Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Remove fork Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> --------- Signed-off-by: Vasiliy Rumyantsev <4119114+xBazilio@users.noreply.github.com> Signed-off-by: Ben Ye <benye@amazon.com> Signed-off-by: GitHub <noreply@github.com> Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> Co-authored-by: Vasiliy Rumyantsev <4119114+xBazilio@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: yeya24 <yeya24@users.noreply.github.com> Co-authored-by: Saswata Mukherjee <saswataminsta@yahoo.com> * cherry pick store gateway fix to release 0.30 (thanos-io#6089) * Fix: Failure to close BlockSeriesClient cause store-gateway deadlock (thanos-io#6086) * Fix: Failure to close BlockSeriesClient cause store-gateway deadlock Signed-off-by: Alan Protasio <alanprot@gmail.com> * Adding tests Signed-off-by: Alan Protasio <alanprot@gmail.com> * reverting the change on get series Signed-off-by: Alan Protasio <alanprot@gmail.com> * fix lint Signed-off-by: Alan Protasio <alanprot@gmail.com> --------- Signed-off-by: Alan Protasio <alanprot@gmail.com> * update changelog Signed-off-by: Ben Ye <benye@amazon.com> --------- Signed-off-by: Alan Protasio <alanprot@gmail.com> Signed-off-by: Ben Ye <benye@amazon.com> Co-authored-by: Alan Protasio <alanprot@gmail.com> * fix changelog entries Signed-off-by: Ben Ye <benye@amazon.com> * docs: improving the description for tsdb.retention on the receiver Signed-off-by: Victor Fernandes <victorhbfernandes@gmail.com> * Receiver: Use `intern` package when reallocating label strings (thanos-io#5926) * Cleanup go mod Signed-off-by: Matej Gera <matejgera@gmail.com> * Use string interning for labels realloc method Signed-off-by: Matej Gera <matejgera@gmail.com> * Enhance label realloc benchmarks Signed-off-by: Matej Gera <matejgera@gmail.com> * Make interning optional; put behind hiddend flag Signed-off-by: Matej Gera <matej.gera@coralogix.com> * Update CHANGELOG Signed-off-by: Matej Gera <matej.gera@coralogix.com> * Address feedback - Fix wrong condition - Adjust benchmarks Signed-off-by: Matej Gera <matej.gera@coralogix.com> --------- Signed-off-by: Matej Gera <matejgera@gmail.com> Signed-off-by: Matej Gera <matej.gera@coralogix.com> Signed-off-by: Matej Gera <38492574+matej-g@users.noreply.github.com> * Updaing README with drawing fixes and minor wording clarification (thanos-io#6078) * New drawing and wording for Thanos other deployment models Signed-off-by: Jonah Kowall <jkowall@kowall.net> * New drawing and wording for Thanos other deployment models Signed-off-by: Jonah Kowall <jkowall@kowall.net> * Added comments to README.md and updated the quick-tutorial.md with the same diagram updates and text to match Signed-off-by: Jonah Kowall <jkowall@kowall.net> * Ran make docs Signed-off-by: Jonah Kowall <jkowall@kowall.net> --------- Signed-off-by: Jonah Kowall <jkowall@kowall.net> * Compact: Remove spam of replica label removed log (thanos-io#6088) * Remove spam of replica label removed log Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Reduce amount of removed replica label instead of removing it Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Reformat code Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> --------- Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Store: Don't error when no stores are matched (thanos-io#6082) It's normal and not an error if a query does not match due to no downstream stores. This is common when querying with external labels and tiered query servers. This bug was introduced in thanos-io#5296 Fixes: thanos-io#5862 Signed-off-by: SuperQ <superq@gmail.com> * docs: Fix must have Ruler alerts definition (thanos-io#6058) * Fix must have Ruler alerts definition ThanosRuler missing rule intervals metric used the wrong comparator sign, confusing users trying to create the rule. Signed-off-by: Maxim Muzafarov <m.muzafarov@gmail.com> * Update docs/components/rule.md Co-authored-by: Saswata Mukherjee <saswataminsta@yahoo.com> Signed-off-by: Maxim Muzafarov <m.muzafarov@gmail.com> --------- Signed-off-by: Maxim Muzafarov <m.muzafarov@gmail.com> Co-authored-by: Saswata Mukherjee <saswataminsta@yahoo.com> * Fix conflicts Signed-off-by: haanhvu <haanh6594@gmail.com> * Specify overwriting behavior in flag and add validation Signed-off-by: haanhvu <haanh6594@gmail.com> * Add log and doc Signed-off-by: haanhvu <haanh6594@gmail.com> * Mixins(Rule): Fix query for long rule evaluations (thanos-io#6103) * mixin(Rule): Fix query for long rule evaluations Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Update changelog Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> --------- Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> --------- Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com> Signed-off-by: haanhvu <haanh6594@gmail.com> Signed-off-by: Ben Ye <benye@amazon.com> Signed-off-by: Pradyumna Krishna <git@onpy.in> Signed-off-by: Kartik-Garg <kartik.garg@infracloud.io> Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> Signed-off-by: Daniel Mellado <dmellado@redhat.com> Signed-off-by: Pablo Moncada Isla <pablo.moncada@masmovil.com> Signed-off-by: cyip <cyip@jackhenry.com> Signed-off-by: Kama Huang <kamatogo13@gmail.com> Signed-off-by: Alan Protasio <alanprot@gmail.com> Signed-off-by: Vasiliy Rumyantsev <4119114+xBazilio@users.noreply.github.com> Signed-off-by: GitHub <noreply@github.com> Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com> Signed-off-by: Victor Fernandes <victorhbfernandes@gmail.com> Signed-off-by: Matej Gera <matejgera@gmail.com> Signed-off-by: Matej Gera <matej.gera@coralogix.com> Signed-off-by: Matej Gera <38492574+matej-g@users.noreply.github.com> Signed-off-by: Jonah Kowall <jkowall@kowall.net> Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: SuperQ <superq@gmail.com> Signed-off-by: Maxim Muzafarov <m.muzafarov@gmail.com> Signed-off-by: Sebastian Rabenhorst <sebastian.rabenhorst@shopify.com> Co-authored-by: Filip Petkovski <filip.petkovsky@gmail.com> Co-authored-by: Ha Anh Vu <75315486+haanhvu@users.noreply.github.com> Co-authored-by: Ben Ye <benye@amazon.com> Co-authored-by: Pradyumna Krishna <git@onpy.in> Co-authored-by: Kartik-Garg <kartik.garg@infracloud.io> Co-authored-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> Co-authored-by: wanjunlei <53003665+wanjunlei@users.noreply.github.com> Co-authored-by: wanjunlei <wanjunlei@yujnify.com> Co-authored-by: Daniel Mellado <1313475+danielmellado@users.noreply.github.com> Co-authored-by: Pablo Moncada <pmoncadaisla@gmail.com> Co-authored-by: Chantel Yip <52993239+sshantel@users.noreply.github.com> Co-authored-by: cyip <cyip@jackhenry.com> Co-authored-by: Kama Huang <121007071+kama910@users.noreply.github.com> Co-authored-by: Alan Protasio <alanprot@gmail.com> Co-authored-by: Vasiliy Rumyantsev <4119114+xBazilio@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: yeya24 <yeya24@users.noreply.github.com> Co-authored-by: Saswata Mukherjee <saswataminsta@yahoo.com> Co-authored-by: Victor Fernandes <victorhbfernandes@gmail.com> Co-authored-by: Matej Gera <38492574+matej-g@users.noreply.github.com> Co-authored-by: Jonah Kowall <jkowall@kowall.net> Co-authored-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Co-authored-by: Ben Kochie <superq@gmail.com> Co-authored-by: Maxim Muzafarov <m.muzafarov@gmail.com> Co-authored-by: haanhvu <haanh6594@gmail.com>

* Store: Support disable cache index header file. Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * Store: add a seprate flag to disable caching index header file Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * Tools: add cleanup API for bucket web Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * resolve conversation Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * resolve confilcts Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * change the flag to `--cache-index-header` Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * Wrap mem writer in file writer Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * update CHANGELOG Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * update CHANGELOG Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> * fix bug Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> --------- Signed-off-by: wanjunlei <wanjunlei@kubesphere.io> Co-authored-by: wanjunlei <wanjunlei@yujnify.com>

pull-request-size bot added the size/XXL label Oct 8, 2022

wanjunlei force-pushed the store branch from 00ae8a9 to a0ffb13 Compare October 8, 2022 09:44

yeya24 reviewed Oct 8, 2022

View reviewed changes

pull-request-size bot added size/XL and removed size/XXL labels Oct 9, 2022

wanjunlei force-pushed the store branch from 3d741a6 to 2020b4f Compare October 9, 2022 06:20

fpetkovski reviewed Oct 11, 2022

View reviewed changes

wanjunlei force-pushed the store branch from 2020b4f to e11c5b8 Compare October 11, 2022 09:57

matej-g reviewed Oct 17, 2022

View reviewed changes

pull-request-size bot added size/L and removed size/XL labels Oct 20, 2022

wanjunlei force-pushed the store branch 3 times, most recently from 02f9857 to 6fce0ba Compare October 20, 2022 05:51

yeya24 reviewed Oct 20, 2022

View reviewed changes

pull-request-size bot added size/XL and removed size/L labels Oct 21, 2022

wanjunlei force-pushed the store branch 2 times, most recently from 0712d69 to 98e303a Compare October 21, 2022 08:29

wanjunlei force-pushed the store branch 4 times, most recently from 6821ace to c2a4819 Compare December 19, 2022 03:29

fpetkovski reviewed Dec 20, 2022

View reviewed changes

cmd/thanos/store.go Outdated Show resolved Hide resolved

wanjunlei force-pushed the store branch 2 times, most recently from d8ef85f to b87ce1a Compare January 19, 2023 03:25

wanjunlei force-pushed the store branch from b87ce1a to f923c26 Compare January 19, 2023 08:25

matej-g previously approved these changes Jan 19, 2023

View reviewed changes

wanjunlei added 9 commits January 28, 2023 10:27

Store: Support disable cache index header file.

c6b6114

Signed-off-by: wanjunlei <wanjunlei@kubesphere.io>

Store: add a seprate flag to disable caching index header file

016e7a3

Signed-off-by: wanjunlei <wanjunlei@kubesphere.io>

Tools: add cleanup API for bucket web

d626e4d

Signed-off-by: wanjunlei <wanjunlei@kubesphere.io>

resolve conversation

fd085ad

Signed-off-by: wanjunlei <wanjunlei@kubesphere.io>

resolve confilcts

0a6b07b

Signed-off-by: wanjunlei <wanjunlei@kubesphere.io>

change the flag to --cache-index-header

73f0e94

Signed-off-by: wanjunlei <wanjunlei@kubesphere.io>

Wrap mem writer in file writer

afb9653

Signed-off-by: wanjunlei <wanjunlei@kubesphere.io>

update CHANGELOG

9b079ef

Signed-off-by: wanjunlei <wanjunlei@kubesphere.io>

update CHANGELOG

1f04ccb

Signed-off-by: wanjunlei <wanjunlei@kubesphere.io>

wanjunlei dismissed matej-g’s stale review via 30a06a9 January 28, 2023 02:30

wanjunlei force-pushed the store branch 3 times, most recently from 2190f4e to 39ceed0 Compare January 28, 2023 02:55

fix bug

4be4ab3

Signed-off-by: wanjunlei <wanjunlei@kubesphere.io>

wanjunlei force-pushed the store branch from 39ceed0 to 4be4ab3 Compare January 28, 2023 03:12

fpetkovski approved these changes Jan 28, 2023

View reviewed changes

yeya24 merged commit 1b12ab9 into thanos-io:main Jan 28, 2023

tao12345666333 mentioned this pull request Feb 27, 2023

store: fix --no-cache-index-header flag #6169

Merged

2 tasks

yeya24 mentioned this pull request Jul 6, 2023

index header: Remove memWriter from fileWriter #6509

Merged

2 tasks

		Default("./data").StringVar(&sc.dataDir)

		cmd.Flag("disable-caching-index-header-file", "Disable caching purposes (index-header, in-mem cache items and meta.jsons).").

Store: Support disable cache index header file. #5773

Store: Support disable cache index header file. #5773

Conversation

wanjunlei commented Oct 8, 2022 • edited Loading

Changes

Verification

yeya24 commented Oct 8, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wanjunlei commented Oct 9, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fpetkovski Oct 11, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yeya24 left a comment

Choose a reason for hiding this comment

wanjunlei commented Oct 21, 2022

fpetkovski commented Dec 18, 2022

wanjunlei commented Dec 19, 2022

fpetkovski left a comment

Choose a reason for hiding this comment

wanjunlei commented Dec 26, 2022

yeya24 commented Jan 19, 2023

wanjunlei commented Jan 19, 2023

matej-g left a comment

Choose a reason for hiding this comment

yeya24 commented Jan 19, 2023

fpetkovski left a comment

Choose a reason for hiding this comment

yeya24 commented Jan 28, 2023

wanjunlei commented Oct 8, 2022 •

edited

Loading

yeya24 commented Oct 8, 2022 •

edited

Loading

fpetkovski Oct 11, 2022 •

edited

Loading