Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compactor: weird directory structure #1661

Closed
Reamer opened this issue Oct 17, 2019 · 1 comment · Fixed by #1666
Closed

compactor: weird directory structure #1661

Reamer opened this issue Oct 17, 2019 · 1 comment · Fixed by #1666

Comments

@Reamer
Copy link

Reamer commented Oct 17, 2019

Thanos, Prometheus and Golang version used:

/ $ /bin/thanos --version
thanos, version 0.8.1 (branch: HEAD, revision: a328f4562f77deefcd990330681a62f7de049e67)
  build user:       root@my_company-thanos-1-build
  build date:       20191016-07:10:48
  go version:       go1.13.1

Object Storage Provider:
S3 with a minio setup

What happened:
Compaction directory structure is weird when a label includes a /. The additional / produces a subfolder.

ls -l /data/compact/
total 20
drwxr-sr-x    4 10003300 10003300      4096 Oct 17 13:03 0@{cluster="s-openshift.my_company.de",prometheus="openshift-monitoring
drwxr-sr-x    4 10003300 10003300      4096 Oct 17 13:03 300000@{cluster="s-openshift.my_company.de",prometheus="openshift-monitoring
drwxr-sr-x    4 10003300 10003300      4096 Oct 17 13:03 3600000@{cluster="s-openshift.my_company.de",prometheus="openshift-monitoring

What you expected to happen:

A nice folder without weird subfolders to chunks

ls -l /data/compact/3600000@{cluster="s-openshift.my_company.de",prometheus="openshift-monitoring/my_company",prometheus_replica="_agg_replica_"}:
total 36
drwxr-sr-x    2 10003300 10003300      4096 Oct 17 13:03 01DQC4N9T0YNFZZR27RD5GCAG5
drwxr-sr-x    2 10003300 10003300      4096 Oct 17 13:03 01DQC4PQ6NTRH3BJYQGJ1S6CKK
drwxr-sr-x    2 10003300 10003300      4096 Oct 17 13:03 01DQC4R5NBDYY241MHJW0MN74D
drwxr-sr-x    2 10003300 10003300      4096 Oct 17 13:03 01DQC4SYR49E14SAN1245ZZZA2
drwxr-sr-x    2 10003300 10003300      4096 Oct 17 13:03 01DQC4VGDG3Y9YS46FZTZ9EPJR
drwxr-sr-x    2 10003300 10003300      4096 Oct 17 13:03 01DQC4WYWR0ZY2PR6ZTXTHJQ1C
drwxr-sr-x    2 10003300 10003300      4096 Oct 17 13:03 01DQC4YB870WD2X7DD9V09SYGG

How to reproduce it (as minimally and precisely as possible):

  • Create a Prometheus cluster with prometheus-operator and thanos
  • start a compaction
  • check folder structure

Anything else we need to know:

I think, at the moment it doesn't affect the functionality

@Reamer Reamer changed the title compactor: weird Directory structure compactor: weird directory structure Oct 17, 2019
bwplotka added a commit that referenced this issue Oct 18, 2019
Fixes: #1661

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
@bwplotka
Copy link
Member

Thanks for reporting!

Wow, it is interesting indeed (: I fixed it here: #1666

It should not have any bad consequences, but still, it's safer to sanitize that. Now it will be resolution@hash.

bwplotka added a commit that referenced this issue Oct 18, 2019
Fixes: #1661

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
bwplotka added a commit that referenced this issue Oct 21, 2019
Fixes: #1661

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
bwplotka added a commit that referenced this issue Oct 21, 2019
…s. (#1666)

* Fixed compactor tests; Moved to full e2e compact test; Cleaned metrics.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

* Removed block after each compaction group run.

Fixes: #1499

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

* Moved to label hash for dir names for compactor groups.

Fixes: #1661

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

* Addressed comments, rebased.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
GiedriusS pushed a commit that referenced this issue Oct 28, 2019
…s. (#1666)

* Fixed compactor tests; Moved to full e2e compact test; Cleaned metrics.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

* Removed block after each compaction group run.

Fixes: #1499

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

* Moved to label hash for dir names for compactor groups.

Fixes: #1661

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

* Addressed comments, rebased.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
brancz pushed a commit to brancz/objstore that referenced this issue Jan 28, 2022
…s. (#1666)

* Fixed compactor tests; Moved to full e2e compact test; Cleaned metrics.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

* Removed block after each compaction group run.

Fixes: thanos-io/thanos#1499

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

* Moved to label hash for dir names for compactor groups.

Fixes: thanos-io/thanos#1661

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>

* Addressed comments, rebased.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants