Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compactor: failing to create directory, with error "file name too long" #1491

Closed
ingvarm-gr opened this issue Sep 4, 2019 · 1 comment
Closed

Comments

@ingvarm-gr
Copy link

Thanos: v0.6.0
Prometheus: v2.11.2

thanos:v0.6.0

What happened

We started the thanos-compactor, after a while it crashes and ends up in a crashloop

What you expected to happen

Thanos-compactor running and compacting data

How to reproduce it (as minimally and precisely as possible):

Don't know.

Full logs to relevant components

Unfortunately, I can't egress the full logs, but the failure log message looks like:
level=error ts=2019-09-04T10:11:14.436861213Z caller=main.go:199 msg="running command failed" err="error executing compaction: compaction failed: compaction failed for group 0@{prometheus="REDACTED/prometheus-REDACTED",prometheus_replica="prometheus-prometheus-REDACTED-0"}: download block 01DCQ2K02CKMAD9N4R6ZA5VZW9: create dir: mkdir /local/state/compact/0@{prometheus="datadev/prometheus-REDACTED",prometheus_replica="prometheus-prometheus-REDACTED-0"}/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9/01DCQ2K02CKMAD9N4R6ZA5VZW9: file name too long"

Anything else we need to know

@bwplotka
Copy link
Member

bwplotka commented Sep 4, 2019

Hi, Thanks for reporting. Looks like some bug, but users run 0.6.0 compactor just fine, so it must be specific to your environment.

How you configure Thanos compact? What object storage do you use? This looks like a duplicate of #844 So can you provide more data to that ticket instead?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants