Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't know how to use this tool, need enhance documentation #31

Open
yangyang919 opened this issue Aug 25, 2021 · 3 comments
Open

Don't know how to use this tool, need enhance documentation #31

yangyang919 opened this issue Aug 25, 2021 · 3 comments

Comments

@yangyang919
Copy link

yangyang919 commented Aug 25, 2021

Hi, we are a bunch of data scientists. We are trying to leverage prometheus metrics data to do AIOPS. Metrics data are already stored in AWS S3 via Thanos sidecar approach. To make data consumable by Spark, need a tool to convert data. Looks like this tool can be helpful, but it's really hard to understand how to use this tool given neither of us have Go programming experiences...

For instance, we are testing below commands:
go run ./cmd/obslytics export --input-config='{"endpoint":"127.0.0.1:10901","tls_config":{"insecure_skip_verify":true}}' --match="container_cpu_load_average_10s" --resolution=2h --output-config-file=container_cpu_load_average_10s.parquet --debug --min-time="2021-08-24T20:00:00+00:00" --max-time="2021-08-25T20:00:00+00:00"

The console gets error codes:
level=error ts=2021-08-25T09:32:26.912354Z caller=main.go:104 msg="running command failed" err="unsupported Reader type " exit status 1

No idea why, something related to StoreAPI. We deployed Thanos sidecar and Querier, and then kubectl port-forward querier pod like this: kubectl port-forward thanos-addons-query-7fc47d8cff-tf4rg 10901:10902

@yangyang919
Copy link
Author

Update:
By reading source codes, we made it work finally. First port-forward Thanos StoreAPI, then run command:
go run ./cmd/obslytics export --input-config='{"type":"STOREAPI","endpoint":"127.0.0.1:10901","tls_config":{"insecure_skip_verify":true}}' --output-config='{"type":"parquet", "path":"metrics/up", "storage":{"type":"s3", "config": {"bucket": "xxx", "endpoint": "s3.us-east-1.amazonaws.com", "insecure": true, "access_key": "key", "secret_key": "secret"}}}' --match="up" --resolution=2h --min-time="2021-08-24T23:59:59Z" --max-time="2021-08-26T00:59:59Z" --debug

@bwplotka
Copy link
Member

Nice 👍🏽

@bwplotka
Copy link
Member

Yup, docs are needed. Help wanted!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants