-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Documentation, examples? #295
Comments
Would be nice to get some |
@setevoy2 Traditional database is not a good solution for chunk storage, typically cloud object storage is better for this task. You can use local file(boltdb) as index table, or Google's big table. Any reason you are not looking for DynamoDB? |
@yubozhao Yah, I agreed about traditional databases. Just wondering if Loki can (will can) deal with something else as storages. |
I want to second the request for better documentation and examples. I've read through this blog post about loki's scalability but cant any documentation on how to actually achieve this. |
hi @Bfoster-melrok I agree with more documentations. I am not very clear what documentations you are looking for. From what you wrote, I think you are looking for :
Is that correct? |
@yubozhao Almost,
|
For example setup without using ksonnet, you could use helm --dry-run to print out the raw YAML files. That should be a good starting point. You can also use helm as well. Documentation for configuration is definitely lagging behind. ATM you would have to dig in the code and find what you want. |
I have tried the helm chart and it only seems to be able to do a single loki instance setup with local storage |
It would be nice to get a bit more information about how Loki scales. As far as I understand you are supposed to run multiple instances of Loki. Each of these instances taking on one(or possibly more) of the roles of distributor, ingestor or querier. However I cannot find how this should be achieved. |
Need docs |
It's worse than that, Loki cluster requires also hashicorp consul cluster + an AWS storage (S3 or dynamodb). S3 can be emulated using multiple opensource solutions(like minio, edgefs, etc..) - but not really tested to the best of my knowledge. Anyhow - that means lots more overhead than should be!! |
If you don't have heavy workload, I would advice not use the consul to setup a multi instance Loki. As far as S3/Dynamo those stuff, refer to the upstream support. Those open source projects are not part of the support AFAIK. |
Detailed docs help users try Loki easy and developers support the development. |
agree especially docs on how to scrape logs from other docker containers (when promtail is running in docker itself) is required. |
I just looked through different issues here to learn more about how to deploy loki in a production environment as I could not find documentation. My current understanding is:
Are my findings correct so far? |
Yes we're going to make scaling on helm possible as we don't want you to have to handle this case:
Unless you're running at huge scale. |
Thanks for the information. So the only additional system one needs is a consul cluster then. |
This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions. |
[release-5.8] Backport PR grafana#12503 and grafana#12698
Hi!
First of all - thanks for the Loki - really great.
But still few questions, please.
1. "federation" analog?
Is there any way to configure Loki's instances in the same way as Prometheus uses its
federation
ability?I.e. - I have a Stage environment, with 3 AWS EC2 instances running.
On each EC2 - I have Prom's exporters running + Prometheus itself to collect metrics from local exporters.
Then Prometheus exposes them via its :9090/federation to the central Prometheus instance.
So I'd like to make the same with Loki - collect logs with
promtail
and Loki on some environment's instance, and pass them to the "central" Loki on the host with Prometheus server and Grafana.Is it possible? Can't see any docs about such approach, unfortunately.
UPD: "Solved" this: Loki and Grafana runs on one host,
promtail
"worker" - on another, and just hasurl: http://loki.example.com:3100/api/prom/push
.2. Loki's storages
Found only one doc where storages configuration was mentioned - here.
Can I use local filesystem instead of AWS S3? I think so - just map new volume via Docker like:
Is it correct? And the same for indexes.
And is there any way to use MySQL/Postgres etc as chunks storage?
And is there any way to use something else instead of DynamoDB etc as indexes storage?
3. Documentation, examples?
And finally - the main question... Docs :-)
Where can I see
storage_config
description, available options etc for the Loki, for example? Or -scrape_configs
for thepromtail
? The only way I found since yesterday was an NGINX's "example" here.Or it is absolutely similar to the Prometheus?
Thanks in advance and sorry for posting all question to the same issue.
The text was updated successfully, but these errors were encountered: