-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
v3 helm chart doesn't support native ingress in distributed mode #12582
Comments
trying to make sure I understand, you want to avoid running any kind of gateway pod like nginx and have the ingress do the routing directly to Loki backend services? is this something that worked before and we broke it? (I'm newer to helm but just did a lot of work on the chart changes) |
@slim-bean exactly! - An ingress object standalone. Some users will have that ingress object served via an nginx-ingress-controller pod but others will use something like an ALB to serve the ingress directly (such as myself). It's an alternative completely to running the gateway (for OSS users removes the need to run 2 sets of nginx deployments as well). I'm not sure if it worked on the old loki-distributed chart (we've always used the SSD chart but are keen to migrate to the distributed mode of the main v2 chart) but it worked on the SSD version. I think it was introduced in #1585 and evolved on over time. In the tpl at https://github.com/jkroepke/loki/blob/main/production/helm/loki/templates/_helpers.tpl#L521 and below it dynamically generates the ingress object path routes depending on if the deployment mode and I think that section only has SSD & single binary modes not the new Distributed mode. To resolve it would be a case of mapping the various read paths to the right query service, ruler to the ruler, ingest to ingest etc based on the path (whereas at the moment it's either read/write/backend). |
I will fix it by the end of the week |
@rknightion could you please check the latest chart version with the fix |
@vlad-diachenko works great! I had been overwriting the ingress object (with my own that I'd written) after deploying with helm. The only endpoint that I was defining that this MR doesn't is |
Describe the bug
When deploying loki in distributed mode on the v3 helm chart and using native ingress (no gateway) the ingress object created points everything to a backend service "loki" which does not exist.
It's as if it's assuming single binary mode rather than distributed.
To Reproduce
Set up loki in SSD mode and get it functioning with native ingress.
Switch to distributed mode in the chart values file (and apply other relevant config).
Observe the diff in the rules section from helm showing the chart will point to a non-existent "loki" service rather than the distributed specific backend services.
Expected behavior
That distributed mode sets up the correct routes to the backend services.
The text was updated successfully, but these errors were encountered: