Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not specifying query-frontend downstream URL doesn't log any error #1863

Closed
adityacs opened this issue Mar 29, 2020 · 5 comments
Closed

Not specifying query-frontend downstream URL doesn't log any error #1863

adityacs opened this issue Mar 29, 2020 · 5 comments

Comments

@adityacs
Copy link
Contributor

adityacs commented Mar 29, 2020

Describe the bug
If we don't specify frontend-downstream URL, there is no error logged. When we try to send any request to query-frontend, the request hangs and there is no error returned to client.

To Reproduce
Steps to reproduce the behavior:

  1. Start Loki with target as query-fronted without specifying frontend.downstream-url

Expected behavior
Error returned to client

/cc @cyriltovena

@adityacs adityacs changed the title Not specifying query- frontend downstream URL doesn't log any error Not specifying query-frontend downstream URL doesn't log any error Mar 29, 2020
@adityacs
Copy link
Contributor Author

adityacs commented Mar 29, 2020

If there is no downstream URL, cortex by default configures GRPC Roundtrip. I am not sure how that flow works with Cortex. Is loki expected to behave same way?

Am I missing something?

@cyriltovena
Copy link
Contributor

Yes by default it uses GRPC unless you provide that downstream URL.

For sure we could warn the users if there is no connected GRPC client ready to process item in the frontend queue. But that's more work for Cortex.

@adityacs
Copy link
Contributor Author

Thanks @cyriltovena . Yeah, it would be good to warn user. So, he will be aware of what is not configured

@owen-d
Copy link
Member

owen-d commented Mar 30, 2020

More explicitly, the query frontend operates in one of two fashions:

  1. with frontend.downstream-url. This just proxies requests over http to said url.
  2. without (1). This way it defaults to grpc as a pull service. This requires the as-of-yet undocumented Document frontend worker #1867. In this form, the frontend instantiates per-tenant queues that downstream queriers pull queries from via grpc. In your example configuration, it's enqueueing queries, but nothing is dequeueing them :)

@adityacs
Copy link
Contributor Author

adityacs commented Mar 30, 2020

I was trying to figure out how GRPC thing works without providing any address to connect to 😉. I couldn't get any thing. Now it's clear to me

Thanks @owen-d

@owen-d owen-d closed this as completed Mar 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants