Logstash Input/Output

[krainboltgreene]

Hello, my services all use logstash and it would be fantastic if I could have loki ingest or receive data from logstash.

I've been told that loki receives an HTTP request, does anyone have the exact specification of this HTTP request?

[DanCech]

Loki accepts application/json with a simple format outlined here https://github.com/grafana/loki/blob/master/docs/api.md#rest-api

[krainboltgreene]

Fantastic, so it looks like I have transform @timestamp into ts, and the rest of my data into line?

Question 1: Is line a string or can it be a json structure?
Question 2: Is line required or can I just pass along the rest of my json?

[DanCech]

1. line is a string, you can pass multiple labels for each log stream, but the individual log lines are strings
2. line is required, if you have multiple fields that are specific to the log line you'd want to encode them together into a string

[krainboltgreene]

I see, so it looks like @timestamp -> ts and message -> line.

[DanCech]

Absolutely, and labels would come from any fields that identify the log stream, host, syslog_facility, etc. Note that labels is a prometheus label string {label="value",...} which is then embedded in the json document.

[ytzelf]

I see, so it looks like @timestamp -> ts and message -> line.

@krainboltgreene were you successful in using logstash's http output to route logs to Loki? If so, would you mind sharing your pipeline configuration? Thqnks a lot!

[ioanc]

@krainboltgreene were you able to send logs to Loki from logstash?
could you please share the pipeline configuration?
thank you

[krainboltgreene]

No, and frankly I won't. I grossly overestimated my need for logstash. It's now a huge pain and I wish I hadn't picked it.