Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] AWS CloudWatch Input add multiline support #23575

Open
gomclifton opened this issue Jan 19, 2021 · 6 comments
Open

[Filebeat] AWS CloudWatch Input add multiline support #23575

gomclifton opened this issue Jan 19, 2021 · 6 comments
Labels
enhancement Team:Integrations Label for the Integrations team

Comments

@gomclifton
Copy link

Mirroring request for S3 multiline input, #23350

Describe the enhancement:
At the moment Filebeat AWS CloudWatch Input doesn't offer multiline support

Describe a specific use case for the enhancement or feature:
In AWS Cloudwatch streaming to SQS there might are occasions for log files that hold multiline log lines like for instance AWS Hadoop or other Java products.

We currently have AWS SQS logs in Cloudwatch that need multiline processing in order to get into ELK pipeline.

https://www.elastic.co/guide/en/beats/filebeat/master/filebeat-input-s3.html
https://github.com/elastic/beats/blob/master/x-pack/filebeat/input/s3/config.go#L12
Would it be possible to add multiline support please like there exist for logs input already??
https://www.elastic.co/guide/en/beats/filebeat/master/multiline-examples.html
https://github.com/elastic/beats/blob/master/filebeat/input/log/config.go#L34
https://github.com/elastic/beats/blob/master/filebeat/input/log/config.go#L70

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 19, 2021
@andresrc andresrc added the Team:Integrations Label for the Integrations team label Jan 20, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 20, 2021
@ifmusic
Copy link

ifmusic commented Oct 13, 2021

Is there any workaround at least for this? I tried using logstash multiline codec but it refuses to work when filebeat is the input.
So, no multiline in filebeat input, no multiline in logstash. I'm not sure how to workaround this one

@botelastic
Copy link

botelastic bot commented Apr 5, 2023

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Apr 5, 2023
@karlgrz
Copy link

karlgrz commented Jun 23, 2023

Any thoughts on this one? Would be helpful. Thanks! 👋

@botelastic botelastic bot removed the Stalled label Jun 23, 2023
@DesAWSume
Copy link

Waiting on solution as well, please bring the thread up to date

@limberger
Copy link

Hello! is there a solution?
Problem is the continuation line beguns with timestamp ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Team:Integrations Label for the Integrations team
Projects
None yet
Development

No branches or pull requests

9 participants