[Filebeat 7.14.2] Adding log_group_name_prefix in aws-cloudwatch input

[chyeyun]

This is the example which is my AWS cloudwatch log groups.

/aws/eks/cluster
/aws/eks/de-cluster
/aws/eks/mo-cluster
aws-tower-logs/Cloud
aws-tower-logs/CloudLogs
...

I'm trying to get all log groups, and I tested to get log groups which are started '/aws' at first.
filebeat.yml configuration:

filebeat:
 inputs:
  - type: aws-cloudwatch
    log_group_name_prefix: /aws/           # or /aws or /
    region_name: ap-northeast-2

But filebeat fetchs only one log group, for instance '/aws/eks/mo-cluster' just like this:

2021-11-17T01:59:01.541Z INFO [aws-cloudwatch] awscloudwatch/input.go:155 aws-cloudwatch input worker for log group: '/aws/eks/mo-cluster' has started

When I apply log_group_name_prefix: aws, it's same.
There is no other log group after that and no error message.

It was confirmed in the issue #21528 that the reason the log_group_name_prefix option was applied was to get multiple logs.
So What should I put values on 'log_group_name_prefix' to get all log groups( I have 100~200 log groups)?

***bug issue
I think this 'workerOnce.Do()' line in code seems like a bug.
https://github.com/elastic/beats/blame/fd322dad6ceafec40c84df4d2a0694ea357d16cc/x-pack/filebeat/input/awscloudwatch/input.go#L151

• I have 'CloudwatchFullAccess' Iam Policy # @platforms

[chyeyun]

How can I add Labels?

[chyeyun]

@platforms

[chyeyun]

@kaiyan-sheng .
Hello, I added comment about this issue on #21528 issue. Please check this problem.

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[chyeyun]

@kaiyan-sheng
Thanks for your support. And I have a question, Which version is the merged version?

[kaiyan-sheng]

@chyeyun 8.0.0 would be the version 🙂