[ECS] [request]: Support log driver fluentd-buffer-limit option in FireLens

[zhonghui12]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request
FireLens uses the Fluentd docker log driver to send logs from the container to Fluent Bit/Fluentd. Some of our customers found that under high throughput, the default buffer inside of Docker could run out of memory. We could add an option like log-driver-buffer-limit which would set the fluentd-buffer-limit field.

Which service(s) is this request for?
ECS EC2, and ECS Fargate

Additional context
The feature is already released in ECS EC2: https://github.com/aws/amazon-ecs-agent/releases/tag/v1.55.0 and will be supported in ECS Fargate soon.

[zhonghui12]

Note:

This feature is different for #964 because this would provide a way for customers to configure options in fluentd log driver. But #964 aims to work on a fluent bit config option.

[zhonghui12]

This feature has been launched and fully supported in ECS EC2 and ECS fargate now: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_firelens.html#firelens-docker-buffer-limit.

[lucasgadams]

Is something like this supported for fluent bit as well?