Investigate the possibility to use fluentbit instead of fluentd

[gemagomez]

Overview

Let's investigate whether it makes sense to change fluentd for fluentbit[1] and proceed accordingly.

[1] https://aws.amazon.com/blogs/opensource/centralized-container-logging-fluent-bit/

[cPu1]

Fluent Bit's deployment is similar to Fluentd's in that it requires a ClusterRole for accessing pods and some logging configuration. Access to CloudWatch logging is also covered by enabling the cloudWatch addon policy in the ClusterConfig. It's much more resource-efficient than Fluentd as stated in the benchmarks in the linked blog post.
However, deploying the exact setup linked in the blog post (streaming via Amazon Kinesis Data Firehose) would be too specific and opinionated for a quickstart component, and would also require manually creating IAM policies (as opposed to just specifying them in iam.withAddonPolicies in the ClusterConfig).

A good reason to continue using Fluentd is its much larger plugin ecosystem. Some users may already be using Fluentd with plugins that are unavailable for Fluent Bit. See aws/containers-roadmap#10 (comment) and aws/containers-roadmap#10 (comment)

[cPu1]

An issue has been opened to either replace Fluentd with Fluent Bit, or use them in tandem: #28