Automatically refer to the appropriate, regional FireLens image with addFirelensLogRouter()

[blimmer]

The documentation suggests using Amazon's own ECR images for the FireLens container. However, image is a required property.

It would be a nice convenience if CDK could determine the destination stack's region and automatically use the proper ecr.Repository.fromRepositoryArn() behavior.

Use Case

Today, I can get this behavior manually, but I had to generate the reference myself:

// Amazon recommends pulling this image from their regional ECR repo
// https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_firelens.html#firelens-using-fluentbit
const awsEcrRepo = ecr.Repository.fromRepositoryArn(
  parent,
  "AWSFireLensRepo",
  "arn:aws:ecr:us-west-2:906394416424:repository/aws-for-fluent-bit",
);
taskDefinition.addFirelensLogRouter("DatadogFirelensLogRouter", {
  image: ecs.ContainerImage.fromEcrRepository(awsEcrRepo, "latest"),
  memoryReservationMiB: 50,
  essential: true,
  firelensConfig: {
    type: ecs.FirelensLogRouterType.FLUENTBIT,
    options: {
      enableECSLogMetadata: true,
      configFileType: ecs.FirelensConfigFileType.FILE,
      configFileValue: "/fluent-bit/configs/parse-json.conf",
    },
  },
});

Proposed Solution

Ideally, image on FirelensLogRouterDefinitionOptions is an optional parameter and defaults to the proper ECR image from this table in the docs.

Other

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[bvtujo]

Thanks for bringing this to our attention! Definitely let us know if you're willing to work on the PR; otherwise someone from our team will pick it up.

[rcollette]

The package @aws-cdk/aws-ecs exports the function obtainDefaultFluentBitECRImage. This will reference the appropriate regional image for fluent bit.

You can view usage here:
https://stackoverflow.com/questions/64299664/how-to-configure-aws-cdk-applicationloadbalancedfargateservice-to-log-parsed-jso/

I will say that it takes a lot of discovery to figure out how to override the default logDriver to enable json line parsing. Definitely either more documentation or a simpler API would be helpful.

[github-actions]

This issue has not received any attention in 1 year. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[rcollette]

keep

[PettitWesley]

Q: I can't find any docs for obtainDefaultFluentBitECRImage.

So I think the AWS ECS FireLens constructs are supposed to live here now: https://github.com/cdklabs/cdk-ecs-service-extensions/

But then the function still seems to only be here: obtainDefaultFluentBitECRImage in this repo.

Are there no docs for this function? Or do I not know the right place to look? Is it not intended for end-user direct use?

cdklabs/cdk-ecs-service-extensions#249

@SomayaB @uttarasridhar

[madeline-k]

Hello @PettitWesley, obtainDefaultFluentBitECRImage is just a free-floating function, so it's not included in the JSII assembly (i.e. it's omitted from the API in non -TS/JS langs), and it's excluded from the docs. I am not sure if that was the original intent for this function or not.

If you want to use it in TS/JS, you can, see the documentation in our source code: https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/aws-ecs/lib/firelens-log-router.ts#L149-L155

I think this feature request is still valid, and this function is only a workaround for TS/JS users.

[PettitWesley]

@madeline-k How would I add it to the JSII assembly so that it will become a real official function?

[madeline-k]

Only Classes, Interfaces, and Enums are included. So it would need to be added to a relevant class. Edit: Or possibly a new class made for this? I don't have a strong opinion yet, would need to spend some time diving into it.