Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ecs: set loggroup retention period for FireLensLogDriver #28258

Open
2 tasks
malatep opened this issue Dec 5, 2023 · 2 comments
Open
2 tasks

ecs: set loggroup retention period for FireLensLogDriver #28258

malatep opened this issue Dec 5, 2023 · 2 comments
Labels
@aws-cdk/aws-ecs Related to Amazon Elastic Container effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2

Comments

@malatep
Copy link

malatep commented Dec 5, 2023

Describe the feature

Hello,

I would like to be able to set the maximum retention for the CloudWatch log group that is created with FireLensLogDriver.

The interface FireLensLogDriverProps does not have the option to set the logRetention.

Similar to how it is currently done in AwsLogDriver using logRetention from AwsLogDriverProps

Thanks

Use Case

Be able to set the maximum retention for the CloudWatch log group

Proposed Solution

No response

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

2.81.0

Environment details (OS name and version, etc.)

Mac OS 13.6.2

@malatep malatep added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Dec 5, 2023
@github-actions github-actions bot added the @aws-cdk/aws-ecs Related to Amazon Elastic Container label Dec 5, 2023
@pahud pahud changed the title [aws_ecs]: set loggroup retention period for FireLensLogDriver ecs: set loggroup retention period for FireLensLogDriver Dec 5, 2023
@pahud
Copy link
Contributor

pahud commented Dec 5, 2023

Yes it would be great to support that.

@pahud pahud added p2 effort/medium Medium work item – several days of effort and removed needs-triage This issue or PR still needs to be triaged. labels Dec 5, 2023
@sakurai-ryo
Copy link
Contributor

Hi @malatep, I think we can set the retention period for the newly created Log Group by specifying log_retention_days in FireLensLogDriverProps.options.

const taskDefinition = new ecs.Ec2TaskDefinition(this, 'TaskDef');
taskDefinition.addContainer('TheContainer', {
  image: ecs.ContainerImage.fromRegistry('example-image'),
  memoryLimitMiB: 256,
  logging: ecs.LogDrivers.firelens({
    options: {
      Name: 'cloudwatch',
      region: 'us-west-2',
      log_group_name: 'firelens-fluent-bit',
      log_stream_prefix: 'from-fluent-bit',
      auto_create_group: 'true',
      log_retention_days: '1',
    },
  }),
});

https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch#configuration-parameters

Please note that the logs:PutRetentionPolicy policy needs to be in the task role.

mergify bot pushed a commit that referenced this issue Dec 19, 2023
This PR resolves the missing `logs:PutRetentionPolicy` permission issue when using `FireLensLogDriver` with CloudWatch Logs.

### Description
When using `FireLensLogDriver` to send logs to CloudWatch Logs, we can specify the retention period for newly created Log Groups by specifying `log_retention_days` in the `FireLensLogDriverProps.options`.
https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch#configuration-parameters
https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.FireLensLogDriverProps.html#options

If you have not added a FluentBit container, CDK will automatically add it to the task definition, and the IAM permissions required for this are added to the task role.
https://github.com/aws/aws-cdk/blob/db22b85c9b2a853aa2f830c182a340f0bcf95d1a/packages/aws-cdk-lib/aws-ecs/lib/base/task-definition.ts#L816
https://github.com/aws/aws-cdk/blob/db22b85c9b2a853aa2f830c182a340f0bcf95d1a/packages/aws-cdk-lib/aws-ecs/lib/firelens-log-router.ts#L170

While `FireLensLogDriver` allows specifying `log_retention_days` for Log Groups, FluentBit cannot set the retention period due to the absence of the `logs:PutRetentionPolicy` policy. Consequently, it results in an `AccessDeniedException`.
To address this, the PR adds the necessary `logs:PutRetentionPolicy` permission to the task role when `log_retention_days` is set in `FireLensLogDriverProps.options`, ensuring FluentBit has the required permissions to set the retention period for Log Groups.

Relates to #28258

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-ecs Related to Amazon Elastic Container effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2
Projects
None yet
Development

No branches or pull requests

3 participants