(aws-lambda): Allow writing Lambda logs to existing Log Group

[Dzhuneyt]

Our project relies heavily on many Lambdas (that back AppSync as Data Sources). We'd love to be able to have a centralized "log" of all these lambdas. Currently, such logs are scattered across multiple Log groups (one per lambda). In other words, the Lambda construct currently creates its own log group and has no way of referring to an existing Log group where Lambda logs should be written.

Proposed Solution

new Function(this, 'fn', {
    code: ...,
    handler: ....,
    logGroup: LogGroup.fromLogGroupArn(...),
})

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[nija-at]

Unfortunately, the Lambda service reports logs to a specific log group, named after the function - https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html.

As far as I know, there is no way to control which log group this goes to.

the Lambda construct currently creates its own log group

Not technically. In the default case, the Lambda service creates this log group. When you specify the logRetention property in the CDK, the CDK creates the log group if it doesn't already exist and adjusts its retention policy; however, the log group name must be what Lambda expects it to be.

Let me know if there's something that I missed here.

[sith]

nija-at, could you explain why this limitation exists.

I have a state machine with multiple lambda functions and I want to have logs in one place rather going and querying them in corresponding lambda log groups.

[Dzhuneyt]

@sith this is a limitation of the AWS Lambda service itself, not something that can be solved with CloudFormation or AWS CDK (which all are built on top of AWS services like Lambda). See this for further information.

If you want to achieve true aggregation of logs for your whole State Machine that is composed of Lambdas that write to multiple LogGroups, you can either query across multiple LogGroups using something like the CloudWatch Logs Insights tool or implement your own logging layer inside the application runtime code (so e.g. replace your console.log() lines with Logger.getInstance().log(message))

[lefnire]

I'm looking at cdk-lib/lambda/Function and I see a {logGroup?: ILogGroup} prop:

The LogGroup where the Lambda function's logs are made available.

I'm wondering if maybe this prop became available after this discussion took place? Unless I'm misunderstanding its functionality. If I'm not mistaken, one can just:

import * as logs from '@aws-cdk/aws-logs';
import * as lambda from '@aws-cdk/aws-lambda';

const aggregatedLogGroup = new logs.LogGroup(this, 'AggregatedLogGroup', {
  logGroupName: '/aws/lambda/aggregated',
});

const myFunction = new lambda.Function(this, 'MyFunction', {
  ...
  logGroup: aggregatedLogGroup,
});

[Dzhuneyt]

@lefnire you are probably mistaking the lambda.logGroup getter function for a prop. The FunctionProps interface (which declares the input configuration for the created lambda) does not accept any logGroup configuration. Only logGroup expiration settings.

[lefnire]

@Dzhuneyt you're absolutely correct. Thanks for pointing that out!

[tilak-puli]

Just an update:

I see a change in CDK API to set a custom log group from v2.110. I am trying to use this and will update after using

https://github.com/aws/aws-cdk/releases/tag/v2.110.0
#28039