Exception caught when writing error log to fileSystem.IO.IOException: Read-only file system : '/var/task/aws-logger-errors.txt'

[pponzano]

Describe the bug

I've introduced serilog to some AWS lambda I wrote... when I run them I got this... any suggestion?

Here's my code

  public static void InitializeLogger()
        {
            AWSLoggerConfig configuration = new AWSLoggerConfig();

            Log.Logger = new LoggerConfiguration()
                .WriteTo.AWSSeriLog(configuration)
                .CreateLogger();

            var serviceCollection = new ServiceCollection();
            ConfigureServices(serviceCollection);

            serviceCollection.AddLogging(loggingBuilder =>
                loggingBuilder.AddSerilog(dispose: true));
        }

and when I log it's something as

   Log.Logger.Information($"Request inserted with Token: {guid}\n");

Any suggestion?
Thanks

Expected Behavior

The fact that the logs are correcly written to the CloudWatch

Current Behavior

This error is written in cloudwatch but no other application logs are written

Reproduction Steps

see up

Possible Solution

No response

Additional Information/Context

No response

AWS .NET SDK and/or Package version used

AWS Lambda core 2.1.0

Targeted .NET Platform

.NET6

Operating System and version

Windows 10

[ashishdhingra]

Hi @pponzano,

Good morning.

Could you please share your .csproj file, appSettings.json and Serilog configuration (if not using appSettings.json)? Looks like you have Serilog:LibraryLogErrors set to true in your config. This flag enables Serilog extension to log any service errors to local log file. When executing in Lambda environment, looks like there are some library errors (such as inability to log to CloudWatch logs may be due to lack of permissions) and it's unable to write to this file since Lambda code is executing with /var/task as current directory.

This setting is typically used for development environment. So you should have this setting enabled instead in appSettings.Development.json, not in production appSettings.json.

Thanks,
Ashish

[pponzano]

Hello,
Sorry for my late response...have been really busy and had to focus on delivery... I don't use appSettings.json
Here's how I configure the logger

 public static void InitializeLogger()
        {
            AWSLoggerConfig configuration = new AWSLoggerConfig();
            
            Log.Logger = new LoggerConfiguration()
                .WriteTo.AWSSeriLog(configuration)
                .CreateLogger();
            
            var serviceCollection = new ServiceCollection();
            ConfigureServices(serviceCollection);

            serviceCollection.AddLogging(loggingBuilder =>
                loggingBuilder.AddSerilog(dispose: true));
        }

and here's the .csproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
    <AWSProjectType>Lambda</AWSProjectType>
    <!-- This property makes the build directory similar to a publish directory and helps the AWS .NET Lambda Mock Test Tool find project dependencies. -->
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>

    <!-- Generate ready to run images during publishing to improve cold start time. -->
    <PublishReadyToRun>true</PublishReadyToRun>	  
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Amazon.Lambda.APIGatewayEvents" Version="2.4.1" />
    <PackageReference Include="Amazon.Lambda.Core" Version="2.1.0" />
    <PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.3.0" />
    <PackageReference Include="AWSSDK.DynamoDBv2" Version="3.7.3.34" />
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
  </ItemGroup>
  <ItemGroup>
    <ProjectReference Include="..\xxx\xxx.csproj" />
    <ProjectReference Include="..\xxx.WriteRequestToDynamoDB\xxx.WriteRequestToDynamoDB.csproj" />
  </ItemGroup>
</Project>

I've not manually set the Serilog:LibraryLogErrors in any part of the code

Thanks

[ashishdhingra]

@pponzano Apologies for late reply. Is it possible for you to share sample minimal code (solution) to reproduce the issue?

[ashishdhingra]

@pponzano Apologies for late reply. Is it possible for you to share sample minimal code (solution) to reproduce the issue?

@pponzano Is this issue resolved? Could you also check if the Lambda IAM role has proper permissions to write to CloudWatch logs?

[iongion]

I am experiencing the same problem, how did you guys solve it ?

This is in appsettings.json

{
    "Logging": {
        "LogLevel": {
            "Default": "Debug",
            "Microsoft.AspNetCore": "Warning",
            "Microsoft.EntityFrameworkCore.Database.Command": "Information"
        }
    },
    "Serilog": {
        "Using": [
            "AWS.Logger.SeriLog"
        ],
        "LogGroup": "/aws/lambda/CustomAPI-Production-ApiHandler",
        "Region": "us-east-1",
        "MinimumLevel": {
            "Default": "Debug"
        },
        "WriteTo": [
            {
                "Name": "AWSSeriLog",
                "Args": {
                    "formatter": "Serilog.Formatting.Json.JsonFormatter, Serilog"
                }
            }
        ]
    }
}

And this is is Program.cs of asp.net core 6.0 with top level statements

builder.Host.UseSerilog((_, loggerConfig) =>
{
    loggerConfig.ReadFrom.Configuration(builder.Configuration);
});

builder.Logging.AddAWSProvider();

I have no other configuration related to logging anywhere else, I don't understand what Serilog:LibraryLogErrors turned on / off is.

And then in the CloudWatch logs I only see exceptions

It is a aspnet core 6.0 AWS SAM (serverless) application, it has all permissions to write to logs, if I am not using Serilog, all appears properly in CloudWatch

Part of the permissions in the sam template file

                   - Action:
                          - "logs:CreateLogGroup"
                          - "logs:CreateLogStream"
                          - "logs:PutLogEvents"
                      Effect: "Allow"

EDIT - I've mitigated it by not using AWS.Logger.Serilog - I'm using the console and now CloudWatch logs contain everything properly, formatted properly as JSON. No need for any additional permissions tinkering.

{
    "Logging": {
        "LogLevel": {
            "Default": "Debug",
            "Microsoft.AspNetCore": "Warning",
            "Microsoft.EntityFrameworkCore.Database.Command": "Information"
        }
    },
    "Serilog": {
        "Using": [ "Serilog.Sinks.Console" ],
        "MinimumLevel": "Debug",
        "WriteTo": [
            {
                "Name": "Console",
                "Args": {
                    "formatter": "Serilog.Formatting.Json.JsonFormatter, Serilog",
                    "textFormatter": "Serilog.Formatting.Json.JsonFormatter, Serilog"
                }
            }
        ],
        "Properties": {
            "Application": "CustomAPI"
        }
    }
}

What is then the benefit of using AWS.Logger.SeriLog instead of the Console if the outcomes are the same ? (aren't they?)

[Rajesh35hsp]

Facing same issue with

"Using": [
"AWS.Logger.SeriLog"
],

but it works with

"Using": [ "Serilog.Sinks.Console" ],

not sure what is the use of AWS.Logger.SeriLog then.

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.