Lambda Test Tool using wrong directory to find configuration JSON files

[BradKnowles]

Describe the bug

When using Microsoft.Extensions.Configuration.Json version 7.0.0 and specifying a path for a JSON file, like so

var logConfig = new ConfigurationBuilder()
    .AddJsonFile("logging.json")
    .Build();

logging.json is not found, even when present in the output directory.

Instead of searching ..\bin\Debug\net6.0\logging.json it uses %USERPROFILE%\.dotnet\tools\.store\amazon.lambda.testtool-6.0\0.13.0\amazon.lambda.testtool-6.0\0.13.0\tools\net6.0\any\logging.json.

If specifying appsettings.json in the AddJsonFile() method, the appsettings.json file from the Mock Lambda Test tool is injected into logConfig instead of the one supplied with the Lambda project.

Expected Behavior

I expect logging.json to be parsed from ..\bin\Debug\net6.0\logging.json and become part of the logConfig configuration as mentioned above.

Current Behavior

logging.json is serached for in %USERPROFILE%\.dotnet\tools\.store\amazon.lambda.testtool-6.0\0.13.0\amazon.lambda.testtool-6.0\0.13.0\tools\net6.0\any\logging.json, is not found, and logConfig remains empty.

Reproduction Steps

• Have the AWS Toolkit installed in Visual Studio
• Create a new solution using the AWS Lambda Project (.NET Core - C#) template
• Choose the Empty Function Blueprint
• Add the Microsoft.Extensions.Configuration.Json version 7.0.0 NuGet package
• Add using Microsoft.Extensions.Configuration; to the top of Function.cs
• Copy and paste this line of code into the FunctionHandler

var logConfig = new ConfigurationBuilder()
    .AddJsonFile("logging.json", false)
    .Build();

• Execute the project using the Mock Lambda Test Tool - You can use "" for the function input.

This will throw an exception saying it can't find logging.json, which is correct since we didn't add the file. However, the error message will show the path it tried to search for the file. Instead of searching ..\AWSLambda1\AWSLambda1\bin\Debug\net6.0\logging.json it uses %USERPROFILE%\.dotnet\tools\.store\amazon.lambda.testtool-6.0\0.13.0\amazon.lambda.testtool-6.0\0.13.0\tools\net6.0\any\logging.json.

Possible Solution

I believe the issue might be in this area. Even though AppBaseCompilationAssemblyResolver(Path.GetDirectoryName(rootAssemblyPath)) uses the correct directory, debugging into .NET source shows that DependencyContext of the assemblyResolver points to the wrong location. This is still a educated guess though, I wasn't able to dig too much further as of yet.

aws-lambda-dotnet/Tools/LambdaTestTool/src/Amazon.Lambda.TestTool/Runtime/LambdaAssemblyLoadContext.cs

Lines 124 to 130 in 630d1d3

	this.assemblyResolver = new CompositeCompilationAssemblyResolver
	(new ICompilationAssemblyResolver[]
	{
	new AppBaseCompilationAssemblyResolver(Path.GetDirectoryName(rootAssemblyPath)),
	new ReferenceAssemblyPathResolver(),
	new PackageCompilationAssemblyResolver()
	});

Additional Information/Context

A workaround does exist. Adding .SetBasePath(Environment.CurrentDirectory) to the ConfigurationBuilder will cause the configuration system to use the correct path.

AWS .NET SDK and/or Package version used

<PackageReference Include="Amazon.Lambda.Core" Version="2.1.0" />
<PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="2.3.1" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="7.0.0" />

Targeted .NET Platform

.NET 6

Operating System and version

Windows 10

[ashishdhingra]

Hi @BradKnowles,

Good morning. Thanks for reporting the issue.

The root path for the file provider used by new ConfigurationBuilder() is the path of the current executing assembly. When using Lambda Test Tool, since Visual Studio is actually executing the Lambda Test tool, this is the path of the test tool, not the path of the current project assembly. Lambda Test Tool loads the current project assemblies using reflection to enable testing Lambda functions using the tool. You could use the workaround .SetBasePath(Environment.CurrentDirectory) as you suggested in combination with the #if DEBUG preprocessor directive. The reason that the Environment.CurrentDirectory would work is that the when Lambda test tool is launched, the current directory is set as the root of the project assemblies.

Thanks,
Ashish

[BradKnowles]

I was doing some more testing and everything works correctly if the project is made from the Lambda ASP.NET Core Web API (AWS) blueprint. Is it somehow hosted differently?

[ashishdhingra]

I was doing some more testing and everything works correctly if the project is made from the Lambda ASP.NET Core Web API (AWS) blueprint. Is it somehow hosted differently?

@BradKnowles The project created from Lambda ASP.NET Core Web API (AWS) blueprint is by default executed (debugged) using the built in IIS Express (Kestrel) server. Hence it works fine. Although you could use Lambda Test Tool and use API Gateway sample requests, test tool is better used while testing simple Lambda functions.

Thanks,
Ashish

[github-actions]

This issue has not received a response in 5 days. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.

[normj]

@ashishdhingra I asked @BradKnowles to submit this. The problem is the test tool's process has its current working directory set to where it was executed so if the Lambda function is looking for files relative to its location thinking that is where the current working directory is then it will fail. We need to make sure before we call the function that we change the Lambda test tool process's current working directory to the location of the Lambda function so the behavior will be the same as it is in the real Lambda service.

[BradKnowles]

Is there any more help I can provide in tracking this down?

[ashishdhingra]

@ashishdhingra I asked @BradKnowles to submit this. The problem is the test tool's process has its current working directory set to where it was executed so if the Lambda function is looking for files relative to its location thinking that is where the current working directory is then it will fail. We need to make sure before we call the function that we change the Lambda test tool process's current working directory to the location of the Lambda function so the behavior will be the same as it is in the real Lambda service.

@normj Changing the current process directory by using the Directory.SetCurrentDirectory() doesn't work. During debugging, when debug breakpoint is reached in Lambda function code, even though the Directory.GetCurrentDirectory() points to the path set (as part of this change, it is set to Lambda assembly directory) before executing the Lambda function, it still points to \Amazon.Lambda.TestTool.BlazorTester\\bin\\Debug\\net8.0\\ directory. This might be because:

• The current default base directory used when invoking IConfigurationBuilder.AddJson() is the one referenced by AppContext.BaseDirectory.
• The value of AppContext.BaseDirectory is a per-application domain property. Its value corresponds to the AppDomain.BaseDirectory property of the current application domain.

So Directory.SetCurrentDirectory() doesn't work. (May be we could implement a workaround to expose some custom environment variable, e.g. AWS_APPCONTEXT_BASE_PATH, which points to Lambda assembly directory and the Lambda function code could probe this environment variable and use IConfigurationBuilder.SetBasePath())

Selecting Copy to Output Directory as Copy always for the JSON file doesn't work. It displays below error:

System.IO.FileNotFoundException: 'The configuration file 'logging.json' was not found and is not optional. The expected physical path was 'C:\Users\<<REDACTED>>\.dotnet\tools\.store\amazon.lambda.testtool-8.0\0.15.2\amazon.lambda.testtool-8.0\0.15.2\tools\net8.0\any\logging.json'.'

It searches for JSON file only in lambda test tool directory, perhaps due to above reasoning.

[normj]

In the current test tool there is not a way to fix this issue. The good news is we are working on a new version of the test tool. The design of new version is the test tool is always out of process so doesn't interfere with how the project is loaded, working directories or dependency resolution. Currently the focus is on Aspire integration with the new tool which you can read about here. aws/integrations-on-dotnet-aspire-for-aws#17

We don't have any documentation yet on how to use the new tooling outside of Aspire and the experience isn't smooth yet. We will make it better. But if you want to try it out here are some rough instructions.

1. Install the new tooling.

dotnet tool install -g amazon.lambda.testtool --prerelease

2. Start the new test tool below I used port 8000 but feel free to change it. This will launch a webpage to add Lambda events to events to debug.

dotnet lambda-test-tool --lambda-emulator-port 8000

3. In your .NET lambda project add a new profile like below. You need to change the 2 <assembly-name> with the name of your assembly for the .NET Lambda project. And replace the <lambda-function-handler> with your function handler. Also note the port 8000 is set for the AWS_LAMBDA_RUNTIME_API environment variable. If you used a different port in step 2 then use that here.

{
  "profiles": {
    "TestToolv2": {
      "commandName": "Executable",
      "environmentVariables": {
        "AWS_LAMBDA_RUNTIME_API": "localhost:8000"
      },
      "commandLineArgs": "exec --depsfile ./<assembly-name>.deps.json --runtimeconfig ./<assembly-name>.runtimeconfig.json  %USERPROFILE%\\.dotnet\\tools\\.store\\amazon.lambda.testtool\\0.0.1-preview\\amazon.lambda.testtool\\0.0.1-preview\\content\\Amazon.Lambda.RuntimeSupport\\net8.0\\Amazon.Lambda.RuntimeSupport.dll <lambda-function-handler>",
      "workingDirectory": ".\\bin\\$(Configuration)\\net8.0",
      "executablePath": "dotnet"
    }
  }
}

Make sure new profile is selected and then push F5 for debugging. You can go back to the test tool's web page and add events to start debugging. Your issue about not find the config files because the working directory is set to where the test to is installed at goes away because the working directory is set by the launchsettings.json file as you can see above.

We plan on making this setup more automatic in the future but hopefully this is enough for you to start trying the new tooling. I'm going to close this because our focus is on the new tooling and this is not an issue with the new tooling.

[github-actions]

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.