java_maven workflow not up-to-date fails on utf-8 encoding issues of Maven output #6075
Description
Description:
This issue is not about the original encoding handling problem, but the fact, that current installer does not include the package which seems to fix that issue.
sam build fails with Error: JavaMavenWorkflow:MavenBuild - 'utf-8' codec can't decode byte 0xdf in position 76813: invalid continuation byte. --debug shows the error originates from \aws_lambda_builders\workflows\java_maven\maven.py", line 31 which is this: LOG.debug("Maven logs: %s", stdout.decode("utf8").strip())
This is weird as there are preexisting issues (ie. #1164 ) related to this, which got closed already. And fun fact is, that the aws_lambda_builders included with CLI installer does not seem to be the same as current content of the repository for that package. at least maven.py now uses an util for decoding maven output, instead direcly using decode() with hardcoded utf-8.
Steps to reproduce:
- install SAM CLI as instructed in https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html#install-sam-cli-instructions
- run
sam.cmd build Function --template project\.aws-sam\temp-template.yaml --build-dir project\.aws-sam\build
Observed result:
SAM build fails after Maven finishes successfully:
2023-10-12 13:02:46,559 | JavaMavenWorkflow:CopySource succeeded
2023-10-12 13:02:46,560 | Running JavaMavenWorkflow:MavenBuild
2023-10-12 13:04:01,892 | JavaMavenWorkflow:MavenBuild raised unhandled exception
Traceback (most recent call last):
File "C:\Soft\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflow.py", line 371, in run
action.execute()
File "C:\Soft\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\java_maven\actions.py", line 39, in execute
self.subprocess_maven.build(self.scratch_dir)
File "C:\Soft\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\java_maven\maven.py", line 31, in build
LOG.debug("Maven logs: %s", stdout.decode("utf8").strip())
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdf in position 76813: invalid continuation byte
Build Failed
If, however, I update the installed maven.py to say LOG.debug("Maven logs: %s", stdout.decode(encoding="utf8", errors="replace").strip()) the whole build passes without issue, also cannot identify where the replacement was done.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:13 min
[INFO] Finished at: 2023-10-12T13:27:13+02:00
[INFO] ------------------------------------------------------------------------
2023-10-12 13:27:15,963 | JavaMavenWorkflow:MavenBuild succeeded
2023-10-12 13:27:15,965 | Running JavaMavenWorkflow:MavenCopyDependency
2023-10-12 13:27:15,965 | Running copy_dependency with scope: runtime
2023-10-12 13:27:21,176 | JavaMavenWorkflow:MavenCopyDependency succeeded
Expected result:
I'd expect Maven output to be copied as-is, and in general, console output encoding should have no impact on the build process...
Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
- OS: win10
sam --version: SAM CLI, version 1.98.0- AWS region: eu-west-1
sam --info
{
"version": "1.98.0",
"system": {
"python": "3.8.8",
"os": "Windows-10-10.0.19041-SP0"
},
"additional_dependencies": {
"docker_engine": "24.0.6",
"aws_cdk": "Not available",
"terraform": "Not available"
},
"available_beta_feature_env_vars": [
"SAM_CLI_BETA_FEATURES",
"SAM_CLI_BETA_BUILD_PERFORMANCE",
"SAM_CLI_BETA_TERRAFORM_SUPPORT",
"SAM_CLI_BETA_RUST_CARGO_LAMBDA"
]
}