Bug: SAM is failing to build python package neo4j

[ernesto-licea-mango]

Description:

I have a sam project with common layer using python 3.9 as runtime for all my lambdas. When i run sam build the process failing due to a packages inside my requirements.txt file (neo4j==5.5.0). This is happening after upgrade aws-sam-cli to version 1.80.0, in version 1.70.0 the process pass without errors

Steps to reproduce:

The logs from sam build --debug are the following:

SAM CLI now collects telemetry to better understand customer needs.

You can OPT OUT and disable telemetry collection by setting the
environment variable SAM_CLI_TELEMETRY=0 in your shell.
Thanks for your help!

Learn More: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-telemetry.html

2023-04-19 13:55:21,120 | Config file location: /myproject/samconfig.toml
2023-04-19 13:55:21,121 | Config file '/myproject/samconfig.toml' does not exist
2023-04-19 13:55:21,137 | Using SAM Template at /myproject/template.yaml
2023-04-19 13:55:21,152 | Using config file: samconfig.toml, config environment: default
2023-04-19 13:55:21,152 | Expand command line arguments to:
2023-04-19 13:55:21,152 | --template_file=/myproject/template.yaml --resource_logical_id=LibLayer --mount_with=READ --build_dir=.aws-sam/build --cache_dir=.aws-sam/cache
2023-04-19 13:55:21,193 | 'build' command is called
2023-04-19 13:55:21,215 | There is no customer defined id or cdk path defined for resource LibLayer, so we will use the resource logical id as the resource id
2023-04-19 13:55:21,216 | 0 stacks found in the template
2023-04-19 13:55:21,230 | There is no customer defined id or cdk path defined for resource LibLayer, so we will use the resource logical id as the resource id
2023-04-19 13:55:21,231 | 1 resources found in the stack
2023-04-19 13:55:21,231 | --base-dir is not presented, adjusting uri lib relative to /myproject/template.yaml
2023-04-19 13:55:21,236 | 1 resources found in the stack
2023-04-19 13:55:21,236 | Instantiating build definitions
2023-04-19 13:55:21,236 | No previous build graph found, generating new one
2023-04-19 13:55:21,236 | Unique Layer build definition found, adding as new (Layer Build Definition: LayerBuildDefinition(LibLayer, /myproject/lib, , 94cca488-8d05-4a8e-ab3b-9ef866a80810, python3.9, ['python3.9'], x86_64, {}), Layer: <samcli.lib.providers.provider.LayerVersion object at 0x7f7330c3d220>)
2023-04-19 13:55:21,236 | Building layer 'LibLayer'
2023-04-19 13:55:21,237 | Loading workflow module 'aws_lambda_builders.workflows'
2023-04-19 13:55:21,238 | Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
2023-04-19 13:55:21,239 | Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
2023-04-19 13:55:21,240 | Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
2023-04-19 13:55:21,241 | Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
2023-04-19 13:55:21,242 | Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
2023-04-19 13:55:21,243 | Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
2023-04-19 13:55:21,243 | Registering workflow 'NodejsNpmEsbuildBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm-esbuild', application_framework=None)'
2023-04-19 13:55:21,245 | Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
2023-04-19 13:55:21,245 | Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
2023-04-19 13:55:21,246 | Registering workflow 'RustCargoLambdaBuilder' with capability 'Capability(language='rust', dependency_manager='cargo', application_framework=None)'
2023-04-19 13:55:21,246 | Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
2023-04-19 13:55:21,265 | Running workflow 'PythonPipBuilder'
2023-04-19 13:55:21,265 | Running PythonPipBuilder:ResolveDependencies
2023-04-19 13:55:21,283 | calling pip download -r /myproject/lib/requirements.txt --dest /tmp/tmpzy4znksp --exists-action i
2023-04-19 13:55:48,327 | Full dependency closure: {unknown==5.5.0(sdist), pytz==2023.3(wheel)}
2023-04-19 13:55:48,327 | initial compatible: {pytz==2023.3(wheel)}
2023-04-19 13:55:48,328 | initial incompatible: {unknown==5.5.0(sdist)}
2023-04-19 13:55:48,328 | Downloading missing wheels: {unknown==5.5.0(sdist)}
2023-04-19 13:55:48,328 | calling pip download --only-binary=:all: --no-deps --platform manylinux2014_x86_64 --implementation cp --abi cp39 --dest /tmp/tmpzy4znksp unknown==5.5.0
2023-04-19 13:55:54,531 | compatible wheels after second download pass: {pytz==2023.3(wheel)}
2023-04-19 13:55:54,531 | Build missing wheels from sdists (C compiling True): {unknown==5.5.0(sdist)}
2023-04-19 13:55:54,531 | calling pip wheel --no-deps --wheel-dir /tmp/tmpzy4znksp /tmp/tmpzy4znksp/neo4j-5.5.0.tar.gz
2023-04-19 13:55:59,842 | compatible after building wheels (no C compiling): {neo4j==5.5.0(wheel), pytz==2023.3(wheel)}
2023-04-19 13:55:59,842 | Build missing wheels from sdists (C compiling False): {unknown==5.5.0(sdist)}
2023-04-19 13:55:59,842 | calling pip wheel --no-deps --wheel-dir /tmp/tmpzy4znksp /tmp/tmpzy4znksp/neo4j-5.5.0.tar.gz
2023-04-19 13:56:05,373 | compatible after building wheels (C compiling): {neo4j==5.5.0(wheel), pytz==2023.3(wheel)}
2023-04-19 13:56:05,373 | Final compatible: {neo4j==5.5.0(wheel), pytz==2023.3(wheel)}
2023-04-19 13:56:05,373 | Final incompatible: set()
2023-04-19 13:56:05,373 | Final missing wheels: {unknown==5.5.0(sdist)}
2023-04-19 13:56:05,423 | PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 51, in execute
package_builder.build_dependencies(
File "/usr/local/lib/python3.9/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 156, in build_dependencies
self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
File "/usr/local/lib/python3.9/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 259, in build_site_packages
raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {unknown==5.5.0(sdist)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/aws_lambda_builders/workflow.py", line 369, in run
action.execute()
File "/usr/local/lib/python3.9/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 57, in execute
raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {unknown==5.5.0(sdist)}

Build Failed
2023-04-19 13:56:05,424 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2023-04-19 13:56:05,433 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2023-04-19 13:56:05,433 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': '99b02333-1cfb-49e6-8d29-419f01859245', 'installationId': '5a5d4851-6fb3-4eef-ae76-a961c56913b0', 'sessionId': '3ab6cb88-b50c-4535-a52c-933903ff1f9e', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.9.16', 'samcliVersion': '1.80.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'metricSpecificAttributes': {'projectType': 'CFN', 'gitOrigin': None, 'projectName': '9a76b440a0db5ffd45f655871fe955a3f7a2de2506333e71709d0667cf697865', 'initialCommit': None}, 'duration': 44271, 'exitReason': 'WorkflowFailedError', 'exitCode': 1}}]}
2023-04-19 13:56:05,433 | Unable to find Click Context for getting session_id.
2023-04-19 13:56:05,435 | Sending Telemetry: {'metrics': [{'events': {'requestId': 'e503f772-2428-424f-9ac7-23dd4b31293b', 'installationId': '5a5d4851-6fb3-4eef-ae76-a961c56913b0', 'sessionId': '3ab6cb88-b50c-4535-a52c-933903ff1f9e', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.9.16', 'samcliVersion': '1.80.0', 'metricSpecificAttributes': {'events': [{'event_name': 'BuildWorkflowUsed', 'event_value': 'python-pip', 'thread_id': 140132771964736, 'time_stamp': '2023-04-19 13:55:21.236', 'exception_name': None}]}}}]}
2023-04-19 13:56:06,827 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
2023-04-19 13:56:06,827 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: PythonPipBuilder:ResolveDependencies - {unknown==5.5.0(sdist)}

Expected result:

The expected result is build my layer dependencies without errors

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Ubuntu 20.04
2. SAM Version: 1.80.0
3. AWS region: us-east-1

[lucashuy]

Hi, thanks for opening this issue. Some of your neo4j credentials have been exposed in the issue. I've updated your issue, but you should still take action to update them.

[lucashuy]

As for the issue, can you provide a reproducible project or set of steps here? I am not able to reproduce this problem using a layer that installs neo4j, and a Lambda function that references that layer. There is a portion of the logs that mention a package named "unknown", wonder if this has something to do with it, and may not be directly related to neo4j.

[ernesto-licea-mango]

Hi, thanks for the quick response,

Step to reproduce the issue

Environment Details

1. OS: ubuntu 20.04
2. SAM Version: 1.80.0
3. AWS region: us-east-1

project structure

├── project-name
      ├── application
      │   ├── __init__.py
      │   └── rest
      │        ├── collections
      │        │   ├── app.py
      │        │   └── __init__.py
      │        ├── channel
      │        │   ├── app.py
      │        │   └── __init__.py
      │        └── __init__.py
      ├── lib
      │    ├── core
      │    │     ├── module1.py
      │    │     ├── module2.py 
      │    │     └── __init__.py
      │    └── requirements.txt
      ├── README.md
      ├── samconfig.toml
      └── template.yaml

template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  Project Description

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 30
    MemorySize: 512

Resources:
  CollectionFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: application/rest/collections
      Handler: app.lambda_handler
      Runtime: python3.9
      Architectures:
        - x86_64
      Layers:
        - !Ref LibLayer

  ChannelFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: application/rest/channel
      Handler: app.lambda_handler
      Runtime: python3.9
      Architectures:
        - x86_64
      Layers:
        - !Ref LibLayer

  LibLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      CompatibleRuntimes:
        - python3.9
      ContentUri: lib
      Description: Provides the base backend shared library and dependencies
      LayerName:
        Fn::Sub: 'my-project-layer'
    Metadata:
      BuildMethod: python3.9

requirements.txt

requests==2.28.2
neo4j==5.5.0

Inside project folder execute the command

sam build --use-container --debug

After build docker image i get the logs shown previously and the process of build project packages fail

If i make a downgrade of aws-sam-cli to version 1.70.0 and execute the same command the process finish success and the folder .aws-sam with my packages is created

[lucashuy]

Hi, thanks for providing the sample project. I can replicate this using the --use-container argument. Would it be possible in your environment to omit that flag for now while we investigate why its failing to resolve the requirements in the container?

[lucashuy]

Hi, this fix is now released in Lambda Builders 1.36.0. SAM CLI will have to be updated to consume this new version in the future.