Skip to content

sam build can't resolve dependencies - tensorflow #2014

Closed
@pasppro

Description

@pasppro

Description

Error when building projects containing tensorflow on a Mac:

Error: PythonPipBuilder:ResolveDependencies - {tensorflow==2.2.0(wheel), grpcio==1.29.0(wheel)}

This could be the same as Issue #1840 but I'm not using PyCharm - just the command line.

Steps to reproduce

(Assumes python 3.6 is already installed)

  1. sam init (Project sam-bld-err, choose python 3.6 runtime and Hello World template)
  2. Create a python virtual environment within the project
    cd sam-bld-err
    virtualenv -p /Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 venv_be
    (Adjust path to python 3.6 as needed)
  3. Activate virtual env and install tensorflow
    source ./venv_be/bin/activate
    pip install tensorflow==2.2.0
  4. Update project requirements
    pip freeze > ./hello_world/requirements.txt
  5. Build the project
    sam build or sam build --use-container (Same results for each)

Observed result

(Sorry - Lots of debug output!)

sam build --debug
Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
'build' command is called
No Parameters detected in the template
2 resources found in the template
Found Serverless function with name='HelloWorldFunction' and CodeUri='hello_world/'
No Parameters detected in the template
Building function 'HelloWorldFunction'
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
calling pip download -r /Users/philsweeney/Documents/AWSProjects/sam-bld-error/hello_world/requirements.txt --dest /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z
Full dependency closure: {tensorflow==2.2.0(wheel), markdown==3.2.2(wheel), setuptools==47.1.1(wheel), tensorboard==2.2.2(wheel), tensorboard-plugin-wit==1.6.0.post3(wheel), scipy==1.4.1(wheel), chardet==3.0.4(wheel), google-auth-oauthlib==0.4.1(wheel), h5py==2.10.0(wheel), opt-einsum==3.2.1(wheel), certifi==2020.4.5.1(wheel), oauthlib==3.1.0(wheel), protobuf==3.12.2(wheel), google-auth==1.16.0(wheel), zipp==3.1.0(wheel), six==1.15.0(wheel), tensorflow-estimator==2.2.0(wheel), importlib-metadata==1.6.0(wheel), termcolor==1.1.0(sdist), gast==0.3.3(wheel), wrapt==1.12.1(sdist), cachetools==4.1.0(wheel), grpcio==1.29.0(wheel), pyasn1==0.4.8(wheel), google-pasta==0.2.0(wheel), requests-oauthlib==1.3.0(wheel), urllib3==1.25.9(wheel), pyasn1-modules==0.2.8(wheel), absl-py==0.9.0(sdist), astunparse==1.6.3(wheel), keras-preprocessing==1.1.2(wheel), idna==2.9(wheel), requests==2.23.0(wheel), rsa==4.0(wheel), wheel==0.34.2(wheel), numpy==1.18.4(wheel), werkzeug==1.0.1(wheel)}
initial compatible: {markdown==3.2.2(wheel), setuptools==47.1.1(wheel), tensorboard==2.2.2(wheel), tensorboard-plugin-wit==1.6.0.post3(wheel), chardet==3.0.4(wheel), google-auth-oauthlib==0.4.1(wheel), opt-einsum==3.2.1(wheel), certifi==2020.4.5.1(wheel), oauthlib==3.1.0(wheel), google-auth==1.16.0(wheel), zipp==3.1.0(wheel), six==1.15.0(wheel), tensorflow-estimator==2.2.0(wheel), importlib-metadata==1.6.0(wheel), gast==0.3.3(wheel), cachetools==4.1.0(wheel), pyasn1==0.4.8(wheel), google-pasta==0.2.0(wheel), requests-oauthlib==1.3.0(wheel), urllib3==1.25.9(wheel), pyasn1-modules==0.2.8(wheel), astunparse==1.6.3(wheel), keras-preprocessing==1.1.2(wheel), idna==2.9(wheel), requests==2.23.0(wheel), rsa==4.0(wheel), wheel==0.34.2(wheel), werkzeug==1.0.1(wheel)}
initial incompatible: {protobuf==3.12.2(wheel), tensorflow==2.2.0(wheel), termcolor==1.1.0(sdist), wrapt==1.12.1(sdist), numpy==1.18.4(wheel), grpcio==1.29.0(wheel), absl-py==0.9.0(sdist), scipy==1.4.1(wheel), h5py==2.10.0(wheel)}
Downloading missing wheels: {tensorflow==2.2.0(wheel), termcolor==1.1.0(sdist), absl-py==0.9.0(sdist), h5py==2.10.0(wheel), protobuf==3.12.2(wheel), wrapt==1.12.1(sdist), numpy==1.18.4(wheel), grpcio==1.29.0(wheel), scipy==1.4.1(wheel)}
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp36m --dest /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z tensorflow==2.2.0
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp36m --dest /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z termcolor==1.1.0
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp36m --dest /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z absl-py==0.9.0
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp36m --dest /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z h5py==2.10.0
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp36m --dest /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z protobuf==3.12.2
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp36m --dest /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z wrapt==1.12.1
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp36m --dest /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z numpy==1.18.4
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp36m --dest /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z grpcio==1.29.0
calling pip download --only-binary=:all: --no-deps --platform manylinux1_x86_64 --implementation cp --abi cp36m --dest /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z scipy==1.4.1
compatible wheels after second download pass: {markdown==3.2.2(wheel), setuptools==47.1.1(wheel), tensorboard==2.2.2(wheel), tensorboard-plugin-wit==1.6.0.post3(wheel), scipy==1.4.1(wheel), chardet==3.0.4(wheel), google-auth-oauthlib==0.4.1(wheel), h5py==2.10.0(wheel), opt-einsum==3.2.1(wheel), certifi==2020.4.5.1(wheel), oauthlib==3.1.0(wheel), protobuf==3.12.2(wheel), google-auth==1.16.0(wheel), zipp==3.1.0(wheel), six==1.15.0(wheel), tensorflow-estimator==2.2.0(wheel), importlib-metadata==1.6.0(wheel), gast==0.3.3(wheel), cachetools==4.1.0(wheel), pyasn1==0.4.8(wheel), google-pasta==0.2.0(wheel), requests-oauthlib==1.3.0(wheel), urllib3==1.25.9(wheel), pyasn1-modules==0.2.8(wheel), astunparse==1.6.3(wheel), keras-preprocessing==1.1.2(wheel), idna==2.9(wheel), requests==2.23.0(wheel), rsa==4.0(wheel), wheel==0.34.2(wheel), numpy==1.18.4(wheel), werkzeug==1.0.1(wheel)}
Build missing wheels from sdists (C compiling True): {termcolor==1.1.0(sdist), wrapt==1.12.1(sdist), absl-py==0.9.0(sdist)}
calling pip wheel --no-deps --wheel-dir /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z/termcolor-1.1.0.tar.gz
calling pip wheel --no-deps --wheel-dir /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z/wrapt-1.12.1.tar.gz
calling pip wheel --no-deps --wheel-dir /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z/absl-py-0.9.0.tar.gz
compatible after building wheels (no C compiling): {markdown==3.2.2(wheel), setuptools==47.1.1(wheel), tensorboard==2.2.2(wheel), tensorboard-plugin-wit==1.6.0.post3(wheel), scipy==1.4.1(wheel), chardet==3.0.4(wheel), google-auth-oauthlib==0.4.1(wheel), h5py==2.10.0(wheel), opt-einsum==3.2.1(wheel), certifi==2020.4.5.1(wheel), oauthlib==3.1.0(wheel), protobuf==3.12.2(wheel), google-auth==1.16.0(wheel), zipp==3.1.0(wheel), six==1.15.0(wheel), tensorflow-estimator==2.2.0(wheel), importlib-metadata==1.6.0(wheel), termcolor==1.1.0(wheel), gast==0.3.3(wheel), cachetools==4.1.0(wheel), pyasn1==0.4.8(wheel), google-pasta==0.2.0(wheel), requests-oauthlib==1.3.0(wheel), urllib3==1.25.9(wheel), pyasn1-modules==0.2.8(wheel), absl-py==0.9.0(wheel), astunparse==1.6.3(wheel), keras-preprocessing==1.1.2(wheel), idna==2.9(wheel), requests==2.23.0(wheel), rsa==4.0(wheel), wheel==0.34.2(wheel), numpy==1.18.4(wheel), werkzeug==1.0.1(wheel)}
Build missing wheels from sdists (C compiling False): {wrapt==1.12.1(sdist)}
calling pip wheel --no-deps --wheel-dir /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z /var/folders/s5/_787jljn6l14fhg2w4wn4d_w0000gn/T/tmplzw2zx5z/wrapt-1.12.1.tar.gz
compatible after building wheels (C compiling): {markdown==3.2.2(wheel), setuptools==47.1.1(wheel), tensorboard==2.2.2(wheel), tensorboard-plugin-wit==1.6.0.post3(wheel), scipy==1.4.1(wheel), chardet==3.0.4(wheel), google-auth-oauthlib==0.4.1(wheel), h5py==2.10.0(wheel), opt-einsum==3.2.1(wheel), certifi==2020.4.5.1(wheel), oauthlib==3.1.0(wheel), protobuf==3.12.2(wheel), google-auth==1.16.0(wheel), zipp==3.1.0(wheel), six==1.15.0(wheel), tensorflow-estimator==2.2.0(wheel), importlib-metadata==1.6.0(wheel), termcolor==1.1.0(wheel), gast==0.3.3(wheel), wrapt==1.12.1(wheel), cachetools==4.1.0(wheel), pyasn1==0.4.8(wheel), google-pasta==0.2.0(wheel), requests-oauthlib==1.3.0(wheel), urllib3==1.25.9(wheel), pyasn1-modules==0.2.8(wheel), absl-py==0.9.0(wheel), astunparse==1.6.3(wheel), keras-preprocessing==1.1.2(wheel), idna==2.9(wheel), requests==2.23.0(wheel), rsa==4.0(wheel), wheel==0.34.2(wheel), numpy==1.18.4(wheel), werkzeug==1.0.1(wheel)}
Final compatible: {markdown==3.2.2(wheel), setuptools==47.1.1(wheel), tensorboard==2.2.2(wheel), tensorboard-plugin-wit==1.6.0.post3(wheel), scipy==1.4.1(wheel), chardet==3.0.4(wheel), google-auth-oauthlib==0.4.1(wheel), h5py==2.10.0(wheel), opt-einsum==3.2.1(wheel), certifi==2020.4.5.1(wheel), oauthlib==3.1.0(wheel), protobuf==3.12.2(wheel), google-auth==1.16.0(wheel), zipp==3.1.0(wheel), six==1.15.0(wheel), tensorflow-estimator==2.2.0(wheel), importlib-metadata==1.6.0(wheel), termcolor==1.1.0(wheel), gast==0.3.3(wheel), wrapt==1.12.1(wheel), cachetools==4.1.0(wheel), pyasn1==0.4.8(wheel), google-pasta==0.2.0(wheel), requests-oauthlib==1.3.0(wheel), urllib3==1.25.9(wheel), pyasn1-modules==0.2.8(wheel), absl-py==0.9.0(wheel), astunparse==1.6.3(wheel), keras-preprocessing==1.1.2(wheel), idna==2.9(wheel), requests==2.23.0(wheel), rsa==4.0(wheel), wheel==0.34.2(wheel), numpy==1.18.4(wheel), werkzeug==1.0.1(wheel)}
Final incompatible: {tensorflow==2.2.0(wheel), h5py==2.10.0(wheel), protobuf==3.12.2(wheel), wrapt==1.12.1(wheel), numpy==1.18.4(wheel), grpcio==1.29.0(wheel), scipy==1.4.1(wheel)}
Final missing wheels: {tensorflow==2.2.0(wheel), grpcio==1.29.0(wheel)}
PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 42, in execute
requirements_path=self.manifest_path,
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 137, in build_dependencies
self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 201, in build_site_packages
raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {tensorflow==2.2.0(wheel), grpcio==1.29.0(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/aws_lambda_builders/workflow.py", line 269, in run
action.execute()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 45, in execute
raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {tensorflow==2.2.0(wheel), grpcio==1.29.0(wheel)}

Build Failed
Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 50308, 'exitReason': 'WorkflowFailedError', 'exitCode': 1, 'requestId': '199413b8-0cd6-41f2-8750-b2189bccb2fa', 'installationId': '27166426-9c3e-4c34-93bc-1f3346ed5909', 'sessionId': '08d96ae3-9148-4850-ad9c-87e77e262c0d', 'executionEnvironment': 'CLI', 'pyversion': '3.6.8', 'samcliVersion': '0.52.0'}}]}
HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: PythonPipBuilder:ResolveDependencies - {tensorflow==2.2.0(wheel), grpcio==1.29.0(wheel)}

Expected result

I expected a successful build.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS: MacOS 10.14.4
  2. sam --version: SAM CLI, version 0.52.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    stage/waiting-for-releaseFix has been merged to develop and is waiting for a release

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions