Pythonpipbuilder: resolve dependencies - list index out of range

[pabloiarriola]

Description:
I am trying to use sam build with the following requirements but its throwing the error: Pythonpipbuilder: resolve dependencies - list index out of range

• pyotp
• ulid-py
• aws_encryption_sdk
• boto3
• requests
• attrs
• cryptography

Steps to reproduce the issue:

1. Create a virtual env.
2. Activate virtual env in a terminal
3. pip install -r requirements.txt
4. sam build

Observed result:
Build Failed
Error: PythonPipBuilder:ResolveDependencies - list index out of range

Expected result:
Build Succeeded

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
Amazon Linux 2 Workspace
Python3.8

[CaptainDriftwood]

I have a similar issue with different requirements. How are we supposed to use Sam with this underlying issue?

[pabloiarriola]

@Jasonca2 thats pretty much the problem, that we cannot use SAM. Have you found any workarounds?

[CaptainDriftwood]

@Jasonca2 thats pretty much the problem, that we cannot use SAM. Have you found any workarounds?

I have not. The issue is Sam, installed on Linux via there recommended installation method, brings in the aws-lambda-builders package as a requirement, not as part of the source code. The lambda-builders package would need to be patched and then aws-Sam updated to use the new version. I can't think of a workaround and would really like it if the Sam team could fix this.

[sriram-mv]

@pabloiarriola @Jasonca2 what version of pip are you on? does sam build --use-container work?

[CaptainDriftwood]

@pabloiarriola @Jasonca2 what version of pip are you on? does sam build --use-container work?

I'm using python3.7.10 and pip v20.1

I'm using the latest updated version of Sam as well.

[jzavgok]

Any news on this? We are having the same issue :(

[pabloiarriola]

Any news on this? We are having the same issue :(

I got it working using sam build --use-container its pretty much the only way I found that works

[pabloiarriola]

@pabloiarriola @Jasonca2 what version of pip are you on? does sam build --use-container work?

I'm using python3.7.10 and pip v20.1

I'm using the latest updated version of Sam as well.

the only difference with me is that im running python 3.8, the only workaround that worked was to use sam build --use-container

[shaunenslin]

Same issue for me as well and I am running on cloud9.
Mine did not work, even with sam build --use-container

[timjohnson6]

I have the same issue. My requirements.txt only has

• requests
• google-api-python-client

everything worked until I added the google lib. no code changes. started with the HelloWorld template
I'm running the latest SAM v1.23.0, pip v21.1, python 3.8.5

adding the --use-container --debug I no longer get the error: Pythonpipbuilder: resolve dependencies - list index out of range
I get whats below

Running workflow 'PythonPipBuilder'
Running PythonPipBuilder:ResolveDependencies
calling pip download -r /tmp/samcli/source/requirements.txt --dest /tmp/samcli/scratch
Full dependency closure: {pyasn1-modules==0.2.8(wheel), certifi==2020.12.5(wheel), setuptools==56.2.0(wheel), pyparsing==2.4.7(wheel), urllib3==1.26.4(wheel), rsa==4.7.2(wheel), google-auth-httplib2==0.1.0(wheel), cachetools==4.2.2(wheel), pyasn1==0.4.8(wheel), google-api-core==1.28.0(wheel), google-auth==1.30.0(wheel), uritemplate==3.0.1(wheel), requests==2.25.1(wheel), idna==2.10(wheel), googleapis-common-protos==1.53.0(wheel), google-api-python-client==2.5.0(wheel), httplib2==0.19.1(wheel), chardet==4.0.0(wheel), protobuf==3.17.0(wheel), pytz==2021.1(wheel), six==1.16.0(wheel), packaging==20.9(wheel)}
initial compatible: {pyasn1-modules==0.2.8(wheel), certifi==2020.12.5(wheel), setuptools==56.2.0(wheel), pyparsing==2.4.7(wheel), urllib3==1.26.4(wheel), rsa==4.7.2(wheel), google-auth-httplib2==0.1.0(wheel), cachetools==4.2.2(wheel), pyasn1==0.4.8(wheel), google-api-core==1.28.0(wheel), uritemplate==3.0.1(wheel), six==1.16.0(wheel), requests==2.25.1(wheel), idna==2.10(wheel), googleapis-common-protos==1.53.0(wheel), google-api-python-client==2.5.0(wheel), httplib2==0.19.1(wheel), chardet==4.0.0(wheel), pytz==2021.1(wheel), google-auth==1.30.0(wheel), packaging==20.9(wheel)}
initial incompatible: {protobuf==3.17.0(wheel)}
Downloading missing wheels: {protobuf==3.17.0(wheel)}
calling pip download --only-binary=:all: --no-deps --platform manylinux2014_x86_64 --implementation cp --abi cp38 --dest /tmp/samcli/scratch protobuf==3.17.0
compatible wheels after second download pass: {pyasn1-modules==0.2.8(wheel), certifi==2020.12.5(wheel), setuptools==56.2.0(wheel), pyparsing==2.4.7(wheel), urllib3==1.26.4(wheel), rsa==4.7.2(wheel), google-auth-httplib2==0.1.0(wheel), cachetools==4.2.2(wheel), pyasn1==0.4.8(wheel), google-api-core==1.28.0(wheel), google-auth==1.30.0(wheel), uritemplate==3.0.1(wheel), requests==2.25.1(wheel), idna==2.10(wheel), googleapis-common-protos==1.53.0(wheel), google-api-python-client==2.5.0(wheel), httplib2==0.19.1(wheel), chardet==4.0.0(wheel), pytz==2021.1(wheel), six==1.16.0(wheel), packaging==20.9(wheel)}
Build missing wheels from sdists (C compiling True): set()
compatible after building wheels (no C compiling): {pyasn1-modules==0.2.8(wheel), certifi==2020.12.5(wheel), setuptools==56.2.0(wheel), pyparsing==2.4.7(wheel), urllib3==1.26.4(wheel), rsa==4.7.2(wheel), google-auth-httplib2==0.1.0(wheel), cachetools==4.2.2(wheel), pyasn1==0.4.8(wheel), google-api-core==1.28.0(wheel), google-auth==1.30.0(wheel), uritemplate==3.0.1(wheel), requests==2.25.1(wheel), idna==2.10(wheel), googleapis-common-protos==1.53.0(wheel), google-api-python-client==2.5.0(wheel), httplib2==0.19.1(wheel), chardet==4.0.0(wheel), pytz==2021.1(wheel), six==1.16.0(wheel), packaging==20.9(wheel)}
Build missing wheels from sdists (C compiling False): set()
compatible after building wheels (C compiling): {pyasn1-modules==0.2.8(wheel), certifi==2020.12.5(wheel), setuptools==56.2.0(wheel), pyparsing==2.4.7(wheel), urllib3==1.26.4(wheel), rsa==4.7.2(wheel), google-auth-httplib2==0.1.0(wheel), cachetools==4.2.2(wheel), pyasn1==0.4.8(wheel), google-api-core==1.28.0(wheel), google-auth==1.30.0(wheel), uritemplate==3.0.1(wheel), requests==2.25.1(wheel), idna==2.10(wheel), googleapis-common-protos==1.53.0(wheel), google-api-python-client==2.5.0(wheel), httplib2==0.19.1(wheel), chardet==4.0.0(wheel), pytz==2021.1(wheel), six==1.16.0(wheel), packaging==20.9(wheel)}
Final compatible: {pyasn1-modules==0.2.8(wheel), certifi==2020.12.5(wheel), setuptools==56.2.0(wheel), pyparsing==2.4.7(wheel), requests==2.25.1(wheel), idna==2.10(wheel), googleapis-common-protos==1.53.0(wheel), urllib3==1.26.4(wheel), rsa==4.7.2(wheel), google-auth-httplib2==0.1.0(wheel), cachetools==4.2.2(wheel), google-api-python-client==2.5.0(wheel), httplib2==0.19.1(wheel), chardet==4.0.0(wheel), pyasn1==0.4.8(wheel), google-api-core==1.28.0(wheel), uritemplate==3.0.1(wheel), six==1.16.0(wheel), pytz==2021.1(wheel), google-auth==1.30.0(wheel), packaging==20.9(wheel)}
Final incompatible: {protobuf==3.17.0(wheel)}
Final missing wheels: {protobuf==3.17.0(wheel)}
PythonPipBuilder:ResolveDependencies failed
Traceback (most recent call last):
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 39, in execute
    package_builder.build_dependencies(
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 137, in build_dependencies
    self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 207, in build_site_packages
    raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {protobuf==3.17.0(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 269, in run
    action.execute()
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 45, in execute
    raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {protobuf==3.17.0(wheel)}
Builder workflow failed
Traceback (most recent call last):
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 39, in execute
    package_builder.build_dependencies(
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 137, in build_dependencies
    self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path)
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 207, in build_site_packages
    raise MissingDependencyError(packages_without_wheels)
aws_lambda_builders.workflows.python_pip.packager.MissingDependencyError: {protobuf==3.17.0(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 269, in run
    action.execute()
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 45, in execute
    raise ActionFailedError(str(ex))
aws_lambda_builders.actions.ActionFailedError: {protobuf==3.17.0(wheel)}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/__main__.py", line 117, in main
    builder.build(
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/builder.py", line 125, in build
    return workflow.run()
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 76, in wrapper
    func(self, *args, **kwargs)
  File "/usr/local/opt/sam-cli/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 276, in run
    raise WorkflowFailedError(workflow_name=self.NAME, action_name=action.NAME, reason=str(ex))
aws_lambda_builders.exceptions.WorkflowFailedError: PythonPipBuilder:ResolveDependencies - {protobuf==3.17.0(wheel)}
2021-05-21 22:33:41,412 | Build inside container returned response {"jsonrpc": "2.0", "id": 1, "error": {"code": 400, "message": "PythonPipBuilder:ResolveDependencies - {protobuf==3.17.0(wheel)}"}}

Build Failed
2021-05-21 22:33:41,597 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': '70cf581c-be97-4fd5-8984-f5f3e4b3312c', 'installationId': 'bbb02c7a-d67b-484c-8cd7-992b210db59e', 'sessionId': 'a630fd26-f155-4223-97cc-ca2d94b428b1', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.7.10', 'samcliVersion': '1.23.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 4780, 'exitReason': 'BuildInsideContainerError', 'exitCode': 1}}]}
2021-05-21 22:33:41,955 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: PythonPipBuilder:ResolveDependencies - {protobuf==3.17.0(wheel)}

[justinmk3]

Final compatible: {...}
Final incompatible: {protobuf==3.17.0(wheel)}
Final missing wheels: {protobuf==3.17.0(wheel)}

For "native" (non-pure-python) modules, --use-container must get a wheel, or else build the package natively in-container. Both can fail for various reasons which I don't fully understand, but are decided by pip based on the machine architecture, package definition, and package resources.

Similar issue for protobuf is aws/aws-sam-cli#2861 . Workaround: pin to an older version. Thus you could try adding this to your requirements.txt:

protobuf==3.16.0

[ItayGoren]

im having the same problem. it started out of nowhere 2 days ago.
i now tried to create an empty template with only a function which points to src/ which only has requirements.txt which contains only dataclasses-json==0.5.2.
i get the same error.
we can't build our code in container because we need to use private code artifacts.

[ash0x0]

Just happened out of the blue here too. It doesn't really relate to requirements because this was working just an hour ago and I haven't made any changes to requirements. I even reverted to the last build and it doesn't work anymore.

[robot-apocalypse]

I ran into the same issue; in my case it seems to be related to PyPDF2. If I remove that from the requirements.txt file, the build works. I'm guessing it's because it's a fairly old library and has not been compiled for Python3.8?

--use-container isn't a good workaround for me; I get "Requirements file not found: /tmp/samcli/source/requirements.txt." I suspect it is because my dev environment is itself a container, as the requirements files are where they're supposed to be.