ansible-builder failed to build EE for cisco.aci because of Double requirement given

[hiyokotaisa]

Issue

ansible-builder returned the following error when trying to build EE for AAP 2.1 with cisco.aci collection

+ pip3 install --cache-dir=/output/wheels -r /tmp/src/requirements.txt
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
ERROR: Double requirement given: python_dateutil (from -r /tmp/src/requirements.txt (line 19)) (already in python-dateutil>=2.7.0 (from -r /tmp/src/requirements.txt (line 5)), name='python-dateutil')
[3/3] STEP 1/5: FROM registry.redhat.io/ansible-automation-platform-21/ee-supported-rhel8:latest
Error: error building at STEP "RUN assemble": error while running runtime: exit status 1

Step to reproduce

1. Using following version of podman and ansible-builder

# podman --version
podman version 3.4.2

# ansible-builder --version
1.0.1

2. Prepare the following files(execution-environment.yml/requirements.yml) before the build:

# cat execution-environment.yml
---
version: 1

build_arg_defaults:
  EE_BASE_IMAGE: 'registry.redhat.io/ansible-automation-platform-21/ee-supported-rhel8:latest'
dependencies:
  galaxy: requirements.yml

# cat requirements.yml
---
collections:
  - name: cisco.aci
    version: 2.1.0

3. build the EE with ansible-builder command

  # ansible-builder build -v 3

Additional Info

• This issue only occurs when specifying cisco.aci version 2.1.0 collection for requirements.yml.
• There is no issue when specifying version 2.0.0
• 2.1.0 has following requirements.txt but 2.0.0 does not

lxml
cryptography
pyOpenSSL
python_dateutil
xmljson

• When using ee-minimal-rhel8, the build has been completed without issue
• We found there is old pip version in EE_BUILDER_IMAGE provided from registry.redhat.io. There is no issue when specifying EE_BUILDER_IMAGE: 'quay.io/ansible/ansible-builder:latest'. So I think we need to upgrade pip in builder image.

# podman run -it registry.redhat.io/ansible-automation-platform-21/ansible-builder-rhel8:latest pip3 --version
pip 20.2.4 from /usr/lib/python3.8/site-packages/pip (python 3.8)

# podman run -it quay.io/ansible/ansible-builder:latest pip3 --version
pip 22.0.3 from /usr/local/lib/python3.8/site-packages/pip (python 3.8)

[Shrews]

Yes, upgrading pip is one way to go. The maintainers of this repo do not have control over the AAP container images, so we can't address that. However, we've proposed #359 which should help address the situation. We can backport this to 2.1, 2.0, and maybe 1.4.x, but you are using a very old version of ansible-builder which will not see the fix, if we should merge this. So you will need to upgrade.

[Shrews]

Oops, ignore my comment about versions. I was confusing runner and builder versions. Sorry about that.