Remove helix-scripts/ from Docker containers

[dougbu]

I noticed we might not have a customer driver for installing helix-scripts/ in the Docker containers intended for Helix testing (e.g., mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-amd64 aka https://github.com/dotnet/dotnet-buildtools-prereqs-docker/blob/main/src/alpine/3.15/helix/amd64/Dockerfile) while looking at #1423. References seem limited to our own ValidateHelix use, which only check the helix-scripts/ version. If we don't have a customer driver, removing some headaches from our Docker story by reducing the Python mess we push into the images may be a big win.

Wins would come from

• reducing the scope of what we need to keep up to date elsewhere in the Improve Container Images Security and Sustainability #1313 epic
• avoiding interactions (especially conflicts) w/ whatever our customers do in these containers
• removing some work item failures (again, like pip fails to build wheels for cffi (gcc not found) in ubuntu.1804.armarch.open using image alpine-3.15-helix-arm32v7 #1423) we cause — even when the customer's testing makes no use of Python
• as a result of ^^, increasing customer satisfaction w/ testing in Docker containers w/ Helix

Possible approaches include

• do a deeper investigation into how Docker and Helix interact, to confirm whether we use helix-scripts/ outside our own validations
• test some scenarios w/o the helix-scripts/ installation. this might be a bit time-consuming b/c it'll likely involve a custom helix-scripts/ that puts fewer commands into the Docker execution script (see write_commands_to_file(...) in dockerhelper.py, and new images from dotnet/dotnet-buildtools-prereqs-docker:
  • does test reporting depend on having helix-scripts/ in the container❔
  • does on-agent retry depend on having helix-scripts/ in the container❔
  • does sending attachments from the tests (e.g., dump files and additional logs) back to AzDO depend on having helix-scripts/ in the container❔

For everything in the "scenarios" list above, I can think of ways to avoid the dependency like relying on the mount points for external access to new files the tests generate and restarting the container to run tests again. Just not sure what options we currently use.

If we find the current driver is our own validations, removal might be as simple as

1. skip verify_helix_version(...) in validate.py when $env:HELIX_CORRELATION_PAYLOAD is set
2. remove sudo $HELIX_PYTHONPATH -m pip install --disable-pip-version-check -r $HELIX_SCRIPT_ROOT/runtime_python_requirements.txt from the commands we run in Docker containers
3. remove helix-scripts/ installations from the relevant dotnet/dotnet-buildtools-prereqs-docker Dockerfiles.

Of course, "simple as" is a fair amount of manual steps…

---

If customers scenarios require helix-scripts/ in the Docker images, a fallback would be similar to what I did for #1371, isolating our Python requirements from whatever customers need. That is, let customer requirements drive how they use Python while we use our own stuff for the Helix wrappers (whatever they are) in the containers.

---

A slight benefit of this removal (again, if we can do it) will be separating ourselves from component governance and compliance decisions our customers make. This is not a primary goal b/c our choices are a rising tide that floats all boats. On the other hand, a fix here will allow customers to have direct conversations w/ the PM drivers when compliance issues w/ specific containers become untenable.

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Removed our team's Python infrastructure into a from all *-helix-* Docker images. This should have minimal impact on customers because existing python3 executables and system-installed Python packages in the images are not changing. However, if you use python3 in your pipelines and Helix jobs inside Docker containers, some Python packages may be missing. Please let us know if you hit issues and are unable to use a venv (preferably in the Helix work item folder or build root).