Fix missing setuptools dep

[joeyorlando]

What this PR does

tldr; I think we should install setuptools into our engine Dockerfile + in our CI env because Python 3.12 no longer installs distutils by default. This should unblock us from being able to merge #4656 and #4555.

More details

I would like to be able to merge #4656 and #4555. However, in both of these PRs setuptools is being removed from requirements-dev.txt (here and here). This leads to things breaking (CI job #1 and #2) because of:

File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/polymorphic/__init__.py", line 9, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

(☝️ from the pkg_resources docs.. "The pkg_resources module distributed with setuptools provides an API for Python libraries")

Python 3.12 made a change to no longer pre-install distutils (relevant release notes):

PEP 632: Remove the distutils package. See the migration guide for advice replacing the APIs it provided. The third-party Setuptools package continues to provide distutils, if you still require it in Python 3.12 and beyond.

gh-95299: Do not pre-install setuptools in virtual environments created with venv. This means that distutils, setuptools, pkg_resources, and easy_install will no longer available by default; to access these run pip install setuptools in the activated virtual environment.

Additionally, setuptools is in pip-tools UNSAFE_PACKAGES list (related GitHub issue), hence why I think Dependabot is removing it in #4656 and #4555.

Checklist

• Unit, integration, and e2e (if applicable) tests updated
• Documentation added (or pr:no public docs PR label added if not required)
• Added the relevant release notes label (see labels prefixed w/ release:). These labels dictate how your PR will
  show up in the autogenerated release notes.

[joeyorlando]

also, bump requests because of this message I was seeing when running uv pip compile:

warning: `requests==2.32.0` is yanked (reason: "Yanked due to conflicts with CVE-2024-35195 mitigation").

https://nvd.nist.gov/vuln/detail/CVE-2024-35195

[joeyorlando]

I don't quite understand why it's dropping the [redis] portion here 🤔

[matiasb]

FYI, astral-sh/uv#1595 (I don't have a strong opinion to preserve the extra, though)

[joeyorlando]

thanks for the reference 👍 (seems like including it/excluding it doesn't make a big difference)

[joeyorlando]

similar to this comment

[joeyorlando]

I guess there's not much we can do here? I assume this means that apscheduler and opentelemetry-instrumentation both have a dependency on setuptools?

• https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/opentelemetry-instrumentation/pyproject.toml#L29
• https://github.com/agronholm/apscheduler/blob/028506a816c74ee05951717c0e45d2e6ad32773e/setup.py#L41

[matiasb]

FYI, astral-sh/uv#1595 (I don't have a strong opinion to preserve the extra, though)

[matiasb]

if setuptools is being added through requirements, does it need to be here too? (same above)

[joeyorlando]

I guess technically no, but I think setuptools is only being included in requirements.txt/requirements-dev.txt because of two transitive dependencies. If those dependencies drop their requirement on setuptools we'll need this anyways 🤔

Python 3.12's release notes mention:

python/cpython#95299: Do not pre-install setuptools in virtual environments created with venv. This means that distutils, setuptools, pkg_resources, and easy_install will no longer available by default; to access these run pip install setuptools in the activated virtual environment.