otel-0.40b release breaks 0.39b installation #3382

ggainey · 2023-07-13T20:55:01Z

We (the https://github.com/pulp/pulpcore project) have project-releases that specify otel-release-requirements like so:

opentelemetry-distro[otlp]>=0.38b0,<=0.39b0                                                  
opentelemetry-exporter-otlp-proto-http>=1.17.0,<=1.18.0                                      
opentelemetry-instrumentation-django>=0.38b0,<=0.39b0                                        
opentelemetry-instrumentation-wsgi>=0.38b0,<=0.39b0

With the release of 0.40b today, those became uninstallable due to unrecoverable pip-version-conflicts.

Investigation shows that (for example) "opentelemetry-distro[otlp]<=0.39b0,>=0.38b0" can't be installed into a clean environment:

(dep-tests) (main) ~/github/opentelemetry-python $ pip3 install --no-cache-dir --dry-run "opentelemetry-distro[otlp]<=0.39b0,>=0.38b0"
Collecting opentelemetry-distro[otlp]<=0.39b0,>=0.38b0
  Downloading opentelemetry_distro-0.39b0-py3-none-any.whl (3.3 kB)
Collecting opentelemetry-api~=1.12
  Downloading opentelemetry_api-1.19.0-py3-none-any.whl (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.5/57.5 kB 4.4 MB/s eta 0:00:00
Collecting opentelemetry-instrumentation==0.39b0
  Downloading opentelemetry_instrumentation-0.39b0-py3-none-any.whl (24 kB)
Collecting opentelemetry-sdk~=1.13
  Downloading opentelemetry_sdk-1.19.0-py3-none-any.whl (103 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 103.9/103.9 kB 15.9 MB/s eta 0:00:00
Collecting opentelemetry-exporter-otlp==1.18.0
  Downloading opentelemetry_exporter_otlp-1.18.0-py3-none-any.whl (7.0 kB)
Collecting opentelemetry-exporter-otlp-proto-grpc==1.18.0
  Downloading opentelemetry_exporter_otlp_proto_grpc-1.18.0-py3-none-any.whl (18 kB)
Collecting opentelemetry-exporter-otlp-proto-http==1.18.0
  Downloading opentelemetry_exporter_otlp_proto_http-1.18.0-py3-none-any.whl (17 kB)
Requirement already satisfied: setuptools>=16.0 in /home/ggainey/.virtualenvs/dep-tests/lib/python3.11/site-packages (from opentelemetry-instrumentation==0.39b0->opentelemetry-distro[otlp]<=0.39b0,>=0.38b0) (65.5.1)
Collecting wrapt<2.0.0,>=1.0.0
  Downloading wrapt-1.15.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.9/78.9 kB 268.4 MB/s eta 0:00:00
Collecting backoff<3.0.0,>=1.10.0
  Downloading backoff-2.2.1-py3-none-any.whl (15 kB)
Collecting deprecated>=1.2.6
  Downloading Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)
Collecting googleapis-common-protos~=1.52
  Downloading googleapis_common_protos-1.59.1-py2.py3-none-any.whl (224 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 224.5/224.5 kB 41.1 MB/s eta 0:00:00
Collecting grpcio<2.0.0,>=1.0.0
  Downloading grpcio-1.56.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.2/5.2 MB 112.3 MB/s eta 0:00:00
Collecting opentelemetry-exporter-otlp-proto-common==1.18.0
  Downloading opentelemetry_exporter_otlp_proto_common-1.18.0-py3-none-any.whl (17 kB)
Collecting opentelemetry-proto==1.18.0
  Downloading opentelemetry_proto-1.18.0-py3-none-any.whl (52 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 52.6/52.6 kB 386.6 MB/s eta 0:00:00
Collecting opentelemetry-sdk~=1.13
  Downloading opentelemetry_sdk-1.18.0-py3-none-any.whl (101 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 101.6/101.6 kB 481.2 MB/s eta 0:00:00
Requirement already satisfied: requests~=2.7 in /home/ggainey/.virtualenvs/dep-tests/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-http==1.18.0->opentelemetry-exporter-otlp==1.18.0->opentelemetry-distro[otlp]<=0.39b0,>=0.38b0) (2.31.0)
Requirement already satisfied: protobuf<5.0,>=3.19 in /home/ggainey/.virtualenvs/dep-tests/lib64/python3.11/site-packages (from opentelemetry-proto==1.18.0->opentelemetry-exporter-otlp-proto-grpc==1.18.0->opentelemetry-exporter-otlp==1.18.0->opentelemetry-distro[otlp]<=0.39b0,>=0.38b0) (4.23.4)
Collecting importlib-metadata~=6.0
  Downloading importlib_metadata-6.8.0-py3-none-any.whl (22 kB)
Collecting opentelemetry-api~=1.12
  Downloading opentelemetry_api-1.18.0-py3-none-any.whl (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.3/57.3 kB 435.8 MB/s eta 0:00:00
Collecting opentelemetry-semantic-conventions==0.39b0
  Downloading opentelemetry_semantic_conventions-0.39b0-py3-none-any.whl (26 kB)
Collecting typing-extensions>=3.7.4
  Downloading typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Collecting zipp>=0.5
  Downloading zipp-3.16.1-py3-none-any.whl (7.2 kB)
ERROR: Exception:
Traceback (most recent call last):
  File "/home/ggainey/.virtualenvs/dep-tests/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/home/ggainey/.virtualenvs/dep-tests/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ggainey/.virtualenvs/dep-tests/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 400, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "/home/ggainey/.virtualenvs/dep-tests/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/home/ggainey/.virtualenvs/dep-tests/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ggainey/.virtualenvs/dep-tests/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 373, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ggainey/.virtualenvs/dep-tests/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 227, in _attempt_to_pin_criterion
    raise InconsistentCandidate(candidate, criterion)
pip._vendor.resolvelib.resolvers.InconsistentCandidate: Provided candidate LinkCandidate('https://files.pythonhosted.org/packages/cc/37/db7ba97e676af155f5fcb1a35466f446eadc9104e25b83366e8088c9c926/importlib_metadata-6.8.0-py3-none-any.whl (from https://pypi.org/simple/importlib-metadata/) (requires-python:>=3.8)') does not satisfy SpecifierRequirement('importlib-metadata~=6.0'), SpecifierRequirement('importlib-metadata~=6.0.0')

It looks to me like the collision happens because the dependency Collecting opentelemetry-api~=1.12 finds 0.19, which wants importlib-metadata==6.8.0, while the rest of 0.39b wants importlib-metadata~=6.0.0 which disallows 6.8.0.

(I absolutely could be wrong about exactly where in the otel deps "something" is demanding 6.8.0 - but that's the general shape of the problem.)

Describe your environment
I am on Fedora38, python/3.11, pip-3.22, FWIW. This has been reproduced in several different environments now (not unexpected, it's not environmental, it's an inherently-unrecoverable version-conflict)

Steps to reproduce

Create a clean python virtual environment: mkvirtualenv otel-version-problem
Issue the following: pip3 install --no-cache-dir --dry-run "opentelemetry-distro[otlp]==0.39b0"
Watch it fail to install
Be Sad.

What is the expected behavior?
0.39b should remain installable, even when newer versions are available.

The text was updated successfully, but these errors were encountered:

tammy-baylis-swi · 2023-07-20T16:36:25Z

I got a similar pip InconsistentCandidate error when I set up my own test container that installs opentelemetry-exporter-otlp-proto-grpc. I was able to stop the error by doing pip install importlib_metadata>=6.0.0 before exporter install. The error returns if I do pip install importlib_metadata== 5.2.0. I think the error comes from this less-strict Otel API requirement for importlib-metadata introduced in 92bfd08 to work regardless of Python version (this comment). cc @ocelotl

ocelotl · 2023-08-10T12:56:02Z

Investigating...

ocelotl · 2023-08-10T14:46:22Z

First find: a solution could be to add opentelemetry-api <= 1.18 in your requirements file.

ocelotl · 2023-08-10T15:09:59Z

Second find: Interesting, installing like this works:

pip install opentelemetry-api

pip freeze
Deprecated==1.2.14
importlib-metadata==6.8.0
opentelemetry-api==1.19.0
wrapt==1.15.0
zipp==3.16.2

pip3  install --no-cache-dir "opentelemetry-distro[otlp]==0.39b0"

pip freeze
backoff==2.2.1
certifi==2023.7.22
charset-normalizer==3.2.0
Deprecated==1.2.14
googleapis-common-protos==1.60.0
grpcio==1.56.2
idna==3.4
importlib-metadata==6.0.1
opentelemetry-api==1.18.0
opentelemetry-distro==0.39b0
opentelemetry-exporter-otlp==1.18.0
opentelemetry-exporter-otlp-proto-common==1.18.0
opentelemetry-exporter-otlp-proto-grpc==1.18.0
opentelemetry-exporter-otlp-proto-http==1.18.0
opentelemetry-instrumentation==0.39b0
opentelemetry-proto==1.18.0
opentelemetry-sdk==1.18.0
opentelemetry-semantic-conventions==0.39b0
protobuf==4.24.0
requests==2.31.0
typing_extensions==4.7.1
urllib3==2.0.4
wrapt==1.15.0
zipp==3.16.2

Notice that after running pip3 install --no-cache-dir "opentelemetry-distro[otlp]==0.39b0" this happened:

opentelemetry-api==1.19 was uninstalled and opentelemetry-api==1.18 was installed instead.
importlib-metadata==6.8.0 was uninstalled and importlib-metadata==6.0.1 was installed instead.

It seems like pip dependency solver can handle the case when opentelemetry-api is installed first but it can't handle the case where it isn't.

ocelotl · 2023-08-10T15:59:11Z

I think this is being caused by this pip issue. I downgraded pip to 20.1 and was able to run pip3 install --no-cache-dir --dry-run "opentelemetry-distro[otlp]==0.39b0" which produced this virtual environment:

backoff==2.2.1
certifi==2023.7.22
charset-normalizer==3.2.0
Deprecated==1.2.14
googleapis-common-protos==1.60.0
grpcio==1.56.2
idna==3.4
importlib-metadata==6.8.0
opentelemetry-api==1.19.0
opentelemetry-distro==0.39b0
opentelemetry-exporter-otlp==1.18.0
opentelemetry-exporter-otlp-proto-common==1.18.0
opentelemetry-exporter-otlp-proto-grpc==1.18.0
opentelemetry-exporter-otlp-proto-http==1.18.0
opentelemetry-instrumentation==0.39b0
opentelemetry-proto==1.18.0
opentelemetry-sdk==1.19.0
opentelemetry-semantic-conventions==0.40b0
protobuf==4.24.0
requests==2.31.0
typing-extensions==4.7.1
urllib3==2.0.4
wrapt==1.15.0
zipp==3.16.2

ocelotl · 2023-08-10T16:44:55Z

Will try with specifying the requirement for importlib-metadata with >= instead of ~=.

ocelotl · 2023-08-14T12:02:09Z

Will try with specifying the requirement for importlib-metadata with >= instead of ~=.

I think this should work, I ran an experiment.

First I checked out v1.19.0 of opentelemetry-api and manually created a package:

tigre@hilleman:~/github/ocelotl/opentelemetry-python/opentelemetry-api$ python3 -m build .

That produced dist/opentelemetry_api-1.19.0.tar.gz.

I created a private pypi server in my laptop, and after moving opentelemetry_api-1.19.0.tar.gz to its packages directory I ran it like this:

(venv) tigre@hilleman:~/sandbox/venv$ pypi-server run -p 8081 packages/

In another virtual environment, I tried the failing command reported in this issue:

(testvenv) tigre@hilleman:~/sandbox$ pip3 install --extra-index-url http://127.0.0.1:8081/simple/ --trusted-host 127.0.0.1 --no-cache-dir "opentelemetry-distro[otlp]==0.39b0"

Got this error:

ERROR: Exception:
Traceback (most recent call last):
  File "/home/tigre/sandbox/testvenv/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/home/tigre/sandbox/testvenv/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tigre/sandbox/testvenv/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 377, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "/home/tigre/sandbox/testvenv/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/home/tigre/sandbox/testvenv/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tigre/sandbox/testvenv/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 427, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tigre/sandbox/testvenv/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 254, in _attempt_to_pin_criterion
    raise InconsistentCandidate(candidate, criterion)
pip._vendor.resolvelib.resolvers.InconsistentCandidate: Provided candidate LinkCandidate('https://files.pythonhosted.org/packages/cc/37/db7ba97e676af155f5fcb1a35466f446eadc9104e25b83366e8088c9c926/importlib_metadata-6.8.0-py3-none-any.whl (from https://pypi.org/simple/importlib-metadata/) (requires-python:>=3.8)') does not satisfy SpecifierRequirement('importlib-metadata~=6.0.0

I "created" a new 1.20.0 opentelemetry-api package with these changes:

tigre@hilleman:~/github/ocelotl/opentelemetry-python/opentelemetry-api$ git diff
diff --git a/opentelemetry-api/pyproject.toml b/opentelemetry-api/pyproject.toml
index f659d7775..d03e870df 100644
--- a/opentelemetry-api/pyproject.toml
+++ b/opentelemetry-api/pyproject.toml
@@ -28,7 +28,7 @@ dependencies = [
     "Deprecated >= 1.2.6",
     # FIXME This should be able to be removed after 3.12 is released if there is a reliable API
     # in importlib.metadata.
-    "importlib-metadata ~= 6.0",
+    "importlib-metadata >= 6.0",
 ]
 dynamic = [
     "version",
diff --git a/opentelemetry-api/src/opentelemetry/version.py b/opentelemetry-api/src/opentelemetry/version.py
index 270451efb..9592f5e83 100644
--- a/opentelemetry-api/src/opentelemetry/version.py
+++ b/opentelemetry-api/src/opentelemetry/version.py
@@ -12,4 +12,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-__version__ = "1.20.0.dev"
+__version__ = "1.20.0"

Swapped the 1.20.0 for the 1.19.0 package in the packages directory, restarted the private pypi server and tried installing again:

...
Collecting opentelemetry-api~=1.12 (from opentelemetry-distro[otlp]==0.39b0)                                                                                                                  
  Downloading http://127.0.0.1:8081/packages/opentelemetry_api-1.20.0.tar.gz (56 kB)
...

No error this time, pip was able to do the downgrade of opentelemetry-api to 1.18.0 and importlib-metadata to 6.0.1:

(testvenv) tigre@hilleman:~/sandbox$ pip freeze
backoff==2.2.1
certifi==2023.7.22
charset-normalizer==3.2.0
Deprecated==1.2.14
googleapis-common-protos==1.60.0
grpcio==1.57.0
idna==3.4
importlib-metadata==6.0.1
opentelemetry-api==1.18.0
opentelemetry-distro==0.39b0
opentelemetry-exporter-otlp==1.18.0
opentelemetry-exporter-otlp-proto-common==1.18.0
opentelemetry-exporter-otlp-proto-grpc==1.18.0
opentelemetry-exporter-otlp-proto-http==1.18.0
opentelemetry-instrumentation==0.39b0
opentelemetry-proto==1.18.0
opentelemetry-sdk==1.18.0
opentelemetry-semantic-conventions==0.39b0
protobuf==4.24.0
requests==2.31.0
typing_extensions==4.7.1
urllib3==2.0.4
wrapt==1.15.0
zipp==3.16.2

This is functionally equivalent to the previous state but it works around a bug in pypi. Fixes open-telemetry#3382

ocelotl · 2023-08-17T19:40:25Z

@aabmass pointed out that this fix is missing an upper boundary. Added this boundary in #3406. Used the same method to test with the upper boundary, same results. 👍

ggainey added the bug Something isn't working label Jul 13, 2023

decko mentioned this issue Jul 13, 2023

New release of opentelemetry-python break the dependency tree. pulp/pulpcore#4042

Closed

ggainey mentioned this issue Jul 14, 2023

Pin importlib-metadata to avoid an unsolvable dependency resolution by pip pulp/pulpcore#4036

Merged

jonapich mentioned this issue Jul 17, 2023

(🎁) Abillity to constrain transitive dependencies python-poetry/poetry#4991

Closed

2 tasks

ggainey mentioned this issue Jul 25, 2023

[noissue]: Update importlib-metadata requirement from <=6.0.1,>=6.0.1 to >=6.0.1,<=6.7.0 pulp/pulpcore#4092

Closed

ocelotl self-assigned this Aug 7, 2023

ocelotl added a commit to ocelotl/opentelemetry-python that referenced this issue Aug 14, 2023

Change importlib-metadata requirement specification

7f22e0c

This is functionally equivalent to the previous state but it works around a bug in pypi. Fixes open-telemetry#3382

ocelotl mentioned this issue Aug 14, 2023

Change importlib-metadata requirement specification #3403

Merged

ocelotl closed this as completed in #3403 Aug 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

otel-0.40b release breaks 0.39b installation #3382

otel-0.40b release breaks 0.39b installation #3382

ggainey commented Jul 13, 2023

tammy-baylis-swi commented Jul 20, 2023

ocelotl commented Aug 10, 2023

ocelotl commented Aug 10, 2023

ocelotl commented Aug 10, 2023

ocelotl commented Aug 10, 2023

ocelotl commented Aug 10, 2023

ocelotl commented Aug 14, 2023

ocelotl commented Aug 17, 2023 •

edited

Loading

otel-0.40b release breaks 0.39b installation #3382

otel-0.40b release breaks 0.39b installation #3382

Comments

ggainey commented Jul 13, 2023

tammy-baylis-swi commented Jul 20, 2023

ocelotl commented Aug 10, 2023

ocelotl commented Aug 10, 2023

ocelotl commented Aug 10, 2023

ocelotl commented Aug 10, 2023

ocelotl commented Aug 10, 2023

ocelotl commented Aug 14, 2023

ocelotl commented Aug 17, 2023 • edited Loading

ocelotl commented Aug 17, 2023 •

edited

Loading