-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conflicting Dependencies #3579
Comments
@lukesneeringer ISTM the issue is that
Here is the latest state of things:
|
I cannot reproduce. For $ /opt/Python-3.6.0/bin/virtualenv /tmp/gcp-3579-0.25
Using base prefix '/opt/Python-3.6.0'
New python executable in /tmp/gcp-3579-0.25/bin/python3.6
Also creating executable in /tmp/gcp-3579-0.25/bin/python
Installing setuptools, pip, wheel...done.
$ /tmp/gcp-3579-0.25/bin/pip install google-cloud==0.25.0
Collecting google-cloud==0.25.0
...
Successfully installed cachetools-2.0.0 certifi-2017.4.17 chardet-3.0.4 dill-0.2.6 future-0.16.0 gapic-google-cloud-datastore-v1-0.15.3 gapic-google-cloud-error-reporting-v1beta1-0.15.3 gapic-google-cloud-logging-v2-0.91.3 gapic-google-cloud-pubsub-v1-0.15.4 gapic-google-cloud-spanner-admin-database-v1-0.15.3 gapic-google-cloud-spanner-admin-instance-v1-0.15.3 gapic-google-cloud-spanner-v1-0.15.3 gapic-google-cloud-speech-v1-0.15.3 google-auth-1.0.1 google-auth-httplib2-0.0.2 google-cloud-0.26.0 google-cloud-bigquery-0.25.0 google-cloud-bigtable-0.25.0 google-cloud-core-0.25.0 google-cloud-datastore-1.1.0 google-cloud-dns-0.25.0 google-cloud-error-reporting-0.25.0 google-cloud-language-0.25.0 google-cloud-logging-1.1.0 google-cloud-monitoring-0.25.0 google-cloud-pubsub-0.26.0 google-cloud-resource-manager-0.25.0 google-cloud-runtimeconfig-0.25.0 google-cloud-spanner-0.25.0 google-cloud-speech-0.26.0 google-cloud-storage-1.2.0 google-cloud-translate-0.25.0 google-cloud-videointelligence-0.25.0 google-cloud-vision-0.25.0 google-gax-0.15.13 google-resumable-media-0.1.1 googleapis-common-protos-1.5.2 grpc-google-iam-v1-0.11.1 grpcio-1.4.0 httplib2-0.10.3 idna-2.5 oauth2client-3.0.0 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-error-reporting-v1beta1-0.15.3 proto-google-cloud-logging-v2-0.91.3 proto-google-cloud-pubsub-v1-0.15.4 proto-google-cloud-spanner-admin-database-v1-0.15.3 proto-google-cloud-spanner-admin-instance-v1-0.15.3 proto-google-cloud-spanner-v1-0.15.3 proto-google-cloud-speech-v1-0.15.3 protobuf-3.3.0 pyasn1-0.2.3 pyasn1-modules-0.0.9 requests-2.18.1 rsa-3.4.2 six-1.10.0 urllib3-1.21.1 For $ /opt/Python-3.6.0/bin/virtualenv /tmp/gcp-3579-0.26
Using base prefix '/opt/Python-3.6.0'
New python executable in /tmp/gcp-3579-0.26/bin/python3.6
Also creating executable in /tmp/gcp-3579-0.26/bin/python
Installing setuptools, pip, wheel...done.
$ /tmp/gcp-3579-0.26/bin/pip install google-cloud==0.26.0
Collecting google-cloud==0.26.0
...
Successfully installed cachetools-2.0.0 certifi-2017.4.17 chardet-3.0.4 dill-0.2.6 future-0.16.0 gapic-google-cloud-datastore-v1-0.15.3 gapic-google-cloud-error-reporting-v1beta1-0.15.3 gapic-google-cloud-logging-v2-0.91.3 gapic-google-cloud-pubsub-v1-0.15.4 gapic-google-cloud-spanner-admin-database-v1-0.15.3 gapic-google-cloud-spanner-admin-instance-v1-0.15.3 gapic-google-cloud-spanner-v1-0.15.3 gapic-google-cloud-speech-v1-0.15.3 google-auth-1.0.1 google-auth-httplib2-0.0.2 google-cloud-0.26.0 google-cloud-bigquery-0.25.0 google-cloud-bigtable-0.25.0 google-cloud-core-0.25.0 google-cloud-datastore-1.1.0 google-cloud-dns-0.25.0 google-cloud-error-reporting-0.25.0 google-cloud-language-0.25.0 google-cloud-logging-1.1.0 google-cloud-monitoring-0.25.0 google-cloud-pubsub-0.26.0 google-cloud-resource-manager-0.25.0 google-cloud-runtimeconfig-0.25.0 google-cloud-spanner-0.25.0 google-cloud-speech-0.26.0 google-cloud-storage-1.2.0 google-cloud-translate-0.25.0 google-cloud-videointelligence-0.25.0 google-cloud-vision-0.25.0 google-gax-0.15.13 google-resumable-media-0.1.1 googleapis-common-protos-1.5.2 grpc-google-iam-v1-0.11.1 grpcio-1.4.0 httplib2-0.10.3 idna-2.5 oauth2client-3.0.0 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-error-reporting-v1beta1-0.15.3 proto-google-cloud-logging-v2-0.91.3 proto-google-cloud-pubsub-v1-0.15.4 proto-google-cloud-spanner-admin-database-v1-0.15.3 proto-google-cloud-spanner-admin-instance-v1-0.15.3 proto-google-cloud-spanner-v1-0.15.3 proto-google-cloud-speech-v1-0.15.3 protobuf-3.3.0 pyasn1-0.2.3 pyasn1-modules-0.0.9 requests-2.18.1 rsa-3.4.2 six-1.10.0 urllib3-1.21.1 |
Please double-check the repro steps before deciding this is not reproducible. Pip install succeeds, but running a pip-installed executable that requires google-cloud 0.25.0 fails. The repro steps above have a nice easy way to reproduce that error. |
Ah, my bad! Confirmed:
Any reason that A picture of the installed packages may give a hint:
|
I am still sleuthing, but I'm pretty sure the issue is that our "stable" packages have looser bounds:
so we get newer version of those packages that depend on |
OK, after clicking into all of the
|
Could this be related to pip doesn't resolve dependencies correctly? |
@duggelz In this case, it's just our fault for being too "loose" with our restrictions on the "stable" packages. UPDATE: It would be nice if |
OK I'm going to send some fixes for this right now. I just pushed two branches:
corresponding to the existing "broken" tags:
I will send PRs (#3584, #3586, #3587, #3588) against them to create |
We've got similar issues in
I confirmed it "stops" there, the previous release was "pre-stable"
|
OK the following releases have come as a result:
I am closing this as "mostly fixed". @nhammond In the current form |
Thanks @dhermes, I understand not wanting to overwrite the release even if it leaves it in a somewhat broken state. Given that this just happened a few months ago (#3331), are there any safeguards against repeating the mistake? It will become quite frustrating to users if this breaks in the same mode with every new release of google-cloud. |
I actually wondered that to myself as well. It'd involve starting with all matches for packages in a range, e.g.
yields
then we just install all combinations and then "check" if OK (e.g. via
@jonparrott is actually on the PyPA (packaging authority) so he would be great here, but he is OOO for the next week or so. |
The combinatorial explosion is real. I recommended (internally) that we just test against 1) the latest of combination of "everything", 2) the earliest version of "everything" that still satisfies the dependency specs, and 3) a handful (or less) of particular combinations that are important. I think this would catch most of the problems with a minimum amount (2x or 3x) of the testing resources. And some kind of abbreviated test where we don't actually run the tests, we just import everything, might mitigate that as well. |
I agree with @duggelz's suggestion of installing all of the latest stuff and running a sanity check import and then another test to install the latest umbrella with the minimum versions of its dependencies. We could do this as a pre-release check. I would also vote in favor of killing the umbrella package. It's going to become more and more useless as we go on. I'm also in favor of figuring out a plan to v1.0.0 google-cloud-core (& merge it with gax) so that we can stop breaking packages because of that. |
The moment we had more sub-packages (like |
I was always suspicious of the "give me everything" usecase for the umbrella package, so +1 for losing it. |
This is reposted from an internal ticket. (Googlers: b/63062329)
Relevant PRs:
(Added by @dhermes for posterity)
Issue summary:
Here is the traceback:
Workaround: Customer has reverted to 0.23.0 to build.
Very similar to #3331.
The text was updated successfully, but these errors were encountered: