Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin all versions in requirements.txt to prevent against breakages #2648

Closed
wants to merge 19 commits into from
Closed

Pin all versions in requirements.txt to prevent against breakages #2648

wants to merge 19 commits into from

Conversation

KevinVillela
Copy link
Contributor

Cirq appears to be incompatible with the latest SymPy release of 1.5: https://github.com/sympy/sympy/releases/tag/sympy-1.5. While we're here, might as well pin em all.
Fixes #2646
@vtomole This is a PR to replace #2647 because I messed up that PRs commit history something fierce.

@googlebot googlebot added the cla: yes Makes googlebot stop complaining. label Dec 16, 2019
Copy link
Collaborator

@vtomole vtomole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mpharrigan
Copy link
Collaborator

If we do this, we should remove the part of setup.py that reads in the requirements.txt file and uses it directly, xref quantumlib/OpenFermion-Cirq#352 xref https://packaging.python.org/discussions/install-requires-vs-requirements/

We should also open an issue to fix Cirq to work with the latest sympy

@KevinVillela
Copy link
Contributor Author

Done. Also opened #2650.

@vtomole
Copy link
Collaborator

vtomole commented Dec 16, 2019

@mpharrigan Why don't we want to use it directly?

@mpharrigan
Copy link
Collaborator

Please see the packaging.python.org link above, but in summary:

  1. requirements.txt is for setting up a reproducible environment
  2. install_requires is for listing the bare minimum reqs to get something working
  3. (me editorializing:) it's crucial especially for a developer-oriented library like cirq (i.e. not an end-user application) to support some degree of freedom in the developer environment. I don't appreciate it when libraries keep over-writing / re-installing my version of numpy. You can get into even trickier situations (see linked openfermion-cirq issue) where too many pins can make it impossible to create a consistent environment when technically it would work fine

@vtomole vtomole requested a review from Strilanc December 16, 2019 22:18
@vtomole vtomole mentioned this pull request Dec 16, 2019
Copy link
Contributor

@Strilanc Strilanc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please only pin the major version (the first non-zero number). Otherwise we will create incompatibilities with other libraries. When the minor version changes and breaks us... we just have to take that and fix it, I think. If a particular library keeps doing it, we can pin it down for real.

@KevinVillela
Copy link
Contributor Author

Done. Is this change still desirable now that the breakage is fixed btw? Or do we want to just take breakages as they come and fix them on the fly?

@mpharrigan
Copy link
Collaborator

for reference: fixed by #2655

# install_requires should not be overly specific.
# Change any == pinned deps to >=.
# https://packaging.python.org/discussions/install-requires-vs-requirements/
requirements = [r.replace('==', '>=') for r in requirements]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this change. Our requirements.txt file should be the literal source of truth. This also causes the code installed on user machines to be broken, e.g. it uses the bad version of sympy.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I suggested this change. I agree that as written it has sympy issues. In general, I'd advocate for never being overly specific on which versions are required for a developer-oriented library. Usually the way python packages can handle this is by pinning things in requirements.txt so e.g. CI runs under a consistent environment and users who don't care can pip install -r requirements.txt and get a good environment and leaving install_requires with a looser set of restrictions so pip install cirq has a low probability of upgrading or downgrading an existing install of a package (particularly heavy-weight ones like matplotlib or numpy).

If you insist on having one set of version specs for these two use cases, each restriction in requirements.txt should have a corresponding reason associated with it. If the reason is "there's a bug in the most recent release" then that's good and a future developer knows the condition under which it is safe to change the version spec. If the reason is "well I seem to have pandas 0.25.1 and I can't be bothered to check older or newer versions" ... that's less compelling

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are going to have reasons required for restrictions in requirements.txt, seems like this PR isn't useful anymore, because the SymPy stuff was fixed. I can revert if that's the direction we want to go in.

@vtomole vtomole self-requested a review December 18, 2019 21:41
@vtomole
Copy link
Collaborator

vtomole commented Dec 18, 2019

Forgot to revert my approval after @mpharrigan's comment.

@KevinVillela
Copy link
Contributor Author

Seems like right now we will want to only pin requirements when necessary, and since SymPy is fixed there are nothing we need to actually pin. I'm going to revert this for now, but feel free to let me know if I should bring it back!

@mpharrigan
Copy link
Collaborator

Thanks @KevinVillela ! I think that's the right call for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes Makes googlebot stop complaining.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cirq tests appear to be failing
5 participants