Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When formatting a requirement, only lowercase its name. #2123

Merged
merged 1 commit into from
May 23, 2018

Conversation

jfly
Copy link
Contributor

@jfly jfly commented May 2, 2018

This fixes #2113.

This bug was introduced in
jazzband/pip-tools#452 as a band-aid fix to
jazzband/pip-tools#431. Pipenv then copied
that code in 2553ebc#diff-b56b95ccea8595a0f6f24ea753842976, and inherited this latent bug.

Maybe the right fix is for pypa/packaging to lowercase the name? There's
a comment here
https://github.com/pypa/packaging/blob/16.8/packaging/requirements.py#L86
about normalizing the requirement's name, which might be what this is
referring to.

Testing

Demo of how I tested this out:

Before

Note the lowercase "cpython" in the output.

# Clear pipenv cache
$ rm -r /home/jeremy/.cache/pipenv/

# Test for bug
$ PIPENV_PACKAGES="gevent -i https://python-packages.honordev.com/build/dev/" "/home/jeremy/.local/share/virtualenvs/external-api-kWcvfjzK/bin/python" "/usr/lib/python3.6/site-packages/pipenv/resolver.py" --pre --verbose
RESULTS:
[{"hashes": ["sha256:0141b44a1eea5c3d2dec9e75da3bf0a51153bc6f9ef6e631a1e18460541d2ac1", "sha256:11108689372672000cf5393451cde5b4583b8394a747c1faf56ea5afcca687ef", "sha256:135c4cac0e3b35ba2497acb38ed739606cb2ca1acbec4c052fb4b7f527abfbfb", "sha256:1b75ddcdadce3174d8b0a5ea3f46af36274fe0275a47a73ee3e5be94c0471be5", "sha256:26878e0432cab7cdd85ed08caa0c277d38164a067ea2586c210a49948db9df28", "sha256:2b2be6b0991c17826f4f7a6ee0be3a62ae2825c92158d9a8ae567b399452a579", "sha256:42816718f9039c54f32a8bffe333a63e6e1cc9c673151dd07390b7e7d874bcf4", "sha256:6217c69fbac737d62b67bb07434a61a437373f4211f594ade9210f5a2d053607", "sha256:66574654c43c655beed59825b00e3a5c006c4d9a0b767e3a34ba256246e7df9d", "sha256:83127cd5bab985ad35f7523af32d32ce8f437834a30938b1f5a5b3ff76268eef", "sha256:86b6cf0a5f9d88ff85c6e9c595e5fdc8cb38249efe660cf6fbe0eeb50df241e9", "sha256:967c8df8192ab388e985e964f7c7608dc4a967c081c7b042228c56aeac276e78", "sha256:a140b720c0836e6121092f34649c750e7cff7fdda6b9169b001cac581246612a", "sha256:b3f41829ff9c2d2fde2c71a646b53b9c7f2b62908ee3601ad2d17eaa6c8b89b9", "sha256:c541e1d4f738c9760335a988b47527b0e816d62ce88eb09a61382e551b96dea5", "sha256:dbbe5285ce4d788b8b4643a7ee74544999537661b882b269a3a7bc1cff24b695", "sha256:e2808a40a5747c13e634edbdd42032dfe5f930a39eddd6f235703886cdf486bc", "sha256:e39617b8ae37b840e5b6c149ed5ce26cd5d0d2b348dd4410b1f584d7eac4b333", "sha256:ebeb962bd19c4297067e50c8786115900022564b90a91a13adc77296c6698df3", "sha256:f2e3f638e6bbede7295447c6ed8864ae151c45e85875996299996f4136d231df"], "version": "1.3b1", "name": "gevent"}, {"hashes": ["sha256:09ef2636ea35782364c830f07127d6c7a70542b178268714a9a9ba16318e7e8b", "sha256:0fef83d43bf87a5196c91e73cb9772f945a4caaff91242766c5916d1dd1381e4", "sha256:1b7df09c6598f5cfb40f843ade14ed1eb40596e75cd79b6fa2efc750ba01bb01", "sha256:1fff21a2da5f9e03ddc5bd99131a6b8edf3d7f9d6bc29ba21784323d17806ed7", "sha256:42118bf608e0288e35304b449a2d87e2ba77d1e373e8aa221ccdea073de026fa", "sha256:50643fd6d54fd919f9a0a577c5f7b71f5d21f0959ab48767bd4bb73ae0839500", "sha256:58798b5d30054bb4f6cf0f712f08e6092df23a718b69000786634a265e8911a9", "sha256:5b49b3049697aeae17ef7bf21267e69972d9e04917658b4e788986ea5cc518e8", "sha256:75c413551a436b462d5929255b6dc9c0c3c2b25cbeaee5271a56c7fda8ca49c0", "sha256:769b740aeebd584cd59232be84fdcaf6270b8adc356596cdea5b2152c82caaac", "sha256:ad2383d39f13534f3ca5c48fe1fc0975676846dc39c2cece78c0f1f9891418e0", "sha256:b417bb7ff680d43e7bd7a13e2e08956fa6acb11fd432f74c97b7664f8bdb6ec1", "sha256:b6ef0cabaf5a6ecb5ac122e689d25ba12433a90c7b067b12e5f28bdb7fb78254", "sha256:c2de19c88bdb0366c976cc125dca1002ec1b346989d59524178adfd395e62421", "sha256:c7b04a6dc74087b1598de8d713198de4718fa30ec6cbb84959b26426c198e041", "sha256:f8f2a0ae8de0b49c7b5b2daca4f150fdd9c1173e854df2cce3b04123244f9f45", "sha256:fcfadaf4bf68a27e5dc2f42cbb2f4b4ceea9f05d1d0b8f7787e640bed2801634"], "version": "0.4.13", "name": "greenlet", "markers": "platform_python_implementation == 'cpython'"}]

After

Note the correctly capitalized "CPython" in the output:

# Clear pipenv cache
$ rm -r /home/jeremy/.cache/pipenv/

# Test for bug
$ PIPENV_PACKAGES="gevent -i https://python-packages.honordev.com/build/dev/" "/home/jeremy/.local/share/virtualenvs/external-api-kWcvfjzK/bin/python" "/usr/lib/python3.6/site-packages/pipenv/resolver.py" --pre
RESULTS:
[{"hashes": ["sha256:0141b44a1eea5c3d2dec9e75da3bf0a51153bc6f9ef6e631a1e18460541d2ac1", "sha256:11108689372672000cf5393451cde5b4583b8394a747c1faf56ea5afcca687ef", "sha256:135c4cac0e3b35ba2497acb38ed739606cb2ca1acbec4c052fb4b7f527abfbfb", "sha256:1b75ddcdadce3174d8b0a5ea3f46af36274fe0275a47a73ee3e5be94c0471be5", "sha256:26878e0432cab7cdd85ed08caa0c277d38164a067ea2586c210a49948db9df28", "sha256:2b2be6b0991c17826f4f7a6ee0be3a62ae2825c92158d9a8ae567b399452a579", "sha256:42816718f9039c54f32a8bffe333a63e6e1cc9c673151dd07390b7e7d874bcf4", "sha256:6217c69fbac737d62b67bb07434a61a437373f4211f594ade9210f5a2d053607", "sha256:66574654c43c655beed59825b00e3a5c006c4d9a0b767e3a34ba256246e7df9d", "sha256:83127cd5bab985ad35f7523af32d32ce8f437834a30938b1f5a5b3ff76268eef", "sha256:86b6cf0a5f9d88ff85c6e9c595e5fdc8cb38249efe660cf6fbe0eeb50df241e9", "sha256:967c8df8192ab388e985e964f7c7608dc4a967c081c7b042228c56aeac276e78", "sha256:a140b720c0836e6121092f34649c750e7cff7fdda6b9169b001cac581246612a", "sha256:b3f41829ff9c2d2fde2c71a646b53b9c7f2b62908ee3601ad2d17eaa6c8b89b9", "sha256:c541e1d4f738c9760335a988b47527b0e816d62ce88eb09a61382e551b96dea5", "sha256:dbbe5285ce4d788b8b4643a7ee74544999537661b882b269a3a7bc1cff24b695", "sha256:e2808a40a5747c13e634edbdd42032dfe5f930a39eddd6f235703886cdf486bc", "sha256:e39617b8ae37b840e5b6c149ed5ce26cd5d0d2b348dd4410b1f584d7eac4b333", "sha256:ebeb962bd19c4297067e50c8786115900022564b90a91a13adc77296c6698df3", "sha256:f2e3f638e6bbede7295447c6ed8864ae151c45e85875996299996f4136d231df"], "version": "1.3b1", "name": "gevent"}, {"hashes": ["sha256:09ef2636ea35782364c830f07127d6c7a70542b178268714a9a9ba16318e7e8b", "sha256:0fef83d43bf87a5196c91e73cb9772f945a4caaff91242766c5916d1dd1381e4", "sha256:1b7df09c6598f5cfb40f843ade14ed1eb40596e75cd79b6fa2efc750ba01bb01", "sha256:1fff21a2da5f9e03ddc5bd99131a6b8edf3d7f9d6bc29ba21784323d17806ed7", "sha256:42118bf608e0288e35304b449a2d87e2ba77d1e373e8aa221ccdea073de026fa", "sha256:50643fd6d54fd919f9a0a577c5f7b71f5d21f0959ab48767bd4bb73ae0839500", "sha256:58798b5d30054bb4f6cf0f712f08e6092df23a718b69000786634a265e8911a9", "sha256:5b49b3049697aeae17ef7bf21267e69972d9e04917658b4e788986ea5cc518e8", "sha256:75c413551a436b462d5929255b6dc9c0c3c2b25cbeaee5271a56c7fda8ca49c0", "sha256:769b740aeebd584cd59232be84fdcaf6270b8adc356596cdea5b2152c82caaac", "sha256:ad2383d39f13534f3ca5c48fe1fc0975676846dc39c2cece78c0f1f9891418e0", "sha256:b417bb7ff680d43e7bd7a13e2e08956fa6acb11fd432f74c97b7664f8bdb6ec1", "sha256:b6ef0cabaf5a6ecb5ac122e689d25ba12433a90c7b067b12e5f28bdb7fb78254", "sha256:c2de19c88bdb0366c976cc125dca1002ec1b346989d59524178adfd395e62421", "sha256:c7b04a6dc74087b1598de8d713198de4718fa30ec6cbb84959b26426c198e041", "sha256:f8f2a0ae8de0b49c7b5b2daca4f150fdd9c1173e854df2cce3b04123244f9f45", "sha256:fcfadaf4bf68a27e5dc2f42cbb2f4b4ceea9f05d1d0b8f7787e640bed2801634"], "version": "0.4.13", "name": "greenlet", "markers": "platform_python_implementation == 'CPython'"}]

TODO

  • Report this upstream to pip-tools: format_requirement is lowercasing markers, which violates PEP 508 jazzband/pip-tools#660.
  • Is it possible to write a test for this? I don't know the existing test infrastructure well enough to know where such a test would go, or what it would look like.
  • As you can see in my testing above, because of pipenv's caching, this problem won't simply go away with a release of a new version of pipenv (people will continue using buggy cache files). Do we have a mechanism for clearing the cache on a pipenv upgrade? Or will we just have to make a release note advising people to clear their pipenv caches?

@techalchemy
Copy link
Member

/cc @jtratner on the cache question

@techalchemy
Copy link
Member

Thanks for the PR, I think I typed a comment about this on mobile but I have no idea if I sent t or if that was on the issue.

Probably related -- #2125

@techalchemy
Copy link
Member

@jfly ok, so first of all thanks a ton for digging into this. To answer your questions

  1. We would want to upstream this, although I'm not sure if it's correct to only lowercase the requirement name (what if the user puts 'windows' for their `sys_platform', do we really want to be strict?)
  2. Test would look something like this and live in tests/integration/test_install_markers.py:
@pytest.mark.markers
@flaky
def test_platform_python_implementation_marker(PipenvInstance, pypi):
    """Markers should be converted during locking to help users who input this incorrectly
    """
    with PipenvInstance(pypi=pypi) as p:
        import platform
        with open(p.pipfile_path, 'w') as f:
            contents = """
[packages]
pytz = {{version = "*", platform_python_implementation = "== '{0}'"}}
            """.format(platform.python_implementation()).strip()
            f.write(contents)

        c = p.pipenv('install')
        assert c.return_code == 0

        assert p.lockfile['default']['funcsigs']['markers'] == "os_name == '{0}'".format(platform.python_implementation())
  1. This approach gets around the caching problem because it leaves things uppercased -- then we can just update the lockfile code to do something like this (you need to do this in two places):
    python_implementations = ('CPython', 'IronPython', 'Jython', 'PyPy',)
    for dep in results:
        # Add version information to lockfile.
        lockfile['develop'].update(
            {dep['name']: {'version': '=={0}'.format(dep['version'])}}
        )
        # Add Hashes to lockfile
        lockfile['develop'][dep['name']]['hashes'] = dep['hashes']
        # Add index metadata to lockfile.
        if 'index' in dep:
            lockfile['develop'][dep['name']]['index'] = dep['index']
        # Add PEP 508 specifier metadata to lockfile if dep isnt top level
        # or top level dep doesn't itself have markers
        if 'markers' in dep:
            if dep['name'] in dev_packages and not any(key in dev_packages[dep['name']] for key in allowed_marker_keys):
                continue
            implementation = dep['markers'].get('platform_python_implementation', '')
            if implementation:
                marker = first(impl for impl in python_implementations if lower(impl) == lower(implementation))
            else:
                marker = dep['markers']
            lockfile['develop'][dep['name']]['markers'] = marker

@jfly
Copy link
Contributor Author

jfly commented May 7, 2018

  1. We would want to upstream this ...

Ok, I've reported this to them in jazzband/pip-tools#660.

  1. ... what if the user puts 'windows' for their `sys_platform', do we really want to be strict?

If we want to be PEP 508 compliant, then yes, I think we do. If I'm understanding this correctly, then for the same reasons I mentioned in #2113 (comment), I believe that since sys_platform is an environment marker, it should be compared using regular (case sensitive) Python string equality.

  1. Test would look something like this and live in tests/integration/test_install_markers.py:

Thanks for the pointer. I messed around with that a bit, and came up with the following test:

@pytest.mark.markers
@flaky
@pytest.mark.markers
@flaky
def test_platform_python_implementation_marker(PipenvInstance, pypi):
    """Markers should be converted during locking to help users who input this incorrectly
    """
    with PipenvInstance(pypi=pypi) as p:
        with open(p.pipfile_path, 'w') as f:
            contents = """
[packages]
pytz = {{version = "*", platform_python_implementation = "== '{0}'"}}
            """.format(platform.python_implementation()).strip()
            f.write(contents)

        c = p.pipenv('install')
        assert c.return_code == 0

        #import pdb; pdb.set_trace()#<<<
        assert p.lockfile['default']['pytz']['markers'] == "platform_python_implementation == '{0}'".format(platform.python_implementation())

However, this test actually passes even without any changes to the Pipenv source code, so it's definitely not testing the bug I'm trying to fix =( I stepped through the Pipenv code during this test, and the reason it's not triggering the bug is because we end up on this line of resolver.py:

for dependency in self.repository.get_dependencies(ireq):

and self.repository.get_dependencies(ireq) returns an empty array. To reproduce this bug, we'd need to go into the line which invokes format_requirement:

self.dependency_cache[ireq] = sorted(format_requirement(ireq) for ireq in dependencies)
.

I looked a bit into how to trigger that code, and the only solution I can think of would be to add a new package to the tests/pypi directory. That new package could be something like gevent, which has a dependency on greenlet and uses the platform_python_implementation marker. Should I give that a shot?

  1. This approach gets around the caching problem because it leaves things uppercased -- then we can just update the lockfile code to do something like this (you need to do this in two places):

Hmm... do you mean "it leaves things lowercased"? That looks like it could work, but it feels pretty silly to me to first lowercase a string, and then do a lookup later on to figure out it is actually supposed to be capitalized. It also doesn't do the right thing for other marker types such as sys_platform, right?

@jfly
Copy link
Contributor Author

jfly commented May 14, 2018

Whoops, I don't know how I accidentally closed this, sorry for the noise!

@jfly
Copy link
Contributor Author

jfly commented May 19, 2018

Ping. Any thoughts on how to proceed with this? Thanks!

@techalchemy
Copy link
Member

Yeah sorry for the delay, feel free to add the requisite wheels and make sure we are handling this case properly but I'll probably merge this soon either way if tests still pass

This fixes pypa#2113.

This bug was introduced in
<jazzband/pip-tools#452> as a band-aid fix to
<jazzband/pip-tools#431>. Pipenv then copied
that code in <pypa@2553ebc#diff-b56b95ccea8595a0f6f24ea753842976>, and inherited this latent bug.

Maybe the right fix is for pypa/packaging to lowercase the name? There's
a comment here
<https://github.com/pypa/packaging/blob/16.8/packaging/requirements.py#L86>
about normalizing the requirement's name, which might be what this is
referring to.

To test this, I invented a new, very simple python package called
`depends-on-marked-package`. The setup.py for this package is just:

```python
import setuptools

setuptools.setup(
    name="depends-on-marked-package",
    version="0.0.1",
    packages=setuptools.find_packages(),
    install_requires=['pytz; platform_python_implementation=="CPython"'],
)
```

This is a simplified version of gevent's setup.py's install_requires upon greenlet.
@jfly
Copy link
Contributor Author

jfly commented May 23, 2018

Ok, I've updated my PR with a test! Rather than adding all of gevent and greenlet, I invented a new, super simple package called depends-on-marked-package. Its setup.py is simply:

    import setuptools
    
    setuptools.setup(
        name="depends-on-marked-package",
        version="0.0.1",
        packages=setuptools.find_packages(),
        install_requires=['pytz; platform_python_implementation=="CPython"'],
    )
Without my change to how we format markers, I see the test fail
➜  ~/honor/pipenv git:(issue-2113) pipenv run pytest -v -s tests/integration/test_install_markers.py::test_platform_python_implementation_marker
Loading .env environment variables…
=========================== test session starts ============================
platform linux -- Python 3.6.5, pytest-3.5.1, py-1.5.3, pluggy-0.6.0 -- /home/jeremy/.local/share/virtualenvs/pipenv-ia304dsC/bin/python
cachedir: .pytest_cache
rootdir: /home/jeremy/honor/pipenv, inifile:
plugins: xdist-1.22.2, tap-2.2, forked-0.2, flaky-3.4.0, pypi-0.1.1
collected 1 item                                                           

tests/integration/test_install_markers.py::test_platform_python_implementation_marker 127.0.0.1 - - [22/May/2018 23:11:04] "GET /simple/depends-on-marked-package/ HTTP/1.1" 200 379
127.0.0.1 - - [22/May/2018 23:11:04] "GET /simple/depends-on-marked-package/ HTTP/1.1" 200 379
127.0.0.1 - - [22/May/2018 23:11:04] "GET /depends-on-marked-package/depends_on_marked_package-0.0.1-py2.py3-none-any.whl HTTP/1.1" 200 1215
127.0.0.1 - - [22/May/2018 23:11:04] "GET /simple/pytz/ HTTP/1.1" 200 330
127.0.0.1 - - [22/May/2018 23:11:04] "GET /simple/pytz/ HTTP/1.1" 200 330
127.0.0.1 - - [22/May/2018 23:11:04] "GET /pytz/pytz-2018.3-py2.py3-none-any.whl HTTP/1.1" 200 509950
127.0.0.1 - - [22/May/2018 23:11:04] "GET /simple/pytz/ HTTP/1.1" 200 330
127.0.0.1 - - [22/May/2018 23:11:04] "GET /pytz/pytz-2018.3-py2.py3-none-any.whl HTTP/1.1" 200 509950
127.0.0.1 - - [22/May/2018 23:11:04] "GET /pytz/pytz-2018.3.tar.gz HTTP/1.1" 200 314214
127.0.0.1 - - [22/May/2018 23:11:04] "GET /simple/depends-on-marked-package/ HTTP/1.1" 200 379
127.0.0.1 - - [22/May/2018 23:11:04] "GET /depends-on-marked-package/depends_on_marked_package-0.0.1-py2.py3-none-any.whl HTTP/1.1" 200 1215
127.0.0.1 - - [22/May/2018 23:11:05] "GET /simple/depends-on-marked-package/ HTTP/1.1" 200 379
127.0.0.1 - - [22/May/2018 23:11:05] "GET /depends-on-marked-package/depends_on_marked_package-0.0.1-py2.py3-none-any.whl HTTP/1.1" 200 1215
$ pipenv install
Installing dependencies from Pipfile.lock (475c0b)…
Ignoring pytz: markers 'platform_python_implementation == "cpython"' don't match your environment
Looking in indexes: http://127.0.0.1:34209/simple
To activate this project's virtualenv, run the following:
 $ pipenv shell

Creating a virtualenv for this project…
Pipfile: /tmp/pipenv-1_gk8urw-project/Pipfile
Using /home/jeremy/.local/share/virtualenvs/pipenv-ia304dsC/bin/python (3.6.5) to create virtualenv…
Already using interpreter /home/jeremy/.local/share/virtualenvs/pipenv-ia304dsC/bin/python
Using real prefix '/usr'
New python executable in /tmp/pipenv-1_gk8urw-project/.venv/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /tmp/pipenv-1_gk8urw-project/.venv
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (475c0b)!

127.0.0.1 - - [22/May/2018 23:11:10] "GET /simple/depends-on-marked-package/ HTTP/1.1" 200 379
127.0.0.1 - - [22/May/2018 23:11:10] "GET /simple/depends-on-marked-package/ HTTP/1.1" 200 379
127.0.0.1 - - [22/May/2018 23:11:10] "GET /depends-on-marked-package/depends_on_marked_package-0.0.1-py2.py3-none-any.whl HTTP/1.1" 200 1215
127.0.0.1 - - [22/May/2018 23:11:10] "GET /simple/pytz/ HTTP/1.1" 200 330
127.0.0.1 - - [22/May/2018 23:11:10] "GET /simple/pytz/ HTTP/1.1" 200 330
127.0.0.1 - - [22/May/2018 23:11:10] "GET /pytz/pytz-2018.3-py2.py3-none-any.whl HTTP/1.1" 200 509950
127.0.0.1 - - [22/May/2018 23:11:10] "GET /simple/depends-on-marked-package/ HTTP/1.1" 200 379
127.0.0.1 - - [22/May/2018 23:11:10] "GET /depends-on-marked-package/depends_on_marked_package-0.0.1-py2.py3-none-any.whl HTTP/1.1" 200 1215
127.0.0.1 - - [22/May/2018 23:11:10] "GET /simple/pytz/ HTTP/1.1" 200 330
127.0.0.1 - - [22/May/2018 23:11:10] "GET /pytz/pytz-2018.3-py2.py3-none-any.whl HTTP/1.1" 200 509950
127.0.0.1 - - [22/May/2018 23:11:10] "GET /pytz/pytz-2018.3.tar.gz HTTP/1.1" 200 314214
127.0.0.1 - - [22/May/2018 23:11:10] "GET /simple/depends-on-marked-package/ HTTP/1.1" 200 379
127.0.0.1 - - [22/May/2018 23:11:10] "GET /depends-on-marked-package/depends_on_marked_package-0.0.1-py2.py3-none-any.whl HTTP/1.1" 200 1215
$ pipenv install
Installing dependencies from Pipfile.lock (f3d552)…
Ignoring pytz: markers 'platform_python_implementation == "cpython"' don't match your environment
Looking in indexes: http://127.0.0.1:34123/simple
To activate this project's virtualenv, run the following:
 $ pipenv shell

Creating a virtualenv for this project…
Pipfile: /tmp/pipenv-zy98n74f-project/Pipfile
Using /home/jeremy/.local/share/virtualenvs/pipenv-ia304dsC/bin/python (3.6.5) to create virtualenv…
Already using interpreter /home/jeremy/.local/share/virtualenvs/pipenv-ia304dsC/bin/python
Using real prefix '/usr'
New python executable in /tmp/pipenv-zy98n74f-project/.venv/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /tmp/pipenv-zy98n74f-project/.venv
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (f3d552)!

FAILED

================================= FAILURES =================================
________________ test_platform_python_implementation_marker ________________

PipenvInstance = <class 'tests.integration.conftest._PipenvInstance'>
pypi = <pytest_pypi.serve.Server object at 0x7fe9024e5588>

    @pytest.mark.markers
    @flaky
    def test_platform_python_implementation_marker(PipenvInstance, pypi):
        """Markers should be converted during locking to help users who input this incorrectly
        """
        with PipenvInstance(pypi=pypi) as p:
            with open(p.pipfile_path, 'w') as f:
                contents = """
    [packages]
    depends-on-marked-package = "*"
                """.strip()
                f.write(contents)
    
            c = p.pipenv('install')
            assert c.return_code == 0
    
            # depends-on-marked-package has an install_requires of 'pytz; platform_python_implementation=="CPython"'
            # Verify that that marker shows up in our lockfile unaltered.
>           assert p.lockfile['default']['pytz']['markers'] == "platform_python_implementation == 'CPython'"
E           assert "platform_pyt... == 'cpython'" == "platform_pyth... == 'CPython'"
E             - platform_python_implementation == 'cpython'
E             ?                                    ^^
E             + platform_python_implementation == 'CPython'
E             ?                                    ^^

tests/integration/test_install_markers.py:54: AssertionError
===Flaky Test Report===

test_platform_python_implementation_marker failed (1 runs remaining out of 2).
	<class 'AssertionError'>
	assert "platform_pyt... == 'cpython'" == "platform_pyth... == 'CPython'"
  - platform_python_implementation == 'cpython'
  ?                                    ^^
  + platform_python_implementation == 'CPython'
  ?                                    ^^
	[<TracebackEntry /home/jeremy/honor/pipenv/tests/integration/test_install_markers.py:54>]
test_platform_python_implementation_marker failed; it passed 0 out of the required 1 times.
	<class 'AssertionError'>
	assert "platform_pyt... == 'cpython'" == "platform_pyth... == 'CPython'"
  - platform_python_implementation == 'cpython'
  ?                                    ^^
  + platform_python_implementation == 'CPython'
  ?                                    ^^
	[<TracebackEntry /home/jeremy/honor/pipenv/tests/integration/test_install_markers.py:54>]

===End Flaky Test Report===
======================== 1 failed in 10.56 seconds =========================
With my change, the test does pass!
➜  ~/honor/pipenv git:(issue-2113) ✗ pipenv run pytest -v -s tests/integrati➜  ~/honor/pipenv git:(issue-2113) ✗ pipenv run pytest -v -s tests/integration/test_install_markers.py::test_platform_python_implementation_marker
Loading .env environment variables…
============================================================================================================= test session starts =============================================================================================================
platform linux -- Python 3.6.5, pytest-3.5.1, py-1.5.3, pluggy-0.6.0 -- /home/jeremy/.local/share/virtualenvs/pipenv-ia304dsC/bin/python
cachedir: .pytest_cache
rootdir: /home/jeremy/honor/pipenv, inifile:
plugins: xdist-1.22.2, tap-2.2, forked-0.2, flaky-3.4.0, pypi-0.1.1
collected 1 item                                                                                                                                                                                                                              

tests/integration/test_install_markers.py::test_platform_python_implementation_marker 127.0.0.1 - - [22/May/2018 23:14:11] "GET /simple/depends-on-marked-package/ HTTP/1.1" 200 379
127.0.0.1 - - [22/May/2018 23:14:11] "GET /simple/depends-on-marked-package/ HTTP/1.1" 200 379
127.0.0.1 - - [22/May/2018 23:14:11] "GET /depends-on-marked-package/depends_on_marked_package-0.0.1-py2.py3-none-any.whl HTTP/1.1" 200 1215
127.0.0.1 - - [22/May/2018 23:14:11] "GET /simple/pytz/ HTTP/1.1" 200 330
127.0.0.1 - - [22/May/2018 23:14:11] "GET /simple/pytz/ HTTP/1.1" 200 330
127.0.0.1 - - [22/May/2018 23:14:11] "GET /pytz/pytz-2018.3-py2.py3-none-any.whl HTTP/1.1" 200 509950
127.0.0.1 - - [22/May/2018 23:14:11] "GET /simple/depends-on-marked-package/ HTTP/1.1" 200 379
127.0.0.1 - - [22/May/2018 23:14:11] "GET /depends-on-marked-package/depends_on_marked_package-0.0.1-py2.py3-none-any.whl HTTP/1.1" 200 1215
127.0.0.1 - - [22/May/2018 23:14:11] "GET /simple/pytz/ HTTP/1.1" 200 330
127.0.0.1 - - [22/May/2018 23:14:11] "GET /pytz/pytz-2018.3.tar.gz HTTP/1.1" 200 314214
127.0.0.1 - - [22/May/2018 23:14:11] "GET /pytz/pytz-2018.3-py2.py3-none-any.whl HTTP/1.1" 200 509950
127.0.0.1 - - [22/May/2018 23:14:11] "GET /simple/depends-on-marked-package/ HTTP/1.1" 200 379
127.0.0.1 - - [22/May/2018 23:14:11] "GET /simple/pytz/ HTTP/1.1" 200 330
127.0.0.1 - - [22/May/2018 23:14:11] "GET /depends-on-marked-package/depends_on_marked_package-0.0.1-py2.py3-none-any.whl HTTP/1.1" 200 1215
127.0.0.1 - - [22/May/2018 23:14:11] "GET /pytz/pytz-2018.3-py2.py3-none-any.whl HTTP/1.1" 200 509950
$ pipenv install
Installing dependencies from Pipfile.lock (9e5824)…
To activate this project's virtualenv, run the following:
 $ pipenv shell

Creating a virtualenv for this project…
Pipfile: /tmp/pipenv-9_61p45n-project/Pipfile
Using /home/jeremy/.local/share/virtualenvs/pipenv-ia304dsC/bin/python (3.6.5) to create virtualenv…
Already using interpreter /home/jeremy/.local/share/virtualenvs/pipenv-ia304dsC/bin/python
Using real prefix '/usr'
New python executable in /tmp/pipenv-9_61p45n-project/.venv/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /tmp/pipenv-9_61p45n-project/.venv
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (9e5824)!

PASSED
===Flaky Test Report===

test_platform_python_implementation_marker passed 1 out of the required 1 times. Success!

===End Flaky Test Report===

========================================================================================================== 1 passed in 5.34 seconds ===========================================================================================================

The way I've set up this test, I imagine it will only pass in a CPython environment. I'm not sure if that will be a problem or not.

@techalchemy
Copy link
Member

Thanks for all your hard work on this!

@techalchemy techalchemy merged commit 14ffd3c into pypa:master May 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

markers 'platform_python_implementation == "cpython"' don't match your environment when installing gevent
2 participants