Inconsistent sys_platform markers when running pipenv install

[ptink]

A sys_platform marker is occasionally being erroneously added to Pipfile.lock when installing certain libraries.

$ python -m pipenv.help output

Pipenv version: '2018.05.18'

Pipenv location: 'C:\\Users\\~\\AppData\\Local\\Programs\\Python\\Python36\\lib\\site-packages\\pipenv'

Python location: 'C:\\Users\\~\\AppData\\Local\\Programs\\Python\\Python36\\python.exe'

Other Python installations in PATH:

• 3.6: C:\Users\~\AppData\Local\Programs\Python\Python36\python.exe

• 3.6.5: C:\Users\~\AppData\Local\Programs\Python\Python36\python.exe

• 3.6.5: C:\Windows\py.exe

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.5',
 'os_name': 'nt',
 'platform_machine': 'AMD64',
 'platform_python_implementation': 'CPython',
 'platform_release': '10',
 'platform_system': 'Windows',
 'platform_version': '10.0.15063',
 'python_full_version': '3.6.5',
 'python_version': '3.6',
 'sys_platform': 'win32'}

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: removed
• SHELL: C:\Program Files\Git\usr\bin\bash.exe
• LANG: en_US.UTF-8
• PWD: C:/dev/git/test

---

Contents of Pipfile ('C:\dev\git\test\Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]

[dev-packages]
sphinx = "*"
pinocchio = "*"

[requires]
python_version = "3.6"

Contents of Pipfile.lock ('C:\dev\git\test\Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "5587158cdb18980b643efee96ff76ea597e3df3bfc877dfeb769a2d5ccad57b2"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.6"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {},
    "develop": {
        "alabaster": {
            "hashes": [
                "sha256:2eef172f44e8d301d25aff8068fddd65f767a3f04b5f15b0f4922f113aa1c732",
                "sha256:37cdcb9e9954ed60912ebc1ca12a9d12178c26637abdf124e3cde2341c257fe0"
            ],
            "version": "==0.7.10"
        },
        "babel": {
            "hashes": [
                "sha256:6778d85147d5d85345c14a26aada5e478ab04e39b078b0745ee6870c2b5cf669",
                "sha256:8cba50f48c529ca3fa18cf81fa9403be176d374ac4d60738b839122dfaaa3d23"
            ],
            "version": "==2.6.0"
        },
        "certifi": {
            "hashes": [
                "sha256:13e698f54293db9f89122b0581843a782ad0934a4fe0172d2a980ba77fc61bb7",
                "sha256:9fa520c1bacfb634fa7af20a76bcbd3d5fb390481724c597da32c719a7dca4b0"
            ],
            "version": "==2018.4.16"
        },
        "chardet": {
            "hashes": [
                "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
                "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
            ],
            "version": "==3.0.4"
        },
        "colorama": {
            "hashes": [
                "sha256:463f8483208e921368c9f306094eb6f725c6ca42b0f97e313cb5d5512459feda",
                "sha256:48eb22f4f8461b1df5734a074b57042430fb06e1d61bd1e11b078c0fe6d7a1f1"
            ],
            "markers": "sys_platform == 'win32'",
            "version": "==0.3.9"
        },
        "docutils": {
            "hashes": [
                "sha256:02aec4bd92ab067f6ff27a38a38a41173bf01bed8f89157768c1573f53e474a6",
                "sha256:51e64ef2ebfb29cae1faa133b3710143496eca21c530f3f71424d77687764274",
                "sha256:7a4bd47eaf6596e1295ecb11361139febe29b084a87bf005bf899f9a42edc3c6"
            ],
            "version": "==0.14"
        },
        "idna": {
            "hashes": [
                "sha256:156a6814fb5ac1fc6850fb002e0852d56c0c8d2531923a51032d1b70760e186e",
                "sha256:684a38a6f903c1d71d6d5fac066b58d7768af4de2b832e426ec79c30daa94a16"
            ],
            "version": "==2.7"
        },
        "imagesize": {
            "hashes": [
                "sha256:3620cc0cadba3f7475f9940d22431fc4d407269f1be59ec9b8edcca26440cf18",
                "sha256:5b326e4678b6925158ccc66a9fa3122b6106d7c876ee32d7de6ce59385b96315"
            ],
            "version": "==1.0.0"
        },
        "jinja2": {
            "hashes": [
                "sha256:74c935a1b8bb9a3947c50a54766a969d4846290e1e788ea44c1392163723c3bd",
                "sha256:f84be1bb0040caca4cea721fcbbbbd61f9be9464ca236387158b0feea01914a4"
            ],
            "version": "==2.10"
        },
        "markupsafe": {
            "hashes": [
                "sha256:a6be69091dac236ea9c6bc7d012beab42010fa914c459791d627dad4910eb665"
            ],
            "version": "==1.0"
        },
        "packaging": {
            "hashes": [
                "sha256:e9215d2d2535d3ae866c3d6efc77d5b24a0192cce0ff20e42896cc0664f889c0",
                "sha256:f019b770dd64e585a99714f1fd5e01c7a8f11b45635aa953fd41c689a657375b"
            ],
            "version": "==17.1"
        },
        "pinocchio": {
            "hashes": [
                "sha256:1c73db81274d5c8cb50de768fb5735c6c6702891094acb4b08c0920b3758befd",
                "sha256:bc53568703bc8e22d0b96010be657a5ebc6ca445defa45878568a0aef992c343"
            ],
            "index": "pypi",
            "version": "==0.4.2"
        },
        "pygments": {
            "hashes": [
                "sha256:78f3f434bcc5d6ee09020f92ba487f95ba50f1e3ef83ae96b9d5ffa1bab25c5d",
                "sha256:dbae1046def0efb574852fab9e90209b23f556367b5a320c0bcb871c77c3e8cc"
            ],
            "version": "==2.2.0"
        },
        "pyparsing": {
            "hashes": [
                "sha256:0832bcf47acd283788593e7a0f542407bd9550a55a8a8435214a1960e04bcb04",
                "sha256:281683241b25fe9b80ec9d66017485f6deff1af5cde372469134b56ca8447a07",
                "sha256:8f1e18d3fd36c6795bb7e02a39fd05c611ffc2596c1e0d995d34d67630426c18",
                "sha256:9e8143a3e15c13713506886badd96ca4b579a87fbdf49e550dbfc057d6cb218e",
                "sha256:b8b3117ed9bdf45e14dcc89345ce638ec7e0e29b2b579fa1ecf32ce45ebac8a5",
                "sha256:e4d45427c6e20a59bf4f88c639dcc03ce30d193112047f94012102f235853a58",
                "sha256:fee43f17a9c4087e7ed1605bd6df994c6173c1e977d7ade7b651292fab2bd010"
            ],
            "version": "==2.2.0"
        },
        "pytz": {
            "hashes": [
                "sha256:65ae0c8101309c45772196b21b74c46b2e5d11b6275c45d251b150d5da334555",
                "sha256:c06425302f2cf668f1bba7a0a03f3c1d34d4ebeef2c72003da308b3947c7f749"
            ],
            "version": "==2018.4"
        },
        "requests": {
            "hashes": [
                "sha256:421cfc8d9dde7d6aff68196420afd86b88c65d77d8da9cf83f4ecad785d7b9d6",
                "sha256:cc408268d0e21589bcc2b2c248e42932b8c4d112f499c12c92e99e2178a6134c"
            ],
            "version": "==2.19.0"
        },
        "six": {
            "hashes": [
                "sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9",
                "sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb"
            ],
            "version": "==1.11.0"
        },
        "snowballstemmer": {
            "hashes": [
                "sha256:919f26a68b2c17a7634da993d91339e288964f93c274f1343e3bbbe2096e1128",
                "sha256:9f3bcd3c401c3e862ec0ebe6d2c069ebc012ce142cce209c098ccb5b09136e89"
            ],
            "version": "==1.2.1"
        },
        "sphinx": {
            "hashes": [
                "sha256:85f7e32c8ef07f4ba5aeca728e0f7717bef0789fba8458b8d9c5c294cad134f3",
                "sha256:d45480a229edf70d84ca9fae3784162b1bc75ee47e480ffe04a4b7f21a95d76d"
            ],
            "index": "pypi",
            "version": "==1.7.5"
        },
        "sphinxcontrib-websupport": {
            "hashes": [
                "sha256:68ca7ff70785cbe1e7bccc71a48b5b6d965d79ca50629606c7861a21b206d9dd",
                "sha256:9de47f375baf1ea07cdb3436ff39d7a9c76042c10a769c52353ec46e4e8fc3b9"
            ],
            "version": "==1.1.0"
        },
        "urllib3": {
            "hashes": [
                "sha256:a68ac5e15e76e7e5dd2b8f94007233e01effe3e50e8daddf69acfd81cb686baf",
                "sha256:b5725a0bd4ba422ab0e66e89e030c806576753ea3ee08554382c14e685d117b5"
            ],
            "version": "==1.23"
        }
    }
}

---

Expected result

When running pipenv install --dev to install libraries, one of which has a sys_platform specific dependency and the other that has the same dependency regardless of sys_platform. I would expect the dependency in question to be added to Pipfile.lock without any sys_platform markers

Actual result

Pipfile.lock output can vary without Pipfile contents changing, sometimes the sys_platform marker is present, occasionally it isn't

Steps to replicate

Create a Pipfile with two libraries with the same sub-dependency but different sys_platform requirements (e.g. sphinx and pinocchio, sphinx only requires the dependency colorama on win32, pinocchio requires colorama regardless of platform)

1. run pipenv install, observe output of Pipfile.lock
2. delete Pipfile.lock
3. run pipenv install again, observe output
4. repeat, the "markers": "sys_platform == 'win32'", will sometimes show in Pipfile.lock, occasionally it won't. I believe the marker should never appear, as pinocchio always requires colorama.

I am running pipenv via git-bash on Windows, however I have tested and the same thing happens via powershell

[techalchemy]

Thanks for the bug report! I agree with you. This is a bizarre issue and I really have no clue what is going on with it. I know of one other issue like it (installing vcrpy in our tests fails nondeterministically, so we skip it currently).

[ptink]

It's worth mentioning that it seems to confuse the output of pipenv graph as well, which will report colorama as installed via pinocchio after pipenv install is run on Linux even if the marker was present and restricting it to win32 only (the "ignore" message is displayed) - meaning that it wasn't actually installed (This was how I discovered the bug, pinocchio complained that the library was missing despite pipenv graph's insistence)

[techalchemy]

Graph shows the dependency tree whether it’s all present or not

[ptink]

Ah, well in that case possibly an update of the documentation would be in order. According to the docs:

graph will print a pretty graph of all your installed dependencies.

[uranusjr]

Documentation pull requests tend to be accepted quite quickly :)

[techalchemy]

@ptink it already does show a graph of your installed dependencies, and it shows a ? next to the ones that are not installed but that it detects as being required

[techalchemy]

I am 99% sure this is fixed by #2384

[techalchemy]

closing -- should be resolved

[whyboris]

This issue is still lingering in the Next Project
Perhaps it should be moved to the "Done" column ?
https://github.com/pypa/pipenv/projects/8