Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 5023 Fix hash link generation #5024

Merged
merged 5 commits into from
Apr 1, 2022
Merged

Issue 5023 Fix hash link generation #5024

merged 5 commits into from
Apr 1, 2022

Conversation

matteius
Copy link
Member

@matteius matteius commented Mar 30, 2022

The issue

Fixes: #5023

The fix

Re-organize the logic and fallback to generating the hash from the link, if pypi is not applicable, and there are no applicable candidates. This also applies the index restrictions to the hash finder so it too can find the right hashes.

Pipfile.lock Before

{
    "_meta": {
        "hash": {
            "sha256": "dd84b90e2afa892488dc59130dc48898573afeee5ba3aa45d8c64c54be6be39c"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.10"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            },
            {
                "name": "downloadpytorch",
                "url": "https://download.pytorch.org/whl/cu113/",
                "verify_ssl": true
            },
            {
                "name": "downloadpytorch-390",
                "url": "https://download.pytorch.org/whl/",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "torch": {
            "index": "downloadpytorch-390",
            "version": "==1.11.0+cu113"
        },
        "typing-extensions": {
            "hashes": [
                "sha256:1a9462dcc3347a79b1f1c0271fbe79e844580bb598bafa1ed208b94da3cdcd42",
                "sha256:21c85e0fe4b9a155d0799430b0ad741cdce7e359660ccbd8b530613e8df88ce2"
            ],
            "markers": "python_version >= '3.6'",
            "version": "==4.1.1"
        }
    },
    "develop": {}
}

Pilfile.lock After

{
    "_meta": {
        "hash": {
            "sha256": "a22d7bb22d610cc3aad6f7c9c0a955c862618ad63d3a5f1ada90f4a4e0d6e9cd"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.10"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            },
            {
                "name": "downloadpytorch",
                "url": "https://download.pytorch.org/whl/cu113/",
                "verify_ssl": true
            },
            {
                "name": "downloadpytorch-390",
                "url": "https://download.pytorch.org/whl/",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "torch": {
            "hashes": [
                "sha256:7fd4751bbf39bbb04ec6116c7243ce6528aded4197afcf380537340e1eebd19a",
                "sha256:a68c33657a546131eb9bc44e2a98d2fa704aafae861460b051b82813852ccb44",
                "sha256:b6a799bdb6ee3d914e5e62bddb4276d4a10248c1af4f2d217738e5f9ee27485b",
                "sha256:ddc57495195aa2456e78bfc7d8d3f45dabbb8b7b268b3b5dbed4f0e4db492f33",
                "sha256:e4bb14d953db9aad5bdb945a328410638721d77e3e622d0a8d77063c01daf40b",
                "sha256:e9126b0a5d95704bee40a9d0ef1cbd82d8dc7863e4638a376bef702dfd659370",
                "sha256:e9df65c1fb2d80283b276114878fd38f411b70880e0b406c451d000e6159f451",
                "sha256:f56333470daea3c97078b37607e0035cccf72fc5c36fd84546e1a4b8d9944f2b"
            ],
            "index": "downloadpytorch",
            "version": "==1.11.0+cu113"
        },
        "typing-extensions": {
            "hashes": [
                "sha256:1a9462dcc3347a79b1f1c0271fbe79e844580bb598bafa1ed208b94da3cdcd42",
                "sha256:21c85e0fe4b9a155d0799430b0ad741cdce7e359660ccbd8b530613e8df88ce2"
            ],
            "markers": "python_version >= '3.6'",
            "version": "==4.1.1"
        }
    },
    "develop": {}
}

The checklist

  • Associated issue
  • A news fragment in the news/ directory to describe this fix with the extension .bugfix, .feature, .behavior, .doc. .vendor. or .trivial (this will appear in the release changelog). Use semantic line breaks and name the file after the issue number or the PR #.

@matteius
Copy link
Member Author

matteius commented Apr 1, 2022

I'd like to add a test, though I am not going to depend on this 2GB wheel file example, and I am surprised that the existing CI passed without this change. So I don't have a great alternate example to go on, as to why this was such an edge case to the torch example but not the CI since much of the CI uses a private pypi server.

@@ -723,19 +732,21 @@ def _get_hashes_from_pypi(self, ireq):
return None

def collect_hashes(self, ireq):
if ireq.link:
link = ireq.link
if link.is_vcs or (link.is_file and link.is_existing_dir()):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Factor out this base case of set() and move the other case to the end of this method for reasons that involve applying index restrictions first. We will also consider the ireq.link preferable to the ireq.original_link in the resulting refactor since adding in treatment of the ireq.link was important for this example edge case.

@matteius matteius requested a review from oz123 April 1, 2022 01:58
@oz123 oz123 merged commit 45f3237 into main Apr 1, 2022
@oz123 oz123 deleted the issue-5023-hash-links branch April 1, 2022 09:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

hash generation of index restricted package missing from the the Pipfile.lock
2 participants