Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix file path for shard_num 1 with mllama converter #35053

Merged
merged 1 commit into from
Dec 10, 2024

Conversation

strangiato
Copy link
Contributor

@strangiato strangiato commented Dec 3, 2024

What does this PR do?

mllama conversion with shard_num 1 expects the file to be consolidated.pth but the Llama 3.2 11B uses consolidated.00.pth instead.

Fixes #35049

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@qubvel
Copy link
Member

qubvel commented Dec 3, 2024

Hey @strangiato, for llama vision models the file is consolidated.pth

see
https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct/tree/main/original

@strangiato
Copy link
Contributor Author

strangiato commented Dec 4, 2024

It seems there is a discrepancy between the files that is published on HuggingFace for the .pth format, and the files published on meta directly (https://www.llama.com/llama-downloads/)

If you download the files directly form meta they are using consolidated.00.pth not consolidated.pth.

@qubvel
Copy link
Member

qubvel commented Dec 5, 2024

Ok, then it's better to support both, smth like

path = "consloidated.pth" if os.path.exists("consloidated.pth") else "consloidated.00.pth"

@strangiato strangiato force-pushed the 35049-consolodated-path-fix branch from 01800a8 to 8208593 Compare December 5, 2024 21:29
@strangiato
Copy link
Contributor Author

Updated the code to check if the 00.pth file exists and falls back to the non-numbered one in a similar what to what you suggested.

@qubvel qubvel requested a review from zucchini-nlp December 6, 2024 12:19
Copy link
Member

@zucchini-nlp zucchini-nlp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks !

@qubvel qubvel merged commit 80f2b16 into huggingface:main Dec 10, 2024
9 checks passed
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@strangiato strangiato deleted the 35049-consolodated-path-fix branch December 10, 2024 23:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect hardcoded consolidated.pth path for Llama 3.2 11B Vision+Instruct Model
4 participants