Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

common: download from URL, improve parallel download progress status #6537

Open
phymbert opened this issue Apr 8, 2024 · 6 comments · May be fixed by #6831
Open

common: download from URL, improve parallel download progress status #6537

phymbert opened this issue Apr 8, 2024 · 6 comments · May be fixed by #6831
Assignees
@TevinWang
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed split GGUF split model sharding

Comments

@phymbert
Copy link
Collaborator

phymbert commented Apr 8, 2024 •
edited
Loading

Context

When downloading a sharded model, files are downloaded in parallel, it was added in:

The progressions of each download conflict:
image

Need to properly implement CURLOPT_NOPROGRESS for parallel download.

Example in #6515:

main --hf-repo ggml-org/models \
  --hf-file grok-1/grok-1-q4_0-00001-of-00009.gguf \
  --model   models/grok-1-q4_0-00001-of-00009.gguf \
  -ngl 64
   --prompt "I believe the meaning of life is"
@phymbert phymbert added enhancement New feature or request help wanted Extra attention is needed good first issue Good for newcomers split GGUF split model sharding labels Apr 8, 2024
@phymbert phymbert changed the title common: download from URL, improve parallel download progress statrus common: download from URL, improve parallel download progress status Apr 8, 2024
@TevinWang
Copy link

Hi! Can I take on this issue as a first contribution?

@phymbert
Copy link
Collaborator Author

Sure, good luck, please ping here for any question

@phymbert
Copy link
Collaborator Author

Hi @TevinWang, are you still working on this issue ?

@TevinWang
Copy link

Yes, sorry, still working on it!

@phymbert
Copy link
Collaborator Author

Thanks, no worries, take your time.

@TevinWang TevinWang linked a pull request Apr 22, 2024 that will close this issue
Open
@TevinWang
Copy link

@phymbert Just created a PR, #6831, let me know if there are any changes needed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@TevinWang TevinWang

Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed split GGUF split model sharding
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

common : fix parallel shard download interleaving output jhs-panda/llama.cpp
2 participants
@phymbert @TevinWang