feat: add progress_format support for machine-readable JSON output #3654
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Adds
progress_formatsupport tohuggingface_hub, enabling machine-readable JSON progress output similar to huggingface/tokenizers#1921 and huggingface/datasets#7920.Motivation
When using
huggingface_hubin automated pipelines, web backends, or UI applications, it is useful to emit machine-readable progress instead of ANSI progress bars. This PR adds the sameprogress_formatoption that was implemented in tokenizers and datasets.Changes
New Functions
set_progress_format(format: str): Set global progress formatget_progress_format() -> str: Get current progress formatSupported Formats
JSON Format
When
progress_format="json", emits JSON every 5% progress change or at completion:{"stage": "Downloading model.safetensors", "current": 1024, "total": 4096, "percent": 25.0}Usage Example
Implementation Details
io.StringIO()when format is "json"disable=True)huggingface_hub.utilsBackward Compatibility
Cross-Reference
This implementation mirrors the approach from: