Add progress_format support for machine-readable progress output

[podarok]

Summary

Adds support to , enabling machine-readable JSON progress output similar to huggingface/tokenizers#1921.

Motivation

When using datasets in automated pipelines or UI applications, it's useful to emit machine-readable progress instead of ANSI progress bars. This PR adds the same progress_format option that was implemented in tokenizers.

Changes

New Functions

• set_progress_format(format: str): Set global progress format
• get_progress_format() -> str: Get current progress format

Supported Formats

1. "tqdm" (default): Interactive progress bars
2. "json": Machine-readable JSON lines to stderr
3. "silent": No output

JSON Format

When progress_format="json", emits JSON every 5% progress change or completion:

{"stage":"Processing","current":50,"total":100,"percent":50.0}

Usage Example

from datasets import load_dataset
from datasets.utils import set_progress_format

# Enable JSON output
set_progress_format("json")

# Progress will now be emitted as JSON lines
dataset = load_dataset("Goader/kobza", split="train", streaming=True)
for sample in dataset:
    process(sample)

Implementation Details

• Suppresses visual output using io.StringIO() when format is "json"
• Keeps progress tracking active (unlike disable=True)
• Emits JSON to stderr every 5% progress change
• Exports new functions from datasets.utils

Cross-Reference

This implementation mirrors the approach from:

• huggingface/tokenizers#1921

Testing

Tested with:

from datasets.utils import set_progress_format, tqdm

set_progress_format('json')
for i in tqdm(range(100), desc='Test'):
    process(i)
# Outputs: {"stage":"Test","current":10,"total":100,"percent":10.0}

Checklist

• New functions added to datasets.utils.tqdm
• Functions exported from datasets.utils.__init__
• JSON format emits to stderr
• Visual output suppressed when format="json"
• Progress tracking remains active
• Cross-referenced with tokenizers#1921