docs: Expand training customization examples

[behroozazarkhalili]

Summary

This PR addresses issue #4379 by expanding the Training Customization documentation section with 5 new comprehensive examples, rather than removing it.

Resolves #4379

Changes Made

New Examples Added (5):

1. Custom Callbacks - Shows how to add custom callbacks for logging, monitoring, or early stopping
2. Custom Evaluation Metrics - Demonstrates defining custom metrics to track during training
3. Mixed Precision Training - Explains bf16/fp16 usage for speed and memory optimization
4. Gradient Accumulation - Shows how to simulate larger batch sizes with limited GPU memory
5. Custom Data Collator - Demonstrates custom data preprocessing and padding strategies

Documentation Improvements:

• Updated introduction for better clarity and consistency
• All examples follow the same pattern as existing examples
• All code examples verified against the codebase
• Proper imports and configuration options validated

Statistics

• Original examples: 5
• New examples: 5
• Total examples: 10 (doubled!)
• Lines added: ~150

Verification

✅ All imports verified against codebase
✅ All config options verified in DPOConfig
✅ DataCollatorForPreference import path corrected
✅ Consistent code style with existing examples
✅ Examples apply to most/all trainers as stated

Test Plan

• Verified all imports exist in the codebase
• Validated config parameters against DPOConfig
• Ensured consistent formatting with existing examples
• Checked that examples follow DPOTrainer pattern as stated in intro

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[behroozazarkhalili]

I've addressed both review comments:

1. Mixed precision section: Added clarification that bf16=True is the default in TRL. Updated the example to show when/how to override defaults for older GPUs or to disable mixed precision.

2. Gradient accumulation section: Moved from customization guide to the reducing memory usage guide (reducing_memory_usage.md), as it's primarily a memory optimization technique. Expanded the examples to include DPO, SFT, and Reward trainers.

Ready for re-review!

[sergiopaniego]

Thanks!!
I'd aim on reducing the code snippets content, only including the relevant part. Otherwise it's complicated to understand what's been added. A good example of this is the subsection Use the accelerator cache optimizer, where it directly points to the added param.
Btw the optimize_device_cache is no longer part of the codebase so that subsection can actually be removed :)