[Serialization] Fix: remove context manager when loading shards + handle mlx weights

[hanouticelina]

This PR fixes some failing tests in transformers when using the hfh loading helpers introduced in #2687.

See: https://app.circleci.com/pipelines/github/huggingface/transformers/113474/workflows/1a1ecbcd-26dc-4d3e-872d-79e5aedf04f2/jobs/1516443/tests#failed-test-1

• in test_model_from_pretrained_from_mlx : mlx weights should be loaded as pt according to this comment : https://github.com/huggingface/transformers/blob/3d213b57fe74302e5902d68ed9478c3ad1aaa713/src/transformers/modeling_utils.py#L3966

• in test_resume_training_with_safe_checkpoint : The UnboundLocalError occurred because in the context manager _load_shard_into_memory, the finally block tried to delete state_dict before it was created due to probably an exception in the try block. Instead of fixing this directly in the context manager, I'd rather remove it since a direct load-delete pattern is simpler and achieves the same memory management goals.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Wauplin]

Looks reasonable :)

[hanouticelina]

🎉 tests are now passing in transformers : https://github.com/huggingface/transformers/runs/34374642187
(the failing tests seem unrelated).
I think we are good to merge and release :)

cc @LysandreJik