Prototype changes to create fake checkpoints with empty storages

[mikaylagawarecki]

New version of #128162, opening just to show dependencies to prevent colab OOM in pytorch/torchtune#1315

with some changes to FakeTensor __reduce_ex__ (or a subclass of it, TBD how we'll actually do this) such as those here, we can do things like

with FakeTensorMode():
    m = nn.Linear(3, 5, dtype=torch.float16, device='cuda')

sd = m.state_dict()
for v in sd.values():
    v.untyped_storage()._serialize_data = False
    
torch.save(sd, 'bla.pt')
print(torch.load('bla.pt', weights_only=True))
# OrderedDict([('weight', tensor([[0., 0., 0.],
#        [0., 0., 0.],
#        [0., 0., 0.],
#        [0., 0., 0.],
#        [0., 0., 0.]], device='cuda:0', dtype=torch.float16)), ('bias', tensor([0., 0., 0., 0., 0.], device='cuda:0', dtype=torch.float16))])

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/133272

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit e5a7330 with merge base 5b7b3e4 ():

NEW FAILURE - The following job has failed:

• pull / cuda12.1-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) (gh)
  inductor/test_aot_inductor.py::AOTInductorTestABICompatibleCuda::test_conv_freezing_abi_compatible_cuda

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.