[AOTI] Remove the original model weights in Python deployment

[desertfire]

Summary: Fixes #1302. Because AOTI-compiled model contains a copy of model weights, if we also keep the eager model weights, it will be a wast of GPU memory and even triggers OOMs. This PR releases the corresponding eager model weights in the AOTI-Python deployment path.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1337

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 978baa3 with merge base 9480258 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[desertfire]

@Jack-Khuu , is there any known problem with CI? Many of the failures here seem unrelated to my change.

[Jack-Khuu]

Yeah, we're checking with the DevInfra folk

[Jack-Khuu]

Heads up CI is broken in pt/pt and at higher level

https://github.com/NVIDIA/nvidia-container-toolkit/releases/tag/v1.17.0

[byjlw]

@desertfire it should be fixed now. We just need to churn through the tests

[mikekgfb]

Summary: Fixes #1302. Because AOTI-compiled model contains a copy of model weights, if we also keep the eager model weights, it will be a waste of GPU memory and even triggers OOMs. This PR releases the corresponding eager model weights in the AOTI-Python deployment path.

In the long term we might ask whether there's a point in going to all the expense of building the model if we just need the config. The model build process had a config_only bool argument with the intent of allowing suppression of actual model build (that was never implemented though).

This is particularly noteworthy (aka expensive because GGUF requires much more than weight mmaps) when building a model from GGUF, as @metascroy pointed out a long time ago when he implemented the GGUF reader.