FLUX memory management improvements

[RyanJDick]

Summary

This PR contains several improvements to memory management for FLUX workflows.

It is now possible to achieve better FLUX model caching performance, but this still requires users to manually configure their ram/vram settings. E.g. a vram setting of 16.0 should allow for all quantized FLUX models to be kept in memory on the GPU.

Changes:

• Check the size of a model on disk and free the requisite space in the model cache before loading it. (This behaviour existed previously, but was removed in https://github.com/invoke-ai/InvokeAI/pull/6072/files. The removal did not seem to be intentional).
• Removed the hack to free 24GB of space in the cache before loading the FLUX model.
• Split the T5 embedding and CLIP embedding steps into separate functions so that the two models don't both have to be held in RAM at the same time.
• Fix a bug in InvokeLinear8bitLt that was causing some tensors to be left on the GPU when the model was offloaded to the CPU. (This class is getting very messy due to the non-standard state_dict handling in bnb.nn.Linear8bitLt. )
• Tidy up some dtype handling in FluxTextToImageInvocation to avoid situations where we hold references to two copies of the same tensor unnecessarily.
• (minor) Misc cleanup of ModelCache: improve docs and remove unused vars.

Future:
We should revisit our default ram/vram configs. The current defaults are very conservative, and users could see major performance improvements from tuning these values.

QA Instructions

I tested the FLUX workflow with the following configurations and verified that the cache hit rates and memory usage matched the expected behaviour:

• ram = 16 and vram = 16
• ram = 16 and vram = 1
• ram = 1 and vram = 1

Note that the changes in this PR are not isolated to FLUX. Since we now check the size of models on disk, we may see slight changes in model cache offload patterns for other models as well.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• Documentation added / updated (if applicable)

[brandonrising]

Lgtm