[Request] Preview model disk size usage

[czlr]

Request: As a casual user without much knowledge in LLMs, it would be nice to know upfront how much disk space the models need.

Currently: The various posts and docs only mention that llama 3.1 comes in different variants: 8B, 70B, 405B; regular vs instruct; etc. But they don't mention things like required disk size usage, or minimum system specs to run things smoothly. (Unless I accidentally missed those details)
I can only start a download blindly and monitor while it's going. The 70B model also seems to come in 17GB .pth parts -- I can't see how many parts are remaining either.

[DAOZHENREN]

For the 70B model, it may take 131GB

[DAOZHENREN]

And 15GB for the 8B model. The regular and instruct are equal in size, but the latter is finetuned toward chat and is safer(less toxic).

[vonpetersenn]

I have a related question: if the model has 70B parameters, wouldn't we expect the size of the model to be 70*4=280 GB?
A Float32 parameter takes up 4 bytes of storage. Are the parameters of LLaMA Float16-numbers?

[DAOZHENREN]

I think that you have misunderstood the 'B' in the 70B. Actually, it means billion.

[ashwinb]

@vonpetersenn Yes they are float16 -- specifically "bf16" (Brain Float 16) format.

[vonpetersenn]

thank you very much!