Load model from archive

[ekcasey]

Summary

• Supports loading archived models in the local store via CLI tool or distribution.Client.
• Support packaging models to archive file via CLI tool or tarball.Target

CLI Usage

model-distribution-tool load

Introduces load command which loads model from the archive at into the store and optionally applies .

./bin/model-distribution-tool load [--tag <tag>] <path>`

model-distribution-tool package --file

Adds --file <path> flag to package command, which results in the model being written to a TAR archive at the given <path> instead of being pushed to the registry.

> ./bin/model-distribution-tool package --file /tmp/model.tar ./assets/dummy.gguf

LoadModel with distribution.Client

func (c *Client) LoadModel(tag string, rc io.ReadCloser, progressWriter io.Writer) error

Expects to read a TAR archive from rc formatted as described below and applies the given tag, writing progress to progressWriter.

Format

package and load expect/produce models with any number of blobs entries with name blobs/sha256/<hash> and 1 manifest.json. Example:

> tar tfv /tmp/model.tar
d---------  0 0      0           0 Dec 31  1969 blobs
d---------  0 0      0           0 Dec 31  1969 blobs/sha256
-rw-rw-rw-  0 0      0        2016 Dec 31  1969 blobs/sha256/c7790a0a70161f1bfd441cf157313e9efb8fcd1f0831193101def035ead23b32
-rw-rw-rw-  0 0      0         851 Dec 31  1969 blobs/sha256/f1017b51ac729af1cc57f8907347b1308678c8b7ad2ca29ad0098fccfdfa17b8
-rw-rw-rw-  0 0      0         400 Dec 31  1969 manifest.json

TODO

• report load progress
• update README

[doringeman]

Let's not crash on cancellation.
I hit this by cancelling a docker model package with a large GGUF.

Suggested change

	log.Fatalf("Error reading blobs from stream: %v", err)
	if strings.Contains(err.Error(), "unexpected EOF") {
	c.log.Infof("Model load interrupted (likely cancelled): %v", err)
	return "", fmt.Errorf("model load interrupted: %w", err)
	}
	return "", fmt.Errorf("reading blob from stream: %w", err)```

[xenoscopic]

Looks good, but would errors.Is(err, io.ErrUnexpectedEOF) work here?

[doringeman]

Sure! Changed in e3f196c.
Also tested locally with with model-runner, all good.

[xenoscopic]

LGTM, though I agree we should try to find an alternative to the Fatalf before merging.