[Blocked by #103|#76] Make ztoc creation (toc and zinfo) to accept io.Reader instead of filename

[djdongjin]

Is your feature request related to a problem? Please describe.

Currently when soci index creates ztoc (toc and zinfo), it passes a filename to accepts a filename to ztoc.BuildZtoc instead of io.Reader.

It might be good to let index creation create a io.Reader and pass to ztoc creation, which will use to create both toc and zinfo.

soci-snapshotter/ztoc/ztoc_builder.go

Line 30 in b3a62fe

func BuildZtoc(gzipFile string, span int64, buildToolIdentifier string) (*Ztoc, error) {

It also avoids creating a tmp file only for ztoc creation (specifically, required by gzip zinfo implmented in c)

Describe the solution you'd like

Change filename usage in index/ztoc creation to using io.Reader.

There is a potential blocker that compression/gzip_zinfo.go (the underlying c code) cannot handle io.Reader in c. See more discussion in #76 (specifically this relevant comment #72 (review))

Describe alternatives you've considered

Keep filename usage unchanged.

Additional context

It's not high priority, since we're also refactoring/decoupling ztoc/compression (#103). I think this change won't impact #103 and has lower priority than #103.

[Kern--]

I think it's worth highlighting as an acceptance criteria that after this change TocBuilder should receive an uncompressed tarball and have no knowledge of compression.

[djdongjin]

updated the description.

[djdongjin]

This one is blocked on:

#103: after ztoc/compression decouple, we'll have a better unstanding of the decoupled structure.
#76: we use filename string in zinfo c code. If we have c code, it seems unavoidable to use filename, since c/cgo doesn't support io.Reader.