Layer.Digest() takes an absurdly long time?

[nkubala]

I have some code that retrieves the layers of an image and unpacks them individually on disk. Running this on a normal image takes ~2 seconds:

for _, layer := range layers {
    contents, _ := layer.Uncompressed()
    return unpackTar(tar.NewReader(contents))
}

INFO[0000] getting image for name daemon://gcr.io/google-appengine/python:latest 
INFO[0010] time elapsed retrieving image from daemon: 10.660759s 
INFO[0011] time elapsed retrieving layer: 0.349497s     
INFO[0011] time elapsed retrieving layer: 0.012715s     
INFO[0011] time elapsed retrieving layer: 0.012462s     
INFO[0011] time elapsed retrieving layer: 0.655020s     
INFO[0011] time elapsed retrieving layer: 0.251492s     
INFO[0012] time elapsed retrieving layer: 0.179278s     
INFO[0012] time elapsed retrieving layer: 0.199690s     
INFO[0012] time elapsed retrieving layer: 0.015826s     
INFO[0012] time elapsed retrieving layer: 0.015350s     
INFO[0012] time elapsed retrieving layer: 0.149267s     
INFO[0012] time elapsed retrieving layer: 0.014786s     
INFO[0012] time elapsed retrieving layer: 0.068905s     
INFO[0012] time elapsed retrieving image layers: 1.926415s 

However, simply examining each layer's digest before unpacking it causes this to run 100x (!) slower:

for _, layer := range layers {
    contents, _ := layer.Uncompressed()
    digest, _ := layer.Digest()
    logrus.Infof("layer digest: %s", digest.String())
    return unpackTar(tar.NewReader(contents))
}

INFO[0000] getting image for name daemon://gcr.io/google-appengine/python:latest 
INFO[0009] time elapsed retrieving image from daemon: 9.474433s 
INFO[0028] layer digest: sha256:38bb3b20b49df09debb87f05c759576dad591fcad33abf98946807768fc104c1 
INFO[0028] time elapsed retrieving layer: 19.039635s    
INFO[0028] layer digest: sha256:73d61ed8939d064acd93f76542fe93dad812592b7765f9ab0ba9debfaf011e33 
INFO[0028] time elapsed retrieving layer: 0.029642s     
INFO[0028] layer digest: sha256:c7d2240f3922a3220514e7948a0741c37a5cc4f9261c19c659510f5d6625a040 
INFO[0028] time elapsed retrieving layer: 0.029149s     
INFO[0105] layer digest: sha256:806d4a514886c9371f85c5853301fc8244eca185b18c58eda8f43ecf4e32f5d6 
INFO[0106] time elapsed retrieving layer: 77.511757s    
INFO[0134] layer digest: sha256:b4212b01e8417c41203b21657d441f5d8067cd805c6806758933c1b49cd49a7c 
INFO[0135] time elapsed retrieving layer: 29.029258s    
INFO[0168] layer digest: sha256:4d65a203c7ef29d305e9581ba49660997dd27f28c55a87f328a093adb8119528 
INFO[0168] time elapsed retrieving layer: 33.135218s    
INFO[0203] layer digest: sha256:d44c17d01fec073c16faaed07f6a2766cee85318af95c04ee02fb8d4e423e434 
INFO[0203] time elapsed retrieving layer: 35.679781s    
INFO[0203] layer digest: sha256:7afb00c46634e31816767136e542ace4dc39716a33678d973365757f0ee36a8a 
INFO[0203] time elapsed retrieving layer: 0.030645s     
INFO[0203] layer digest: sha256:85ad11bf9955eac62feab20e9389cfdb59e0ca001f04829fd24c76bec622254e 
INFO[0203] time elapsed retrieving layer: 0.031065s     
INFO[0208] layer digest: sha256:ae958273090762cfd4600654a8c272a7452353e7952bdaa4a9dc29be24688241 
INFO[0208] time elapsed retrieving layer: 4.696723s     
INFO[0208] layer digest: sha256:e852b61d2cf646dd19564cd0da014049e85e50e34416261a0659d46979884edb 
INFO[0208] time elapsed retrieving layer: 0.029123s     
INFO[0208] layer digest: sha256:0cf65f49aa89d82cfda8137b5368015d483c5eba15866c55681aa6396c31ab27 
INFO[0208] time elapsed retrieving layer: 0.121465s     
INFO[0208] time elapsed retrieving image layers: 199.365291s 

[nkubala]

An update: this is happening because Daemon stores the layers uncompressed (as they are in the original image), so computing the digest requires each layer to be compressed first, and then read again. Still seems like compressing a layer shouldn't take 77 seconds, but I'll close this for now.

[jonjohnsonjr]

This uses the gzip.BestCompression level for compression:
https://golang.org/pkg/compress/gzip/#pkg-constants
https://github.com/google/go-containerregistry/blob/master/v1/v1util/zip.go#L34

We should probably make that a parameter and benchmark how they affect certain ops.