Skip to content
This repository has been archived by the owner on Aug 14, 2020. It is now read-only.

Annotate manifest hash #237

Merged
merged 2 commits into from
Jan 26, 2017
Merged

Annotate manifest hash #237

merged 2 commits into from
Jan 26, 2017

Conversation

cgonyeo
Copy link
Member

@cgonyeo cgonyeo commented Jan 20, 2017

There are two commits in here. In order they:

  1. Add an annotation to produced ACIs with a hash of the original docker manifest (or sometimes the app ID depending on the version of the HTTP API used)
  2. tweaks the library API so that callers can provide a list of manifest hashes they already have

These two changes should allow for rkt to provide a list of docker manifest hashes (obtained by ACI manifest annotations) for images it already has but wants docker2aci to check for an update for. If docker2aci pulls the manifest for an image and it's in the list of images already owned by the caller, it doesn't have to download the image.

This is part of the work for rkt/rkt#2937

@cgonyeo cgonyeo force-pushed the annotate-manifest-hash branch from 29f2b5d to f461ffd Compare January 21, 2017 00:25
@cgonyeo
Copy link
Member Author

cgonyeo commented Jan 21, 2017

Some test is unhappy, I'll fix it on Monday

@@ -53,15 +53,15 @@ func NewFileBackend(file *os.File, debug, info log.Logger) *FileBackend {
}
}

func (lb *FileBackend) GetImageInfo(dockerURL string) ([]string, *common.ParsedDockerURL, error) {
func (lb *FileBackend) GetImageInfo(dockerURL string) ([]string, string, *common.ParsedDockerURL, error) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this return signature is getting messy, it is not obvious what []string, string is. Can we document or name the return values?

@@ -53,13 +53,13 @@ import (
// BuildACI takes a Docker layer, converts it to ACI and returns its output
// path and its converted ImageManifest.
type Docker2ACIBackend interface {
GetImageInfo(dockerUrl string) ([]string, *common.ParsedDockerURL, error)
BuildACI(layerIDs []string, dockerURL *common.ParsedDockerURL, outputDir string, tmpBaseDir string, compression common.Compression) ([]string, []*schema.ImageManifest, error)
GetImageInfo(dockerUrl string) ([]string, string, *common.ParsedDockerURL, error)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment applies here as above. can we document what []string, string are?

@@ -346,7 +347,7 @@ func GenerateEmptyManifest(name string) (*schema.ImageManifest, error) {
}, nil
}

func GenerateManifestV22(name string, dockerURL *common.ParsedDockerURL, config *typesV2.ImageConfig, imageDigest string, lowerLayers []*schema.ImageManifest, debug log.Logger) (*schema.ImageManifest, error) {
func GenerateManifestV22(name, manhash string, dockerURL *common.ParsedDockerURL, config *typesV2.ImageConfig, imageDigest string, lowerLayers []*schema.ImageManifest, debug log.Logger) (*schema.ImageManifest, error) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

woah, this is getting immense ;-) at least we could make this a newline separated arglist and squash imageDigest with name, manhash. Also some documentation would welp. I am becoming concerned about the size of this arglist.

@cgonyeo cgonyeo force-pushed the annotate-manifest-hash branch 2 times, most recently from 6676468 to 651980b Compare January 23, 2017 21:56
@cgonyeo
Copy link
Member Author

cgonyeo commented Jan 23, 2017

Fixed the test, and added some comments to hopefully address the nits.

@euank
Copy link
Contributor

euank commented Jan 23, 2017

I'd prefer using the upstream 'sha256' hash if we can since it'll make things a bit nicer for the user to read.

It also has one small benefit! If the user does rkt fetch docker://foo@sha256:.... and we annotated that, we can know that we don't need to do any network traffic at all.

@cgonyeo
Copy link
Member Author

cgonyeo commented Jan 23, 2017

I agree with @euank, so please don't merge this until I can update that logic. I've got the rkt PR based on this ready though, so as soon as this goes in the rest of rkt/rkt#2937 will hopefully go pretty quickly.

@cgonyeo cgonyeo force-pushed the annotate-manifest-hash branch from 651980b to 8de20ca Compare January 24, 2017 21:09
@cgonyeo
Copy link
Member Author

cgonyeo commented Jan 24, 2017

This now should have the behavior I want, but #238 should be merged first so I don't have to include that commit in this PR.

Derek Gonyeo added 2 commits January 25, 2017 10:34
This commit notes the sha256 hash of manifests when fetching images for
v2.1 or v2.2, and records either this or the app ID in an annotation in
the manifest of the produced ACI.
This commit allows a user of the library to provide a list of manifest
hashes, presumably generated from annotations left on images by prior
invocations of docker2aci, for images it already has. If docker2aci
fetches a manifest and the hash for it matches one of the elements in
the user provided list, docker2aci will stop working, and return with no
error and no produced ACIs.
@cgonyeo cgonyeo force-pushed the annotate-manifest-hash branch from 8de20ca to 065228f Compare January 25, 2017 18:34
@cgonyeo
Copy link
Member Author

cgonyeo commented Jan 25, 2017

Since @lucab helped me with vendoring, I just removed the vendoring commits and re-pushed this.

@cgonyeo
Copy link
Member Author

cgonyeo commented Jan 26, 2017

Bump on this, I have a PR for rkt once this is done :)

Copy link

@s-urbaniak s-urbaniak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cgonyeo cgonyeo merged commit 1f3b030 into appc:master Jan 26, 2017
@cgonyeo cgonyeo deleted the annotate-manifest-hash branch January 26, 2017 18:17
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants