Attempt to retrieve manifest from local repo for manifest requests to proxy

[raphaelzoellner]

Comprehensive Summary of your change

Harbor Proxies will serve manifests located in the local repository if the digest of the local manifest matches the digest of the remote manifest.

Issue being fixed

Fixes #21122

Please indicate you've done the following:

• Well Written Title and Summary of the PR
• Label the PR as needed. "release-note/ignore-for-release, release-note/new-feature, release-note/update, release-note/enhancement, release-note/community, release-note/breaking-change, release-note/docs, release-note/infra, release-note/deprecation"
• Accepted the DCO. Commits without the DCO will delay acceptance.
• Made sure tests are passing and test coverage is added if needed.
• Considered the docs impact and opened a new docs issue or PR with docs changes if needed in website repository.

[codecov]

Codecov Report

Attention: Patch coverage is 55.76923% with 23 lines in your changes missing coverage. Please review.

Project coverage is 66.10%. Comparing base (c8c11b4) to head (dd9af57).
Report is 312 commits behind head on main.

Files with missing lines	Patch %	Lines
src/controller/proxy/controller.go	64.44%	14 Missing and 2 partials ⚠️
src/server/middleware/repoproxy/proxy.go	0.00%	5 Missing ⚠️
src/controller/proxy/local.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main   #21123       +/-   ##
===========================================
+ Coverage   45.36%   66.10%   +20.73%     
===========================================
  Files         244     1049      +805     
  Lines       13333   114679   +101346     
  Branches     2719     2867      +148     
===========================================
+ Hits         6049    75804    +69755     
- Misses       6983    34733    +27750     
- Partials      301     4142     +3841     

Flag	Coverage Δ
unittests	66.10% <55.76%> (+20.73%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/controller/proxy/local.go	35.89% <0.00%> (ø)
src/server/middleware/repoproxy/proxy.go	4.42% <0.00%> (ø)
src/controller/proxy/controller.go	28.70% <64.44%> (ø)

... and 1285 files with indirect coverage changes

[stonezdj]

what you changed in this part is already covered previous line 202

[raphaelzoellner]

Previous line 202 is the return of a manifest list if it could be fetched from the cache.

harbor/src/controller/proxy/controller.go

Line 202 in 62d1d93

return true, &ManifestList{content, string(desc.Digest), contentType}, nil

But since manifests are not cached (in redis) this logs a not found error in previous line 190 for GET .../manifests/{tag} requests referencing a tag.

harbor/src/controller/proxy/controller.go

Line 190 in 62d1d93

log.Debugf("Digest is not found in manifest list cache, key=cache:%v", manifestListKey(art.Repository, art))

and returns false (when a is nil, because the artifact with the matching digest in the local repository must not have an associated tag) and without a manifest in line 194.

harbor/src/controller/proxy/controller.go

Line 194 in 62d1d93

return a != nil && string(desc.Digest) == a.Digest, nil, nil

This then leads to the manifest not being served from the local repository

harbor/src/server/middleware/repoproxy/proxy.go

Lines 186 to 210 in 62d1d93

	if useLocal {
	if man != nil {
	w.Header().Set(contentLength, fmt.Sprintf("%v", len(man.Content)))
	w.Header().Set(contentType, man.ContentType)
	w.Header().Set(dockerContentDigest, man.Digest)
	w.Header().Set(etag, man.Digest)
	if r.Method == http.MethodGet {
	_, err = w.Write(man.Content)
	if err != nil {
	return err
	}
	}
	return nil
	}
	next.ServeHTTP(w, r)
	return nil
	}
	log.Debugf("the tag is %v, digest is %v", art.Tag, art.Digest)
	if r.Method == http.MethodHead {
	err = proxyManifestHead(ctx, w, proxyCtl, p, art, remote)
	} else if r.Method == http.MethodGet {
	log.Warningf("Artifact: %v:%v, digest:%v is not found in proxy cache, fetch it from remote repo", art.Repository, art.Tag, art.Digest)
	err = proxyManifestGet(ctx, w, proxyCtl, p, art, remote)
	}

My intention with this part in new lines 223-234 was to check prior to the manifest list cache if a manifest with the digest already exists in the local repository and if found return it.

Wouldn't this extend the current behavior for that case?

[raphaelzoellner]

Sorry, I'm new to harbor and I think my proposed implementation would skip other middlewares, which might be a problem.

harbor/src/server/registry/route.go

Lines 51 to 67 in 62d1d93

	// manifest
	root.NewRoute().
	Method(http.MethodGet).
	Path("/*/manifests/:reference").
	Middleware(metric.InjectOpIDMiddleware(metric.ManifestOperationID)).
	Middleware(repoproxy.ManifestMiddleware()).
	Middleware(contenttrust.ContentTrust()).
	Middleware(vulnerable.Middleware()).
	HandlerFunc(getManifest)
	root.NewRoute().
	Method(http.MethodHead).
	Path("/*/manifests/:reference").
	Middleware(metric.InjectOpIDMiddleware(metric.ManifestOperationID)).
	Middleware(repoproxy.ManifestMiddleware()).
	Middleware(contenttrust.ContentTrust()).
	Middleware(vulnerable.Middleware()).
	HandlerFunc(getManifest)

I will close this PR.

I think to solve this one would need to make sure that the initally proxied and cached artifact in the local repository will be referenced by the tag.

[stonezdj]

The manifestListKey is only cache the manfest List, not used to cache a normal manifest

[raphaelzoellner]

Sorry, updating this comment was a mistake. I will revert it.