Discuss about how to check mirror registry health

[changweige]

Nydus has merged two features about deeply integrating nydusd with Dragonfly

• use Dragonfly as a pull-through cache a.k.a mirror registry
• work with Dragonfly as HTTP proxy useHTTPS accessing HTTPS registry with the capability to fallback to the original HTTPS registry.

For Dragonfly mirror mode, nydusd will continue to pull blob data from the next mirror if the current mirror fails, but there is no mechanism to recover access to the previously failed mirror.

To solve this, I can think of two methods:

1. add a health check endpoint to dragonfly
2. periodically get repo tags. If nydusd can successfully get repo tags from the failed mirror, it is treated recovered.

[changweige]

Referring to this discussion, it seems that there is a standard HTTP endpoint in distribution spec to check registry health
distribution/distribution#629

[changweige]

This might be the correct way

A properly secured registry should return 401 when the "/v2/" endpoint is hit
without credentials. The response should include a WWW-Authenticate
challenge, providing guidance on how to authenticate, such as with basic auth
or a token service. If the load balancer has health checks, it is recommended
to configure it to consider a 401 response as healthy and any others as down.
This will secure your registry by ensuring that configuration problems with
authentication don't accidentally expose an unprotected registry. If you're
using a less sophisticated load balancer, such as Amazon's Elastic Load
Balancer, that doesn't allow one to change the healthy response code, health
checks can be directed at "/", which will always return a 200 OK response.

[imeoer]

This might be the correct way

Should also ensure that all registry providers (harbor, ghcr, docker hub, acr...) have implemented the same behavior.

[changweige]

This might be the correct way

Should also ensure that all registry providers (harbor, ghcr, docker hub, acr...) have implemented the same behavior.

It comes from OCI distribution spec, and looks very reliable.https://github.com/distribution/distribution/blob/main/docs/deploying.md#importantrequired-http-headers

[changweige]

The checker code has been merged.