Metadata: Retry dockerhub fetch on metadata post publish fail

[bnchrch]

Problem

The dockerhub api can be stale and not return an image even though its published

From this slack discussion
https://airbytehq-team.slack.com/archives/C03VDJ4FMJB/p1700182656932749

Solution

Metadata validation will retry up to 3 times 30 seconds apart to find the image

Future

We may be able to rework this to work off the local instance of docker which downloads these images much faster

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)	Visit Preview		Nov 24, 2023 0:50am

[alafanechere]

Nice improvement 👍
I'd appreciate:

• Not swallowing DockerHub API errors by being more specific about the status code we handle (404)
  *A unit test of is_image_on_dockerhubto prevent future regression

[alafanechere]

Can we retry on 404 only and raise if the response status is not 200 or 404? The current implementation will swallow DockerHub API errors and end up wit the "Image does not exists" error message, even if the failure is not related to the existence/absence of the image.

try:
  response = requests.get(tag_url, headers=headers)
  response.raise_for_status()
  if response.ok:
    break
except requests.HTTPError as e:
  if e.status_code == 404:
    continue
  raise e

[bnchrch]

Im worried that dockerhub during intermittent issues is not throwing just a 404. But could also throw a 500 or a 401.

Is that a valid fear?

also wdyt about the retry then an the end if still failing raising an error with the error message so we dont swallow it?