Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Image Syncer Job as GitHub Action #11384

Closed
3 tasks done
Tracked by #11627
akiioto opened this issue Jul 18, 2024 · 3 comments
Closed
3 tasks done
Tracked by #11627

Implement Image Syncer Job as GitHub Action #11384

akiioto opened this issue Jul 18, 2024 · 3 comments
Assignees
Labels
area/ci Issues or PRs related to CI related topics

Comments

@akiioto
Copy link
Contributor

akiioto commented Jul 18, 2024

Description

Migrate the image-syncer job from Prow to GitHub Actions by creating a reusable workflow. This migration is necessary to address the issue of exceeding DockerHub's 100 requests per hour limit and to decentralize the maintenance of image synchronization. Each team will manage their own external-images.yaml file. Detailed instructions will be provided to ensure a smooth transition. The reusable workflow will run on GitHub Actions, leveraging different IP addresses to mitigate rate limit issues.

The reusable workflow enables team to use their own secret to access Docker Hub. If no secret is provided, the anonymous download will be used. The credentials to the registry have to be restricted to the reusable workflow and not be shared with the teams.

Reasons

  1. Decentralized Maintenance: Allowing each team to manage their own configuration file and credentials reduces the central team's burden and increases autonomy for teams.
  2. Scalability: GitHub Actions runners use different IP addresses, which can help distribute the load and avoid rate limits.
  3. Efficiency: Teams can approve their own repository changes, streamlining the process and reducing review times for the central team.

Postponed, as probably dockerhub will not be an issue with GitHub as a runner.
Required to do performance test and decide
Rate Limit Issue: The current Prow-based solution frequently hits DockerHub's 100 requests per hour limit, causing failures.

Acceptance Criteria

  • A reusable GitHub Actions workflow is created and available for teams to use.
  • Comprehensive documentation is provided for teams to integrate the workflow into their repositories.
  • The workflow is verified to work correctly, syncing images as expected.

Attachments

@akiioto akiioto added the area/ci Issues or PRs related to CI related topics label Jul 18, 2024
@KacperMalachowski
Copy link
Contributor

KacperMalachowski commented Aug 9, 2024

The're is a agreement between Docker and GIthub: actions/runner-images#1445 (comment)

Agreement is already made: actions/runner-images#1445 (comment)

@TorstenD-SAP TorstenD-SAP changed the title Migrate Image Syncer Job from Prow to GitHub Actions Implement Image Syncer Job as GitHub Action Aug 9, 2024
@dekiel dekiel self-assigned this Aug 16, 2024
@dekiel
Copy link
Contributor

dekiel commented Aug 26, 2024

We've made a decision to define a common location for .yml file with definition of images to sync. The users will not have a way to define their own location. Having files in the same location in every repository using it, will let us collect information for these files or react on changes in the list of images to sync. Implementing such change in the future would require all teams align their tooling and file to common location.

@dekiel
Copy link
Contributor

dekiel commented Aug 27, 2024

An amd64Only flag in image-syncer does not prevent syncing image index or image manifest for different architecture. At present a flag just enable syncing image manifest if a source image reference points to it. @Sawthis would like to remove this flag. This will be planned in other issue as this is not a blocker for migration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ci Issues or PRs related to CI related topics
Projects
None yet
Development

No branches or pull requests

3 participants