Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvement/build cicd #406

Merged
merged 5 commits into from
Jan 22, 2025
Merged

Conversation

maslyankov
Copy link
Contributor

@maslyankov maslyankov commented Jan 22, 2025

Rate Limiting and Workflow Improvements

This PR addresses the "429 Too Many Requests" errors encountered during image manifest creation and improves the overall reliability of the deployment workflow.

Key Changes

1. Sequential Manifest Processing

  • Create base manifest first, then process platform-specific manifests one at a time
  • Added 30-second delays between manifest operations to respect rate limits
  • Clear logging of each step in the process

2. Enhanced Retry Mechanism

  • Implemented exponential backoff starting at 10 seconds
  • Maximum of 5 retry attempts per operation
  • Added descriptive logging for retry attempts and failures

3. Improved Git Operations

  • Added proper Git configuration for GitHub Actions bot
  • Implemented retry mechanism for Git push operations
  • Added rebase strategy for handling concurrent updates

4. Better Visibility

  • Added descriptive echo statements before sleep commands
  • Improved logging throughout the workflow
  • Clear status messages for each operation

Testing

The changes have been tested with multiple platforms (amd64, arm64, armv7, armv6) and handle rate limiting gracefully.

Impact

  • More reliable manifest creation and pushing
  • Better handling of GitHub API rate limits
  • Clearer visibility into workflow progress
  • More resilient to concurrent updates

Related issues

- Updated artifact naming to include sanitized platform name and added error handling for missing files.
- Changed the download artifact step to use a pattern for better flexibility.
- Configured Git settings for the action and implemented a rebase strategy before committing version updates.
- Added logic to conditionally update the version for the multi platform based on the environment.
- Improved push logic with retry mechanism for better reliability.

This update streamlines the deployment process and ensures version consistency across platforms.
- Updated job conditions to improve handling of workflow events, specifically for 'workflow_run' scenarios.
- Enhanced failure handling in the CI job by explicitly exiting with an error code.
- Adjusted the 'information' job to trigger on additional event types, ensuring better integration with release processes.
- Streamlined conditions for the 'publish_addon' job to ensure it only runs when all dependencies succeed.

These changes enhance the robustness and reliability of the deployment process.
- Introduced a retry mechanism with exponential backoff for the manifest creation and image inspection steps to improve reliability.
- Implemented batch processing for manifest creation to handle digests in smaller groups, reducing the risk of failures during the process.
- Added logging for batch processing to provide better visibility into the workflow execution.

These changes enhance the robustness of the deployment process, ensuring smoother multi-platform deployments.
- Added logging statements to indicate wait periods during manifest creation and image inspection steps.
- Introduced consistent wait times (30 seconds after manifest creation, 10 seconds between platform inspections, and 5 seconds before retrying pushes) to improve stability and reliability of the deployment process.

These changes aim to provide better visibility into the workflow execution and ensure smoother multi-platform deployments.
@kellerza kellerza merged commit 02f085f into kellerza:main Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants