Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] Fix workflows stable for ARM64 tests #2553

Merged
merged 2 commits into from
Jan 3, 2025
Merged

[ci] Fix workflows stable for ARM64 tests #2553

merged 2 commits into from
Jan 3, 2025

Conversation

VietND96
Copy link
Member

@VietND96 VietND96 commented Jan 3, 2025

User description

Thanks for contributing to the Docker-Selenium project!
A PR well described will help maintainers to quickly review and merge it

Before submitting your PR, please check our contributing guidelines, applied for this repository.
Avoid large PRs, help reviewers by making them as simple and short as possible.

Description

Motivation and Context

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • I have read the contributing document.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

PR Type

Bug fix, Tests


Description

  • Fixed Selenium test to include explicit wait for title.

  • Increased no_output_timeout to 60 minutes in CircleCI jobs.

  • Removed redundant conditional logic in Kubernetes test command.

  • Enhanced ARM64 test stability in CI workflows.


Changes walkthrough 📝

Relevant files
Tests
__init__.py
Added explicit wait in Selenium test                                         

tests/SeleniumTests/init.py

  • Added explicit wait for page title in test_title.
  • Improved test reliability by ensuring title is loaded.
  • +2/-0     
    Configuration changes
    config.yml
    Updated CircleCI configuration for ARM64 stability             

    .circleci/config.yml

  • Increased no_output_timeout to 60 minutes for Docker and Kubernetes
    tests.
  • Removed redundant conditional logic in Kubernetes test command.
  • Improved CI workflow stability for ARM64 tests.
  • +2/-5     

    💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

    Copy link

    qodo-merge-pro bot commented Jan 3, 2025

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
    🧪 PR contains tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Test Timeout

    The WebDriverWait timeout value is using a constant WEB_DRIVER_WAIT_TIMEOUT but its value is not shown in the diff. Verify the timeout is appropriate for slower CI environments.

    wait = WebDriverWait(self.driver, WEB_DRIVER_WAIT_TIMEOUT)
    wait.until(EC.title_is('The Internet'))

    Copy link

    qodo-merge-pro bot commented Jan 3, 2025

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Score
    General
    Add explicit error handling for timeout scenarios to improve test debugging

    Add error handling for the WebDriverWait timeout to provide more informative test
    failures. Currently, if the title doesn't appear within the timeout period, the
    error message won't be descriptive.

    tests/SeleniumTests/init.py [62-64]

    -wait = WebDriverWait(self.driver, WEB_DRIVER_WAIT_TIMEOUT)
    -wait.until(EC.title_is('The Internet'))
    -self.assertTrue(self.driver.title == 'The Internet')
    +try:
    +    wait = WebDriverWait(self.driver, WEB_DRIVER_WAIT_TIMEOUT)
    +    wait.until(EC.title_is('The Internet'))
    +    self.assertTrue(self.driver.title == 'The Internet')
    +except TimeoutException:
    +    self.fail(f"Page title 'The Internet' not found after {WEB_DRIVER_WAIT_TIMEOUT} seconds. Current title: {self.driver.title}")
    • Apply this suggestion
    Suggestion importance[1-10]: 8

    Why: Adding explicit timeout handling with a descriptive error message would significantly improve test debugging and maintenance. The current code would fail with a generic timeout exception, making it harder to diagnose issues.

    8
    Possible issue
    Implement a timeout mechanism for test retries to prevent indefinite execution

    Add a timeout check in the while loop to prevent infinite execution if tests
    consistently fail. Currently, the loop will keep retrying without a maximum time
    limit.

    .circleci/config.yml [315-318]

    +start_time=$(date +%s)
    +timeout=3600  # 1 hour timeout
     N=3
     while [ $N -gt 0 ]; do
    +  current_time=$(date +%s)
    +  if [ $((current_time - start_time)) -gt $timeout ]; then
    +    echo "Test execution exceeded timeout of 1 hour"
    +    exit 1
    +  fi
       output=$(eval "PLATFORMS=${PLATFORMS} NAME=${IMAGE_REGISTRY} VERSION=${BRANCH} BUILD_DATE=${BUILD_DATE} TEST_UPGRADE_CHART=false make chart_test_autoscaling_${TEST_STRATEGY} && make test_video_integrity")
       status=$?
    • Apply this suggestion
    Suggestion importance[1-10]: 5

    Why: While adding a timeout check could prevent hanging tests, it's less critical since CircleCI already has a 60-minute timeout configured via 'no_output_timeout'. The suggestion provides a redundant safety measure.

    5

    @VietND96 VietND96 force-pushed the ci-arm64 branch 2 times, most recently from cc2c034 to 5bc1328 Compare January 3, 2025 05:51
    Copy link

    qodo-merge-pro bot commented Jan 3, 2025

    CI Failure Feedback 🧐

    (Checks updated until commit 32996d3)

    Action: Rerun workflow when failure

    Failed stage: Authenticate GitHub CLI for PR [❌]

    Failure summary:

    The action failed because the GitHub authentication token lacks the required 'read:org' permission
    scope. The error occurred during the gh auth login command, which requires proper authorization to
    access organizational resources.

    Relevant error logs:
    1:  ##[group]Operating System
    2:  Ubuntu
    ...
    
    28:  SecurityEvents: write
    29:  Statuses: write
    30:  ##[endgroup]
    31:  Secret source: Actions
    32:  Prepare workflow directory
    33:  Prepare all required actions
    34:  Getting action download info
    35:  Download action repository 'actions/checkout@main' (SHA:cbb722410c2e876e24abbe8de2cc27693e501dcb)
    36:  Complete job name: Rerun workflow when failure
    ...
    
    48:  show-progress: true
    49:  lfs: false
    50:  submodules: false
    51:  set-safe-directory: true
    52:  env:
    53:  GH_CLI_TOKEN: ***
    54:  GH_CLI_TOKEN_PR: ***
    55:  RUN_ID: 12601251314
    56:  RERUN_FAILED_ONLY: true
    ...
    
    119:  ##[group]Run sudo apt update
    120:  �[36;1msudo apt update�[0m
    121:  �[36;1msudo apt install gh�[0m
    122:  shell: /usr/bin/bash -e {0}
    123:  env:
    124:  GH_CLI_TOKEN: ***
    125:  GH_CLI_TOKEN_PR: ***
    126:  RUN_ID: 12601251314
    127:  RERUN_FAILED_ONLY: true
    ...
    
    164:  0 upgraded, 0 newly installed, 0 to remove and 48 not upgraded.
    165:  ##[group]Run echo "$GH_CLI_TOKEN_PR" | gh auth login --with-token
    166:  �[36;1mecho "$GH_CLI_TOKEN_PR" | gh auth login --with-token�[0m
    167:  shell: /usr/bin/bash -e {0}
    168:  env:
    169:  GH_CLI_TOKEN: ***
    170:  GH_CLI_TOKEN_PR: ***
    171:  RUN_ID: 12601251314
    172:  RERUN_FAILED_ONLY: true
    173:  RUN_ATTEMPT: 1
    174:  ##[endgroup]
    175:  error validating token: missing required scope 'read:org'
    176:  ##[error]Process completed with exit code 1.
    

    ✨ CI feedback usage guide:

    The CI feedback tool (/checks) automatically triggers when a PR has a failed check.
    The tool analyzes the failed checks and provides several feedbacks:

    • Failed stage
    • Failed test name
    • Failure summary
    • Relevant error logs

    In addition to being automatically triggered, the tool can also be invoked manually by commenting on a PR:

    /checks "https://github.com/{repo_name}/actions/runs/{run_number}/job/{job_number}"
    

    where {repo_name} is the name of the repository, {run_number} is the run number of the failed check, and {job_number} is the job number of the failed check.

    Configuration options

    • enable_auto_checks_feedback - if set to true, the tool will automatically provide feedback when a check is failed. Default is true.
    • excluded_checks_list - a list of checks to exclude from the feedback, for example: ["check1", "check2"]. Default is an empty list.
    • enable_help_text - if set to true, the tool will provide a help message with the feedback. Default is true.
    • persistent_comment - if set to true, the tool will overwrite a previous checks comment with the new feedback. Default is true.
    • final_update_message - if persistent_comment is true and updating a previous checks message, the tool will also create a new message: "Persistent checks updated to latest commit". Default is true.

    See more information about the checks tool in the docs.

    @VietND96 VietND96 force-pushed the ci-arm64 branch 7 times, most recently from 8423ab4 to 31a5b54 Compare January 3, 2025 16:20
    Signed-off-by: Viet Nguyen Duc <nguyenducviet4496@gmail.com>
    @VietND96 VietND96 merged commit 710783e into trunk Jan 3, 2025
    17 of 26 checks passed
    @VietND96 VietND96 deleted the ci-arm64 branch January 3, 2025 19:45
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    1 participant