Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dashboard state not updating #48

Closed
prescottprue opened this issue Dec 10, 2019 · 7 comments · Fixed by #101
Closed

Dashboard state not updating #48

prescottprue opened this issue Dec 10, 2019 · 7 comments · Fixed by #101

Comments

@prescottprue
Copy link
Contributor

In the following cases the dashboard remains in the "running" state (as if it was never notified):

This results in test runs being indicated as taking 2+ hours within the dashboard even when the Github Action itself is done in less than 10 minutes (it shuts off due to the cypress command shutting down due to the failed test)

@bahmutov
Copy link
Contributor

I am not sure I understand the situation - do you have public action to show this? The test fails, but the Cypress does not post completion to the Dashboard? Or does the Cypress Test Runner crash and leaves the job hanging?

@prescottprue
Copy link
Contributor Author

prescottprue commented Jan 2, 2020

I don't have a public action for this yet, but will work on making one. More the first - Cypress does not post completion in the failure case. Also, during a run if "Cancel Check Suite" is clicked within actions, Cypress does not post completion to the Dashboard.

@prescottprue
Copy link
Contributor Author

prescottprue commented Feb 24, 2020

Just found that there is an option in Github Actions called fail-fast which controls builds in the matrix failing if another build in the matrix fails - I have found that setting this to false (i.e. fail-fast: false) fixes the first bullet above (i.e. if a test fails the dashboard does not continue indicating running since the rest of the tests run).

@bahmutov We may want to add that to the README as the default setup since I don't believe that it would be common to want all matrix builds to fail in the case of one failing (which happens from a test failure). As mentioned, this prevents the dashboard from hanging in the "in progress" state. Although leaving the default setting of true can be mentioned as a way to keep from using as many runner minutes

@github-actions
Copy link

github-actions bot commented Mar 3, 2020

🎉 This issue has been resolved in version 1.23.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

@jamime
Copy link

jamime commented May 7, 2020

@bahmutov @prescottprue

The fail-fast approach is causing an issue for me.

I have a project that uses 20 runners (our maximum concurrency). When we receive a pr from a fork it won't receive the cypress dashboard secret token so it will be limited to 1 runner but the remaining 19 runners remain occupied. Our suite takes ~3 hours with one runner.

I tried to change the matrix size if the PR was raised from a fork, but it does not seem possible.

Is there another way that we could resolve the dashboard hanging instead of using fail-fast?

Update
I did find a workaround. However, I still feel like this should be handled without fail-fast: false because on non-forks the runners that finish early will still be kept alive incurring additional cost.

@dgreene1
Copy link

dgreene1 commented Feb 6, 2021

Should this issue still be referenced in the readme if the issue is truly solved and closed?

76660486niuyigmailcom added a commit to 76660486niuyigmailcom/wa- that referenced this issue Oct 5, 2023
name: Cypress Tests
on: [push]
jobs:
  cypress-run:
    runs-on: ubuntu-latest
    # Runs tests in parallel with matrix strategy https://docs.cypress.io/guides/guides/parallelization
    # https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs
    # Also see warning here https://github.com/cypress-io/github-action#parallel
    strategy:
      fail-fast: false # cypress-io/github-action#48
      matrix:
        containers: [1, 2] # Uses 2 parallel instances
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Cypress run
        # Uses the official Cypress GitHub action https://github.com/cypress-io/github-action
        uses: cypress-io/github-action@v6
        with:
          # Starts web server for E2E tests - replace with your own server invocation
          # https://docs.cypress.io/guides/continuous-integration/introduction#Boot-your-server
          start: npm start
          wait-on: 'http://localhost:3000' # Waits for above
          # Records to Cypress Cloud 
          # https://docs.cypress.io/guides/cloud/projects#Set-up-a-project-to-record
          record: true
          parallel: true # Runs test in parallel using settings above
        env:
          # For recording and parallelization to work you must set your CYPRESS_RECORD_KEY
          # in GitHub repo → Settings → Secrets → Actions
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
          # Creating a token https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@h2oearth
Copy link

h2oearth commented Feb 7, 2024

It seems like the fail-fast:false is here to stay in favour of https://www.cypress.io/blog/2023/02/15/cypress-cloud-auto-cancellation-is-now-available-via-cli. It would be nice if the customers under the Team plan could have these features for free!

Johnson1s added a commit to Johnson1s/docs-1 that referenced this issue Aug 29, 2024
…ey 6c2f2ab5-666a-4c9f-a744-60f0808350b7

name: Cypress Tests
on: [push]
jobs:
  cypress-run:
    runs-on: ubuntu-latest
    # Runs tests in parallel with matrix strategy https://docs.cypress.io/guides/guides/parallelization
    # https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs
    # Also see warning here https://github.com/cypress-io/github-action#parallel
    strategy:
      fail-fast: false # cypress-io/github-action#48
      matrix:
        containers: [1, 2] # Uses 2 parallel instances
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Cypress run
        # Uses the official Cypress GitHub action https://github.com/cypress-io/github-action
        uses: cypress-io/github-action@v6
        with:
          # Starts web server for E2E tests - replace with your own server invocation
          # https://docs.cypress.io/guides/continuous-integration/introduction#Boot-your-server
          start: npm start
          wait-on: 'http://localhost:3000' # Waits for above
          # Records to Cypress Cloud 
          # https://docs.cypress.io/guides/cloud/projects#Set-up-a-project-to-record
          record: true
          parallel: true # Runs test in parallel using settings above
        env:
          # For recording and parallelization to work you must set your CYPRESS_RECORD_KEY
          # in GitHub repo → Settings → Secrets → Actions
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
          # Creating a token https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
kannadasantvr added a commit to kannadasantvr/VSCODE that referenced this issue Oct 24, 2024
on: [push]
jobs:
  cypress-run:
    runs-on: ubuntu-latest
    # Runs tests in parallel with matrix strategy https://docs.cypress.io/guides/guides/parallelization
    # https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs
    # Also see warning here https://github.com/cypress-io/github-action#parallel
    strategy:
      fail-fast: false # cypress-io/github-action#48
      matrix:
        containers: [1, 2] # Uses 2 parallel instances
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Cypress run
        # Uses the official Cypress GitHub action https://github.com/cypress-io/github-action
        uses: cypress-io/github-action@v6
        with:
          # Starts web server for E2E tests - replace with your own server invocation
          # https://docs.cypress.io/guides/continuous-integration/introduction#Boot-your-server
          start: npm start
          wait-on: 'http://localhost:3000' # Waits for above
          # Records to Cypress Cloud
          # https://docs.cypress.io/guides/cloud/projects#Set-up-a-project-to-record
          record: true
          parallel: true # Runs test in parallel using settings above
        env:
          # For recording and parallelization to work you must set your CYPRESS_RECORD_KEY
          # in GitHub repo → Settings → Secrets → Actions
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
          # Creating a token https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants