GitHub Actions job marked as SUCCESS but CYPRESS tests fail

[Monica89898]

My GitHub Actions run has always been marked a Success even when my cypress spec file test blocks marked as failed.
Below is the screenshot of how my yml file roughly looks like

I’ve been stuck with this issue for last two months, need suggestions on how to resolve them.
Appreciate any help.

[MikeMcC399]

@Monica89898

Cypress no longer supports Node.js 16. You need to upgrade to

• Node.js 18.x or
• Node.js 20.x and above

The screenshot of your yml file is not complete and does not show any call to cypress-io/github-action.

Instead of pasting a screenshot, it would be better if you pasted the text of the yml file, including the call to cypress-io/github-action. You can format the code using triple backticks like this:

```yml
name: example-basic
# This workflow represents a set of basic End-to-End tests
on:
  push:
    branches:
      - 'master'
and so on
```

We may also need to see your log files.

[Monica89898]

Hi @MikeMcC399 , Below is my yml
name: Smoke Action
on:
push:
branches: [main, master]
schedule:
- cron: "0 * * * *"
jobs:
e2e:
timeout-minutes: 60
runs-on: "Runner supported by organization"
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: "18"
- name: set registry
shell: bash
run: npm config set registry https://repo1.uhc.com/artifactory/api/npm/npm-virtual
- name: set strict-ssl to false
shell: bash
run: npm config set strict-ssl false
- name: Install dependencies
shell: bash
run: |
npm install
- name: env variables & Run Tests
shell: bash
run: |
export secretVariable=${{ secrets.keyname }} // adding some secret keys
npm run Scriptname
- name: Output test failure
if: success() || failure()
run: |
echo "failed_run=${true}" >> $GITHUB_OUTPUT
- name: Uploading Artifacts HTML Report
uses: actions/upload-artifact@v2
if: always()
with:
name: HTML-Report
path: cypress/reports/html
retention-days: 14

[MikeMcC399]

@Monica89898

Thank you for providing your complete yml file. Your workflow apparently is not using the Cypress GitHub Action. An example of the use of the action would be

name: End-to-end tests
on: push
jobs:
  cypress-run:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      # Install npm dependencies, cache them correctly
      # and run all Cypress tests
      - name: Cypress run
        uses: cypress-io/github-action@v6

I understand from your workflow, that you call Cypress using the step

- name: env variables & Run Tests
shell: bash
run: |
export secretVariable=${{ secrets.keyname }} // adding some secret keys
npm run Scriptname

This is outside of the scope of this issue list, which is for reporting bugs or making enhancement requests for the Cypress JavaScript GitHub Action cypress-io/github-action. The issue list is not for general use of GitHub Actions with Cypress without the action.

The only comment I can give you is that you have outdated versions of other JavaScript GitHub Actions. You might want to update these. I don't expect this would resolve your problem though:

- uses: actions/checkout@v2
- uses: actions/setup-node@v2
- uses: actions/upload-artifact@v2

[Monica89898]

@MikeMcC399 as per you suggession, I've upgraded node version and I tried to use cypress-io/github-action@v6 but then job failed saying "cypress-io/github-action@v6" is not allowed to be used within Organization.
So our company Actions marketplace, and I found one for cypress. The issue I faced by using this action is that it initially running the spec file that matches the patten specified in scriptName inside package.json, followed by running all the spec files that are within repository.
In logs I can see it searched for below pattern and runs all the scripts.
│ Searched: cypress/e2e/**/*.cy.{js,jsx,ts,tsx} │

[MikeMcC399]

@Monica89898

You can find many examples in the README and https://github.com/cypress-io/github-action/tree/master/.github/workflows about how to use cypress-io/github-action successfully.

Normally, if you are using cypress-io/github-action as it is intended, you would not be using a script to run Cypress.

example-config.ym shows how to select specs.

I am going to close this issue now, as it does not concern any bug in the action.

If you think you have found a bug when using the action, please open a new issue and provide your new workflow.