viewport not getting recognized in CI

[silasjoisten]

Current behavior

packages:

"cypress": "^8.4.1",
"@cypress/vite-dev-server": "^2.1.0",
"@cypress/vue": "^2.2.3",
"@testing-library/cypress": "^8.0.1",

node version: 14.17.6

We setup Cypress Testing for e2e. Locally the viewportHeight and viewportWidth in the cypress.json is getting recognized and it works like a charm. => Tests are passing.

When we push to Github our GithubAction will execute yarn test:e2e which is described in the package.json as the following:

yarn build && concurrently --success=first -k "yarn serve" "cypress run --headless --config-file=cypress.json"

The weird behavior is that sometimes the tests are passing (probably the viewport option is getting recognized) and sometimes not. The test actually logs in a user and clicks a logout button in the sidebar navigation which is not visible in Mobile. (collapsed sidebar)

We tried a lot and also the solutions described in: #2102 but it did not work as well.

The behavior is flacky sometimes it fails sometimes it works.

First commit (passing) from screenshot above:

Second commit (failing) from screenshot above:

The Test which is failing caused by the invisible Logout Button collapsed in the sidebar.

The code from the test which fails:

  it('User can log out', () => {
    cy.login()

    cy.get('button')
      .contains('Ausloggen')
      .click()

    cy.location().should((loc) => {
      expect(loc.pathname).to.eq('/login')
    })
  })

Desired behavior

No response

Test code to reproduce

Create a Github Action with the following:

# ...
    tests:
        name: "Tests"

        runs-on: "ubuntu-latest"

        strategy:
            matrix:
                node-version:
                    - "14.17.6"

        steps:
            -   if: github.actor != 'dependabot[bot]'
                name: "Checkout"
                uses: "actions/checkout@v2.3.4"
                with:
                  ref: "${{ github.head_ref }}"
                  token: "${{ secrets.DATANA_BOT_TOKEN }}"

            -   if: github.actor == 'dependabot[bot]'
                name: "Checkout"
                uses: "actions/checkout@v2.3.4"
                with:
                    ref: "${{ github.head_ref }}"

            -
                name: "Install Node.js"
                uses: "actions/setup-node@v2.4.0"
                with:
                    node-version: "${{ matrix.node-version }}"

            -
                name: "Determine yarn cache directory"
                id: "determine-yarn-cache-directory"
                run: "echo \"::set-output name=directory::$(yarn cache dir)\""

            -
                name: "Cache dependencies installed with yarn"
                uses: "actions/cache@v2.1.6"
                with:
                    path: "${{ steps.determine-yarn-cache-directory.outputs.directory }}"
                    key: "node-${{ matrix.node-version }}-yarn-${{ hashFiles('frontend/yarn.lock') }}"
                    restore-keys: "node-${{ matrix.node-version }}-yarn-"

            -
                name: "Install dependencies with yarn"
                run: "yarn install"

            -
                name: "Cypress End to end tests"
                run: "yarn test:e2e"

package.json

{
  "name": "frontend-datapool",
  "license": "UNLICENSED",
  "version": "0.0.0",
  "scripts": {
    "dev": "vite --mode dev",
    "build": "vite build",
    "serve": "vite preview",
    "api": "json-server --watch db.json --port 3004 --delay 450",
    "lint:script": "eslint \"src/**/*.{ts,vue}\"",
    "lint:tsc": "vue-tsc --noEmit",
    "lint": "concurrently 'yarn lint:tsc' 'yarn lint:script'",
    "test:e2e": "yarn build && concurrently --success=first -k \"yarn serve\" \"cypress run --headless --config-file=cypress.json\"",
    "cypress:open": "./node_modules/.bin/cypress open"
  },
  "dependencies": {
    "@headlessui/vue": "^1.4.0",
    "@heroicons/vue": "^1.0.4",
    "@intlify/vite-plugin-vue-i18n": "^2.4.0",
    "@tailwindcss/forms": "^0.3.3",
    "axios": "^0.21.1",
    "vue": "^3.2.6",
    "vue-debounce": "^3.0.1",
    "vue-i18n": "^9.1.7",
    "vue-router": "4",
    "vuex": "^4.0.2"
  },
  "devDependencies": {
    "@cypress/vite-dev-server": "^2.1.0",
    "@cypress/vue": "^2.2.3",
    "@testing-library/cypress": "^8.0.1",
    "@types/node": "^16.7.6",
    "@typescript-eslint/eslint-plugin": "^4.31.1",
    "@typescript-eslint/parser": "^4.31.1",
    "@vitejs/plugin-vue": "^1.6.1",
    "@vue/compiler-sfc": "^3.2.6",
    "autoprefixer": "^10.3.1",
    "concurrently": "^6.2.1",
    "cypress": "^8.4.1",
    "eslint": "^7.32.0",
    "eslint-config-prettier": "^8.3.0",
    "eslint-config-standard-with-typescript": "^20.0.0",
    "eslint-plugin-cypress": "^2.12.1",
    "eslint-plugin-import": "^2.24.0",
    "eslint-plugin-node": "^11.1.0",
    "eslint-plugin-promise": "^4.3.1",
    "eslint-plugin-vue": "^7.16.0",
    "json-server": "^0.16.3",
    "postcss": "^8.3.6",
    "tailwindcss": "^2.2.7",
    "typescript": "^4.3.2",
    "vite": "^2.5.4",
    "vue-tsc": "^0.2.2"
  }
}

cypress.json

{
  "testFiles": "**/*.spec.ts",
  "componentFolder": "src",
  "video":false,
  "baseUrl": "http://127.0.0.1:5000",
  "viewportWidth": 1440,
  "viewportHeight": 900
}

Cypress Version

8.4.1

Other

No response

[jennifer-shehane]

The viewportHeight and viewportWidth have nothing to do with the height and width passed to the launchOptions args. The height and width passed to the launchOptions changes the screen size when testing headlessly, so this would affect the video and screenshot sizes. The viewportHeight and viewportWidth change the size of the application under test within the iframe that Cypress is running - the thing to the right of the reporter.

The application under test in your case will render at a size of 1440x900, if you want to override that for any test, like making your app render at a mobile size, you could set cy.viewport() to do this or set viewportHeight and viewportWidth in any config overrides during the test.

Let me know if this helps.

[silasjoisten]

Thank you for the detailed answer. Is there any global option to set that and override this via cy.viewport(); ? The option in the configuration is not getting recognized somehow. That makes it so strange it is working sometimes and sometimes not.

[silasjoisten]

We fixed it. By calling on every test case we wrote cy.viewport(). But this can't be the real solution if we want to have tests for desktop size global. Is there a Flobal option?

[silasjoisten]

@jennifer-shehane is here anything new?

[silasjoisten]

So we finally figured out, that the rendered HTML was too slow. So we increased defaultCommandTimeout": 10000 in our cypress.json. And that fixed it. It had nothing todo with the viewport at all. Thank you i am closing that.