FRS-5 Testing (#5)

* Updated FRS with testing section

* Basic Cypress configuration

* Change cypress github action image

* Added component test setup

* Added basic E2E tests

* Finished E2E tests

* Fixed missing config in action

* Add wait to test

* Try index select

* Add focus

* Add logging

* Include debug logs

* Include debug logs

* Update github action

* Fix action

* different assertion

* New github action

* New github action

* Remove wait-on

* Following nextjs guide for starting server

* Move start-server and test to dev deps

* Added caching

* Break tests to check artifacts

* Final Github actions

* Completed BoardFrom component tests

* Remove debug mode from E2E github action

* Added test for BoardGrid component

* Updated FRS

* Added hardcoded functionality for Layout props.

* Tided up TS

* Added basic getLayoutProps test

* Set layout constants

* Updated FRS

* Added Github issue links

.github/workflows/e2e_test.yml

@@ -0,0 +1,54 @@
+name: End-to-end tests
+on:
+  push:
+    branches:
+      - main
+
+jobs:
+  cypress-run:
+    runs-on: ubuntu-latest
+    name: Run Cypress E2E tests
+    steps:
+      - name: Check out Git repository
+        uses: actions/checkout@v4
+
+      - name: Cache
+        uses: actions/cache@v3
+        with:
+          path: |
+            ~/.npm
+            ${{ github.workspace }}/.next/cache
+          key: ${{ runner.os }}-nextjs-${{ hashFiles('**/package-lock.json') }}-${{ hashFiles('**/*.js', '**/*.jsx', '**/*.ts', '**/*.tsx') }}
+          restore-keys: |
+            ${{ runner.os }}-nextjs-${{ hashFiles('**/package-lock.json') }}-
+
+      - name: Set up Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: 18
+          cache: "npm"
+
+      - name: Install Node.js dependencies
+        run: npm ci
+
+      - name: Build
+        run: npm run build --if-present
+
+      - name: Run Cypress Tests
+        run: npm run cypress:ci
+
+      - name: Upload Photo Artifacts
+        uses: actions/upload-artifact@v4
+        if: failure()
+        with:
+          name: cypress-screenshots
+          path: cypress/screenshots
+          if-no-files-found: ignore
+
+      - name: Upload Video Artifacts
+        uses: actions/upload-artifact@v4
+        if: failure()
+        with:
+          name: cypress-videos
+          path: cypress/videos
+          if-no-files-found: ignore

.github/workflows/lint-and-test.yml

@@ -1,4 +1,5 @@
 # Copied from https://github.com/marketplace/actions/lint-action
+# And: https://nextjs.org/docs/pages/building-your-application/deploying/ci-build-caching#github-actions
 
 name: Lint and Test
 
@@ -17,25 +18,34 @@ jobs:
 
     steps:
       - name: Check out Git repository
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
+
+      - name: Cache
+        uses: actions/cache@v3
+        with:
+          path: |
+            ~/.npm
+            ${{ github.workspace }}/.next/cache
+          key: ${{ runner.os }}-nextjs-${{ hashFiles('**/package-lock.json') }}-${{ hashFiles('**/*.js', '**/*.jsx', '**/*.ts', '**/*.tsx') }}
+          restore-keys: |
+            ${{ runner.os }}-nextjs-${{ hashFiles('**/package-lock.json') }}-
 
       - name: Set up Node.js
         uses: actions/setup-node@v4
         with:
           node-version: 18
-          cache: "npm"
 
       - name: Install Node.js dependencies
         run: npm ci
 
+      - name: Build
+        run: npm run build --if-present
+
       - name: Lint
         uses: wearerequired/lint-action@v2
         with:
           eslint: true
           prettier: true
 
-      - name: Build
-        run: npm run build --if-present
-
       - name: Test
         run: npm test

README.md

@@ -10,17 +10,21 @@ Full documentation can be found in the [documentation](documentation) folder.
 
 ## Dependencies
 
-- [Next.js](https://nextjs.org/) `14.1.0`
+- [Next.js](https://nextjs.org/) `14.1.*`
 - [React](https://react.dev/) `18.*`
 - [react-hexgrid](https://github.com/Hellenic/react-hexgrid) `2.0.0@beta`
-  - [emotion](https://emotion.sh/docs/introduction) `11.11.3` (dependency not imported by react-hexgrid)
-- [Zod](https://zod.dev/) `3.22.4`
+  - [emotion](https://emotion.sh/docs/introduction) `11.11.*` (dependency not imported by react-hexgrid)
+- [Zod](https://zod.dev/) `3.22.*`
 
 ### Dev Dependencies
 
 - [Typescript](https://www.typescriptlang.org/) `v5.*`
 - [Tailwind CSS](https://tailwindcss.com/) `v3.3.*`
-- [ESLint](https://eslint.org/) `8.56.0`
-- [Prettier](https://prettier.io/) `3.2.5`
-- [pre-commit](https://pre-commit.com/) `3.5.0`
-- [vitest](https://vitest.dev/) `1.2.2`
+- [ESLint](https://eslint.org/) `8.56.*`
+- [Prettier](https://prettier.io/) `3.2.*`
+- [pre-commit](https://pre-commit.com/) `3.5.*`
+- [vitest](https://vitest.dev/) `1.2.*`
+- [Cypress](https://www.cypress.io/) `13.6.*`
+- [start-server-and-test](https://github.com/bahmutov/start-server-and-test) `2.0.3`
+- [testing-library/react](https://testing-library.com/docs/react-testing-library/intro) `14.2.*`
+- [testing-library/user-event](https://testing-library.com/docs/user-event/intro) `14.5.*`

cypress.config.ts

@@ -0,0 +1,16 @@
+import { defineConfig } from "cypress";
+
+export default defineConfig({
+  e2e: {
+    setupNodeEvents(on, config) {
+      // implement node event listeners here
+    },
+  },
+
+  component: {
+    devServer: {
+      framework: "next",
+      bundler: "webpack",
+    },
+  },
+});

cypress/e2e/app.cy.ts

@@ -0,0 +1,31 @@
+import { ALGORITHM_COASTAL, ALGORITHM_RANDOM } from "../../src/lib/constants";
+
+describe("BoardForm", () => {
+  it("should display a random board on submit", () => {
+    cy.visit("http://localhost:3000/");
+
+    // Assert form is in base state.
+    cy.get('[data-testid="use-seafarers-checkbox"]').should("not.be.checked");
+    cy.get('[data-testid="rand-algorithm-selector"] option:selected').should(
+      "have.value",
+      ALGORITHM_RANDOM,
+    );
+    cy.get('[data-testid="rand-algorithm-selector"] option').should(
+      "not.have.value",
+      ALGORITHM_COASTAL,
+    );
+
+    // Select the seafarers checkbox. force: true is required due to div overlaying input.
+    cy.get('[data-testid="use-seafarers-checkbox"]').check({ force: true });
+
+    // Assert it's possible to select a seafarers algorithm.
+    cy.get('[data-testid="rand-algorithm-selector"]').select(ALGORITHM_COASTAL);
+
+    cy.get('[data-testid="submit-button"]').click();
+    // Top level node of an SVG is a single <g>, so to check hexagons we have to go a level lower.
+    cy.get('[data-testid="hexgrid-svg"]')
+      .children()
+      .first()
+      .should("have.descendants", "g.hexagon-group");
+  });
+});

cypress/fixtures/example.json

@@ -0,0 +1,5 @@
+{
+  "name": "Using fixtures to represent data",
+  "email": "hello@cypress.io",
+  "body": "Fixtures are a great way to mock data for responses to routes"
+}

cypress/support/commands.ts

@@ -0,0 +1,37 @@
+/// <reference types="cypress" />
+// ***********************************************
+// This example commands.ts shows you how to
+// create various custom commands and overwrite
+// existing commands.
+//
+// For more comprehensive examples of custom
+// commands please read more here:
+// https://on.cypress.io/custom-commands
+// ***********************************************
+//
+//
+// -- This is a parent command --
+// Cypress.Commands.add('login', (email, password) => { ... })
+//
+//
+// -- This is a child command --
+// Cypress.Commands.add('drag', { prevSubject: 'element'}, (subject, options) => { ... })
+//
+//
+// -- This is a dual command --
+// Cypress.Commands.add('dismiss', { prevSubject: 'optional'}, (subject, options) => { ... })
+//
+//
+// -- This will overwrite an existing command --
+// Cypress.Commands.overwrite('visit', (originalFn, url, options) => { ... })
+//
+// declare global {
+//   namespace Cypress {
+//     interface Chainable {
+//       login(email: string, password: string): Chainable<void>
+//       drag(subject: string, options?: Partial<TypeOptions>): Chainable<Element>
+//       dismiss(subject: string, options?: Partial<TypeOptions>): Chainable<Element>
+//       visit(originalFn: CommandOriginalFn, url: string, options: Partial<VisitOptions>): Chainable<Element>
+//     }
+//   }
+// }

cypress/support/component-index.html

@@ -0,0 +1,14 @@
+<!doctype html>
+<html>
+  <head>
+    <meta charset="utf-8" />
+    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
+    <meta name="viewport" content="width=device-width,initial-scale=1.0" />
+    <title>Components App</title>
+    <!-- Used by Next.js to inject CSS. -->
+    <div id="__next_css__DO_NOT_USE__"></div>
+  </head>
+  <body>
+    <div data-cy-root></div>
+  </body>
+</html>

cypress/support/component.ts

@@ -0,0 +1,39 @@
+// ***********************************************************
+// This example support/component.ts is processed and
+// loaded automatically before your test files.
+//
+// This is a great place to put global configuration and
+// behavior that modifies Cypress.
+//
+// You can change the location of this file or turn off
+// automatically serving support files with the
+// 'supportFile' configuration option.
+//
+// You can read more here:
+// https://on.cypress.io/configuration
+// ***********************************************************
+
+// Import commands.js using ES2015 syntax:
+import "./commands";
+
+// Alternatively you can use CommonJS syntax:
+// require('./commands')
+
+import { mount } from "cypress/react18";
+
+// Augment the Cypress namespace to include type definitions for
+// your custom command.
+// Alternatively, can be defined in cypress/support/component.d.ts
+// with a <reference path="./component" /> at the top of your spec.
+declare global {
+  namespace Cypress {
+    interface Chainable {
+      mount: typeof mount;
+    }
+  }
+}
+
+Cypress.Commands.add("mount", mount);
+
+// Example use:
+// cy.mount(<MyComponent />)

cypress/support/e2e.ts

@@ -0,0 +1,20 @@
+// ***********************************************************
+// This example support/e2e.ts is processed and
+// loaded automatically before your test files.
+//
+// This is a great place to put global configuration and
+// behavior that modifies Cypress.
+//
+// You can change the location of this file or turn off
+// automatically serving support files with the
+// 'supportFile' configuration option.
+//
+// You can read more here:
+// https://on.cypress.io/configuration
+// ***********************************************************
+
+// Import commands.js using ES2015 syntax:
+import "./commands";
+
+// Alternatively you can use CommonJS syntax:
+// require('./commands')

documentation/functional_requirements_specification.md

@@ -145,6 +145,44 @@ functionality.
 
 ### 5 Testing
 
+Next.js has a number of guides for setting up various
+[testing libraries](https://nextjs.org/docs/app/building-your-application/testing) with the framework.
+
+While using [Jest](https://jestjs.io/) was attempted (due to prior knowledge), initial configuration for Typescript didn't work so
+[Vitest](https://vitest.dev/) should be used for unit testing.
+
+For E2E testing, [Cypress](https://www.cypress.io/) due to prior experience. As it stands Cypress does not support component testing with
+[Server Actions](https://nextjs.org/docs/app/building-your-application/testing/cypress), therefore it's best to stick
+with E2E testing. Component testing can be handled inside Vitest.
+
+See Appendix D for a list of errors found during the testing process.
+
+### 5.1 Unit Testing - Vitest
+
+The main file that should be unit tested is `src/lib/CatanBoardGenerator.ts`, as this file generates the all the
+possible Catan boards. A test suite should be setup with parameterised tests for all the different algorithms.
+
+Additionally, `src/lib/utils.ts` should be tested to confirm the functions work as expected.
+
+### 5.2 Component Testing - Vitest
+
+`<BoardFrom />` should be tested in the following ways:
+
+1. Assert the form renders with the expected fields.
+2. Assert the form select options change when `Use Seafarers` is clicked.
+
+`<BoardGrid />` should be tested in the following ways:
+
+1. Mock `useParams` and assert an `svg` is created.
+
+### 5.3 E2E Testing - Cypress
+
+E2E should cover as many user paths as possible, including those covered in 5.1 and 5.2 for completeness. Given there is
+a single user path in the application:
+
+1. Configure and submit a form, asserting a `svg` is created. This test will implicitly cover much of the functionality
+   of 5.2.
+
 ### 6 Containerisation
 
 #### 6.1 Docker
@@ -204,3 +242,17 @@ Taken from this [document](https://idoc.pub/documents/catan-components-list-wl1p
   - Desert: 1
   - Gold: 2
   - Ocean: 7
+
+### D: Errors from Testing
+
+1. The [Cypress Github Action](https://github.com/cypress-io/github-action) caused a failure on the
+   `should change select values when use-seafarers is checked` test, as it could not find the new options in the select.
+   The test works locally, and fine when a manual github action is setup.
+2. [#6](https://github.com/stuart-bradley/catan-randomiser-js/issues/6): `TypeError: useFormStatus is not a function` when Component testing `<BoardForm />`. This appears to be a Typescript
+   issue, but the functionality is not required for MVP so has been removed.
+3. Different sizes of board change the SVG viewport, moving and scaling the SVG differently. Functionality should be
+   added to change teh `<Layout />` component `size` and `origin` props based on the incoming board.
+4. [#7](https://github.com/stuart-bradley/catan-randomiser-js/issues/7) The Next.js [App Router](https://nextjs.org/docs/app) does not have functionality for
+   [shallow routing](https://nextjs.org/docs/pages/building-your-application/routing/linking-and-navigating#shallow-routing)
+   (which the Page Router does). This means it's not possible to avoid a redirect to update the URL and form state is lost.
+   Again, this is not MVP functionality so can be ignored for now.