SPIKE: Integrate Cypress with PixiJS

[nstclair-cc]

PT-#188565339

Summary

This PR documents findings from a SPIKE aimed at improving Cypress tests for verifying graph point interaction on a PIXIJS-rendered canvas in CODAP. While partial success was achieved in triggering interactions with the canvas, there remain challenges in validating the visual state of PIXIJS-rendered elements.

Unskipped Tests

In graph.spec.ts, the following comment explained why certain tests were previously skipped:

// Skipping this test because Cypress 13 does not support displaying pixijs canvas elements in CI
// cypress-io/cypress#28289
// This test can be unskipped when the above Cypress bug is resolved
// (Locally, this works fine and the tests can be run successfully)

Since the referenced Cypress issue cypress-io/cypress#28289 has been resolved, the tests were unskipped and revalidated for performance in both local and CI environments.

---

Functional Test Code

Below is an updated test example, unskipped and modified during this SPIKE:

context("Test graph plot transitions", () => {
  beforeEach(function () {
    const queryParams = "?mouseSensor"
    const url = `${Cypress.config("index")}${queryParams}`
    cy.visit(url)
    cfm.openLocalDoc("cypress/fixtures/3TableGroups.codap")
    cy.wait(2500)
  })

  plots.forEach(test => {
    it(`${test.testName}`, () => {
      c.getIconFromToolShelf("graph").click()
      c.moveComponent("graph", 1000)
      test.axes.forEach(hash => {
        hash.checks.forEach(check => {
          cy.checkDragAttributeHighlights("table", hash.attribute, check.axis, check.active)
        })
        cy.dragAttributeToTarget("table", hash.attribute, hash.target)
        cy.wait(2000)
      })
    })
  })
})

---

Skipped Test Example

The following test remains skipped as it relies on PIXIJS exposing DOM elements or API hooks for verifying interactions, such as graph point highlighting:

    it.skip("should highlight a selected graph point", () => {
      // This test is the outcome of a SPIKE to explore testing graph interactions.
      // It partially validates interactions but requires further PIXIJS-level support.
      // https://github.com/concord-consortium/codap/pull/1637

      // Select the target table cell
      table.getGridCell(2, 2).should("contain", "African Elephant").click({ force: true })

      // Verify the graph's component title matches the collection name
      c.getComponentTitle("graph").should("contain", collectionName)

      // Re-click the table cell to ensure interaction consistency
      table.getGridCell(2, 2).click({ force: true })

      // Future goal: Validate the highlighted graph point
      // ChatGPT suggests this approach could work if PIXIJS exposes DOM elements
      // or provides API/event hooks that allow direct verification of point states.
      // cy.get('[data-testid="graph"] canvas')
      //   .should('be.visible') // Ensure the canvas is rendered
      // cy.get('[data-testid="graph"]')
      //   .find('svg .below-points-group circle') // Intended to locate graph points
      //   .then((elements) => {
      // Debugging information (e.g. to find out point color or position of points)
      //     cy.log('Highlighted point details:', elements)
      // })
    })

---

Custom Cypress Commands

We propose adding the following command (or something similar) to commands.ts for verifying the color of graph points:

// This command verifies the color of the selected graph point
Cypress.Commands.add("checkGraphPointSelect", (expectedColor) => {
  cy.get('[data-testid="graph"]')
    .find('svg .below-points-group circle') // Target the dots in the graph
    .first() // Assuming the first dot corresponds to the selected point
    .should('have.attr', 'fill', expectedColor); // Verify the fill color
});

Adding Custom Commands in cypress.d.ts

To ensure TypeScript support for the new command, include the following declaration in cypress.d.ts:

declare namespace Cypress {
  interface Chainable<Subject> {
    /**
     * Verifies the color of the selected graph point
     * @param expectedColor - The expected color value (e.g., '#0000FF' for blue)
     */
    checkGraphPointSelect(expectedColor: string): Chainable<void>;
  }
}

---

Goals for Future Enhancements

1. Testing Graph Point Interactions:

• Enable validations for graph point states (e.g., highlighting, selection) in tests.
• Leverage PIXIJS capabilities to expose DOM elements or provide API/event hooks.

2. Improved Canvas Interaction:

• Investigate methods to validate tooltips or other visual cues triggered by canvas interactions.
• Enhance mousemove coordinate targeting for better precision in hover simulations.

3. Collaborating with Developers:

• Work with the development team to introduce hooks or debugging modes in PIXIJS for more accessible state verification.
• Explore APIs that log graph interactions or expose them for testing purposes.

4. Refined Assertions:

• Add commands and assertions to verify specific visual states, such as tooltips or adornments.

---

Next Steps

1. Incorporate Findings: Merge this PR as a reference for further test enhancements.
2. Iterative Improvements: Continue refining Cypress tests based on PIXIJS debugging capabilities.
3. Developer Collaboration:

• Collaborate on exposing internal states or events for easier test validation.
• Discuss potential backend logging or API endpoints to track graph interactions.

---

Additional documentation:

• https://github.com/pixijs/pixijs/wiki/v5-Resources (see cypress automation link)
• Alternative solution: consider using tools like Applitools Eyes, which can capture and compare canvas images to detect visual differences: https://www.devzery.com/post/master-canvas-testing-with-cypress-applitools

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.66%. Comparing base (6f3b40e) to head (0b9e92b).
Report is 32 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1637       +/-   ##
===========================================
+ Coverage   67.88%   85.66%   +17.78%     
===========================================
  Files         600      600               
  Lines       30405    30414        +9     
  Branches     8351     8355        +4     
===========================================
+ Hits        20639    26053     +5414     
+ Misses       9611     4043     -5568     
- Partials      155      318      +163     

Flag	Coverage Δ
cypress	75.20% <ø> (+31.21%)	⬆️
jest	52.59% <ø> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests
• JS Bundle Analysis - Avoid shipping oversized bundles

[cypress]

codap-v3    Run #5314

Run Properties:   Passed #5314  •  433efc72cf: Merge pull request #1637 from concord-consortium/188565339-pixiJS-automation-spi...

Project	codap-v3
Branch Review	main
Run status	Passed #5314
Run duration	24m 50s
Commit	433efc72cf: Merge pull request #1637 from concord-consortium/188565339-pixiJS-automation-spi...
Committer	Natalya St. Clair
View all properties for this run ↗︎

---

Test results
Failures	0
Flaky	0
Pending	20
Skipped	0
Passing	236
View all changes introduced in this branch ↗︎

[kswenson]

👍 Enabling the skipped tests is fine. Adding tests that validate dots in graphs will require canvas techniques that can be determined later (if appropriate).

[kswenson]

This won't work because there are no DOM elements that correspond to the points, SVG or otherwise.

[scytacki]

Since I was tagged. I'd say the PR is good and the description is correct. 👍