ACI - Update logic for pulling runs on Runs page and Specs list

[warrensplayer]

Current behavior

The Specs list page and the Runs page use different methods for querying for runs to show the relevant Cloud data on those pages. The Debug page introduced a third way by pulling runs that map to the local Git tree.

Expected bahavior

The Specs list page and Runs page will also use the local Git tree to find the most relevant runs that map to the local developer's checked out code.

What the Debug page does...

The Debug page uses the local Git tree to determine which runs to show for debugging. This is done by pulling the latest 100 git hashes (i.e. shas) and passing them to Cypress Cloud via the GraphQL endpoint runsByCommitShas in the RelevantRunsDatasource. The Cloud returns a list of runs that map to the commit hashes that were passed and the runs are filtered down to the set of "relevant" runs for the Debug page.

Specs List

The Specs List page currently pulls cloud data using the following GraphQL fields from CloudProjectSpec:

• averageDuration
• isConsideredFlaky
• flakyStatus
• specsRun

It only sends this part of the GraphQL query if a Git branch is available.

Polling
It also leverages a polling mechanism using the subscription startPollingForSpecs. This polls the Cloud every 30 seconds (or can be changed by a value returned from the Cloud query) and returns a timestamp in a field from cloudLatestRunUpdateSpecData.mostRecentUpdate. This value is the latest updated time for the runs it is interested in. The front end tracks changes in this timestamp and queries the Cloud again if it changes.

Updates
The existing fields should be swapped out for new fields that leverage a set of cloudRunIds:

• averageDurationForRunIds
• isConsideredFlakyForRunIds
• flakyStatusForRunIds
• specRunsForRunIds

The cloudRunIds are the id field from the CloudRun type, not the runNumber that is shown in the UI. The App should pass as many ids as possible up to a max of 100 to these endpoints. specRunsForRunIds will return results for as many matching CloudSpecRuns as it finds. The App should use the latest 4 for the Run Dots.

Runs page

The Runs page currently pulls the 10 most recent runs for the Project and shows them in reverse chronological order. If a user stays on the page, then any new runs that start while on the page will get added to the top of the list.

Polling
Polling is done on the front end in RunsContainer.vue source. This polls every 15 seconds but has a strange behavior if the page has already been viewed and Urql has cached the results. The timeout used starts 15 seconds after the component is mounted. If the original GraphQL query has already been cached, then it can appear that the results are delayed by 15 seconds.

Updates

If the most recent 100 Git hashes is found with the GitDataSource, the Runs page should leverage the same pattern as the Debug page to pull the 10 most recent runs based on the local Git tree. This should use the polling being done in RelevantRunsDatasource to prevent additional polling queries to the Cloud. To simplify this change, the page can just always show the most recent 10 runs. Thus, if a polling cycle occurs while on the page and a new run is started, then that would be the latest run with 9 more below it.

If no local Git hashes are found, then the page should default back to pulling the most recent 10 runs for the Cloud project. Update the logic so there is no delay in pulling the latest data if Urql has already cached the results.

Acceptance Criteria

• Specs List should pull data with the new fields based on CloudRunIds
  • Flaky badge, tooltip, and run dots should all be updated
• Specs List should still show empty run dots if no local Git data is available
• Validate the that Cloud is still leveraging the batching function when fetching spec data
• Runs page should show latest 10 runs for local Git tree if available
• Runs page should fallback to existing logic of latest 10 runs for the Cloud project if local Git data is not available
• When either page is using local Git data to resolve which runs to use, they should leverage the existing polling in RelevantRunsDataSource and not introduce additional polling.

[warrensplayer]

Hey team! Please add your planning poker estimate with Zenhub @astone123 @jordanpowell88 @lmiller1990 @marktnoonan @mike-plummer

[lmiller1990]

This does not seem to be working for me, at least as I understand it. I created a branch, modified a test, and pushed. The UI never updates with the latest run. This is still the case, even if I restart the runner.

Here's a long video of it:
https://github.com/cypress-io/cypress/assets/19196536/8e8e3e97-2a5d-4fbf-b0e1-a6d014300f71

Here is my test project: https://github.com/cypress-io/aci-test-projects
Here is the project in Cypress Cloud: https://cloud.cypress.io/projects/3926de/runs?branches=%5B%5D&committers=%5B%5D&flaky=%5B%5D&page=1&status=%5B%5D&tags=%5B%5D&timeRange=%7B%22startDate%22%3A%221970-01-01%22%2C%22endDate%22%3A%222038-01-19%22%7D

Do I need a specific Cloud preview environment perhaps, or did I miss something else in the test plan?

[lmiller1990]

I tried again without a git repo, and it's working great - updating as expected, etc. I wonder if I'm doing something incorrectly, or there's a bug / something missing that breaks when you've got a git repo.

Working w/o git repo:

ok.mov

[warrensplayer]

@lmiller1990 I watched your video and not sure why that is not working for you. You do not need a specific Cloud environment nor do you need to use a binary of the app for this to work. I will recreate your scenario and make sure I did not break something critical.

[warrensplayer]

@lmiller1990 Here is long video showing my tests using the cypress-example-kitchensink repo.
My steps:

• I created a brand new org in Cypress cloud and attached my local project as a new Cloud project
• I started recording the video while in the middle of recording my first run on master
• The cypress run finished
• I switched branches to a branch (new-branch) that had a failing test on it
• Recorded a run on that branch and watched the changes from the Specs, Runs, and Debug pages
• That run completed
• Switched back to master to show the states on the different pages
• Then switched back to new-branch to show the states of the pages there

Sorry the video is blurry. I had to trim it down and save it 720p to get it under 100MB.

new_aci_logic_trimmed_720.mov

[lmiller1990]

I will test again. The one thing I'm doing differently is creating a new branch with git checkout -b .... I wonder if this makes a difference.

[cypress-bot]

Released in 12.15.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v12.15.0, please open a new issue.