breaking: upgrade cy.readFile() to be a query command #25595

BlueWinds · 2023-01-26T17:04:41Z

Additional details

This PR updates cy.readFile() to be a query command. It is a breaking change because users will no longer be able to overwrite readFile with Cypress.Commands.overwrite(). No other impact on user's tests is anticipated.

cy.readFile() already matched the behavior of other queries, by reading the file from disk until upcoming assertions pass. This update therefore is not a behavior change - it simply brings the readFile code execution path in line with other queries which do the same thing.

// Rereads file from disk until assertion passes
cy.readFile('foo.json').should('eql', { user: 'bar', id: 123 })

// Requeries element from DOM until assertion passes
cy.get('#div').should('have.class', 'active')

// Does not reread file from disk
cy.fixture('foo.json').should('eql', { user: 'bar', id: 123 })

Despite there being no user-facing changes, this update is a prerequisite for #25296. In that pull request, assertions will no longer "belong" to previous commands - they execute entirely independently. That means that the only way for a command to update the subject in response to failed assertions is to be a query - .readFile() can no longer be a special case of a "query-like command that isn't a query."

When reviewing / aiming to understand the changes, suggest starting with this comment: #25595 (comment).

Steps to test

No behavior has changed, and tests required no updates (see comments on each individual change for why it is code-clarity related and not functional).

How has the user experience changed?

Previously, if readFile timed out, users would get the message

`cy.readFile(foo.txt)` timed out retrying after 4000ms.

They now instead get the message

Timed out retrying after 4000ms: `cy.readFile(foo.txt)` timed out.

PR Tasks

Have tests been added/updated?
Has the original issue (or this PR, if no issue exists) been tagged with a release in ZenHub? (user-facing changes only)
Has a PR for user-facing changes been opened in cypress-documentation? .readFile() is now a query cypress-documentation#5017
[n/a] Have API changes been updated in the type definitions?

cypress · 2023-01-26T17:27:27Z

6 flaky tests on run #43697 ↗︎

0	4284	879	0	6

Details:

Merge branch 'release/13.0.0' into issue-25134-readfile-as-query
Project: cypress	Commit: `96e75d6d7b`
Status: Passed	Duration: 15:49 💡
Started: Feb 1, 2023 8:54 PM	Ended: Feb 1, 2023 9:10 PM

commands/net_stubbing.cy.ts • 3 flaky tests • 5x-driver-webkit

View Output Video

Test
network stubbing > intercepting request > can delay and throttle a StaticResponse
... > with `times` > only uses each handler N times
... > stops waiting when an xhr request is canceled

cypress/cypress.cy.js • 3 flaky tests • 5x-driver-webkit

View Output Video

Test
... > correctly returns currentRetry
... > correctly returns currentRetry
... > correctly returns currentRetry

_{This comment has been generated by cypress-bot as a result of this
project's GitHub integration settings.}

…[run ci]

BlueWinds · 2023-01-26T17:06:17Z

packages/driver/cypress/e2e/commands/files.cy.js

@@ -14,6 +14,14 @@ describe('src/cy/commands/files', () => {
  })

  describe('#readFile', () => {
+    it('really works', () => {


Moved the simplest tests higher up in the file. It felt off that we started with detailed assertions about the interaction with the backend before just using the command plainly.

BlueWinds · 2023-01-26T17:08:13Z

packages/driver/cypress/e2e/commands/files.cy.js

@@ -80,7 +88,7 @@ describe('src/cy/commands/files', () => {
      .resolves(okResponse)

      cy.readFile('foo.json').then(() => {
-        expect(retries).to.eq(1)
+        expect(retries).to.eq(2)


When mocked, the new version always retries one more time than the non-query version. This occurs because the first invocation triggers the promise which reads the file from disk and throws an error; it is always asynchronous, even when mocked.

BlueWinds · 2023-01-26T17:10:43Z

packages/driver/cypress/e2e/commands/files.cy.js

@@ -176,10 +178,6 @@ describe('src/cy/commands/files', () => {

        this.logs = []

-        cy.on('fail', () => {


Having an on('fail') handler in the beforeEach() block makes it hard to debug actual failures, since if you comment out a test's onFail block, the test will suddenly pass.

The only change this makes is that in some tests we see three logs rather than two, because "the stub has been called" log is included (where previously it would have been ignored because of this handler-removal).

BlueWinds · 2023-01-26T17:13:32Z

packages/driver/cypress/e2e/e2e/origin/commands/files.cy.ts

@@ -64,11 +64,11 @@ context('cy.origin files', { browser: '!webkit' }, () => {
      })

      cy.shouldWithTimeout(() => {
-        const { consoleProps } = findCrossOriginLogs('readFile', logs, 'foobar.com')
+        const log = findCrossOriginLogs('readFile', logs, 'foobar.com')


No functional change here. While working on these changes, I found it easier to read test failures when they were of the form "unable to read consoleProps from null" rather than "unable to destructure intermediate object" (the actual errors are significantly more verbose and harder to read than the above).

BlueWinds · 2023-01-26T17:29:10Z

packages/driver/src/cy/commands/files.ts

+      })
+    }
+
+    let fileResult: any = null


The way this now works is basically a state machine, with three pieces of state:

The contents of the file read from disk (can be null or have a value)

The currently active promise, waiting for a response from the server (can be null or be a promise)

The most recent error (always has a value)

The state can be updated in three different places.

When the query function executes (return () => {), if we have a fileResult, return it. Otherwise, create a new filePromise if we don't already have one (createFilePromise()) and throw mostRecentError.

When filePromise resolves (.then((result) => {), we set the contents to fileResult, and clear filePromise.

When filePromise rejects:
a. If it rejects ((.catch((err) => {)) with "file doesn't exist", we set fileResult to "no file exists" and clear filePromise.
b. Otherwise, we set the error to mostRecentError, and clear filePromise.

If an upcoming assertion fails, we clear fileResult.

The end result is that we query the disk for a file, and throw a "timed out" error until we get back a result (including "no file exists"); if the server responds with an unexpected error, we start throwing that instead of "timed out" and retry.

If it succeeds, we start returning that file as the result.

If an upcoming assertion fails (including the implicit "file should exist" assertion), we clear the result and head back into the retry loop.

BlueWinds · 2023-01-26T17:42:39Z

packages/driver/cypress/e2e/commands/assertions.cy.js

@@ -652,18 +652,21 @@ describe('src/cy/commands/assertions', () => {
        cy.get('button:first', { timeout: 500 }).should('have.class', 'does-not-have-class')
      })

-      it('has a pending state while retrying for commands with onFail', (done) => {
+      it('has a pending state while retrying for commands with onFail', function (done) {


Similar to the other test changes, this one relied on the state after the first retry; we no longer wait for a server response before "retrying" - so the test needs to be more patient.

chrisbreiding

Looks good! Just one little suggestion for the error message you can take or leave. I had originally thought a command like readFile couldn't be made into a query since queries have to be synchronous, so it's interesting to see how you've made it work.

Should this be considered a breaking change? Since readFile is now a query, any users overwriting it as a command will no longer be able to do so.

packages/driver/src/cypress/error_messages.ts

packages/driver/src/cy/commands/files.ts

BlueWinds · 2023-01-30T16:05:15Z

Looks good! Just one little suggestion for the error message you can take or leave. I had originally thought a command like readFile couldn't be made into a query since queries have to be synchronous, so it's interesting to see how you've made it work.

That was definitely my initial thought as well; I spent some time while working on this PR wondering if I'd dropped the ball a bit with queries and they could be async after all, but I think the answer remains the same - it's a hard limitation in the design that can be hacked around, but not changed.

I've been considering if it might be possible to provide a utility function to do this in a more generic way - the signature would be something like

createQueryCallback(generatePromise, initialError)

where you call it with a function that returns a promise and "the error if we don't have another error" and it returns a query function. It's a bit more complicated to make it generic than it seems, which is why I didn't end up going down that route in this pull request, but I think the possibility is there.

Should this be considered a breaking change? Since readFile is now a query, any users overwriting it as a command will no longer be able to do so.

Good call, I forgot about that side effect. Will update changelog.

…le-as-query

cli/CHANGELOG.md

Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>

emilyrohrbough · 2023-01-30T17:52:37Z

@BlueWinds Can you create a release-13.0.0 branch to base this off of?

The base branch was changed.

BlueWinds · 2023-01-30T18:38:57Z

@BlueWinds Can you create a release-13.0.0 branch to base this off of?

Done.

Also created the docs PR - cypress-io/cypress-documentation#5017.

chrisbreiding · 2023-01-30T18:47:04Z

scripts/semantic-commits/parse-changelog.js

@@ -2,12 +2,7 @@
 const fs = require('fs')


Would these changes be useful in develop? Maybe they could be cherry-picked into another PR against develop?

@emilyrohrbough , thoughts?

I have a branch in progress for this.

cli/CHANGELOG.md

Co-authored-by: Matt Henkes <mjhenkes@gmail.com>

Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> Co-authored-by: Matt Henkes <mjhenkes@gmail.com>

jennifer-shehane · 2023-08-29T17:25:00Z

Released in Cypress 13.0.0.

chore: cy.readFile() is now a query [run ci]

82425e6

Fix for assertion test that relied on counting retries of readFile() …

3345971

…[run ci]

BlueWinds force-pushed the issue-25134-readfile-as-query branch from a3b3c12 to 3345971 Compare January 26, 2023 17:40

BlueWinds commented Jan 26, 2023

View reviewed changes

BlueWinds marked this pull request as ready for review January 26, 2023 18:16

emilyrohrbough changed the title ~~chore: cy.readFile() is now a query [run ci]~~ chore: cy.readFile() is now a query Jan 26, 2023

emilyrohrbough changed the title ~~chore: cy.readFile() is now a query~~ feat: upgrade cy.readFile() to be a query command Jan 26, 2023

chrisbreiding self-requested a review January 26, 2023 21:15

chrisbreiding previously approved these changes Jan 30, 2023

View reviewed changes

packages/driver/src/cypress/error_messages.ts Show resolved Hide resolved

packages/driver/src/cy/commands/files.ts Show resolved Hide resolved

Merge remote-tracking branch 'origin/develop' into issue-25134-readfi…

1cd59da

…le-as-query

BlueWinds changed the title ~~feat: upgrade cy.readFile() to be a query command~~ breaking: upgrade cy.readFile() to be a query command Jan 30, 2023

BlueWinds added 3 commits January 30, 2023 08:11

Update changelog entry to clarify that this is breaking

7c50dfb

Remove linebreak to please changelog linter

1616186

Additional changelog fix

29b1b7e

BlueWinds force-pushed the issue-25134-readfile-as-query branch from c67120c to 29b1b7e Compare January 30, 2023 16:46

emilyrohrbough reviewed Jan 30, 2023

View reviewed changes

cli/CHANGELOG.md Outdated Show resolved Hide resolved

Update cli/CHANGELOG.md

25c0d55

emilyrohrbough reviewed Jan 30, 2023

View reviewed changes

cli/CHANGELOG.md Outdated Show resolved Hide resolved

emilyrohrbough and others added 2 commits January 30, 2023 11:40

fix changelog parsing for breaking change

58f6fa7

Update cli/CHANGELOG.md

ee4e87c

Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>

BlueWinds changed the base branch from develop to release/13.0.0 January 30, 2023 18:05

emilyrohrbough mentioned this pull request Jan 30, 2023

Cypress 13.0.0 Release #25630

Closed

BlueWinds mentioned this pull request Jan 30, 2023

.readFile() is now a query cypress-io/cypress-documentation#5017

Merged

BlueWinds requested review from chrisbreiding and emilyrohrbough January 30, 2023 18:44

chrisbreiding reviewed Jan 30, 2023

View reviewed changes

chrisbreiding approved these changes Jan 30, 2023

View reviewed changes

This was referenced Jan 30, 2023

Chains of query commands do not retry anything before an assertion #25134

Open

breaking: .should() and .and() are now queries. #25296

Closed

emilyrohrbough mentioned this pull request Jan 30, 2023

chore: fix changlelog section parsing and reference right ENV #25633

Merged

BlueWinds requested a review from mjhenkes January 31, 2023 20:49

mjhenkes reviewed Jan 31, 2023

View reviewed changes

cli/CHANGELOG.md Outdated Show resolved Hide resolved

Update cli/CHANGELOG.md

b3a0b57

Co-authored-by: Matt Henkes <mjhenkes@gmail.com>

mjhenkes approved these changes Jan 31, 2023

View reviewed changes

Blue F added 2 commits February 1, 2023 08:46

Merge branch 'release/13.0.0' into issue-25134-readfile-as-query

fa514d5

Merge branch 'release/13.0.0' into issue-25134-readfile-as-query

96e75d6

BlueWinds merged commit 1efd90d into release/13.0.0 Feb 1, 2023

BlueWinds deleted the issue-25134-readfile-as-query branch February 1, 2023 20:59

BlueWinds pushed a commit that referenced this pull request Feb 7, 2023

breaking: upgrade cy.readFile() to be a query command (#25595)

2e3b203

Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> Co-authored-by: Matt Henkes <mjhenkes@gmail.com>

emilyrohrbough mentioned this pull request Apr 4, 2023

Update to be Query Commands #26422

Open

jennifer-shehane mentioned this pull request Jul 5, 2023

breaking: v13.0.0 #27040

Merged

3 tasks

cypress-io locked as resolved and limited conversation to collaborators Aug 29, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

breaking: upgrade cy.readFile() to be a query command #25595

breaking: upgrade cy.readFile() to be a query command #25595

BlueWinds commented Jan 26, 2023 •

edited

Loading

cypress bot commented Jan 26, 2023 •

edited

Loading

BlueWinds Jan 26, 2023

BlueWinds Jan 26, 2023

BlueWinds Jan 26, 2023

BlueWinds Jan 26, 2023

BlueWinds Jan 26, 2023 •

edited

Loading

BlueWinds Jan 26, 2023

chrisbreiding left a comment

BlueWinds commented Jan 30, 2023

emilyrohrbough commented Jan 30, 2023

BlueWinds commented Jan 30, 2023

chrisbreiding Jan 30, 2023

BlueWinds Jan 30, 2023

emilyrohrbough Jan 30, 2023

jennifer-shehane commented Aug 29, 2023

		@@ -176,10 +178,6 @@ describe('src/cy/commands/files', () => {

		this.logs = []

		cy.on('fail', () => {

breaking: upgrade cy.readFile() to be a query command #25595

breaking: upgrade cy.readFile() to be a query command #25595

Conversation

BlueWinds commented Jan 26, 2023 • edited Loading

Additional details

Steps to test

How has the user experience changed?

PR Tasks

cypress bot commented Jan 26, 2023 • edited Loading

6 flaky tests on run #43697 ↗︎

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BlueWinds Jan 26, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chrisbreiding left a comment

Choose a reason for hiding this comment

BlueWinds commented Jan 30, 2023

emilyrohrbough commented Jan 30, 2023

BlueWinds commented Jan 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jennifer-shehane commented Aug 29, 2023

BlueWinds commented Jan 26, 2023 •

edited

Loading

cypress bot commented Jan 26, 2023 •

edited

Loading

BlueWinds Jan 26, 2023 •

edited

Loading