Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hydration errors in Cypress when using React 18, Next, and server components #27204

Open
liambutler opened this issue Jul 5, 2023 · 33 comments
Labels
E2E Issue related to end-to-end testing existing workaround Triaged Issue has been routed to backlog. This is not a commitment to have it prioritized by the team.

Comments

@liambutler
Copy link

liambutler commented Jul 5, 2023

Current behavior

I’ve got a Next app running React 18. In the next.config.js, it has been configured with appDir to support Server Components. When running Cypress and accessing the page, I’m getting the following React errors:

  • (uncaught exception)Error: Hydration failed because the initial UI does not match what was rendered on the server. Warning: Expected server HTML to contain a matching <script> in <head>. See more info here: https://nextjs.org/docs/messages/react-hydration
  • 
(uncaught exception)Error: There was an error while hydrating. Because the error happened outside of a Suspense boundary, the entire root will switch to client rendering.

However, I do not see these hydration issues when accessing the page via browser, so it’s surprising to see them when running Cypress.

Steps to replicate:

  1. Clone repo and run npm install
  2. Run npm run e2e
  3. Visit http://localhost:3000 in your browser and check for console errors
  4. Run the spec app.cy.ts

Here's what I see in Cypress:
Screenshot 2023-07-05 at 16 14 00

It's not there when visiting the app via browser:
Screenshot 2023-07-05 at 16 21 08

The error will no longer appear if you remove the <script> tag on line 10 of layout.tsx (link to repro code)
Screenshot 2023-07-05 at 16 14 33

Seeing as this script tag isn't causing errors when accessing via the browser, I can only conclude that it's being caused by how Cypress is loading this page when React Server Components are present

The error occurs both when the app is running in dev mode, and when you visit the application in production mode after running build and start. In production mode, the errors are minified:

  • (uncaught exception)Error: Minified React error #418; visit https://reactjs.org/docs/error-decoder.html?invariant=418
  • (uncaught exception)Error: Minified React error #423; visit https://reactjs.org/docs/error-decoder.html?invariant=423

I’ve seen this issue raised in a few places. The suggestion I usually see is to prevent the tests from failing due to this uncaught exception (eg here), or to catch the issue in the application (Cypress docs)

As far as I have managed to understand, there should never be a hydration mismatch in a server rendered component since there is no client rendering to match it to. And when accessing the app via the browser, there are no mismatch errors. So I can only conclude that the issue stems from how Cypress is loading the page.

Desired behavior

Cypress should not be causing uncaught exceptions. Any exceptions that occur should also be triggered under the same conditions via browser

Test code to reproduce

https://github.com/liambutler/cypress-react-next-hydration-issue

Cypress Version

12.16.0

Node version

18.16.0

Operating System

MacOS 13.4.1

Debug Logs

No response

Other

Edit 8th November 2023- since updating Next.js from 13.4.2 to 13.5.6, we are also seeing this uncaught exception:

This also only appears when Cypress is being injected into the browser

@marktnoonan
Copy link
Contributor

Hi Liam 👋 - based on the error and the fact that it happens based on simply including a script tag in the server rendered <head>, I have a bit of a guess about what's going on.

The key factor may be that Cypress adds a script tag to the <head> of pages that are visited through Cypress. The Cypress <script> sets up window.Cypress and some other critical stuff, and is injected at the top of the <head> before the page is served. It's important that it be there on initial load since other code further down the page may rely on this global being present.

The <script> tag from your RSC is at the bottom of the head. I suspect that what is happening during hydration is that, since the validator knows to expect a <script> tag, it is parsing the head and finding the one injected by Cypress, and attempting to diff that against the <script></script> from your JSX.

I don't have a workaround at the moment besides ignoring hydration errors, which as you've said it not ideal, they are still there.

I'm assuming the use case here is going to be pretty common - you want to write some inline JS in your <head> so that it is active immediately on page load in your Next app?

@jcmonteiro
Copy link

Hi Mark! Thanks for the reply. Liam and I work together, and I might be able to provide some more context. Your explanation makes total sense. We also observe the problem when adding an empty <style> tag to <head>. From what you've said, it seems like the root cause is the same, i.e., Cypress's looking for its injected <script> tag and finding something of ours instead.

We indeed want an inline JS to execute as soon as the page is loaded. For context, that inline JS will test the browser against a series of modern JS commands, immediately redirecting (location.replace) the session to a simple "please upgrade your browser" page if such commands are not supported. That provides the best experience since we don't need to wait for all the JS to load, potentially breaking the session if the user is running on an old browser.

The <style> tag I mentioned is used to globally set the style of some elements. It is needed because the setup we have to produce our CSS design tokens strips away some properties that we then enforce back via <style dangerouslySetInnerHTML={{ __html: minifiedCss }} />

@marktnoonan
Copy link
Contributor

Thanks @jcmonteiro

Cypress's looking for its injected <script> tag and finding something of ours instead.

It's actually the other way around, Cypress doesn't care about any of this, but my guess is, where React is expecting the innerHtml of <head> to be the same on the client and server, it's not able to ignore the <script> tag added by Cypress. It's not clear to me why Next doesn't throw these errors always, why it only happens when either <style> or <script> is present as you've noticed. But maybe there are some rules/shortcuts for what it checks based on the contents of the JSX.

Cypress adds the script before the HTML is sent over the wire to the browser, in between the application server and the client, so there is no way for the Cypress JS to not be in the HTML sent to the client and evaluated during hydration.

It's similar to the problems that would happen if you are trying to do something like personalize a page by rewriting the HTML on the edge, after it has been rendered on a server. There's a long blog post from Netlify about the hoops that need to be jumped through to make hydration happy in that situation. Probably any tool that modifies the page contents after server-rendering through some middleware is going to have problems like this to solve with RSC.

I can think of a few workarounds, but I'm not too familiar with React and Next so they might not be ideal. Maybe this gives you enough info to work around it on your own, or to feel comfortable ignoring these errors for the time being. I'll flag this for the team as well so we can see if Cypress can avoid triggering this in the first place.

@cacieprins
Copy link
Contributor

Hi Liam and Joāo, Thank you so much for the reproduction case - it was very helpful in troubleshooting. I think I have a workaround for you. Can you try to wrap the style and script elements in a React.Suspense component?

I tested this in the reproduction case along with some small edits to more closely fit your use case, and it seems to no longer throw the hydration error when running in Cypress.

<Suspense>
  <script dangerouslySetInnerHTML={{ __html : `if(window.location.pathname === '/') { window.location.replace('http://localhost:3000/redirected') }` }}></script>
  <style dangerouslySetInnerHTML={{ __html: `body { color: #676 }`}} ></style>
</Suspense>

Please let us know if that helps!

@jcmonteiro
Copy link

jcmonteiro commented Jul 10, 2023

Hi Cacie, you are welcome and thank you for the reply.

Although the <Suspense> can help, I don't see it as a viable option for us. The reason is that our strategy for putting the "browser validation" <script> in the top-level <head> is that it can run before any JS is loaded (including the one that resolves the <Suspense>). As for the <style> element, I fear that wrapping it in a <Suspense> might lead to unforeseen issues in a larger application.

@cacieprins
Copy link
Contributor

I was worried about that too, so I did some digging. It turns out that Suspense contents do get rendered by SSR, so that code will execute before your main application code. You can see this come over the wire in network dev tools as a pair of comments.

The only other path forward I can see for this issue is configuring your build system inject the desired <script> and <style> tags outside of the react tree, similar to the other build artifacts. This would prevent hydration from trying to associate a React script component with the scripts that cypress injects. I'm not familiar enough with next.js to aid with this, however.

@jcmonteiro
Copy link

We will likely continue to catch the warnings and discard them in our Cypress tests, but it's not ideal since we might end up discarding true hydration errors. Could you keep this issue open so the community (ourselves included) can track its progress?

@nagash77 nagash77 added E2E Issue related to end-to-end testing Triaged Issue has been routed to backlog. This is not a commitment to have it prioritized by the team. labels Jul 11, 2023
@karlhorky
Copy link
Contributor

karlhorky commented Jul 21, 2023

I'm experiencing the 418 and 423 hydration errors with Cypress tests (currently suspecting usage of the cy.window() API via gatsby-cypress) as well also with Gatsby and a new canary version of React (18.3.0-canary-7118f5dd7-20230705)

Next.js uses canary React internally, which may be one of the reasons that it is visible there.

But if this indicates a general hydration problem with React 18.3.0, then Cypress may have problems with many more applications when a non-canary version of React 18.3.0 is released.

References:

@karlhorky
Copy link
Contributor

karlhorky commented Jul 21, 2023

Workaround

In case you're wondering how exactly to disable the uncaught exception errors in Cypress as Liam and João mentioned:

@liambutler in PR description: prevent the tests from failing due to this uncaught exception (eg here),

@jcmonteiro in comment 1629028523: We will likely continue to catch the warnings and discard them in our Cypress tests

...here's an example which was working for us (add block 1 before the test code that causes the failure, add block 2 after the test code that causes the failure):

// 1. Disable Cypress uncaught exception failures from React hydration errors
Cypress.on('uncaught:exception', (err) => {
  if (
    err.message.includes('Minified React error #418') ||
    err.message.includes('Error: Minified React error #423')
  ) {
    return false;
  }
  // Enable uncaught exception failures for other errors
});

// 2. Re-enable Cypress uncaught exception failures from React hydration errors
Cypress.on('uncaught:exception', () => {});

@minimit
Copy link

minimit commented Aug 7, 2023

Happends also if you put in the <head> tag a stylesheet link <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;700;900&display=swap" rel="stylesheet"/>

@liambutler
Copy link
Author

@cacieprins would it be possible to remove the 'existing workaround' tag from this ticket? I believe that the exceptions are causing issues for my tests despite the Cypress.on('uncaught:exception' catch detailed above

@jcmonteiro
Copy link

I will also add that the workaround interferes with the tests in a way that errors might go unnoticed. Thus, I also advocate for removing the existing workaround tag since it undermines the purpose of using Cypress.

@marktnoonan
Copy link
Contributor

I've been interpreting the "existing workaround" tag as referring to the use of <Suspense> mentioned here and other options around building mentioned here.

I don't see squashing the errors as a workaround - that's just ignoring errors - but I do see structuring your application in a way that it won't yell about injecting JS needed for testing as a workaround, which while inconvenient might not be too painful.

I'm interested in the reasons people might not find those options acceptable/feasible - and if you have tried any of them before choosing to ignore the error. Eg if somebody has tried the <Suspense> solution, did you find it didn't solve the problem, or it solved the problem but introduced unwanted side effects?

@jcmonteiro
Copy link

jcmonteiro commented Aug 27, 2023

In our use case, the issue with the <Suspense> is the one I mentioned previously.

The reason is that our strategy for putting the "browser validation" <script> in the top-level is that it can run before any JS is loaded (including the one that resolves the )

This is just our use case. In any case, adding the <Suspense> changes the execution flow substantially, so I would recommend against it, given the wide variety of use cases that are difficult to foresee.

@marktnoonan
Copy link
Contributor

Thanks @jcmonteiro - for clarity, what I am interested in is: have you verified that it does or would in fact cause a problem, given our understanding the <Suspense> is resolved during SSR and comes over the wire with the initial payload HTML, causing no need to wait for JS to load on the client to deliver the contents?

I'm not saying that you should accept/implement the workarounds, it's valid to be cautious. But at the moment I can't tell for sure which of these we are dealing with:

Option 1: <Suspense> works, but is a workaround that you find unacceptable due to an abundance of caution around possible unexpected side effects later (I think this is where we are)

Option 2: <Suspense> doesn't solve the problem because the information in this comment is incorrect and SSR is not resolving the contents on the server in your case

@cacieprins also mentioned "configuring your build system inject the desired <script> and <style> tags outside of the react tree, similar to the other build artifacts" so I'd have a similar question about that option: is this something that you tried and it didn't work, or didn't try (for whatever reason)?

@liambutler
Copy link
Author

liambutler commented Sep 6, 2023

Hi @marktnoonan

You're right in that we're at option 1. I've wrapped the scripts in <Suspense> and no longer see the exceptions. I've also tested the application on older browsers to ensure that the compatibility check script still works (see this earlier comment from @jcmonteiro ), and all still works as expected.

However, I'm getting a lot of pushback from the team around making a change like this to our layout.tsx, and I can see their point. Should we be making fundamental changes to our application in order to have a workaround for an issue caused by our choice of e2e tool? It's not an easy sell. (I haven't tried @cacieprins' suggestion, but the pushback would be much the same)

@karlhorky
Copy link
Contributor

Should we be making fundamental changes to our application in order to have a workaround for an issue caused by our choice of e2e tool? It's not an easy sell.

Yeah, I would guess this is enough for many teams to switch from Cypress to Playwright. I know that I would, if I was in that situation.

@brian-mann
Copy link
Member

brian-mann commented Sep 8, 2023

Hey @liambutler ... @marktnoonan brought this to my attention and I am recommending and testing out an approach where we automatically cleanup our injections before any of your react code runs, which may actually fix this issue and not require any changes on your side. We're going to put together a POC and may want ya'll to try out a development version of this fix to confirm it works.

This is a very simple example:

<html>
  <head>
    <script type="text/javascript">window.foo = {}; document.currentScript.remove()</script>

    <script type="text/javascript">console.log(window.foo)</script>
  </head>
</html>

@liambutler
Copy link
Author

Hi @brian-mann, thanks for looking into this!

Happy to help test any fix builds you want to try out. Let me know

@0420syj
Copy link

0420syj commented Sep 12, 2023

Happends also if you put in the <head> tag a stylesheet link <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;700;900&display=swap" rel="stylesheet"/>

Hi @minimit
In my case, I had the same problem when using the following.

<head>
  <link
    rel="stylesheet"
    as="style"
    crossOrigin="anonymous"
    href="https://cdnjs.cloudflare.com/ajax/libs/pretendard/1.3.8/static/pretendard.css"
  />
</head>

Changing the rel attribute to preload solved the problem.

<head>
  <link
    rel="preload"
    as="style"
    crossOrigin="anonymous"
    href="https://cdnjs.cloudflare.com/ajax/libs/pretendard/1.3.8/static/pretendard.css"
  />
</head>

Hope this works for you !

@marktnoonan
Copy link
Contributor

Hi folks, I've confirmed that the modification suggested by @brian-mann to have Cypress clean up the script element as the last step in its execution does not solve this issue, though I think it will be part of the final solution.

Reading through the thread in the React repo that @karlhorky linked to (facebook/react#24430) was interesting, in that people reported hydration errors with the likes of Loom and other browser extensions that modify the DOM, not just when testing with Cypress.

This makes sense based on the the nature of the problem. And it seems like the React team is aware of these consequences of stricter hydration rules and has worked to mitigate them in previous releases. I'm curious what will happen when 18.3.0 is stable. One thing we can test is a recent canary build that seems to fix a similar hydration problem for one person here: remix-run/remix#4822 (comment)

To summarize: We will look into the long term way for Cypress to handle this kind of error, or to at least confirm that it is handled upstream in React.

The "existing workaround" label doesn't mean that we won't work on it, just that there are ways to become unblocked in the meantime and allow your tests to run.

@dkasper-was-taken
Copy link
Contributor

Tested example repo with latest 18.3.0-canary-9ba1bbd65-20230922, hydration error still present.

@jordanpowell88 jordanpowell88 removed their assignment Nov 27, 2023
@bahmutov
Copy link
Contributor

bahmutov commented Dec 4, 2023

Hitting this problem sometimes. React should respect an element's attribute something like react-ignore-when-hydrating=true in this case.

@vahidvdn
Copy link

Workaround

Cypress.on("uncaught:exception", (err) => {
  // Cypress and React Hydrating the document don't get along
  // for some unknown reason. Hopefully, we figure out why eventually
  // so we can remove this.
  if (
    /hydrat/i.test(err.message) ||
    /Minified React error #418/.test(err.message) ||
    /Minified React error #423/.test(err.message)
  ) {
    return false;
  }
});

One important note regarding this, it should be in the e2e.ts in the cypress/support directory. Otherwise it won't work

@n-l-i
Copy link

n-l-i commented Jan 24, 2024

Hi folks, I've confirmed that the modification suggested by @brian-mann to have Cypress clean up the script element as the last step in its execution does not solve this issue, though I think it will be part of the final solution.

Reading through the thread in the React repo that @karlhorky linked to (facebook/react#24430) was interesting, in that people reported hydration errors with the likes of Loom and other browser extensions that modify the DOM, not just when testing with Cypress.
. . .

In that thread they also discuss the workaround of modifying the code so that the <head> is not hydrated, only the <body>. This works rather well for me. See the example in this comment: facebook/react#24430 (comment)

@BKG01
Copy link

BKG01 commented Feb 5, 2024

Workaround

Cypress.on("uncaught:exception", (err) => {
  // Cypress and React Hydrating the document don't get along
  // for some unknown reason. Hopefully, we figure out why eventually
  // so we can remove this.
  if (
    /hydrat/i.test(err.message) ||
    /Minified React error #418/.test(err.message) ||
    /Minified React error #423/.test(err.message)
  ) {
    return false;
  }
});

One important note regarding this, it should be in the e2e.ts in the cypress/support directory. Otherwise it won't work

This was the simplest solution for me (removing any stylesheet etc... in the head didn't fix the error). (Using remix with tailwind).
One thing to note is that all my tests were failing due to this error, it wasn't just an error that could be ignored (which may not be obvious given the above). I'll use the workaround until the react team have a more permanent solution. Thx

@RandallRock123
Copy link

Hi Liam 👋 - based on the error and the fact that it happens based on simply including a script tag in the server rendered <head>, I have a bit of a guess about what's going on.

The key factor may be that Cypress adds a script tag to the <head> of pages that are visited through Cypress. The Cypress <script> sets up window.Cypress and some other critical stuff, and is injected at the top of the <head> before the page is served. It's important that it be there on initial load since other code further down the page may rely on this global being present.

The <script> tag from your RSC is at the bottom of the head. I suspect that what is happening during hydration is that, since the validator knows to expect a <script> tag, it is parsing the head and finding the one injected by Cypress, and attempting to diff that against the <script></script> from your JSX.

I don't have a workaround at the moment besides ignoring hydration errors, which as you've said it not ideal, they are still there.

I'm assuming the use case here is going to be pretty common - you want to write some inline JS in your <head> so that it is active immediately on page load in your Next app?

I have to remember this one, when somthing is only happenning on certain modern browser, I just diabled the extensions, then it worked!

@nelsondev19
Copy link

Hello everyone, I had the same problem and I solved it with the following example:

Next JS version: 14.2.3

/context/StyledComponentsRegistry/index.tsx

"use client";

import { useServerInsertedHTML } from "next/navigation";

export function StyledComponentsRegistry({
  children,
  dynamicStyle,
}: {
  children: React.ReactNode;
  dynamicStyle: string;
}) {
  useServerInsertedHTML(() => {
    return <style>{dynamicStyle}</style>;
  });

  if (typeof window !== "undefined") return <>{children}</>;

  return <>{children}</>;
}

layout.tsx

import { StyledComponentsRegistry } from "@/context/StyledComponentsRegistry";


export default function RootLayout({ children }: Props) {
  const dynamicStyle = `:root {--bg-brand-sel: #fff;`;

  return (
    <html >
      <body>
        <StyledComponentsRegistry dynamicStyle={dynamicStyle}>
            {children}
        </StyledComponentsRegistry>
      </body>
    </html>
  );
}

@yukulelix
Copy link

In my case, using experimentalStudio: true in my e2e config was the issue.

defineConfig({
  e2e: {
    baseUrl: "http://localhost:3000",
    experimentalStudio: true,  // <--- Here
  }
 })

Removing it solved my hydration issues.

@douglasgf
Copy link

Even applying the solutions mentioned here didn't solve it.
When running Cypress, the website doesn't renerise everything and the Login option doesn't appear.

J-HernandezM added a commit to J-HernandezM/ananda-web that referenced this issue Jul 11, 2024
…ironment using React.Suspense

The issue apparently has to do with the way cypress loads the page causing the hydration error.
Source: cypress-io/cypress#27204
@dwjohnston
Copy link

@cacieprins solution solved this for me.

I had multiple, apparently unrelated issue relating to:

  • Links being detached from DOM before clicking them
  • Client components from external libraries failing to render

Wrapping a Suspense boundary around a component (utterances comments) that would imperatively attach a script to to the DOM in a useEffect solved all the issues for me.

Here's the component:

https://github.com/dwjohnston/blacksheepcode/blob/bd50bd5433628e54060f5c84d81e288b87ed44da/src/components/PostComments/PostComments.tsx#L1-L56

The fix:

https://github.com/dwjohnston/blacksheepcode/blob/ebc330a751e835d378e96fa31a1fa50034774f6c/src/components/BlogPostFrame/BlogPostFrame.tsx#L13-L15

It's a little confusing about why this would work - because it doesn't really seem to be anything to do with SSR - the script is attached in a useEffect - but just happens to solve a bunch of problems for me.

@dwjohnston
Copy link

dwjohnston commented Jul 12, 2024

My investigation suggests that :

  • Any client components that aren't wrapped in a Suspense boundary will cause this issue to occur.
  • Also next Links that occur within a RSC will also cause this issue to occur.

I'll see if I can get a good repro of this up.

@garronej
Copy link

garronej commented Sep 5, 2024

Hi Liam and Joāo, Thank you so much for the reproduction case - it was very helpful in troubleshooting. I think I have a workaround for you. Can you try to wrap the style and script elements in a React.Suspense component?

I tested this in the reproduction case along with some small edits to more closely fit your use case, and it seems to no longer throw the hydration error when running in Cypress.

<Suspense>
  <script dangerouslySetInnerHTML={{ __html : `if(window.location.pathname === '/') { window.location.replace('http://localhost:3000/redirected') }` }}></script>
  <style dangerouslySetInnerHTML={{ __html: `body { color: #676 }`}} ></style>
</Suspense>

Please let us know if that helps!

@cacieprins It helped a hole lot! It seems like many people overlooked it, but it’s a perfectly reasonable solution at least for library authors that needs to make their tool work in Cypress one way or another.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
E2E Issue related to end-to-end testing existing workaround Triaged Issue has been routed to backlog. This is not a commitment to have it prioritized by the team.
Projects
None yet
Development

No branches or pull requests