Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Frontend: Flaky test due to "cannot log after tests are done" error #290

Closed
omarkohl opened this issue Feb 28, 2023 · 8 comments
Closed

Frontend: Flaky test due to "cannot log after tests are done" error #290

omarkohl opened this issue Feb 28, 2023 · 8 comments
Labels
bug Something isn't working
Milestone
v0.4.0

Comments

@omarkohl
Copy link
Contributor 

Cannot log after tests are done. Did you forget to wait for something async in your test?
    Attempted to log "Error getting config TypeError: Cannot read properties of null (reading '_location')
        at Window.get location [as location] (/home/runner/work/cleodora/cleodora/frontend/node_modules/jsdom/lib/jsdom/browser/Window.js:375:79)
        at FetchInterceptor.<anonymous> (/home/runner/work/cleodora/cleodora/frontend/node_modules/@mswjs/interceptors/src/interceptors/fetch/index.ts:50:22)
        at step (/home/runner/work/cleodora/cleodora/frontend/node_modules/@mswjs/interceptors/lib/interceptors/fetch/index.js:59:23)
        at Object.next (/home/runner/work/cleodora/cleodora/frontend/node_modules/@mswjs/interceptors/lib/interceptors/fetch/index.js:40:53)
        at fulfilled (/home/runner/work/cleodora/cleodora/frontend/node_modules/@mswjs/interceptors/lib/interceptors/fetch/index.js:31:58)
        at runNextTicks (node:internal/process/task_queues:61:5)
        at processImmediate (node:internal/timers:437:9)".

      36 |                  }
      37 |             )
    > 38 |             .catch(
         |               ^
      39 |                 (reason) => {
      40 |                     console.log("Error getting config", reason)
      41 |                 }

      at Window.get location [as location] (node_modules/jsdom/lib/jsdom/browser/Window.js:375:79)
      at FetchInterceptor.<anonymous> (node_modules/@mswjs/interceptors/src/interceptors/fetch/index.ts:50:22)
      at step (node_modules/@mswjs/interceptors/lib/interceptors/fetch/index.js:59:23)
      at Object.next (node_modules/@mswjs/interceptors/lib/interceptors/fetch/index.js:40:53)
      at fulfilled (node_modules/@mswjs/interceptors/lib/interceptors/fetch/index.js:31:58)
      at processImmediate (node:internal/timers:437:9)".
      at console.log (node_modules/react-scripts/node_modules/@jest/console/build/BufferedConsole.js:199:10)
      at src/Footer.tsx:38:15

It's probably some sort of race condition.

It happens from time to time e.g. here: https://github.com/cleodora-forecasting/cleodora/actions/runs/4294409786/jobs/7483394228
@omarkohl omarkohl added the bug Something isn't working label Feb 28, 2023
omarkohl added a commit that referenced this issue Mar 16, 2023
@omarkohl
tests(frontend): fix 'cannot log after tests are done' error (#290)
9ce56de 
At least this is an attempt to fix this flaky test as explained here:
https://stackoverflow.com/a/61612357 . It's not clear that this will
actually help, but it's worth a try. The general idea is that the test
should wait until all changes have been processed.
@omarkohl
Copy link
Contributor Author

Issue has not been solved with the above commit. https://github.com/cleodora-forecasting/cleodora/actions/runs/4480272259/jobs/7875344833?pr=338

@omarkohl
Copy link
Contributor Author

Seems to be the same issue http://tech.yipp.ca/reactjs/jest-react-useeffect-executed-test-log-tests-done-forget-wait-something-async-test/

@omarkohl
Copy link
Contributor Author

Maybe related testing-library/react-testing-library#667

omarkohl added a commit that referenced this issue May 17, 2023
@omarkohl
fix(frontend): modify Footer useEffect() to fix flaky test (#290)
d3f7a1d 
Following the instruction found here:
https://devtrium.com/posts/async-functions-useeffect
@omarkohl omarkohl added this to v0.3.0 May 17, 2023
@omarkohl omarkohl moved this to In Progress in v0.3.0 May 17, 2023
@omarkohl
Copy link
Contributor Author

Looking good so far after the latest commit. If there are no failures until the end of May in can be closed.

@omarkohl
Copy link
Contributor Author

It might have become less frequent but the job is still failing https://github.com/cleodora-forecasting/cleodora/actions/runs/5070288233/jobs/9105115234?pr=501

@omarkohl omarkohl removed this from v0.3.0 May 25, 2023
@omarkohl omarkohl moved this to Todo in Implementation Jun 8, 2023
@omarkohl omarkohl moved this from Planned to Backlog in Implementation Jun 8, 2023
@omarkohl
Copy link
Contributor Author

omarkohl commented Jun 9, 2023

Even after running the tests ~500 times locally it did not fail, but it is still failing in GitHub actions. The explanation for that seems to be jestjs/jest#11132 i.e. it only fails in CI but not locally.

@omarkohl omarkohl moved this from Backlog to Planned in Implementation Jun 9, 2023
@omarkohl
Copy link
Contributor Author

omarkohl commented Jun 9, 2023

Interesting comment, but the whole thread might be useful: jestjs/jest#9324 (comment)

@omarkohl omarkohl moved this from Planned to In Progress in Implementation Jun 10, 2023
@omarkohl
Copy link
Contributor Author

I am able to reproduce the issue locally with the following command. The error appears about every 10 executions.

cd frontend
npm run test -- --all --bail --ci --watchAll=false --runInBand

omarkohl added a commit that referenced this issue Jun 13, 2023
@omarkohl
tests(frontend): fix 'Cannot log after tests are done' (#290)
10bd892 
Contrary to what I initially believed the issue does not seem to be in
the Footer.test.tsx but in App.test.tsx . The error message appears
after the Footer.test.tsx output and before the App.test.tsx output. The
App test includes all components but was not checking for the footer
content. My hypothesis is that when the test was done it tore down the
msw handlers making the fetch request for config.json made in the Footer
either abort or not be intercepted by msw. What set me on the right
track was the following issue, in particular this comment
jestjs/jest#9324 (comment) .

Before this fix the tests would fail about once every 10 to 40
executions. After this fix the tests failed due to #561 after about 320
and after about 60 executions. There were no more failures because of
this issue.
omarkohl added a commit that referenced this issue Jun 13, 2023
@omarkohl
tests(frontend): validate the forecast table in App.test.tsx
6960b7d 
This is related to #290 because the same error could have appeared for
this case since nothing was waiting for the table content.
@omarkohl omarkohl moved this from In Progress to Done in Implementation Jun 13, 2023
@omarkohl omarkohl added this to the v0.4.0 milestone Jun 13, 2023
omarkohl added a commit that referenced this issue Jun 15, 2023
@omarkohl
tests(frontend): add delays to msw request handlers
ef81dc8 
This is to simulate real world network situation and flush out bugs such
as #290 and #561 earlier.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
Implementation
Status: Done
Milestone
v0.4.0
Development

No branches or pull requests
1 participant
@omarkohl