Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve flakyness of tests #5265

Closed
1 task done
khendrikse opened this issue Nov 23, 2022 · 2 comments
Closed
1 task done

Improve flakyness of tests #5265

khendrikse opened this issue Nov 23, 2022 · 2 comments
Assignees
Labels
type: chore work needed to keep the product and development running smoothly type: feature code contributing to the implementation of a feature and/or user facing functionality

Comments

@khendrikse
Copy link
Contributor

Which problem is this feature request solving?

A lot of tests on PR's are flaky. This causes devs to have to rerun tests.

It's usually time-outs that cause the tests to fail.

Describe the solution you'd like

During a call we spoke about looking at the following:

  • what tests are most likely to fail
  • restructuring (some people had ideas)
  • trying out Vitest

Pull request (optional)

  • I can submit a pull request.
@khendrikse khendrikse added type: chore work needed to keep the product and development running smoothly type: feature code contributing to the implementation of a feature and/or user facing functionality labels Nov 23, 2022
@danez
Copy link
Contributor

danez commented Nov 24, 2022

Before we can use vitest, we need to migrate all files to ESM.

On thing I had on my mind is the writing of the global config file here. This happens for every test and leads to the tests on windows fail with EPERM error when multiple tests write at the same time I think. So maybe we can disable this in tests or mock the filesystem for these paths or write to a unique location for the test, like a tmp folder.

@danez
Copy link
Contributor

danez commented Dec 12, 2022

The ESM migration is pretty much done, there is this PR: #5298 and after that I already have another PR ready.

I had also more ideas while migrating and getting more insights:

  • same as mentioned above: Mock get-global-config in tests, so that we do not even read the config file, as well as write it.
  • Every netlify dev test does spin up a dev server and runs tests against it. We could optimize and group tests that use the same fixtures. Then only spin up one dev-server for all of these tests and run all the tests against this one dev server. This will also probably reduce the test time by a lot.

@khendrikse khendrikse removed their assignment Apr 3, 2023
@danez danez closed this as completed May 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: chore work needed to keep the product and development running smoothly type: feature code contributing to the implementation of a feature and/or user facing functionality
Projects
None yet
Development

No branches or pull requests

2 participants