Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spike: Wrong test suite identified when running "very_good test" with optimization #401

Closed
renancaraujo opened this issue May 16, 2022 · 4 comments
Labels
bug Something isn't working as expected

Comments

@renancaraujo
Copy link
Contributor

renancaraujo commented May 16, 2022

Description
As described here, when running very_good test with optmization, the internal tooling will identify <cwd>/test/.test_runner.dart as the suite of tests.

Steps To Reproduce

  1. run very_good test without --no-optimization
  2. Any output of failing tests will accuse <cwd>/test/.test_runner.dart as output.

Expected Behavior
Outputs should show the real test suite,

Additional Context
As <cwd>/test/.test_runner.dart is a temporary file generated by mason, it is expecetd to somehow proxy the real suite or somehow allow its recover from some other field (group name without snake case?).
It is important that we are bound to the information passable via the JSON Reporter Protocol.

@renancaraujo renancaraujo added the bug Something isn't working as expected label May 16, 2022
@BeatriceMitchell
Copy link

This needs dedicated research time to identify the core issue before it can be fixed.

@BeatriceMitchell BeatriceMitchell changed the title fix: Wrong test suite identified when running "very_good test" with optimization spike: Wrong test suite identified when running "very_good test" with optimization Jun 3, 2022
@mugbug
Copy link

mugbug commented Jun 10, 2022

Not sure if this helps with the solution but:

I was experiencing the same problem when trying to use very_good test with optimization. I was able to fix this by fixing some of our tests.

Basically the fix was:

  • We use injectable and getIt
  • We were registering our mocks on getIt right on the main() test method, instead of doing so inside a setUp() or setUpAll().
  • Moving those to a setUpAll() seemed to fix this weird failing test message

After that, I just needed to do some getIt.reset(dispose: false) in some places to avoid dependency between tests.

@BeatriceMitchell
Copy link

If you have a simple Bloc test, and the test fails... the assertion is in the package instead of your code

@renancaraujo
Copy link
Contributor Author

renancaraujo commented Apr 24, 2023

Well, since this was open, there were a lot of changes that improved the output of the test optimizer.
Like #397 and #563

These changes show the correct location of the failing tests. So we can consider this one as fixed as possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working as expected
Projects
None yet
Development

No branches or pull requests

3 participants