[test] Add toWarnDev() and toErrorDev() matcher #21581
Conversation
not compatible with older browsers
eps1lon
requested review from
oliviertassinari,
mnajdova and
joshwooding
and removed request for
oliviertassinari and
mnajdova
| // eslint-disable-next-line no-underscore-dangle | ||
| const callback = this._obj; | ||
|
|
||
| if (process.env.NODE_ENV !== 'production') { |
There was a problem hiding this comment.
What's the motivation for a different assertion logic if node env is production?
There was a problem hiding this comment.
In prod it shouldn't match any console calls because they are stripped out. At least so far we don't have any console calls that are kept in prod bundles. Though we don't run tests on the prod bundle so it's not doing anything. Just implementing the semantics of a "to error in development" matcher.
| ]); | ||
| } | ||
| // eslint-disable-next-line no-console | ||
| console[methodName] = logUnexpectedConsoleCalls; |
There was a problem hiding this comment.
Something is off with the overall logic. If you run the tests in watch mode and log something in the console, the tests don't fail. However, if you do the same outside the watch mode, it fails correctly.
We can reproduce the issue with this diff and yarn test:watch, yarn test:unit.
diff --git a/packages/material-ui-lab/src/Alert/Alert.test.js b/packages/material-ui-lab/src/Alert/Alert.test.js
index c4de7927fd..9a88e421c8 100644
--- a/packages/material-ui-lab/src/Alert/Alert.test.js
+++ b/packages/material-ui-lab/src/Alert/Alert.test.js
@@ -5,7 +5,7 @@ import describeConformance from '@material-ui/core/test-utils/describeConformanc
import Alert from './Alert';
import Paper from '@material-ui/core/Paper';
-describe('<Alert />', () => {
+describe.only('<Alert />', () => {
const mount = createMount();
let classes;
@@ -20,4 +20,8 @@ describe('<Alert />', () => {
refInstanceof: window.HTMLDivElement,
skip: ['componentProp'],
}));
+
+ it('should work', () => {
+ console.error('oops');
+ });
});There was a problem hiding this comment.
This is an issue with mocha: mochajs/mocha#4347
There was a problem hiding this comment.
Upvoted, thanks for the reference.
There was a problem hiding this comment.
I'm not going to work on that issue since it's trivial to get working: explicitly expect an error or negate that expectation. A test without assertion is not a placeholder for "no error".
There was a problem hiding this comment.
Yeah, I think that we can wait for mocha to solve it :). It took me some time to understand what was going wrong between the CI in my local env. Now, it's documented
There was a problem hiding this comment.
It kind of was documented in this PR: https://github.com/mui-org/material-ui/blob/45cb77ff02484e608467baded0b280fadd286467/test/README.md#L53-L55
There was a problem hiding this comment.
Should've included yarn test:watch but I never use it. Should've explicitly requested review of this file so that we're all on the same page.
Currently we simply throw in
console.errorandconsole.warncalls. This creates hard to parse stacktraces where the origin of theconsolecall is deeply hidden in the stacktrace. We also have a dedicated helper to fully mock and silenceconsole.errorcalls e.g. https://app.circleci.com/pipelines/github/mui-org/material-ui/14984/workflows/928a1113-8c3b-45a9-8655-db59d54bc14d/jobs/160938/steps. This enables problematic tests suites where only a subset of the messages is asserted or errors are completely ignored.We now provide better stacktraces for unexpected error calls:
Documentation: test/console-error/test/README.md#unexpected-calls-to-consoleerror-or-consolewar
This PR also introduces dedicated
toWarnDevandtoErrorDevhelpers which require that every message is expected. You could still trick it with empty messages but this makes the problem stand out a bit more.Documentation for
toErrorDev: test/console-error/test/README.md#writing-a-test-for-consoleerror-or-consolewarnTest suite for toErrorDev (showcasing stacktraces): test/console-error/test/utils/initMatchers.test.js
/cc @mui-org/core to check out https://github.com/eps1lon/material-ui/blob/test/console-error/test/README.md#unexpected-calls-to-consoleerror-or-consolewar and https://github.com/eps1lon/material-ui/blob/test/console-error/test/README.md#writing-a-test-for-consoleerror-or-consolewarn and comment if there are any questions.