Implement `numPassingAsserts` of testCaseResult #13795

ymqy · 2023-01-21T07:51:05Z

Summary

implement numPassingAsserts to track the number of passing asserts in a single test
refactor the implementation of numPassingAsserts in testResult object
make numPassingAsserts actual value available in custom reporter

Test plan

ci green

SimenB

thanks! can you add a unit or e2e test as well verifying the count is correct for both passing and failing assertions?

ymqy · 2023-01-23T14:33:10Z

Thank you for your feedback. I will definitely add unit or e2e test to verify the correctness of the numPassingAsserts count for both passing and failing assertions. I will make sure to update the pull request with the new test cases and let you know once it's ready for review.

thanks! can you add a unit or e2e test as well verifying the count is correct for both passing and failing assertions?

ymqy · 2023-01-24T15:53:31Z

@SimenB Added unit and e2e tests for numPassingAsserts count. Please let me know if there is anything else I can do to improve the implementation.

ymqy · 2023-01-24T16:33:25Z

e2e/__tests__/customReporters.test.ts

+      ],
+      {
+        env: {
+          JEST_JASMINE: '0',


Is it feasible to hard code jest-circus as the testRunner here? because numPassingAsserts is not implemented in jest-jasmine.

You could use this helper (search the repo for usage examples): https://github.com/facebook/jest/blob/3d7a096680078854b06c213403d063128386ab1f/packages/test-utils/src/ConditionalTest.ts#L18

Thanks, but it will skip all test of file, it might be better to find a way to skip the specific test.

mrazauskas · 2023-01-24T18:33:42Z

packages/test-utils/src/ConditionalTest.ts

@@ -46,3 +46,9 @@ export function onNodeVersions(
    });
  }
 }
+
+export function onNotJestJasmine(testBody: () => void): void {


Perhaps skipTestOnJasmine would be better name in this case?

Suggested change

export function onNotJestJasmine(testBody: () => void): void {

export function skipTestOnJasmine(testBody: () => void): void {

Thank you for the suggestion, I have updated the function name to 'skipTestOnJasmine'

SimenB · 2023-01-25T09:01:40Z

packages/test-utils/src/ConditionalTest.ts

@@ -46,3 +46,9 @@ export function onNodeVersions(
    });
  }
 }
+
+export function skipTestOnJasmine(testBody: () => void): void {


why is a new helper needed? shouldn't skipSuiteOnJasmine be enough? That one also supports snapshots properly

skipSuiteOnJasmine will skip all test of file, or is it better to create a separate test file to verify the assertion count and call skipSuiteOnJasmine in the jasmine runtime environment?

SimenB · 2023-01-25T09:02:18Z

packages/jest-circus/src/legacy-code-todo-rewrite/jestAdapterInit.ts

@@ -238,6 +238,7 @@ const eventHandler = async (event: Circus.Event) => {
      break;
    }
    case 'test_done': {
+      event.test.numPassingAsserts = jestExpect.getState().numPassingAsserts;


should we reset it?

numPassingAsserts is reset to 0 after execution of _addExpectedAssertionErrors, reset method is called within jestExpect.extractExpectedAssertionsErrors method.

https://github.com/facebook/jest/blob/d683aafde24f2cf6f6cb5a0a71069fd6a0a55c36/packages/jest-circus/src/legacy-code-todo-rewrite/jestAdapterInit.ts#L249-L253

SimenB · 2023-01-25T09:02:55Z

How does this work with jest.retryTimes? I assume it'd be confused. Not sure how to deal with it either, so...

ymqy · 2023-01-25T11:00:19Z

How does this work with jest.retryTimes? I assume it'd be confused. Not sure how to deal with it either, so...

Thanks for bringing this to our attention. You're correct that when using jest.retryTimes, numPassingAsserts will only save the assertion count of the last test run. To access the numPassingAsserts of each retried test in a custom reporter, you can access it from the onTestCaseResult event. If you're only interested in the last retried test, you can access it from the onTestFileResult event. This event only saves the testCaseResult of the last retried test.

ymqy · 2023-01-26T14:25:49Z

Tried to run my own project branch (ymqy:feature/numPassingAsserts) in Circle CI(Avoid pushing too many commit records), not sure why it affects the current PR's CI check, Submitted a request to the Circle support team to resolve this issue

SimenB

CI seems happy, so happy to land this. Thanks!

SimenB · 2023-01-26T15:10:08Z

https://github.com/facebook/jest/releases/tag/v29.4.1

SimenB · 2023-01-31T06:53:21Z

I don't work at Meta or help maintain React, so I'm not much help outside of Jest itself.

ymqy · 2023-01-31T06:55:29Z

I don't work at Meta or help maintain React, so I'm not much help outside of Jest itself.

thanks

Minor version bump to get the fix for `numPassingAsserts`: jestjs/jest#13795 Test Plan: CI

Minor version bump to get the fix for `numPassingAsserts`: jestjs/jest#13795 Test Plan: CI DiffTrain build for [2de85d7](2de85d7) [View git log for this commit](https://github.com/facebook/react/commits/2de85d7c712ff0f052d9c92f8129ed476f8ce4d8)

github-actions · 2023-03-03T01:48:59Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

ymqy added 5 commits January 21, 2023 15:42

chore: numPassingAsserts type

f442c15

chore: numPassingAsserts default value

edefeab

chore: increment numPassingAsserts

4173948

chore: numPassingAsserts type

590003b

chore: add numPassingAsserts to testResult

559c28f

facebook-github-bot added the cla signed label Jan 21, 2023

ymqy added 2 commits January 21, 2023 20:17

chore: changelog

a8e0584

chore: update changelog

9ecbb59

ymqy changed the title ~~Implement numPassingAsserts for custom reporter~~ Implement numPassingAsserts for custom reporter Jan 21, 2023

ymqy changed the title ~~Implement numPassingAsserts for custom reporter~~ Implement numPassingAsserts of testResult for custom reporter Jan 21, 2023

ymqy changed the title ~~Implement numPassingAsserts of testResult for custom reporter~~ Implement numPassingAsserts of testResult Jan 22, 2023

SimenB approved these changes Jan 22, 2023

View reviewed changes

ymqy added 2 commits January 23, 2023 19:48

chore: fix ci

d4f62c0

chore: TestEntry numPassingAsserts

a423029

ymqy force-pushed the feature/numPassingAsserts branch from 973f6a6 to a423029 Compare January 23, 2023 12:10

fix: missing numPassingAsserts

cdfd96f

ymqy added 2 commits January 24, 2023 22:44

chore: numPassingAsserts unit test

ba550ce

chore: AssertionCountsReporter e2e test

82e450f

ymqy added 2 commits January 25, 2023 00:18

chore: only test in jest-circus

0418a30

chore: update numPassingAsserts snapshot

94386e1

ymqy commented Jan 24, 2023

View reviewed changes

ymqy added 2 commits January 25, 2023 02:27

chore: onNotJestJasmine helper

249388c

chore: remove hard code

8e13c11

mrazauskas reviewed Jan 24, 2023

View reviewed changes

ymqy added 4 commits January 25, 2023 02:37

chore: rename

e5408ae

chore: update fn name

2aa739f

chore: update export

886a709

chore: update copyright

bae470a

move changelog entry

d683aaf

SimenB reviewed Jan 25, 2023

View reviewed changes

ymqy added 3 commits January 25, 2023 20:51

chore: remove skipTestOnJasmine

e920161

chore: customReportersOnCircus e2e test

7f2f2df

fix: ci error

9d0c78a

ymqy changed the title ~~Implement numPassingAsserts of testResult~~ Implement numPassingAsserts of testCaseResult Jan 26, 2023

ymqy added 2 commits January 26, 2023 12:06

Merge branch 'main' into feature/numPassingAsserts

f3dff25

Update CHANGELOG.md

e5cac79

ymqy mentioned this pull request Jan 26, 2023

Upgrading Jest to v29 and optimizing default configuration facebook/react#26041

Closed

SimenB approved these changes Jan 26, 2023

View reviewed changes

Update CHANGELOG.md

565fef8

SimenB merged commit c78905c into jestjs:main Jan 26, 2023

ymqy deleted the feature/numPassingAsserts branch January 26, 2023 14:41

ms-bot mentioned this pull request Feb 6, 2023

chore(deps): bump 🧳 jest to ^29.7.0 mobsuccess-devops/github-actions-asana#47

Draft

ymqy mentioned this pull request Feb 9, 2023

Migrate testRunner from jasmine2 to jest-circus facebook/react#26144

Merged

kassens added a commit to kassens/react that referenced this pull request Feb 10, 2023

Bump jest version

77841df

Minor version bump to get the fix for `numPassingAsserts`: jestjs/jest#13795 Test Plan: CI

kassens mentioned this pull request Feb 10, 2023

Minor Jest upgrade facebook/react#26150

Merged

kassens added a commit to facebook/react that referenced this pull request Feb 10, 2023

Minor Jest upgrade (#26150)

2de85d7

Minor version bump to get the fix for `numPassingAsserts`: jestjs/jest#13795 Test Plan: CI

ms-bot mentioned this pull request Feb 20, 2023

chore(deps): bump 🧳 jest to ^29.7.0 mobsuccess-devops/github-actions-review#40

Draft

ms-bot mentioned this pull request Feb 27, 2023

chore(deps): bump 🧳 jest to ^29.7.0 mobsuccess-devops/asana-magics#27

Closed

github-actions bot locked as resolved and limited conversation to collaborators Mar 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement `numPassingAsserts` of testCaseResult #13795

Implement `numPassingAsserts` of testCaseResult #13795

ymqy commented Jan 21, 2023 •

edited

Loading

SimenB left a comment

ymqy commented Jan 23, 2023

ymqy commented Jan 24, 2023

ymqy Jan 24, 2023 •

edited

Loading

mrazauskas Jan 24, 2023

ymqy Jan 24, 2023 •

edited

Loading

mrazauskas Jan 24, 2023

ymqy Jan 24, 2023

SimenB Jan 25, 2023

ymqy Jan 25, 2023 •

edited

Loading

SimenB Jan 25, 2023

ymqy Jan 25, 2023 •

edited

Loading

SimenB commented Jan 25, 2023

ymqy commented Jan 25, 2023 •

edited

Loading

ymqy commented Jan 26, 2023

SimenB left a comment

SimenB commented Jan 26, 2023

SimenB commented Jan 31, 2023

ymqy commented Jan 31, 2023

github-actions bot commented Mar 3, 2023

	export function onNotJestJasmine(testBody: () => void): void {
	export function skipTestOnJasmine(testBody: () => void): void {

Implement numPassingAsserts of testCaseResult #13795

Implement numPassingAsserts of testCaseResult #13795

Conversation

ymqy commented Jan 21, 2023 • edited Loading

Summary

Test plan

SimenB left a comment

Choose a reason for hiding this comment

ymqy commented Jan 23, 2023

ymqy commented Jan 24, 2023

ymqy Jan 24, 2023 • edited Loading

Choose a reason for hiding this comment

mrazauskas Jan 24, 2023

Choose a reason for hiding this comment

ymqy Jan 24, 2023 • edited Loading

Choose a reason for hiding this comment

mrazauskas Jan 24, 2023

Choose a reason for hiding this comment

ymqy Jan 24, 2023

Choose a reason for hiding this comment

SimenB Jan 25, 2023

Choose a reason for hiding this comment

ymqy Jan 25, 2023 • edited Loading

Choose a reason for hiding this comment

SimenB Jan 25, 2023

Choose a reason for hiding this comment

ymqy Jan 25, 2023 • edited Loading

Choose a reason for hiding this comment

SimenB commented Jan 25, 2023

ymqy commented Jan 25, 2023 • edited Loading

ymqy commented Jan 26, 2023

SimenB left a comment

Choose a reason for hiding this comment

SimenB commented Jan 26, 2023

SimenB commented Jan 31, 2023

ymqy commented Jan 31, 2023

github-actions bot commented Mar 3, 2023

Implement `numPassingAsserts` of testCaseResult #13795

Implement `numPassingAsserts` of testCaseResult #13795

ymqy commented Jan 21, 2023 •

edited

Loading

ymqy Jan 24, 2023 •

edited

Loading

ymqy Jan 24, 2023 •

edited

Loading

ymqy Jan 25, 2023 •

edited

Loading

ymqy Jan 25, 2023 •

edited

Loading

ymqy commented Jan 25, 2023 •

edited

Loading