Closed as not planned
Description
Specification
At the moment if a test fails on the CI/CD we are able to see some output (which is scraped from our Junit reporting):
Unfortunately, this output does not include stderr/stdout, which is often useful for debugging. While Junit does have the ability to include this information in the xml
report it generates, there are several issues that prevent us from utilising this feature:
- Jest has these problems Running tests in verbose mode with console logging breaks output jest-community/jest-react-reporter#4 (comment)
- Console output is not properly captured unless verbose is set to false
- We can override this for our tests when running on the CI/CD, but it's still not ideal
- Jest can only capture from console output (
console.error
/console.log
), but we want to be able to capture fromprocess.stderr
/process.stdout
(more control over the format + easier to read)
- Junit has these problems console support jest-community/jest-junit#45 (comment)
- The issue with needing to run at least two test files seems to be resolved, at least, I'm observing console output being reported for shards containing only one test file
- The verbosity issue still remains though and is subject to the above bug with Jest being fixed
- Gitlab has these problems https://gitlab.com/gitlab-org/gitlab/-/issues/287664
- Even once the Jest and Junit issues above are resolved, Gitlab itself cannot display console output as it is currently formatted in the Junit output
- Once Gitlab is able to display information from tags then the other issues can be worked around
We need to watch these issues so that we can make use of this information when debugging in the CI/CD.
Additional context
- Running tests in verbose mode with console logging breaks output jest-community/jest-react-reporter#4 (comment) - watch this issue (Jest)
- console support jest-community/jest-junit#45 (comment) - watch this issue (Junit)
- https://gitlab.com/gitlab-org/gitlab/-/issues/287664 - watch this issue (Gitlab)
- Test load balancing using jest's
--shard
option #65 - PR for test load balancing, where this issue was originally looked into and the above issues were discovered
Tasks
- ...
- ...
- ...