Add CI-friendly progress output for tests

[potiuk]

This is the first step to run breeze tests in parallel in CI.
This flag adds "limited progress" output when running tests
which means that the runnig tests will just print few lines with
percent progress and color status indication from last few
progress lines of Pytest output, but when it completes, the whole output is
printed in a CI group - colored depending on status.

The final version (wnen we implement parallel test execution) should
also defer writing the output to until all tests are completed, but
this should be a follow-up PR.

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragement file, named {pr_number}.significant.rst, in newsfragments.

[potiuk]

cc: @Bowrna -> here is a part of the "parallel" test implementation. Not yet complete, but shoudl be closer to implement running tests in parallel and should be easy to parallelise it with #23715

[potiuk]

Example output:

• Initially only the single lines are printed with "progress" (44%)
• but when the whole tests complete, the output of tests is printed in the "foldable" group (In this case "Result of ...." )
• in the output you see a "green" line, but in CI it will be a "folded" group of output (same as in the current tests)
• the color of the group is "green" when tests succeed or "red" if they fail (same as in current tests)

[potiuk]

Similar output for bigger number of tests when we see how the progress works:

[potiuk]

Last version has a bit better-suited output for Parallel CI tests. We see two lines of the output + overall progress for each test "process".

[potiuk]

Note - that this one is optimized for our CI usage. We cannot keep log output in memory, because we often might not have enough memory on GitHub Public runners. That's why each test "run" will simply write the output to a temporary file and we are using this file as the "progress status" source. By writing directly to file rather than getting the logs piped to our "managing" process we save precious memory for the actual test execution.

On the other hand, when we will run two tests in parallel, we cannot interleave the output from both processes directly to stdout because it will be a complete mess.

That's why it is also a bit more "complex" with separate thread monitoring the files and only occasionally prints the last few lines to show the progress (otherwise if someone watches the CI output they might be not aware that something is actually happening there and the tests are progresssing).

Seeing the "coloured" progress from the last two lines is also good so that there is an early indicator if there was an error already during test execution (the number will get red) so that the user can get early feedback and cancel/rerun CI build with a fix-up in case they are actively monitoring the output.

[potiuk]

Tt's green and good next step to get rid of the bash finally

[Bowrna]

@potiuk I didn't make much progress in the issue #23538 as I was stuck on what I have to do next in the PR to proceed further. So I started focussing on other PR's and missed updating any progress on that issue.

Thanks for this PR. I will read this and start focussing on completing this issue.

[potiuk]

@potiuk I didn't make much progress in the issue #23538 as I was stuck on what I have to do next in the PR to proceed further. So I started focussing on other PR's and missed updating any progress on that issue.

Thanks for this PR. I will read this and start focussing on completing this issue.

Yep. I just implemented it as I thought it might be useful for your PR (and got some fun while doing it :)

[potiuk]

PR Looking for some approval here :)

[potiuk]

An approval would be nice :)

[potiuk]

Need it to improve testing speed in CI.

[josh-fell]

Small comment on using the new Enum in other places, but otherwise LGTM.

[github-actions]

The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full test matrix is needed and add the 'full tests needed' label. Then you should rebase it to the latest main or amend the last commit of the PR, and push it with --force-with-lease.