Skip to content

Conversation

@cstamas
Copy link
Member

@cstamas cstamas commented Oct 20, 2025

When --raw-streams is used (especially when combined with options like --quiet and -DforceStdout) there is no guarantee that anything touches terminal. Hence, in case of embedded executor it is quite possible that cached/warm code arrives quickly at the place that would do system out before the thread with FastTerminal finishes system install.

In other words, when --raw-streams are used, we cannot guarantee that system streams are already properly set up. This PR changes that, and makes sure (by triggering a dummy call to terminal), at the cost of "jline3 install lag" for CLI invocation. OTOH, this lag in case of embedded executors does not exists (it exists only on first invocation).

My suspicion that this is the cause of IT instability issues, when Verifier used Toolbox output ends up on Surefire stdout and not on "grabbed" output, as simply streams are not yet set up properly. Also, this usually happens "around the second half" of ITs, when cached and warmed up embedded instance is already uber optimized.

When --raw-streams are used (especially when combined with options
like --quiet and -DforceStdout) there is no guarantee that
anything touches terminal. Hence, in case of embedded executor
it is quite possible that cached/warm code arrives fast at
the place that would do system out before the thread with
FastTerminal finishes.

In other words, when --raw-streams are used, we cannot right
now guarantee that system streams are already set up. This PR
changes that, and makes sure by triggering a dummy call
to terminal, at the cost of "jline3 install lag" for CLI
invocation. OTOH, this lag in case of embedded executors
does not exists (it exists only on first invocation).
@cstamas cstamas added this to the 4.1.0 milestone Oct 20, 2025
@cstamas cstamas requested a review from gnodet October 20, 2025 12:44
@cstamas cstamas self-assigned this Oct 20, 2025
@cstamas cstamas added the bug Something isn't working label Oct 20, 2025
@gnodet
Copy link
Contributor

gnodet commented Oct 20, 2025

When --raw-streams is used (especially when combined with options like --quiet and -DforceStdout) there is no guarantee that anything touches terminal. Hence, in case of embedded executor it is quite possible that cached/warm code arrives quickly at the place that would do system out before the thread with FastTerminal finishes system install.

In other words, when --raw-streams are used, we cannot guarantee that system streams are already properly set up. This PR changes that, and makes sure (by triggering a dummy call to terminal), at the cost of "jline3 install lag" for CLI invocation. OTOH, this lag in case of embedded executors does not exists (it exists only on first invocation).

My suspicion that this is the cause of IT instability issues, when Verifier used Toolbox output ends up on Surefire stdout and not on "grabbed" output, as simply streams are not yet set up properly. Also, this usually happens "around the second half" of ITs, when cached and warmed up embedded instance is already uber optimized.

If I follow correctly, the problem happens when using raw-streams because the system streams are not modified in this case. In the default case, system streams are set and redirect to the logger, which then redirect to the terminal, thereby ensuring the FastTerminal is initialised.
So ensuring the terminal is initialised when using --raw-streams should fix the problem. Another possible way would be to setup system stream wrappers that would simply wait for the terminal to be initialised, before waiting for the stream to delegate, but the current PR looks much simpler.

@cstamas cstamas marked this pull request as ready for review October 21, 2025 09:15
@cstamas
Copy link
Member Author

cstamas commented Oct 21, 2025

Yes, exactly as you wrote. But have to note, that IMO this affects ONLY "repeated embedded executor usage", hence ITs. There, embedded executor keeps "hot" embedded Maven, and as we saw, sporadically happens only (tl;dr toolbox plugin execution returns with empty string but without error, as stdout output landed on surefire out; causing all kind of havoc in ITs as given IT instead deleting own files starts deleting whole local repository). But, when it happens, it causes a range on weird issues and at the end IT failures.

One of these issues were recorded as apache/maven-resolver#1636 but now I am sure this is not a bug per-se, but in fact one of the "symptoms" of this bug (as two or more parallel executing IT make each other fails, by pulling the rug beneath them, as one just nuke local repository).

Make the hacks ONLY if embedded+raw-streams.

Fix toolbox tool to implement proper expectation too.
As stderr will contain warnings on higher Java versions
(guava and other; terminally deprecated method...)
@cstamas cstamas merged commit bb94de0 into apache:master Oct 21, 2025
18 of 19 checks passed
@cstamas cstamas deleted the raw-streams-bug branch October 21, 2025 11:35
cstamas added a commit to cstamas/maven that referenced this pull request Oct 21, 2025
…che#11303)

When `--raw-streams` is used (especially when combined with options like `--quiet` and `-DforceStdout`) there is no guarantee that anything touches terminal. Hence, in case of embedded executor it is quite possible that cached/warm code arrives quickly at the place that would do system out **before** the thread with `FastTerminal` finishes system install.

In other words, when `--raw-streams` are used, we cannot guarantee that system streams are already properly set up. This PR changes that, and makes sure (by triggering a dummy call to terminal), at the cost of "jline3 install lag" for CLI invocation. OTOH, this lag in case of embedded executors does not exists (it exists only on first invocation).

My suspicion that this is the cause of IT instability issues, when Verifier used Toolbox output ends up on Surefire stdout and not on "grabbed" output, as simply streams are not yet set up properly. Also, this usually happens "around the second half" of ITs, when cached and warmed up embedded instance is already uber optimized.
cstamas added a commit that referenced this pull request Oct 21, 2025
) (#11310)

When `--raw-streams` is used (especially when combined with options like `--quiet` and `-DforceStdout`) there is no guarantee that anything touches terminal. Hence, in case of embedded executor it is quite possible that cached/warm code arrives quickly at the place that would do system out **before** the thread with `FastTerminal` finishes system install.

In other words, when `--raw-streams` are used, we cannot guarantee that system streams are already properly set up. This PR changes that, and makes sure (by triggering a dummy call to terminal), at the cost of "jline3 install lag" for CLI invocation. OTOH, this lag in case of embedded executors does not exists (it exists only on first invocation).

My suspicion that this is the cause of IT instability issues, when Verifier used Toolbox output ends up on Surefire stdout and not on "grabbed" output, as simply streams are not yet set up properly. Also, this usually happens "around the second half" of ITs, when cached and warmed up embedded instance is already uber optimized.

Backport of bb94de0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-to-4.0.x bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants