Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Final line of shell output without newline lost when using LineTransformationOutputStream-based decorators #112

Merged
merged 6 commits into from
Jul 25, 2019

Conversation

jglick
Copy link
Member

@jglick jglick commented Jul 23, 2019

Fixing the test from #103, strengthening it in the process. Taking on some urgency since after @Vlatombe’s jenkinsci/kubernetes-plugin#552, this problem would affect any sh step in a Kubernetes agent which prints output without a trailing newline (no longer limited to that minority of builds which deliberately use secrets).

@@ -132,7 +132,7 @@
<dependency>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>credentials-binding</artifactId>
<version>1.16</version>
<version>1.17</version>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To pick up jenkinsci/credentials-binding-plugin#51. This seems to be the reason that, in watch mode, the test was previously failing (output was getting lost) even in situations that did not involve a missing final newline.

pom.xml Outdated Show resolved Hide resolved
pom.xml Outdated Show resolved Hide resolved
It is a bit scary, however, and produces poorly formatted log text like:
missing final newline node=remote watching=true[Pipeline] }
which would confuse, for example, pipeline-cloudwatch-logs.
@jglick jglick changed the title Working on ShellStepTest.missingNewline Fix for ShellStepTest.missingNewline Jul 23, 2019
@jglick jglick changed the title Fix for ShellStepTest.missingNewline Final line of shell output without newline lost when using LineTransformationOutputStream-based decorators Jul 23, 2019
@jglick jglick added the bug label Jul 23, 2019
jglick added a commit to jglick/pipeline-cloudwatch-logs-plugin that referenced this pull request Jul 23, 2019
jglick added a commit to jglick/kubernetes-plugin that referenced this pull request Jul 24, 2019
@jglick jglick requested a review from dwnusbaum July 24, 2019 00:02
@jglick jglick marked this pull request as ready for review July 24, 2019 00:03
@@ -599,6 +652,7 @@ private void handleExit(int exitCode, OutputSupplier output) throws IOException,
getContext().onFailure(new AbortException("script returned exit code " + exitCode));
}
}
listener().getLogger().close();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this call will always be on NewlineSafeTaskListener, right? Is this why we don't want to call super.close() in NewlineSafeTaskListener.$NewlineSafeTaskListener.close()? Ditto for the call below in exited. Also, if we are calling close here now, why didn't we call flush here earlier? Should we have been doing that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this call will always be on NewlineSafeTaskListener, right?

Right.

Is this why we don't want to call super.close() in NewlineSafeTaskListener.$NewlineSafeTaskListener.close()?

Yes.

if we are calling close here now, why didn't we call flush here earlier?

There is no particular need to flush until the end of the step (and close flushes).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no particular need to flush until the end of the step (and close flushes).

My question was unclear, but my confusion is about why we weren't calling flush before this PR, since it looks like this is the end of the step.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see! We were calling flush before, in watch mode, in exited. In non-watch mode, it did not really matter since the log messages were coming from the master, and DefaultStepContext.getListener always closes the per-step listener when the step finishes.

Copy link
Member

@dwnusbaum dwnusbaum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine. I am struggling to keep track of when to flush or not, what overrides of close also close their delegates, and what is getting remoted or not in Pipeline logging in general, so my approval is mostly just based on passing tests and no Java-level issues that I see.

@@ -599,6 +652,7 @@ private void handleExit(int exitCode, OutputSupplier output) throws IOException,
getContext().onFailure(new AbortException("script returned exit code " + exitCode));
}
}
listener().getLogger().close();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no particular need to flush until the end of the step (and close flushes).

My question was unclear, but my confusion is about why we weren't calling flush before this PR, since it looks like this is the end of the step.

@jglick
Copy link
Member Author

jglick commented Jul 25, 2019

I am struggling to keep track of when to flush or not, what overrides of close also close their delegates, and what is getting remoted or not in Pipeline logging in general

So am I!

@dwnusbaum dwnusbaum merged commit 00db760 into jenkinsci:master Jul 25, 2019
@jglick jglick deleted the missingNewline-redux branch July 25, 2019 19:18
@@ -65,6 +65,7 @@
<revision>2.33</revision>
<changelist>-SNAPSHOT</changelist>
<jenkins.version>2.176.1</jenkins.version>
<jenkins-test-harness.version>2.54</jenkins-test-harness.version> <!-- TODO pending plugin-pom update -->
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jglick jglick mentioned this pull request Jul 29, 2019
jglick added a commit to jglick/kubernetes-plugin that referenced this pull request Jul 29, 2019
jglick added a commit to jglick/pipeline-cloudwatch-logs-plugin that referenced this pull request Jul 29, 2019
jglick added a commit to jglick/docker-workflow-plugin that referenced this pull request Jan 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants