Windows 2.11 nightly is intermittently failing

[SethTisue]

https://scala-ci.typesafe.com/job/scala-2.11.x-integrate-windows/128/

  [partest] !!  6 - presentation/doc                          [output differs]

% diff Y:\jenkins\workspace\scala-2.11.x-integrate-windows\test\files\presentation\doc-presentation.log Y:\jenkins\workspace\scala-2.11.x-integrate-windows\test\files\presentation\doc.checkdiff Y:\jenkins\workspace\scala-2.11.x-integrate-windows\test\files\presentation\doc-presentation.log Y:\jenkins\workspace\scala-2.11.x-integrate-windows\test\files\presentation\doc.check
--- doc.check
+++ doc-presentation.log
@@ -1,1 +1,3 @@
 reload: Base.scala, Class.scala, Derived.scala
+Unexpected Derived class comment:None
+Unexpected foo method comment:None

[SethTisue]

this has been failing on and off for a while, even when the same commit was being tested. we just didn't notice the failures, because scala/scala-jenkins-infra#154 . apparently the test is nondeterministic.

[SethTisue]

haha, @retronym and @dragos have been here before: Jason disabled this test once already in scala/scala@8986ee4 ("The test seems to only fail on the windows nightly build"), then Iulian reinstated it in scala/scala@1122e9e

[SethTisue]

I have not had any luck reproducing the failure locally on Mac OS X, even on a heavily loaded machine (heavily enough to more than double the time it takes the test to run).

Nor did I have any luck reproducing it on jenkins-worker-windows-publish by running cd ~jenkins/workspace/scala-2.11.x-integrate-windows; test/partest test/files/presentation/doc. Dozens of passes, zero failures. Same result even if I load the machine.

[retronym]

Triggered a build which should appear at https://scala-ci.typesafe.com/job/scala-2.11.x-integrate-windows/134 soon.

[SethTisue]

optimistically closing.

[som-snytt]

I got it to fail on CI but not on local Ubuntu WSL.

+Unexpected Derived class comment:Some(Body(List(Paragraph(Chain(List(Summary(Text($BaseComment
+)))))))
+)
+Unexpected foo method comment:None

A different test failed on Ubuntu with jdk 14, but passes on jdk 8. Forgot to run --show-diff:

!! 19 - presentation/infix-completion             [output differs]
!! 20 - presentation/infix-completion2            [output differs]

[som-snytt]

I have no memory of my previous comment, but

!!  1 - presentation/doc                          [output differs][duration 12.41s]
% diff /home/jenkins/workspace/scala-2.13.x-validate-main/test/files/presentation/doc.check /home/jenkins/workspace/scala-2.13.x-validate-main/test/files/presentation/doc-presentation.log
@@ -1,2 +1,3 @@
 warning: 1 deprecation (since 2.13.0); re-run with -deprecation for details
 reload: Base.scala, Class.scala, Derived.scala
+Unexpected foo method comment:None

##### Log file '/home/jenkins/workspace/scala-2.13.x-validate-main/test/files/presentation/doc-presentation.log' from failed test #####

warning: 1 deprecation (since 2.13.0); re-run with -deprecation for details
reload: Base.scala, Class.scala, Derived.scala
Unexpected foo method comment:None

https://scala-ci.typesafe.com/job/scala-2.13.x-validate-main/15724/testReport/junit/test.files/presentation/doc/

[som-snytt]

The default timeout is 1 second per operation. But I don't see any .get(timeout), just .get which waits until interruption, so maybe that is the intermittency. I was willing to add retries on timeouts.

[som-snytt]

haha, retronym and dragos have been here before:

haha, it seems I've been here before.

I got fed up at scala/scala#10334 (comment).

I don't even have my usual excuse that my previous comment was pre-pandemic. My other excuse is that I block out anything to do with the presentation compiler. If I make an effort, though, I wonder if they'll give me a "Presentation is Everything!" t-shirt.

[som-snytt]

Leaving a breadcrumb this time back to the current ticket.