Fix passing jvm options #14073
Fix passing jvm options #14073
Conversation
|
@philwalk I wonder whether we correctly interpret the |
|
Also passing env variable to scala3 via coursier is working, because coursier runs the JVM with given java setting, the problem is with our |
BarkingBad
force-pushed
the
passing-jvm-opts-to-scala
branch
from
0f3c819
to
77b4cba
Compare
|
We should support -D and not just -J-D since the scala 2 shell scripts support -D |
BarkingBad
force-pushed
the
passing-jvm-opts-to-scala
branch
from
77b4cba
to
61a3c08
Compare
BarkingBad
force-pushed
the
passing-jvm-opts-to-scala
branch
from
61a3c08
to
c5ef535
Compare
|
@smarter, do you think that we could backport this? |
|
Don't think it's critical enough. |
|
These guys have problem because of the issue
|
BarkingBad
force-pushed
the
passing-jvm-opts-to-scala
branch
from
c5ef535
to
79d9f5f
Compare
|
I'm not opposed to a backport but you'll have to ask the release manager for this release (@Kordyjan ?) to see if it can be done in time. |
|
The first test failure is a result of the hashbang line of Output (click arrow to expand)2021-12-09T15:12:25.7872572Z ===> unglobClasspathVerifyTest for script [out/bootstrap/scala3-compiler-bootstrapped/scala-3.1.2-RC1-bin-SNAPSHOT-nonbootstrapped/test-classes/scripting/unglobClasspath.sc]
2021-12-09T15:12:25.7876437Z bash is [/usr/bin/bash]
2021-12-09T15:12:25.7882719Z [/usr/bin/bash]
2021-12-09T15:12:25.7884079Z [-c]
2021-12-09T15:12:25.7887428Z [SCALA_OPTS= out/bootstrap/scala3-compiler-bootstrapped/scala-3.1.2-RC1-bin-SNAPSHOT-nonbootstrapped/test-classes/scripting/unglobClasspath.sc]
2021-12-09T15:12:26.7941440Z bad option '-classpath dist/target/pack/lib/*' was ignored
2021-12-09T15:12:27.5518229Z exception occurred while compiling , unglobClasspath.sc
2021-12-09T15:12:27.5522489Z Exception in thread "main" java.io.IOException: Is a directory
2021-12-09T15:12:27.5527509Z at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
2021-12-09T15:12:27.5532084Z at java.base/sun.nio.ch.FileDispatcherImpl.read(FileDispatcherImpl.java:48)
2021-12-09T15:12:27.5535828Z at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)
2021-12-09T15:12:27.5539025Z at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:245)
2021-12-09T15:12:27.5542187Z at java.base/sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:229)
2021-12-09T15:12:27.5546078Z at java.base/sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
2021-12-09T15:12:27.5548863Z at java.base/sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:107)
2021-12-09T15:12:27.5551159Z at java.base/sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:101)
2021-12-09T15:12:27.5553412Z at dotty.tools.io.AbstractFile.toByteArray(AbstractFile.scala:176)
2021-12-09T15:12:27.5555467Z at dotty.tools.dotc.util.SourceFile$.apply(SourceFile.scala:277)
2021-12-09T15:12:27.5557449Z at dotty.tools.dotc.core.Contexts$Context.getSource$$anonfun$1(Contexts.scala:276)
2021-12-09T15:12:27.5560476Z at dotty.tools.dotc.util.GenericHashMap.getOrElseUpdate(GenericHashMap.scala:133)
2021-12-09T15:12:27.5563106Z at dotty.tools.dotc.core.Contexts$Context.getSource(Contexts.scala:276)
2021-12-09T15:12:27.5564661Z at dotty.tools.dotc.Run.$anonfun$1(Run.scala:205)
2021-12-09T15:12:27.5566148Z at scala.collection.immutable.List.map(List.scala:246)
2021-12-09T15:12:27.5567775Z at dotty.tools.dotc.Run.compile(Run.scala:205)
2021-12-09T15:12:27.5569296Z at dotty.tools.dotc.Driver.doCompile(Driver.scala:39)
2021-12-09T15:12:27.5571774Z at dotty.tools.scripting.ScriptingDriver.compileAndRun(ScriptingDriver.scala:28)
2021-12-09T15:12:27.5574190Z at dotty.tools.scripting.Main$.main(Main.scala:43)
2021-12-09T15:12:27.5575976Z at dotty.tools.MainGenericRunner$.run$1(MainGenericRunner.scala:230)
2021-12-09T15:12:27.5577992Z at dotty.tools.MainGenericRunner$.main(MainGenericRunner.scala:239)
2021-12-09T15:12:27.5580265Z at dotty.tools.MainGenericRunner.main(MainGenericRunner.scala) |
|
As we are still waiting for #13777, I think we can give it a try. I think if we manage to merge this PR tomorrow and have backporting PR merged no later than Monday noon we can squeeze this fix into 3.1.1-RC2. |
|
Two of the test failures have the same cause (regression of the fix for #10761). -cp*|-classpath*) # partial fix for #10761
# hashbang can combine args, e.g. "-classpath 'lib/*'"
CLASS_PATH="${1#* *}${PSEP}"
let class_path_count+=1
shift
;; |
|
Well, yes, but actually no. I enabled them by adding The third problem was with handling our Now it should work as expected |
|
I also had to remove the ReplTests which cannot work - we are using mock driver in the same JVM context, thus we cannot just pass the |
The /usr/bin/env: invalid option -- 'S'
Try '/usr/bin/env --help' for more information. |
|
It is working from |
|
@philwalk now since I "enabled" scripting tests do you mind adding some assertions when the scripts fail? Many of them have the |
I'm running Ubuntu 18.04, which specifies
It was intentional, based on feedback in this discussion: #12962 . The assumption is that if we're unable to run the test, that's different than if the test is run but fails. If either java or scala are not in the PATH, should we fail the test? Perhaps so. The following environment variables may need to be properly initialized for environments that fail FWIW, the 5 failing tests are running in an environment with JAVA_HOME undefined, and with the first java in the PATH is at |
|
I think we should emulate scala2 behavior, any other approach will add to the migration burden for users. |
|
@BarkingBad Unless every test environment provides bash, and has adequate permissions to execute it, the current behavior seems to be necessary. Perhaps I didn't understand your suggestion. |
Do you have a link to a discussion of this issue? The link you provided currently shows passing tests. |
|
I'm almost ready to push an updated |
The test is passing, but it prints stderr with non zero return codes |
|
I'm away from home for the weekend, but as I looked yesterday on logs there was some stderr pointing to different java versions IIRC, yet I had no time to fix it (it's CI subject problem, not the scala logic) |
|
@BarkingBad
It seems that it might be scala logic, because the Assuming it solves this problem, what's the best way to provide this update? I can do a PR. Here's the click to expand:diff --git a/compiler/test/dotty/tools/scripting/BashScriptsTests.scala b/compiler/test/dotty/tools/scripting/BashScriptsTests.scala
index 1cdb700d5a..470f53acda 100644
--- a/compiler/test/dotty/tools/scripting/BashScriptsTests.scala
+++ b/compiler/test/dotty/tools/scripting/BashScriptsTests.scala
@@ -167,6 +167,7 @@ class BashScriptsTests:
def exists: Boolean = s.toPath.toFile.exists
def name: String = s.toFile.getName
def dropExtension: String = s.reverse.dropWhile(_ != '.').drop(1).reverse
+ def parent(up: Int): String = s.norm.split("/").reverse.drop(up).reverse.mkString("/")
}
extension(p: Path) {
@@ -201,22 +202,21 @@ class BashScriptsTests:
if scalacPath.isFile then scalacPath.replaceAll("/bin/scalac", "")
else envOrElse("SCALA_HOME", "").norm
- lazy val javaHome = envOrElse("JAVA_HOME", "").norm
+ lazy val javaHome = whichJava.parent(2)
lazy val testEnvPairs = List(
+ ("whichJava", whichJava),
("JAVA_HOME", javaHome),
("SCALA_HOME", scalaHome),
("PATH", adjustedPath),
).filter { case (name, valu) => valu.nonEmpty }
- lazy val whichBash: String =
- var whichBash = ""
- if osname.startsWith("windows") then
- whichBash = which("bash.exe")
- else
- whichBash = which("bash")
+ lazy val whichBash: String = whichExe("bash")
+ lazy val whichJava: String = whichExe("java")
- whichBash
+ def whichExe(basename: String): String =
+ val exeName = if (osname.toLowerCase.startsWith("windows")) s"$basename.exe" else basename
+ which(exeName)
def bashCommand(cmdstr: String, additionalEnvPairs: List[(String, String)] = Nil): (Boolean, Int, Seq[String], Seq[String]) = {
var (stdout, stderr) = (List.empty[String], List.empty[String])
|
|
@philwalk can you push directly on top of my branch? I think it is possible for maintainers. If not, maybe you could do some branch with fix and I'll cherry-pick your commit, would that be OK? |
This sets `javaHome` at the head of the PATH for `bashCommands`.
|
I was able to update |
|
@Kordyjan should we backport these changes? |
|
@philwalk apparently this PR broke CI on master: the logs are: Logs
This is strange, becuase there is |
|
Also, test_windows_full failed because of missing artifacts (it didn't come up in PR because there is only fast check), this PR should fix it, and check if test_windows_full is passing #14106 |
I'm investigating now ... The test assumes that the current working directory is the first entry on the classpath, which appears to be the case. I notice that the first line of output has a leading space ahead of I have refactored all of the Also, I changed the code that stripped the "cwd: " off of the first line of script output to get the "output[]" value, and now, rather than strip the "cwd: " prefix, I test whether |
|
The following error is caused by the Cygwin does not yet support coreutils 8.30: $ /usr/bin/env --version
env (GNU coreutils) 8.26
Packaged by Cygwin (8.26-2)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.One fix might be to add this back to -cp*|-classpath*) # partial fix for #10761
# hashbang can combine args, e.g. "-classpath 'lib/*'"
CLASS_PATH="${1#* *}${PSEP}"
let class_path_count+=1
shift
;; |
No description provided.