Scalafmt fails with Build without the bytes

[gergelyfabian]

When running a target like this:

# Important to clean, as the files can stay from a previous bazel build with a lower Bazel version.
bazel clean
bazel run //utils/smth:library.format-test

I receive:

utils/smth/src/main/scala/com/dummy/smth/Smth.scala
diff: /home/gege/.cache/bazel/_bazel_user/7f83a0b1f0940f88aec1f4530f927169/execroot/repo/bazel-out/k8-fastbuild/bin/utils/smth/utils/smth/src/main/scala/com/dummy/smth/Smth.scala.fmt.output: No such file or directory

It seems the action responsible for creating this file executes (I have tested disabling writing the output there, but then obviously it fails due to not generating an output), but at a later step the action running scalafmt from the manifest cannot see the scalafmt outputs.

One difference I can see in the bazel-bin folder between Bazel 6.4.0 and Bazel 7.0rc2 (after running bazel run //utils/smth:library.format-test):

# 6.4.0:
ls bazel-bin/utils/smth/utils/smth/src/main/scala/com/dummy/smth/
Smth.scala.fmt.output
# 7.0rc2:
ls bazel-bin/utils/smth/utils/smth/src/main/scala/com/dummy/smth/
[empty]

[gergelyfabian]

Is this a Bazel bug or a rules_scala one?

[gergelyfabian]

Added reproduction in https://github.com/gergelyfabian/bazel-scala-example/ (master branch):

# Change .bazelversion to 7.0.0rc2 and then run:
bazel run //example-maven:example-maven.format-test

[gergelyfabian]

Checking whether this is a bazel issue:

export BAZELISK_CLEAN=true
bazel --bisect=6.4.0..release-7.0.0rc2 run //example-maven:example-maven.format-test

[gergelyfabian]

Checking whether this is a bazel issue:

export BAZELISK_CLEAN=true
bazel --bisect=6.4.0..release-7.0.0rc2 run //example-maven:example-maven.format-test

Trick is that I had to turn off using a java toolchain and remotejdk_17 first, as remotejdk_17 does not work for some commits as bisecting.

[gergelyfabian]

--- Bisect Result

first bad commit is https://github.com/bazelbuild/bazel/commit/9c96529fca4a135c162e35ce2882834b879438fb

That is:

Build without the bytes by default

by changing the default value of `--remote_download_outputs` to `toplevel`.

[gergelyfabian]

Thank you, bazelisk, indeed this seems to be the answer.
This works:

bazel run --remote_download_all //example-maven:example-maven.format-test

[gergelyfabian]

Forming the question again, scalafmt internal outputs not working with "build without the bytes" is a bug of Bazel or rules_scala?

[gergelyfabian]

Starting from Bazel 7, the default download mode will be changed from --remote_download_all to --remote_download_toplevel. That is, Bazel will no longer download intermediate outputs of remote actions into the local output base. In return, your remote builds will be faster.

"Using a remote cache or even a disk cache will make actions "remote" from this point of view." (thanks @fmeum)

In my case, disk cache is enabled.

[gergelyfabian]

The information I received on the Bazel slack was that if rules_scala emitted correct information (DefaultInfo(runfiles = ...)) for the outputs it generates (the .fmt.output files), then those should be visible also when using Build without the bytes.
Unfortunately I cannot find yet whether the scalafmt phase is generating proper DefaultInfo.

[liucijus]

The information I received on the Bazel slack was that if rules_scala emitted correct information (DefaultInfo(runfiles = ...)) for the outputs it generates (the .fmt.output files), then those should be visible also when using Build without the bytes. Unfortunately I cannot find yet whether the scalafmt phase is generating proper DefaultInfo.

@gergelyfabian could you check if anything might be missing here: https://github.com/bazelbuild/rules_scala/blob/master/scala/private/phases/phase_scalafmt.bzl#L18?

[gergelyfabian]

The current state where I'm at is this. phase_scalafmt seems to be adding the proper information (on the line you quoted):

        # Return a depset containing all the relevant files, so a wrapping `sh_test` can successfully access them.
        return struct(runfiles = depset([manifest] + files + srcs))

Then, we go to the phase_default_info, and runfiles properly includes the .fmt.output files there (added a print before that):

https://github.com/bazelbuild/rules_scala/blob/master/scala/private/phases/phase_default_info.bzl#L46

But, when api.bzl runs (https://github.com/bazelbuild/rules_scala/blob/master/scala/private/phases/api.bzl#L85) it does not get the DefaultInfo that phase_default_info was supposed to return to it. I don't get why.

[gergelyfabian]

Debug output like this:

DEBUG: /home/user/.cache/bazel/_bazel_user/64527517dacf7170f7d914889f83481c/external/io_bazel_rules_scala/scala/private/phases/phase_default_info.bzl:38:10: Returning DefaultInfo with runfiles: depset([<generated file example-maven/example-maven.jar>, <generated file external/maven/com/twitter/algebird-core_2.13/0.13.7/processed_algebird-core_2.13-0.13.7.jar>, <generated file external/maven/com/googlecode/javaewah/JavaEWAH/1.1.7/processed_JavaEWAH-1.1.7.jar>, <generated file external/maven/org/scala-lang/modules/scala-collection-compat_2.13/2.1.6/processed_scala-collection-compat_2.13-2.1.6.jar>, <generated file external/maven/org/scala-lang/scala-library/2.13.12/processed_scala-library-2.13.12.jar>, <generated file external/maven/org/scala-lang/scala-reflect/2.13.12/processed_scala-reflect-2.13.12.jar>, <generated file external/maven/org/typelevel/algebra_2.13/2.0.0/processed_algebra_2.13-2.0.0.jar>, <generated file external/maven/org/typelevel/cats-kernel_2.13/2.0.0/processed_cats-kernel_2.13-2.0.0.jar>, <generated file example-maven/format/example-maven/manifest.txt>, <generated file example-maven/example-maven/src/main/scala/mypackage/Maven.scala.fmt.output>, <source file example-maven/src/main/scala/mypackage/Maven.scala>])
DEBUG: /home/user/.cache/bazel/_bazel_user/64527517dacf7170f7d914889f83481c/external/io_bazel_rules_scala/scala/private/phases/api.bzl:85:22: Detected external providers: {"DefaultInfo": struct(data_runfiles = None, default_runfiles = <unknown object com.google.devtools.build.lib.analysis.Runfiles>, files = depset([<generated file example-maven/example-maven.jar>]), files_to_run = None)}

Debugging code is here: https://github.com/gergelyfabian/rules_scala/tree/bazel-7.0-scalafmt-bug

[gergelyfabian]

Btw. the following command also fails for Bazel 6.4.0: bazel run --remote_download_toplevel //example-maven:example-maven.format-test