Missing class files in classes directory of configuration file

[aappddeevv]

When using sbt-bloop, sbt wants the files in project/target/scala-2.13 and they are placed there. The .bloop/project.json seems to indicate that the outputs will go under the .bloop/project directory. The .bloop config classpaths also point to .bloop/project/scala-2.13/classes so it appears that the .bloop config is not right, at least with respect to where sbt wants the outputs. After compiling with sbt, there are no outputs under .bloop.

[jvican]

@aappddeevv This works as expected. For the record, and I'll write this in the docs in an update I'm preparing, the directory written to the configuration file is to be used exclusively in the case the build tool generating it triggers a compilation. The build tool can trigger a compilation by enabling a bloop-specific BSP option in the InitializeParams, check the use of BloopExtraBuildParams.

Any compilation triggered by other clients will have their own specific classes directory, to avoid collusion and conflicts. In the case of the CLI, we have stable directories. In the case of BSP clients, we have directories that only live during the duration of the BSP connection and afterwards are cleaned.

Hope this has clarified your issue. I agree this deserves a better explanation. I've been given this reply in private several times so I'll make sure I make these invariants crystal clear in our docs.

[aappddeevv]

Im not sure I followed that. If I use sbt-bloop and the config files that sbt created, when sbt compiles via the compile command I would expect it, as a Bsp client, to generate files per its bloop config. I’m assuming that sbt maintains the same connection all the time. I also kept a watch out the .bloop specified output directory and never saw any files placed into the configured destination. They just appeared in sbt’s standard place only.

[jvican]

OK, so there has been a misunderstanding here but the behavior you mentioned here still works as expected. The classes dirs between bloop and sbt are kept separate by design. Bloop needs to own the classes directories to ensure that they are properly isolated should more clients ask for the compilation of the codebase concurrently.

There's a feature called compile offloading that allows sbt to trigger a compile to bloop via BSP and have the class files be placed in the sbt target dirs. It's implemented but still experimental, waiting for some key changes I have to make there. Does this clarify what's happening in your example @aappddeevv ?

[aappddeevv]

That does explain it and we should close this issue and move the conversation to chat if needed. I get that the files wind up where SBT expects them and I did read the compile deduplication notes.

I would have expected the files to appear in the location specified in the .bloop config classesDir location and wherever sbt wants them. I want to run a CLI program that reads the bloop config and grabs the output class files for further processing independent assuming a compile request had been completed by any client.

I may be misunderstanding what the .bloop config classesDir and classPath property is then. It almost sounds like with deduplication, while the files may have appeared there at some point (and I missed them being created), sbt acting as a bsp-client moves them to where it wants to.

Hence, with compile deduplication, my cli needs to issue a compile request to get any class files and bloop will place them into my "cli"'s special location. I can do with them what I want to. If they are already compiled, then its fast assuming my disk is fast.

[jvican]

I would have expected the files to appear in the location specified in the .bloop config classesDir location and wherever sbt wants them.

To summarize, the classes directory in the configuration file is to be exclusively used and consumed by the build tool generating the configurations. That means, if the build tool is not a BSP client triggering compiles to bloop, those directories will be empty.

I want to run a CLI program that reads the bloop config and grabs the output class files for further processing independent assuming a compile request had been completed by any client.

The best way to do this is to trigger compilation via a BSP client. The most likely scenario is that there won't be any changes in the workspace, so bloop will copy the class files of the last known state of the codebase to the classes directories that assigned to you as a BSP client. You get which classes were assigned to you by calling the scalacOptions BSP endpoint once the BSP connection has been initialized.

Hence, with compile deduplication, my cli needs to issue a compile request to get any class files and bloop will place them into my "cli"'s special location. I can do with them what I want to. If they are already compiled, then its fast assuming my disk is fast.

If you do this with a CLI client, you must be aware that you must be the only CLI client running. If there's another CLI client running, then Bloop will see there's potential for a conflict and will create a unique, non-stable directory for you that will be deleted right after the CLI exits. In this case, you do not have access to the directory that Bloop created for you, hence why doing this via a BSP endpoint is the way to go.

Note that in a previous comment I made a reference to "compile deduplication" but meant "compile offloading", which I think it's what you mean here too. I've corrected this mistake in my previous comment.