Fix BSP regressions and improved resource file handling

[lefou]

Fix some BSP regression and some improvements

• Restore proper streams handling; currently mill --bsp is dumping all output on STDERR which is very noisy
• Better handling of the resource file
• Fixed buildinfo parser for changed embedded dependencies format

[lefou]

• While testing this locally, I hit Peculiar cache behavior for external modules #2419

[lihaoyi]

What's this change for? AFAICT, it doesn't modify the security characteristics at all: the existing millProjectModule already uses sys.props to control things, so anyone who has control over the MILL_BSP_WORKER property would also have control over the MILL_UNDER_TEST property

[lefou]

Oh, this seems to be left over. I searched for the issues which later turned out to come from #2419. I'll revert it.

[lihaoyi]

I'm not sure this is the right place to put this, since it is really only used by the BSP logic and not the rest of the code in the repo. Furthermore, it kind of breaks the encapsulation here, which is that SystemStreams is meant to be the stdout/stdin/stderr streams that are provided by the operating system and not include any application-specific values

IMO we should pass bspLog around separately, or put it on our existing Logger class, or use an environment variable or JVM property to pass around the log file path. Looking at the code, the last option (environment variable) seems the most appropriate, since this is really only relevant to one single worker and not the rest of the Mill codebase, so putting it on one of the "global" context objects doesn't make sense

[lefou]

I'm not sure this is the right place to put this, since it is really only used by the BSP logic and not the rest of the code in the repo.

Yeah, makes sense. Let me find a better solution.

Furthermore, it kind of breaks the encapsulation here, which is that SystemStreams is meant to be the stdout/stdin/stderr streams that are provided by the operating system and not include any application-specific values

Here I disagree. Initially, SystemStreams was created to hold the OS provided streams, but after your refactoring, it's rather a class to pass the three typical streams around. The point in the BSP mode is, that we very early redirect and run the rest of Mill with these customizations. If we want SystemStreams to hold the original OS values, we can't use this class further down to pass customized streams around.

[lihaoyi]

Oh I don't mean that it's only meant to pass the oringal OS streams around, more that it's meant to pass some abstraction that looks like them and expose the same interface. So they can get substituted or redirected, but they're still meant to have 3 streams. So any code that receives it can use those three streams as if it's the OS-provided ones, even if they got replaced somewhere along the way

[lefou]

Understood. Let me work out some alternative for bspLog.

[lefou]

@lihaoyi I addressed your review comments and am currently in the process of rebasing/merging to latest main, which is a pita.

[lihaoyi]

Not that familiar with the BSP code, but I think this looks ok

[lefou]

I needed to fix various parts. For example the format of the embedded BSP server impl dependency in the BuildInfo had changed, and some other things, but this should work now. Tested with vscode.