Docs: useCodeCache: true for SEA blob preparation leads to 'broken' executables when creating cross platform SEAs.

[mn4367]

Currently useCodeCache has to be set to false if one wants to create a SEA on platform A for platform B. Otherwise the generated executable will crash on startup.

The original problem and the solution are described in more detail here (postject project).

It might be worth noting this in the documentation for the generation of single executable preparation blobs (sub chapter V8 code cache support) since it isn't obvious. Maybe something similar is also true for useSnapshot.

[joyeecheung]

When code cache or snapshot is generated, the executable needs to be run to produce those. Are you running the executable using a simulator to generate the preparation blob?

[joyeecheung]

I see that the postject issue mentions that it showed up on macOS and from the crash report the executable is translated by Rosetta, during the blob generation V8 probably is not generating code for x64 when run in this mode, or maybe it is but there are some edges not taken care of for cross-compilation as cross-compiling code cache is not really something V8 claims to support anyway. I guess the best we could do for now is probably documenting that snapshot or code cache isn't supported when cross-compiling the SEA.

[mn4367]

Edit: you were to fast, so this is my reply to your first comment.

No, my primary development platform is darwin-arm64, I simply downloaded the target executables (darwin-arm64, darwin-x64, win-x64 and linux-x64) from the Node download page. Then I created the preparation blob and injected it into all four target executables, using the darwin-arm64 version of Node.

With useCodeCache : true they all crashed, when executed on the respective target platform. With false this works fine, so I was able to create SEAs on all four platform for all four platforms, e.g. creating a darwin-arm64 executable on a Windows machine and running it on the Mac machine works without any problems, provided it was signed after injection with the tools mentioned in the postject issue (signing was also done on Windows).

[mn4367]

I see that the postject issue mentions that it showed up on macOS and from the crash report the executable is translated by Rosetta, during the blob generation V8 probably is not generating code for x64 when run in this mode, or maybe it is but there are some edges not taken care of for cross-compilation as cross-compiling code cache is not really something V8 claims to support anyway.

The generated executable was for x64. Upon execution it runs through Rosetta, this is why the stack trace contains x64 related messages.

Edit: I haven't tried if this is only a problem with regard to x64 vs. arm64, e.g. generating on win-x64 would work for darwin-x64. Maybe V8 creates the code cache not only for specific CPU types but also for specific OS platforms.

[mn4367]

I guess the best we could do for now is probably documenting that snapshot or code cache isn't supported when cross-compiling the SEA.

Exactly. Documenting it this way indirectly says that cross platform SEA creation is indeed possible, which, I think, is quite awesome. I was diving deeper into this because I was looking for an alternative to the now deprecated pkg.

[joyeecheung]

Then I created the preparation blob and injected it into all four target executables, using the darwin-arm64 version of Node.

I see, in that case it's less likely to work, the documentation explains it as

Single executable preparation blobs that are injected into the application can be generated using the --experimental-sea-config flag of the Node.js binary that will be used to build the single executable.

We should probably make it clearer that if the executable used to generate the blob isn't also the one being injected, there is a chance that it doesn't work. I think so far things other than code cache and snapshot might work but it's not guaranteed. For example with a different endianness it's probably certainly going to break...

[mn4367]

Endianness could also be a problem, yes. So for now it may suffice to document that with useCodeCache: true 'broken' executables might be created if the target platform is different than the build platform. Unfortunately, I wasn't able to try useSnapshot, but I suspect this leads to the same problem.

For the future, it might be helpful to explain in a separate chapter that cross-platform creation of SEAs is possible (maybe with a short example). In my opinion, this use case is simply too interesting not to be mentioned.

Thanks for looking into this issue!

[joyeecheung]

Marking it as a good first issue - feel free to try cross-platform SEA generation and update the docs with better instructions on how to do it (as mentioned before in this issue #52420 (comment), it should "just work" if you don't use code cache and snapshot options).

[mn4367]

Thanks! Should I create a PR*?

*Due to my other activities, I can't start until the end of June, but otherwise I'm happy to do it.

[joyeecheung]

Yes, unless someone else made it happen before you.