Forwarding compile erros to Vite devserver

[belafonte]

Hi There

We are trying to combine shadowcljs with vite.
Basically shadowcljs should compile all cljs sources to esm and vite should pick that up in conjunction with all the js/tsx/jsx files in the project, then reloading everything in the dev server ( or later build the final release ).

This works in a way at the moment, but there are some issues we are encountering.

HMR of course does not work correctly, since shadow creates one big main.js that gets reloaded.

But the bigger problem is, that compile errors in the HUD of shadow get discarded by vite, since those get injected by shadow into the dom, which vite is not aware of.

Maybe i'am overseeing something, and there is a easy way to access the compile errors and insert them somehow in the lifecycle of vite? Or something is not configured correctly?

Has somebody experience with this kind of setup?

For reference here the shadowcljs config:

{:deps {:aliases [:frontend]}
 :builds
 {:frontend
  {:output-dir "src/leihs/inventory/client/.shadow"
   :target :esm
   :js-options {:js-provider :import}
   :modules {:main {:init-fn leihs.inventory.client.main/init}}
   :devtools {:preloads [leihs.inventory.client.preload]}}}}

[thheller]

Generally I would strongly advise against doing this. It is rarely a good idea to run the build output of one tool through another. The chances of things going wrong are rather high. A slightly more recommended way would to use :js-provider :external.

If you insist on doing this I cannot help you much, since I do not know about the vite side of things. I can clear up some confusion though.

shadow-cljs does indeed only produce a singular main.js file in your setup, it does however not use that file for hot-reload purposes. The file written to disk is only loaded when you actually load the full page. Every other load, triggered by hot-reload or the REPL, is done over the websocket connection. Same for the HUD, all that is triggered over the websocket. That is also essentially the problem with this approach. vite cannot be "inserted" into anything happening over the websocket. vite should also never use its reload mechanism to fully reload the main.js since that reloads everything, instead of just the thing that actually changed.

I suspect that the reason you are not seeing compile errors is that the websocket just plainly fails to begin with. Using vite will mean that certain features will either be broken or just unavailable, there isn't much that can be done about that. If the websocket is actually working it might also be that shadow-cljs actually tries to show the HUD, but vite then immediately destroys it via its hot-reload, potentially because it did an actual full page reload? The console should tell you what is going on.

[belafonte]

Thanks for your answer!

I think this is what happening:

If the websocket is actually working it might also be that shadow-cljs actually tries to show the HUD, but vite then immediately destroys it via its hot-reload, potentially because it did an actual full page reload?

And you are right, having two bundlers compete against each other is rather unwanted.
But actually it was working ok, with shadow producing ESM.
Besides the problem with the errors.

The issue with "external" is that as far as i see, that only target allowed is "browser", which then creates a main.js script and a CommonJS external file.

Loading the main script with defer and the external JS as module, does not work as expected.
shadowBridge is not defined and requires are also not defined, because Vite is ESM first and all our JS is too.
Vite can process CommonJS though, but it is either or.

[thheller]

I think vite has an option to build a regular JS file, can't remember what it was called though. Quick glance at the docs suggests maybe build.lib = "iife"?

[belafonte]

Ah sorry i forget about the :external-index-format :esm option.
I set this up quite some while ago and haven't remembered it anymore.

But i think also when i was setting it up, i hadn't found a solution for the shadowBridge not defined issue beside setting the js-provider to "import and using ESM as target.

I have to get into all the options again and see if i can figure it out.

[thheller]

:js-provider :external is currently only support for :target :browser builds. Switch the targets and it should be fine.

{:deps {:aliases [:frontend]}
 :builds
 {:frontend
  {:output-dir "public/js"
   :target :browser
   :js-options {:js-provider :external :external-index-format :esm :external-index "foo/index.js"}
   :modules {:main {:init-fn leihs.inventory.client.main/init}}
   :devtools {:preloads [leihs.inventory.client.preload]}}}}

[belafonte]

I think, you are right and having Vite compete with shadow is just a bad idea.
Would make more sense, when Vite would trigger the cljs compilation ( via clojure compiler for instance ...? ) and then do the bundling.
But that would mean, to remove shadow-cljs.

Vite has some nice features that would have been handy to have.
But yeah would need different approach.

I found this great blog post by Metosin ( which wasn't published, when i started setting up our project ).
https://www.metosin.fi/blog/2024-09-05-using-shadow-cljs-with-esbuild

Which does what you are recommending and solves it using shadow and esbuild.
I was able to integrate most of my other tooling there too, since vite just uses esbuild under the hood and config is the same.

Thank you for your help!

[thheller]

FWIW I absolutely do not recommend using the build hook approach used in that blog post. Instead personally, I'd setup a file watcher for the target/libs/index.js file and trigger the build when that file changes. Doing it via build hooks just introduces vite into the build pipeline again, which will just make things harder to troubleshoot should things go wrong. :configure can also the potentially called multiple times, so there should be a guard to at least not start the watch multiple times. There is also never a controlled shutdown of the yarn task, so relying on the JVM getting killed which I personally do not like.

Vite has some nice features that would have been handy to have.

I assume you are referencing the CSS support? Or something else? Maybe something I could add to shadow-cljs?

[thheller]

I wrote a new blog post with an example for running external processes. I'd recommend that way over the :build-hooks approach.

[belafonte]

Ah cool! I just went over it.

How the start and stop functions get triggered is probably written in that previous blog post?
Or have a missed something?

But for the release build, a build hook seems reasonable to me? Or would you recommend something else there too?

Maybe there was some misunderstanding too, since the config Metosin posted is not using Vite at all but only esbuild.
esbuild bundles and transformes all the external files when they are built from shadow ( on the build hook )

Concerning the features of Vite that are nice:

On thing is easy css support yes ( but not too important since as you also write in your post tailwind or postcss have their own watchers for example )

But Vite has a very powerful plugin system for its server which is very handy. For instance can you add custom middlewares that intercept requests.
Our API is not under a seperate /api route, but checks the header for its mime type on a specific resource and then either return html or json for instance.
In Vite it is possible to proxy requests which are application/json to the api via those middlewares.

But we also used it to forward specific POST requests for i18n.

[thheller]

How the start and stop functions get triggered ...

Yes, the workflow is explained in the previous article. They are not triggered. You call them.

But for the release build, a build hook seems reasonable to me?

For the release build it makes no sense to combine the builds, which build hooks basically do. Instead just let shadow-cljs fininsh doing its work, and when done let esbuild or whatever do its entirely unrelated work. They are not touching the same files anyway. It is literally running two commands after each other, instead of trying to make the first run the second while it itself is still running. Just complications nobody needs.

Why is vite answering your requests in the first place? Why introduce this proxy stuff, that you undoubtabely not going to have in production? I always recommend making your development setup resemble the production setup as much as possible. I'm not sure how a vite middleware makes life any easier :P

[belafonte]

For the release build it makes no sense to combine the builds, which build hooks basically do. Instead just let shadow-cljs fininsh doing its work, and when done let esbuild or whatever do its entirely unrelated work. They are not touching the same files anyway. It is literally running two commands after each other, instead of trying to make the first run the second while it itself is still running. Just complications nobody needs.

Ah yes that sounds reasonable.

I always recommend making your development setup resemble the production setup as much as possible. I'm not sure how a vite middleware makes life any easier :P

Hehe yeah true, we used the proxy to forward some auto translations which is only possible and needed in dev mode. But its not so important, so we drop it. Was just convenient, that it is possible to do it in Vite.

So now everything goes through the backend server again.