Vite, Bundlers, etc

[leaanthony]

There's quite a bit of work going on around incorporating Vite into Wails templates. Rather than have a bunch of discussion on different PRs and issues, this discussion will serve as the main place to discuss this.

[akhudek]

Here's my setup right now. Using v2-alpha, vite + react + typescript. Dev mode and the bridge seem to work fine. I'm currently making my own type declarations, though the ones in the comments along side the autogenerated code are helpful too.

For production usage, I'm building in library mode using the "umd" format. I have a second html file for production only that references the umd js file in head.

[leaanthony]

Could we auto-change the include so you didn't need to hack around the need for 2 html files?

[akhudek]

Hmm, actually maybe I can get away with one html file. I suppose it won't be able to find the production bundle during dev time. I'll try and get back to you.

[akhudek]

Ok, no, it still loads the prod js in dev mode. If you ensure that the prod dir is empty while doing dev it would likely work, but I feel that the two files is a bit cleaner. Also, in prod mode it would end up pulling in src/main.tsx which isn't what we want.

[leaanthony]

Could we not use a shim to load modules that changes based on build mode?

[akhudek]

You could restore the "js" and "css" options in the wails config, they seem to be ignored now. We'd still need a way to ignore the "src/main.tsx" script tag that needs to be in the html for vite dev mode, but maybe there is a way to do that.

[sircodemane]

This topic is a bit old, but it seemed to be appropriate for some pain points I've come across in the V2 beta. I wanted to introduce a few suggestions, and I'd be happy to help implement them if I'm able to.

1. Add frontend:develop key to wails.json which would point to a watch+bundle process that is common to almost all modern JS frameworks. Currently, the developer experience requires starting wails dev which builds the frontend and watches for dist changes, but still requires manually starting a separate build+watch process to trigger the hot reload for any further changes. I tried connecting frontend:build to the build+watch node script, but that hangs because of the watch process. This means that in order for frontend:develop to work, the wails dev would need to run frontend compilation step and backend compilation step in separate routines so the frontend watcher doesn't hang the entire dev command.
2. bindings.d.ts is referenced in some of the bindings code I looked at, but does not appear to be functioning as I have not seen a bindings.d.ts generated. This would help immensely with TS support
3. Having bindings generated as part of the app startup is both really nice, yet really painful at the same time. It's sort of a too little too late issue, where if the frontend is dependent on these bindings, they don't exist until you run a "dummy" build+start of the backend first to get them generated, particularly in TS frontends. I would love to use some sort of code gen approach, such as wails bindings that can be run either standalone, or as part of wails dev/build. The issue is that binding generation is part of app startup, and would require a significant API/workflow change, as well as parsing. My initial thought on how to approach this off the cuff is have an interfaces/bindings directory and wails:bindings key in wails.json pointing to said directory, and have interfaces written in go that can be parsed into JS/TS. The bindings command can walk that directory and employ some method to parse those interfaces into TS/JS code for bindings instead of the current generateBindings() function in CreateApp(). This can also happen before any frontend compilation, which would be a huge improvement to TS support as well.

These are pretty hefty suggestions, but considering the implications they could have, I feel it would be better to try and suggest them before full release, as they could have big impacts.

[leaanthony]

Hey @codydbentley ! I'll try and answer each point inline:

This topic is a bit old, but it seemed to be appropriate for some pain points I've come across in the V2 beta. I wanted to introduce a few suggestions, and I'd be happy to help implement them if I'm able to.

1. Add frontend:develop key to wails.json which would point to a watch+bundle process that is common to almost all modern JS frameworks. Currently, the developer experience requires starting wails dev which builds the frontend and watches for dist changes, but still requires manually starting a separate build+watch process to trigger the hot reload for any further changes. I tried connecting frontend:build to the build+watch node script, but that hangs because of the watch process. This means that in order for frontend:develop to work, the wails dev would need to run frontend compilation step and backend compilation step in separate routines so the frontend watcher doesn't hang the entire dev command.

Your suggestion is roughly how we wanted v1 to work and the issue there, from a wails perspective, was trying to manage multiple processes. The easiest solution was to just output a message saying "run this in another terminal" - but the issue then is that sometimes the message disappeared in the logs so I added a 1s delay. It was a shonky solution. Happy for an improved dev experience but won't it get confusing? If I open on port 3000, i get the vite server, if it's 34115, then it's the wails server. The main point of doing it this way was to ensure we didn't get more bugs in the vein of "When I run this in the browser it works, when i build it, it doesn't", because now you can test it in the webview itself. One option is to make the desktop app proxy the requests to the framework server. It's feeling pretty complex but again, am open to suggestions.

2. bindings.d.ts is referenced in some of the bindings code I looked at, but does not appear to be functioning as I have not seen a bindings.d.ts generated. This would help immensely with TS support

It should be in frontend/wailsjs/runtime - generated by wails dev.

3. Having bindings generated as part of the app startup is both really nice, yet really painful at the same time. It's sort of a too little too late issue, where if the frontend is dependent on these bindings, they don't exist until you run a "dummy" build+start of the backend first to get them generated, particularly in TS frontends. I would love to use some sort of code gen approach, such as wails bindings that can be run either standalone, or as part of wails dev/build. The issue is that binding generation is part of app startup, and would require a significant API/workflow change, as well as parsing. My initial thought on how to approach this off the cuff is have an interfaces/bindings directory and wails:bindings key in wails.json pointing to said directory, and have interfaces written in go that can be parsed into JS/TS. The bindings command can walk that directory and employ some method to parse those interfaces into TS/JS code for bindings instead of the current generateBindings() function in CreateApp(). This can also happen before any frontend compilation, which would be a huge improvement to TS support as well.

There was a lot of work done on parsing the Go code and it was put on hold due to complexity(It's in v2/pkg/parser). Happy to accept a PR that does that :-) I'm not sure I understand why it's too late - you define a Go struct, you run wails dev and you get the TS files that you can then import and use. The frontend wont be dependent on them until you write code against them. wails dev will keep generating your TS as you write your Go code, so you get them immediately. At least, that's what should be happening...

These are pretty hefty suggestions, but considering the implications they could have, I feel it would be better to try and suggest them before full release, as they could have big impacts.

Thanks for doing so! Let's talk!

[sircodemane]

Thank you for taking the time to respond in such detail! I'll was gonna go inline, but after typing a fair bit, I decided its better to split these out into their own sub discussions for 1 & 3. For point 2, I'll continue here, I've been using the beta.2, let me make sure everything is upgraded. So far all I get is wailsjs/go with bindings. js,models.ts, package.json (and the runtime folder of course). I havent seen a bindings.d.ts yet, just runtime.d.ts

[sircodemane]

Can confirm, there is no bindings.d.ts file being generated with CLI beta.3 + require beta.3. There is only runtime.d.ts, which has no application bindings, just the runtime declarations for window controls and such

[leaanthony]

Ah, not a bindings.d.ts for sure. The library I use doesn't support that, but it does generate "interfaces". I don't know a huge amount about TS but might that work? https://github.com/leaanthony/typescriptify-golang-structs

[sircodemane]

Original Context from @codydbentley:
Add frontend:develop key to wails.json which would point to a watch+bundle process that is common to almost all modern JS frameworks. Currently, the developer experience requires starting wails dev which builds the frontend and watches for dist changes, but still requires manually starting a separate build+watch process to trigger the hot reload for any further changes. I tried connecting frontend:build to the build+watch node script, but that hangs because of the watch process. This means that in order for frontend:develop to work, the wails dev would need to run frontend compilation step and backend compilation step in separate routines so the frontend watcher doesn't hang the entire dev command.

Your suggestion is roughly how we wanted v1 to work and the issue there, from a wails perspective, was trying to manage multiple processes. The easiest solution was to just output a message saying "run this in another terminal" - but the issue then is that sometimes the message disappeared in the logs so I added a 1s delay. It was a shonky solution. Happy for an improved dev experience but won't it get confusing? If I open on port 3000, i get the vite server, if it's 34115, then it's the wails server. The main point of doing it this way was to ensure we didn't get more bugs in the vein of "When I run this in the browser it works, when i build it, it doesn't", because now you can test it in the webview itself. One option is to make the desktop app proxy the requests to the framework server. It's feeling pretty complex but again, am open to suggestions.

Fair points, and this makes me consider a number of things. I think wails dev doing frontend compilation is what set my expectation of an Electron-like experience where a single yarn electron:serve command will watch and recompile both frontend and backend. Had it never done any frontend compilation, I would have likely made the natural assumption that I'll have to do it myself.

As for possible solutions, I think there are two scenarios that might make more sense:

1. wails dev doesn't try to do any frontend install or compilation, only watch for /dist changes and reload, while advertising in docs that it's up to the user to to handle frontend compilation with the framework of their choice (most will probably figure this out on their own). This also has the nice side effect of not relying on frontend comp to succeed for backend to continue (more on why this is better in the bindings generation topic). This might be the easier and more natural solution.
2. Add a frontend:develop key to wails.json which can point to a frontend watch script, and run the frontend compilation step in a goroutine, which wont block the backend from compiling and running (and also generating bindings). I don't believe having separate servers running will be a deal breaker for anybody, especially once the backend compiles and pops up the application window, showing the running frontend. Plus, at least in the case of Vite and VueCLI, their dev/serve commands dont generate files to be served in the dist folder. Templates with frontend:develop would instead be pointing to build-only, non-serving scripts that watch the src files to trigger updates, ex- vite build --watch. This does not spawn a server, its compilation only.

[sircodemane]

Original context from @codydbentley:
Having bindings generated as part of the app startup is both really nice, yet really painful at the same time. It's sort of a too little too late issue, where if the frontend is dependent on these bindings, they don't exist until you run a "dummy" build+start of the backend first to get them generated, particularly in TS frontends. I would love to use some sort of code gen approach, such as wails bindings that can be run either standalone, or as part of wails dev/build. The issue is that binding generation is part of app startup, and would require a significant API/workflow change, as well as parsing. My initial thought on how to approach this off the cuff is have an interfaces/bindings directory and wails:bindings key in wails.json pointing to said directory, and have interfaces written in go that can be parsed into JS/TS. The bindings command can walk that directory and employ some method to parse those interfaces into TS/JS code for bindings instead of the current generateBindings() function in CreateApp(). This can also happen before any frontend compilation, which would be a huge improvement to TS support as well.

Reply from @leaanthony:

There was a lot of work done on parsing the Go code and it was put on hold due to complexity(It's in v2/pkg/parser). Happy to accept a PR that does that :-) I'm not sure I understand why it's too late - you define a Go struct, you run wails dev and you get the TS files that you can then import and use. The frontend wont be dependent on them until you write code against them. wails dev will keep generating your TS as you write your Go code, so you get them immediately. At least, that's what should be happening...

I'll definitely check the package out and see if I can do anything with it! After working with things a bit more, I realize I was having issues with a few things combined that were minor inconveniences, but ultimately it all comes down to type checking and a chicken/egg problem. If type checking is turned off, this is almost a none issue, which is why the vanilla JS template and svelte template work well out of the box. Some might argue that type checking isn't necessary because the editor should help with that, but even with type checking turned off, I can't import or use objects that haven't been compiled yet, and I can't generate the bindings I need because the frontend fails to compile because the bindings haven't been generated yet.

I know this is nitpicky, but even such minor inconveniences could potentially turn off users who want to use TS. I think the simplest and most straight forward way to tackle it is maybe remove frontend compilation from wails dev, or at the very least, put it second to the backend so that bindings are generated before frontend compilation. This would eliminate the chicken and egg issue, and allow type checking. This weekend I should have time to really dive in and see if I can come up with some PRs.

[leaanthony]

Yes, I can see how this is a pain, and it was a compromise against parsing the project source. I'm open to any suggestion to make life easier, but I'd also like to go over how it's currently been setup as I'm not quite clear on the chicken/egg problem:

• You generate a project and run wails dev
• That will compile frontend, then backend, then at startup will generate the runtime and bindings in frontend/wailsjs
• You then decide to add a Go struct as a parameter to a bound method
• On save, the app will auto-rebuild and re-run. frontend/wailsjs/bindings now includes the model in models.ts
• You use this in your frontend code
• Let's say you then update a Go file
• On save, it builds the frontend, and that'll work because models.ts still exists on rebuild (if it doesn't then that's a bug)

The only scenario I see where this can be a problem is if you decide to go the other way and write TS first and then try and hook up to the backend. Sorry for being slow on this one! :-)

[rohanray]

I am facing a similar issue as @codydbentley mentioned in point 1. I am trying to use Sveltekit for the frontend which uses vite internally. For now, I have to restart wails dev command for any frontend changes; prior to which I need to run the frontend build command 'npm run build' per se.

As a work around, I tried using npm-watch command to trigger frontend build after any frontend updates. This also triggers wails dev restart automatically as expected. However, the changes in index.html within the build directory are not picked up by wails. The issue then is the new js files which are generated by vite using some hash filename are not reflected.

I am not sure if I could explain the issue clearly. I will create a separate repo with my set-up i.e. basic sveltekit with wails V2 to support above.

-RR

[leaanthony]

Ah, I think I know why the HTML might not be being picked up....should be an easy fix.

Does your bundler have an easy way to execute a command post build? I would think that's a necessary feature.

What would the best solution be? I'm liking @codydbentley 's idea of build:develop which maybe is ran once at startup in a different thread?

[rohanray]

As of now, I am trying to use static adapter with svelte-kit https://github.com/sveltejs/kit/tree/master/packages/adapter-static; to get a pure client side SPA. The default bundler does not have any feature of running a post build script OOTB. Hence, I am using https://www.npmjs.com/package/npm-watch for the same.

build:develop which maybe is ran once at startup If we have a one-time script which is executed only once at startup; then it won't rebuild the frontend code post any changes in frontend?

[leaanthony]

There's a solution in master. Check the other thread where we are tracking this work.

[rohanray]

@leaanthony - I can confirm that index.html now refreshes post rebuild of frontend with beta 4. Thanks -RR