build: improvements for TS & Webpack & Docker

[pano9000]

Hi,

this PR (still in draft) aims to improve the build process and better utilize TS / Webpack combination

still a little bit work left to do here, but wanted to make this more visible to avoid any "duplicated work", while I work on this.

more details about the what and why and some comparison data, will be provided tomorrow.

if you try to understand my changes before that -> I recommended going through the commits chronologically, instead of looking only at the "Files changed" overview :-)

Edit:

Overview of changes

• concentrate build output into one single folder: "dist", previously we had

  • build (from tsc),
  • app-dist (from webpack), which also "stained" the "src/public" folder with build related output
  • dist (from copy-dist), which literally copied contents of build and app-dist

• introduced a separate tsconfig for transpiling the server side code only

  • previously tsc would transpile server side AND client side code during
  • the transpiled client side code would remain unused/useless, since webpack would transpile the code itslef and then bundle it
    • new tsconfig.build.json now only builds server side code
    • the "general" tsconfig.json still remains valid, for dev purposes

• added build:ts and build:clean (for easily removing build folder)

• improved copy-dist

  • simplified code overall
  • avoid copying of unnecessary and duplicate files/folders

• updated Docker build (Dockerfile)

  • updated to reflect this new building "reality" introduced by changes above
  • make use of copy-dist instead of manual file copying -> more DRY this way
  • get rid of "hybrid" build -> now we build the full project inside Docker (which is what Docker is made for)
  • previously we did a partial build locally, copying the partially build into Docker and then doing the last remaining steps

• updated Docker build (Github CI)

  • since we got rid of the "hybrid" buidling, we need to update the Github CI actions that use Docker

---

[eliandoran]

Now that's a lot of changes. 😮

[pano9000]

hope that's not a problem ;-)
I am almost finished with this PR, just need to still figure out two things:

a) make sure swc will work with Gihub actions (currently it fails, but I have an idea why that might be)
b) make adjustments to the copy-dist.ts file, to update copying of assets (translations, stylesheets, etc.)

[pano9000]

update: I'll probably split out the swc Terser plugin work onto its own PR, so that we can at least have the improved ts/webpack building – and the build speed optimization (which would've been introduced by swc TerserPlugin) can come in later, after some additional investigation on how to handle the most elegant way:

→ it is currently giving me some issues during Github CI, due to it requiring "platform" specific packages (e.g. like @swc/core-darwin-x64) – which do not seem to be installed correctly, when we use npm ci.
Other have solved it by "removing" package-lock.json prior to npm i, but that doesn't seem like a proper "elegant" solution to me.

Potentially will also try to just install the required deps in the CI, after running npm ci – but that would not be as elegant, as I'd like – it would mean extra code per platform.

https://forum.strapi.io/t/package-error-swc-code-darwin-x64-deploying-to-strapi-cloud/37922
swc-project/swc#2898
https://stackoverflow.com/a/77721414
https://nextjs.org/docs/messages/failed-loading-swc

=> will update the PR accordingly this evening

[pano9000]

I'd say I am done with this PR now, just need to do a quick test on a Windows machine as well, to make sure the copy-dist script does not behave a bit unexpectedly and I'll then rebase onto latest develop to avoid merge conflicts.

edit: confirmed on Win11 -> copy-dist works exactly as on my Linux machine :-)

Quick first stats from these changes:

develop branch

Dist Folder size:
    du -sch dist/
    282M    dist/

Number of files:
    find ./dist/ | wc -l
    6321

vs

this PR

    
Dist Folder size:
    du -sch dist/
    172M    dist/

Number of files:    
    find ./dist/ | wc -l
    3781

some more details to follow on what exactly changed later this evening

edit:
Docker building needs to be updated as well, it seem - the CI is failing, but I know where and why it goes wrong -- will fix

[pano9000]

update on the docker build:
I have a working version locally, but need to clean it up a bit, before pushing – the benefit here is that the build process now actually happens inside Docker and not the previous weird "hybrid" way, where part of it was built "locally" and then copied inside Docker for further steps.

=> also I do realize that this PR is getting bigger and bigger, but the issue is: The build process was so messy, you start pulling one thread, and you end up needing to take care of every other little bit as well :-D

[pano9000]

another update on the status here:

Docker build is working now and looks a lot cleaner (and even a tiny bit smaller in size) than before.

Couple of things left to do:

• do some tests with the github CI, to make sure I didn't break anything accidentally :-)

that will probably happen tomorrow and then I can finally open this up for reviews.

I will then also hopefully have a detailed list of changes and comparisons

[pano9000]

quick comment on this change:

the installation of these dependencies was originally added 7 years ago:
9cb9ea6

but I don't see any comment as to why these where added in the first place.
Not sure if 7 years ago they actually were needed for some rason, but currently they are definitely not, as I've confirmed by building the project without these :-)

[pano9000]

Did run a test with a slightly modified main-docker.yml workflow (where I removed the "Login" actions, to not fail the workflow, before it even begins building docker):
https://github.com/pano9000/TriliumNextNotes/actions/runs/13681461722/job/38255065185

CI builds :-)
it finally (of course) fails at pushing the images to the registry though

will provide more details and what exactly I did tomorrow, but marking this one as ready for review already, for everyone to check

[pano9000]

@eliandoran
PR is ready, overview of changes has been detailed in the first comment as well :-)

Looking forward to your review on this :-)

(afterwards I can finally start looking into Electron build improvements as well)