Svelte 5 Release Candidate

[jamestagal]

Hi @jimafisk

I see that Svelte 5 is in release candiate. There are some significant changes but it will be have backward compatibility with version 4. You can read the post on it here.

However I see that you bumped a change to version 3.59.1 so Plenti probably will need to at least move to version 4 before going to 5. I remember you tested it version 4 but found a lot of new dependancies.
What are you plans re version 5 Svelte?

How does this all fit with your explorations with custom Go templates? If implemented would they be an alternative to Svelte templates for larger projects? Just curious 😄

Regards,
Ben

[jimafisk]

Hey @jamestagal,

Great questions, the roadmap is still a bit up in the air, but I'll share my current thoughts here. I think Svelte is one of the best reactive frontend frameworks that exists and it makes manipulating the UI very ergonomic. That said, not all is perfect and after building several large production apps with it, there are definitely some aspects that are challenging. One of the main issues is that there is no top-level await that allows you to SSR dynamic components, which is at the heart of a content-driven framework like Plenti. We hacked around this with component signatures and the allLayouts magic prop, but it has side effects like loading all the app components on initial hydration. This is something we could try to improve, I've made attempts in the past (8e49b8c), but maybe it's not getting at the root of the issue - we're sending an absurd amount of client JS for aspects of a site that are mostly static. Probably the only JS that should actually be produced is related to the interactive aspects of the page, in other frameworks this is sometimes called "islands."

Another challenging aspect of using Svelte in Plenti is that we're loading a 60k line node app (the Svelte compiler) into V8 to create ssr and dom compilation targets. This process is extremely slow and finicky and has memory timeout issues when trying to compile large applications. To me this is a show stopping problem that I've been kicking down the road for a long time. I was hoping the Svelte core team would eventually move towards a Go/Rust compiler to help alleviate this, but it seems like there isn't much of an appetite to ever do this and arguably the project has moved farther away from this reality. Again, there are things we could do here to make build time improvements (component caching would be a big one #300), but we will always have a large amount of code massaging this process, when maybe it would be easier to have targeted code to create a new process that fits with the goals of the Plenti project better.

Plenti probably will need to at least move to version 4 before going to 5. I remember you tested version 4 but found a lot of new dependancies

Yeah the dependencies make things difficult when running the compilation step in V8. Not only that, but npm modules make things like themes harder to implement because inheritance gets messy, so a long-term goal has been to remove npm entirely by default but allow people to roll their own JS pipeline if they'd like (we could also embed esbuild for convenience). Svelte used to wrap all their dependencies in the project, but moved to a more standard JS structure for v4. Unfortunately for us, we're purposely trying to avoid conventional JS project structure so using Svelte 4+ is probably out of the cards. I'm also not entirely sold on runes and Svelte 5 reactivity - I'm sure there are great reasons for the changes, but to me it makes Svelte more similar to React/Vue, and loses some of the edge it has as a beginner framework that is easy for new developers to understand.

How does this all fit with your explorations with custom Go templates? If implemented would they be an alternative to Svelte templates for larger projects?

The templates would replace Svelte in Plenti. This is still just a dream, and would require a lot of work to become reality. I'm running some experiments in a throw away git repo (https://github.com/jimafisk/custom_go_template), but I'm still in the exploratory phase. Ideally it would work similarly to Svelte, but would have a lot less features. That sounds nice, but the Plenti CMS alone is using lots of features in Svelte that would hard to replicate without stores, 2-way binding, etc that Svelte provides. There are all sorts of other things to consider like language servers for IDE support, so it's an endeavor not for the faint of heart. It would provide a path to much faster compilation times and better stability in Plenti overall (we could probably cut 50% of the codebase that's dedicated to supporting a shaky V8 build process).

Summary: Realistically Plenti is staying on Svelte v3 and not migrating to v4 or v5. Ultimately we may move to a custom templating solution that is Svelte-like, and this is something I'm actively working on, but it may still be a ways out from being pulled into the project officially. Does that make sense? Would love to hear your thoughts!

[jamestagal]

Hi @jimafisk
Wow, that sounds like some big changes ahead! Though if you believe that the next big step forward for Plenti - to be a more capable and stable project then I'm on board. To be honest I haven’t experienced the limitations, bottlenecks and pitfalls of Plenti with Svelte that you talk about but that is probably because I haven’t built large apps and had to deal with them. The only ongoing issue I have is the crashing memory leak issues But if Plenti turns into a fully Go project and takes advantage of a single code base, cutting out the V8 complexities/issues and reducing the huge Svelte compiler code then I’m sure it could potentially be even better.

I will makes notes of what I love about Plenti as it stands:

1. Plenti’s CMS. I think the way you have built Plenti to be discoverable in terms of the content is awesome and something I haven’t seen in other projects and something that is core to Plenti.
2. The ability to customise and manipulate the CMS. So doing overrides with schema files by adding custom widgets for instance code blocks for images or html elements. Doing searches over arrays of data with the entity reference feature, moving components around, hiding data fields, booleans for showing and hiding content etc…
3. A tightly coupled JSON data source with Svelte component templates. Being able to easily bind data, to declare variables for each and all sources even when data comes from outside of its component and use them to replace content in predictable ways in a component is just awesome. It is so intuitive and easy to work even when the data source is separate from the template.
4. Ordering components in JSON files to build the structure of the site This feature is huge. It is so nice to build out the structure from within the JSON files.
5. Svelte syntax. It has been really good for me to learn Svelte as a beginner and ease into it with Plenti. I typically use {each} and {if} blocks to conditionally add or remove content and iterate over repeated content. These core features of Svelte make developing a UI really nice and seamless.
6. All your Magic props. The allContent, allLayouts are awesome features I also wanted to mention.

Some questions that spring to mind are:

1. Would it be difficult to implement the same sort functionality equivalent to some Svelte {if} and {each} methods into Go templates?
2. How would JS work for manipulating the DOM? having external JS files and importing them into Go files as needed?
3. Could you build on the idea of “Islands” for this JS ? similar to what Astro has done?
4. Would it be wise to learn Go? How much Go lang would I need to learn?

I know you have worked hard on this project over many years and it must be hard to maintain and also as challenging to build on it to make it even better with the current limitation as you have outlined. I hope you can find a way forward and I will try to support you as much as I can.

Best regards,
Ben

[jimafisk]

Thanks for the thoughtful response @jamestagal! It's really nice hearing your perspective on things that are important to you, I agree with your list as well!

The only ongoing issue I have is the crashing memory leak issues

Using a native Go-based solution would hopefully fix this. We will still need to evaluate JS however, and so far I've been Goja instead of v8go which executes JS more slowly but will potentially be more stable for purposes. So far I've been really impressed with the speed given the way that I'm using it while experimenting, but that could change as I flesh out more features. Either way, we should be able to fix these crashes, and we probably can and should fix this with the current Svelte implementation by passing context that cancels a current build if a new one is sent.

I appreciate your list of things you enjoy about the project 😊. Here are some quick thoughts:

1. Plenti’s CMS. The discoverable aspect of the CMS will still be a central concept and that will not change.
2. The ability to customise and manipulate the CMS. Also still planning on allowing schema overrides and all the features you listed here
3. A tightly coupled JSON data source with Svelte component templates. This would be very similar, there would still be JSON content and layouts that map together. The props and static components will probably get imported inside "fences" and any interactivity that happens in the browser would be defined in <script> tags (similar to astro). I'm not settled on the exact API but that's where I'm leaning and work towards at the moment, my hope is it alleviates some of the confusion about not being about to do things like window.alert() unless you're in an onMount first because SSR compiling does not have access to document or window. Binding to data interactively would likely look a little different, it will probably be more like regular JS (document.querySelector()) which is arguably less ergonomic than Svelte but more standardized with how the rest of the web works and you would hopefully be able to use examples from other resources more easily.
4. Ordering components in JSON files to build the structure of the site This would definitely be the same, I've already built a basic dynamic component option that would allow this.
5. Svelte syntax. This would be the biggest change in the list. Svelte provides so many nice things like stores, awaits, 2-way binding, animations, etc that would be hard for us to compete on features. We would definitely still have similar options like {each} and {if} blocks, but the syntax would be different and I'm still working through in my head how to allow dom manipulation for these instead of just producing HTML. I also need to figure out the interaction between JS and Go for lexical scoping, etc. I'm not sure the templates would be as "nice" as Svelte, but DX is super important to me, so it would have to have good answers to the fundamental things in order to be a viable solution. Also, the hope is that the speed and stability trade-offs are so great that it's well worth the change.
6. All your Magic props. These will all still be there, although I'm hoping that allLayouts won't be necessary. This was always a hack around Svelte's limitation of not being about to render dynamic components via SSR. With the new templates you should be able to just pass a path string to the dynamic comp instead of a component constructor with weird component signatures.

Would it be difficult to implement the same sort functionality equivalent to some Svelte {if} and {each} methods into Go templates?

These would actually look very similar for Plenti users. I'm likely changing the syntax from {#if){:else}{/if} to simply {if}{else}{/if} because I personally never liked the former, but it will not look like Go's standard template package at all because I'm building it custom (just happen to be using Go). The {#each items as item}{/each} block will be changing to {for item of items}{/for} to be more inline with standard JS.

How would JS work for manipulating the DOM? having external JS files and importing them into Go files as needed?

You would still have component files very similar to Svelte with scripts, markup, and style. These would all be scoped together, so it shouldn't be something you have to worry about manually importing yourself. The main difference would be that conceptually you would be generating static HTML and then sprinkling in interactive JS when needed vs making everything a JS component from the start. I having some thoughts about scoping the JS to components to make life easier (similar to scoped CSS) and creating boilerplate helpers when using things like {if} or {for}, but I'm still working through this concept a bit, so more to come here soon hopefully!

Could you build on the idea of “Islands” for this JS ? similar to what Astro has done?

We're not actually taking any concepts directly from Astro, I've haven't really spent any time looking through their implementation, but I've heard them use the terms "islands" in their marketing. Basically the way we're building Svelte today is that we create both an SSR and DOM component for every single aspect of the website. Most of this is just static content that never changes, but we still build and interactive component that hydrates in the DOM and carries all the instructions to surgically manipulate that component if we needed to (even though we don't). Combine this with some of hacks we did to make content-driven dynamic components to work and now we're loading a ton of JS to the browser that can have all sorts of detrimental effects: slow initial pages loads, too many scripts that can block google from crawling clientside routes, hydration errors that make the app seem broken, having to bust client app cache with fingerprinting, canonical routes, etc. I'm just trying to take a step back and simplify this whole architecture because right now we're spending a lot of time solving problems that we're creating by loading too much JS in places it's not needed.

Would it be wise to learn Go? How much Go lang would I need to learn?

You still wouldn't need to have any knowledge of Golang to use Plenti. I've been calling this "custom go templates" because that's what I'm using to implement it, but at the end of the day the Plenti users should feel like they're using a very similar system to what they are using now. The template syntax and capabilities will change a bit, but should be familiar conceptually. They won't have to write any Go code and the logic in the templates will actually be JS (interpreted by Goja).

Ultimately there's still a ton of work I need to do on my end to ensure this is actually a viable direction for the project. That is going to be a several phase process. The experiments so far have been going really well and I see a ton of promise, but everything looks good with simple examples and can become much more challenging when implementing with real world problems. I'll definitely try to keep everyone posted on progress! Thanks for the conversation!

[jamestagal]

Hi @jimafisk

I saw this framework for building HTML with Go called Templ. Would this be something that you could utilise?

Ben

[jimafisk]

Hey @jamestagal,

I've actually been using Templ on a different project recently. It's a nice templating engine for Go, but it doesn't really have a solution for interactive / reactive UI's, so you still need to bring in something like HTMX for that. The way it works is it compiles to Go code so you need Go installed on your computer and you need to build your own binaries - Plenti tries to avoid that and actually only builds the deployed assets (html, css, js). They have some ergonomic improvements in works like scoped styles that need to be figured out before it's something I'd reach for on other projects. Overall, I think it's best suited for backend / full stack apps, but it's an interesting project and I'm definitely keeping an eye on it (not for Plenti, but other purposes).

[jamestagal]

I saw this video on Svelte 5 shipping with smaller bundle sizes and I was hoping this might sway you @jimafisk into reconsidering moving away from Svelte...let me know your thoughts. :)

The hidden benefit of upgrading to Svelte 5 - smaller bundle sizes

Cheers,
Ben