SvelteKit Plugins

[AlecAivazis]

👋 Hello! We finally finished updating Houdini to integrate with the latest +page design and we had to pull a few different tricks to extend SvelteKit to support what we wanted to do. It occurred to me that what we had built was a kind of unofficial plugin framework for kit so I wanted to open this discussion to gauge interest in the off chance someone has the time to read some thoughts on what a native plugin system could look like for SvelteKit.

I tried to find a related issue but couldn’t find anything except for #2611 which sounds like more of an internal organization thing. I think this means that there are realistically 3 scenarios: I missed an issue somewhere; y’all have been thinking about this internally but keeping quiet on purpose; or you have been consciously ignoring it since everything is changing so quickly. Regardless of which, if any, is true, I won’t be upset if you just close this with “too soon” - I totally get it.

The final goal

Before I get into the weeds, it’d probably be helpful to have a general idea what we are doing and why it isn’t really supported by vite’s API (nor should it be). The big challenge was finding a way to implement our inline graphql queries (and +page.gql files). Here is an example of a route using Houdini. Just to be clear, this one file is everything the user needs to write for their route to be SSR’d. The +page.js and load function are generated behind the scenes and totally hidden from the user:

<!-- src/routes/myProfile/+page.svelte -->

<script>
	import { graphql } from '$houdini'

	const info = graphql`
		query UserInfo { 
			viewer { 
				firstName
			}
		}
	`
</script>

{$info.data.viewer.firstName}

Another situation that I struggled with was how to provide our own route-level configuration values exported from +page.js. I think this is pretty self explanatory but it was tricky to find a way to reliably look at the exported values of the file. I tried a few different things (some of them lifted from kit’s manifest logic) but ultimately decided that some simple static analysis on the file would be good enough for our constraints.

In the end, I came up with solutions for both of these situations but it definitely left me with a not-so-great feeling that what I was doing could be done in a better way.

A new integration point

Before the new design the first situation was relatively straight forward to support with the existing svelte preprocessor api since it just required creating a module script in a file that already existed. I think you can probably already see where I’m going with this. In the new world (which i LOVE fwiw), this is significantly harder because we have to fake the existence of a file from the perspective of SvelteKit and vite. The vite part is relatively easy to pull off with their load and resolveId hooks but in order to get Kit to generate the necessary imports for vite to process, the files need to actually exist in the filesystem.

This left us with one choice: monkey patching a bunch of functions in node:fs to do things like include +page.js if readdirSync is called on a directory inside of src/routes/.... It’s not pretty but given the current implementation of Kit, it works surprisingly well. Kudos to @pixelmund for the original idea. If you want to see the actual code, it lives here. I don’t have the exact details in mind, but I suspect basically all of the uncomfortable things we had to do are either things that kit is already doing or is logic that could be avoided all together with the right integration in kit’s manifest and related systems.

All of this is to say that I think a SvelteKit plugin can be vite plugin with some extra bits in the same way a vite plugin is fundamentally a rollup plugin.

What would this actually look like?

I think other libraries like svemix or mdsvex would also benefit from something similar but it’s too easy to get caught in never-ending edge cases and “what if”s so I’m only going to talk about houdini’s specific use case. In short, I think the only thing that’s really missing for us is a way to act on a route directory as a whole. I think of this as a fancy version of vite’s transform hook. Instead of just giving me the contents of a single file, this new hook would let me look at and transform the contents of a route’s +page.svelte, +page.js, +page.server.js, etc as one. Ideally, the function would also be passed the exported values of +page.js before transformation which would let plugin authors look for their own route configuration. I have more specific ideas but it’s not worth getting bogged down on implementation details right now. The most important thing is that we have a way to transform files like +page.js even if they don’t actually exist in the filesystem.

Possible Plugins

I’m sure you can already think of some good examples of pluggable bits of logic a user might want to add to their application. In the same way a svelte preprocessor lets people experiment with new syntax and features, this would provide a way for people to try different things using SvelteKit as a base layer. For example, in #6555, someone asked for a way to track when an application has been hydrated. Another example that would be helpful for a wider audience is our recreation of the old session store out of the current primitives. Implementing both of these is pretty straight forward with just the one hook described above. In my mind, this would be a huge step towards achieving a decent boundary that lets kit’s maintainers focus on building a strong foundation while people experiment and go crazy.

Next Steps

Well if you made it this far, thanks for your time ❤️ I think the first big question is whether this kind of extensibility is desirable or not. Assuming that it is, the next question is whether this needs to be in SvelteKit or if it should just exist as an unofficial third party extension. It’s probably worth mentioning that the current implementation does work for transforming routes but I couldn’t find a way to set kit config values from a vite plugin which feels like an easy win. If there was interest in something like this being part of the core framework, I’d gladly give it a shot and submit a PR. Otherwise I will likely pull these bits out of houdini and into a package for others to use. In which case, do you have any ideas for how I could inject additional configuration into kit from my vite plugin? Atm, I force users to manually define a $houdini alias which I would love to have hidden from their config file.

What do you think? Like I said, I understand this is definitely a post-1.0 thing so I won’t be offended if it’s not worth spending the time thinking through this right now. Just wanted to reach out, say hello, and see if there were any thoughts on what we’ve been doing.

[benmccann]

Can you share what the generated +page.js and load function look like?

do you have any ideas for how I could inject additional configuration into kit from my vite plugin? Atm, I force users to manually define a $houdini alias which I would love to have hidden from their config file.

You can use the Vite config hook to define aliases: https://vitejs.dev/guide/api-plugin.html#config. Though if you could just provide it as a standard npm package that would be preferable to an alias

[AlecAivazis]

If I use the vite config hook then I also will have to configure the typescript side of the alias, right? Either way, the user has to see some kind of configuration unless there's some way to use kit's conveniences.

Can you share what the generated +page.js and load function look like?

Totally! The best place to see all the different permutations are the snapshot tests. Here is the example for only a single, inline query like shown in the first codeblock:

import { load_TestQuery } from "$houdini/stores/TestQuery";
import { getCurrentConfig } from "$houdini/runtime/lib/config";
import { RequestContext } from "$houdini/runtime/lib/network";

export async function load(context) {
	const houdini_context = new RequestContext(context);
	const houdiniConfig = await getCurrentConfig();

	const promises = [];
	const inputs = {};

	inputs["TestQuery"] = {};
	promises.push(load_TestQuery({
		"variables": inputs["TestQuery"],
		"event": context,
		"blocking": false
	}));

	let result = {};
	try {
		result = Object.assign({}, ...(await Promise.all(promises)));
	} catch (err) {
		throw err;
	}
	return {
		...houdini_context.returnValue,
		...result
	};
}

There are a few things that seem a little weird for just a single query without any query hooks or variables, but at least you can see the general gist. For each query in a page, I build up the variables, push the pending promise to a list, and then build up an object with the results of each query. Probably worth mentioning that the load_ functions instantiate new stores on every request (no global stores) and return objects that looks like { QueryName: QueryStoreInstance }

standard npm package that would be preferable to an alias

I wouldn't be able to publish the contents of the $houdini directory to npm because it contains a bunch of generated code for every graphql document found in their project (very similar to what relay's compiler does if you are familiar)

edit:

I realized that all of my examples only import graphql from $houdini which probably confuses that last point a bit. It might help to know that we generate a store for every graphql document. Users can import them from $houdini along with a bunch of other things (for example, enums)

[benmccann]

Ok, so tl;dr: you want to create a load function if one doesn't exist. I think basically what you would need to do to solve this is to turn sync into a Vite plugin. If Vite wrote these files with emitFile then a later Vite plugin in the build process (e.g. a Houdini plugin) could transform the manifest to point at your generated code.

I can't say that this is something that's really on our radar. I do like the idea of sync being a Vite plugin for better interoperability with the ecosystem allowing you to do things like this. For dev and build you'd just add the new plugin to the existing build. svelte-kit sync would be a little trickier. You could have it invoke a Vite build with the plugin, which is pretty straight forward. However, the tricky thing is that you're not really bundling anything. Maybe you could just bundle an empty file or something when invoking Vite to have it spit out the file as a side-effect. You'd probably have to set no-treeshake on those files or something to make sure they're not optimized out as unused.

[benjaminpreiss]

Dear all!

As we are working on improving i18n with inlang, we are looking for a way of adding custom files (like /src/routes/+layout.ts) that can then be processed by the sveltekit plugin in a later step. It seems though, that calling this.emitFile() is not possible in vite serve mode? Are you aware of some projects / people that have done this already?

[AlecAivazis]

@benjaminpreiss if you have time to connect on discord I can walk you through all of the tricks we had to pull to get things to work 👍

[jycouet]

@dominikg @dummdidumm , we talked about it yesterday. 🥳
As promised, here is the discussion link.

[benmccann]

vitejs/vite#11469 has been sent in Vite. Hopefully that will help simplify the Houdini implementation quite a bit

[AlecAivazis]

Oh awesome - thanks for the heads up 🙌

Since this first implementation doesn't support transform, it's not immediately clear to me if this will offer much that's different from reading directly from disk for us (since the file content is already fake from the monkey patch). But either way it's a step in the right direction.

[benmccann]

vitejs/vite#11469 has been merged and will be included in Vite 4.2

[DiskoPete]

Hi @AlecAivazis

Thanks for your detailed explanation.

I would definitely be interested in some sort of plugin system for SvelteKit. 🚀
Hopefully it will become a native feature in the future. 🤞

[jycouet]

With the new fantastic Zero-effort type safety feature, I would like to bring a bit more attention to this discussion.

Yes, as written at the beginning by @AlecAivazis, Houdini generates +page.js for users.
And, we asked users to change their PageData type import like this:

<script>
-  /** @type {import('./$types').PageData} */
+  /** @type {import('./$houdini').PageData} */
  export let data
</script>

Yes, in the background, Houdini is also generating all types

Now, with the new Zero-effort type safety feature, regular kit users will not need to add the type, and that's great!
👉 How could we achieve the same with Houdini?

I believe that the plugin system would make it "official" to generate +page.js and, therefore, its type & the Zero-effort type safety that comes with it. Would be beneficial for everything.

@dummdidumm maybe you could share your view here? 😇

[dummdidumm]

The zero typings are implement in language tools and basically insert the type definitions as if you would write them yourself at the correct positions - you don't see them, but TypeScript does because we have added them under the hood. Making this configurable so that it would also work for ./$houdini imports sounds unrelated to the plugin story here, as this all happens within language tools. I'm not sure how an API could look like that makes that possible, and how to reliably tell language tools "this is a houdini project, you should do import('./$houdini').PageData instead of import('./$types').PageData".

[AlecAivazis]

I think the idea is that if there was a way to modify the $types file that SvelteKit generates, then the language tool integration would work automatically. Instead of generating our own type root we could just piggyback on the one Kit generates and add/modify the definitions for load.

In typing that out, I realized we could just modify the file directly after Kit generates it but that adds to an already uncomfortable number of tricks we have to pull to make things work.

Hopefully this doesn't come off as me asking you all to do a bunch of work so we don't have to. I appreciate everyone's free time and passion ❤️

[SergeantWeb]

Maybe something like that?
https://github.com/SergeantWeb/sveltekit-plugin-system

[david-plugge]

I don't understand what your library is doing, what can you do with it?

[david-plugge]

An example would be nice

[SergeantWeb]

I have updated the library, implementing several improvements, rewrote the README to make it more comprehensive, and included a detailed usage example.

The library let you add some "hooks" locations where custom plugins can inject components or code in your Sveltekit project.

It's similar to the Wordpress hooks system, but applied to Sveltekit and with the possibility to inject components.

Also, I added a discussion tab where you can ask any questions if you still find any aspects unclear 😉️

[clibequilibrium]

Maybe something like that? https://github.com/SergeantWeb/sveltekit-plugin-system

Thanks for sharing ! Is there any way in your opinion to run plugins in isolated contexts almost like in a vm ? for security reasons. I know VS Code uses extensions web host