-
Notifications
You must be signed in to change notification settings - Fork 5.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Durable Objects] Deploy a Svelte site example required for Durable Objects #13062
Comments
@gerhardcit currently creating a DO in a Pages project is technically possible but practically useless since So basically when using Pages (which is what the C3 SvelteKit template does) you practically cannot define a DO in your application, the only current solution is to define the DO in a worker and then bind it to your Pages application (you can do that both in the dashboard and in the toml file, and
Yes... they are currently not for Pages... 😅
AFAIK the do it in the manner I just described, have the DO implemented using a worker and binding (locally and remotely) to your Pages application |
Even if Pages wouldn't have this limitation I struggle a bit to imagine how someone could define a DO in a framework and have it exposed in their worker entry point, since it would have to pass through the framework's build process. I'm not saying that it would be impossible but something that the framework itself would have to have baked-in (unless we introduced some new command/convention/way of uploading DOs...) |
Anyways... to summarize... I think that currently the only valid/practical way to use DOs in a SvelteKit (or Pages in general) project is to define the DOs in workers and bind those to the Pages application. Likely not ideal but I don't imagine there will be any better alternative soon... Anyways yeah I suspect that we don't really document that properly... |
Thanks.. sometimes, when you buy a car or a tool, part of what you want to know is what is CANNOT do. The Cloudflare docus here: provides this example:
so naturally the assumption is, ok, that is a thing. At least https://kit.svelte.dev/docs/adapter-cloudflare tells you specifically that it won't work. I'm repeating myself.. but I was hoping documented examples are actually tested. And is some cases provide a repo link where you can run that code and test it in full context. |
@gerhardcit sorry for the frustration 😓 yeah I get the argument that
isn't the best example to have there! 😓 I'll bring it to the team and see to remove it. Alongside making sure that we properly document the extra hurdles that Pages has in regards to DOs. Anyways, just to clarify, DurableObjects are supported (both in
100%, these things sometimes slip through the cracks (but we're trying our best to avoid that moving forward) sorry about that 🙇 |
I would pay good money to see an example of "define them in a separate worker " and how you would structure and run that. (again.. full context) |
Joining the thread. me and my company are working tightly with Pages, D1 and KV and plan to use DO for all of our realtime logic. Our use case is a collaborative 3D platform and DX is a top priority for us to iterate as fast as possible on realtime logic. So far Miniflare worked great with SvelteKit for KV and D1 with persistance and migraitons. I am eager to see working DO . @gerhardcit there are some clues already on how to make a worker with DO As well as how to even build the DO source code into a worker from SvelteKit project But I agree , a fully working solution would help a lot. There is not that much material on DO out there making many people look away... |
You
در تاریخ دوشنبه ۱۹ فوریهٔ ۲۰۲۴، ۲۳:۱۳ Alexander ***@***.***>
نوشت:
… Joining the thread. me and my company are working tightly with Pages, D1
and KV and plan to use DO for all of our realtime logic. Our use case is a
collaborative 3D platform and DX is a top priority for us to iterate as
fast as possible on realtime logic. So far Miniflare worked great with
SvelteKit for KV and D1 with persistance and migraitons. I am eager to see
working DO .
@gerhardcit <https://github.com/gerhardcit> there are some clues already
on how to make a worker with DO
https://github.com/JoshAshby/joshashby.github.io/blob/fa69dcdd9e9a81c991c702c5335d5251417af1e1/_drafts/sveltekit-durable-objects-continued.md?plain=1#L4
As well as how to even build the DO source code into a worker from
SvelteKit project
https://github.com/kalepail/boilerkit/blob/ec77b6cee8b0ed498002b3e4afa9a1fe9c13b8a9/src/helpers/_mf.ts#L33
But I agree , a fully working solution would help a lot. There is not that
much material on DO...
—
Reply to this email directly, view it on GitHub
<#13062 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ANND2JGGOPIDLFM6KD77OWLYUOTNNAVCNFSM6AAAAABDP3FAAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJTGA3TCOJTHE>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
در تاریخ دوشنبه ۱۹ فوریهٔ ۲۰۲۴، ۲۳:۱۸ Fariborz Samsame <
***@***.***> نوشت:
… You
در تاریخ دوشنبه ۱۹ فوریهٔ ۲۰۲۴، ۲۳:۱۳ Alexander ***@***.***>
نوشت:
> Joining the thread. me and my company are working tightly with Pages, D1
> and KV and plan to use DO for all of our realtime logic. Our use case is a
> collaborative 3D platform and DX is a top priority for us to iterate as
> fast as possible on realtime logic. So far Miniflare worked great with
> SvelteKit for KV and D1 with persistance and migraitons. I am eager to see
> working DO .
>
> @gerhardcit <https://github.com/gerhardcit> there are some clues already
> on how to make a worker with DO
>
> https://github.com/JoshAshby/joshashby.github.io/blob/fa69dcdd9e9a81c991c702c5335d5251417af1e1/_drafts/sveltekit-durable-objects-continued.md?plain=1#L4
>
> As well as how to even build the DO source code into a worker from
> SvelteKit project
> https://github.com/kalepail/boilerkit/blob/ec77b6cee8b0ed498002b3e4afa9a1fe9c13b8a9/src/helpers/_mf.ts#L33
>
> But I agree , a fully working solution would help a lot. There is not
> that much material on DO...
>
> —
> Reply to this email directly, view it on GitHub
> <#13062 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ANND2JGGOPIDLFM6KD77OWLYUOTNNAVCNFSM6AAAAABDP3FAAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJTGA3TCOJTHE>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>
|
I was able to get it working with miniflare. @gerhardcit remember that poc repo ? https://github.com/jculvey/svelte-cf-bindings-poc So here is an updated version of it . in hook.server.ts I added another Miniflare instance specifically for DO's const { Miniflare } = await import('miniflare');
const mf = new Miniflare({
kvNamespaces: ['KV'],
kvPersist: '.wrangler/state/v3/kv',
d1Databases: ['D1'],
d1Persist: '.wrangler/state/v3/d1',
modules: true,
script: ''
});
const doMf = new Miniflare({
modules: true,
durableObjects: { COUNTER: 'Counter' },
durableObjectsPersist: '.wrangler/state/v3/do',
scriptPath: 'durable-objects/counter/index.js'
});
// optionally you can do const counterNamespace = await doMf.getDurableObjectNamespace('COUNTER');
// but I just joined 2 miniflare bindings together
env = Object.assign({}, await mf.getBindings(), await doMf.getBindings()); or const { Miniflare } = await import('miniflare');
const mf = new Miniflare({
kvNamespaces: ['KV'],
kvPersist: '.wrangler/state/v3/kv',
d1Databases: ['D1'],
d1Persist: '.wrangler/state/v3/d1',
modules: true,
durableObjects: { COUNTER: 'Counter' },
durableObjectsPersist: '.wrangler/state/v3/do',
scriptPath: 'durable-objects/counter/index.js'
});
env = await mf.getBindings(); app.d.ts // See https://kit.svelte.dev/docs/types#app
// for information about these interfaces
// and what to do when importing types
import { DrizzleD1Database } from 'drizzle-orm/d1';
declare global {
declare namespace App {
interface Locals {
userId: string | null;
userData: UserData | null;
}
// interface PageData {}
// interface Error {}
interface Platform {
env: {
DB: DrizzleD1Database;
D1: D1Database;
KV: KVNamespace;
COUNTER: DurableObjectNamespace;
};
context: {
waitUntil(promise: Promise<any>): void;
};
caches: CacheStorage & { default: Cache };
}
}
} index.js looks like that , it has a blank worker . No wrangler pages dev needed . // Worker
export default {
async fetch(request, env) {
return new Response(`}`);
}
};
// Durable Object
export class Counter {
constructor(state, env) {
this.state = state;
}
// Handle HTTP requests from clients.
async fetch(request) {
// Apply requested action.
let url = new URL(request.url);
// Durable Object storage is automatically cached in-memory, so reading the
// same key every request is fast.
// You could also store the value in a class member if you prefer.
let value = (await this.state.storage.get('value')) || 0;
console.log(url.pathname);
switch (url.pathname) {
case '/api/increment':
++value;
break;
case '/api/decrement':
--value;
break;
case '/':
// Serves the current value.
break;
default:
return new Response('Not found', { status: 404 });
}
// You do not have to worry about a concurrent request having modified the value in storage.
// "input gates" will automatically protect against unwanted concurrency.
// Read-modify-write is safe.
await this.state.storage.put('value', value);
return new Response(value);
}
} Now in your SvelteKit projects create import { json, type RequestHandler } from '@sveltejs/kit';
export const GET: RequestHandler = async ({ platform, request, cookies }) => {
let url = new URL(request.url);
let name = url.searchParams.get('name');
if (!name) {
return new Response(
'Select a Durable Object to contact by using' +
' the `name` URL query string parameter. e.g. ?name=A'
);
}
// Every unique ID refers to an individual instance of the Counter class that
// has its own state. `idFromName()` always returns the same ID when given the
// same string as input (and called on the same class), but never the same
// ID for two different strings (or for different classes).
let id = platform?.env.COUNTER.idFromName(name);
// Construct the stub for the Durable Object using the ID. A stub is a
// client object used to send messages to the Durable Object.
let obj = platform?.env.COUNTER.get(id!);
// Send a request to the Durable Object, then await its response.
let resp = await obj!.fetch(request.url);
let count = parseInt(await resp.text());
let wasOdd = count % 2 === 0 ? 'is odd' : 'is even';
return json(`Durable Object '${name}' ${count} ${wasOdd}`);
}; run the server and go to You can create as many miniflares per DO script as you like or have them split into multiple files (to deploy with wrangler to production) but locally you can just join them under 1 file. To Deploy it to Cloudflare Networkcreate name = "counter"
compatibility_date = "2024-02-19"
[[migrations]]
tag = "v1"
new_classes = ["Counter"] Run @dario-piotrowicz it would be very nice if it was possible to deploy Durable Objects with Pages without needing a void Worker. |
@clibequilibrium , thanks, that looks great. The key piece of code here is this in
This is what i've been looking for high and low.
and if you can typescript those, export them somehow so they end up in the vite build ouput, then you have something really powerful. |
Also, @dario-piotrowicz , please correct me if I'm wrong. DurableObjects and Pages don't go together correct? I can run a DO in a worker, but not a raw pages project with a functions directory? I tried that as a test, but
|
@clibequilibrium yes indeed it would be! 😅 I do not have any like ETA or idea regarding when/how this could be implemented but I would strongly imagine that with Workers and Pages convergence, defining a DO in a "Pages" app should be possible in the future |
@gerhardcit regarding I'll open an issue in the workers-sdk repo for this and see what people think 👍 PS: can you clarify to me, do you find this simpler/more convenient than running a worker which exposes the DO and bind that DO with your SvelteKit app? because I personally find the latter much clearer/cleaner... unless I'm missing something I also still want to provide you with an example of the above (since you said you'd like to see one), I'll get to write one when I have some spare time on my hands |
DurableObjects and Pages do work together, the issue is that you cannot define a DurableObject inside a Pages application, but you can define the DurableObject inside a worker and use a DurableObject binding to use that durable object inside your Pages application (to clarify, I am not saying to use a service binding to the worker and get the DO, I'm saying that you can bind directly to that DO).
Yes, sort of... You can declare a DO in a worker but in a Pages project it's different... you can declare the DO in a pages project but it won't get deployed, so basically, you can but there's nearly no point in doing so Also the Pages' |
@dario-piotrowicz , Initially I thought binding in the Sveltekit App is the most obvious way. But understanding DurableObjects better now, with Alarms specifically, it is a true "worker" concept. So I think there must be limits when you mix it in to Pages apps. The contradiction though, is the way websockets and DurableObjects are mixed. This feels like someone shot them selves in the foot. You create a true worker component, that fires up and work in the background, but then you mix in with a websocket UI into it.. Somehow that is a weird concept to get my head around. So for what we want to use DurableObjects for, with Alarms etc.. I'm going to run them completely independent and than hack the CORS thing to get to it from another Pages UI that does ALL the other KV, D1 and R2 stuff. I thought I solved it with a Vite UI (Pure Svelte) and a functions folder for my api.. but when I throw Durable Objects in there I had the same issue. |
@gerhardcit if it helps, here's a quick example of using a DO with Pages in advanced mode (only locally): https://github.com/dario-piotrowicz/cf-pages-advanced-mode-do-example/tree/main |
@dario-piotrowicz , thx. that is simple enough.. but I'm not following the practicality of the warning
When you say production, you mean the pages or the worker? Something does not add up somehow. Apologies, but I need to see the link between "here" and "there" which seems to how it will be deployed. It feels like there is a remote URL variable that needs to come in somehow. |
Exactly! sorry if I skipped through that part, basically DOs are currently only like a "pure worker concept" and Pages can somewhat "borrow" them, but not use them independently
Sorry I am not seeing that, could you point me to the specific section that says that? 😅 |
The wording is: " |
sorry for the warning not being too clear 😓 I meant the Pages "production"/actual/remote deployment Basically, you can try to clone that repo and run The issue being that in the Cloudflare dashboard you will not find your DO anywhere, it simply won't get uploaded/recognized (as that is not supported for Pages). The only thing you can do to get the binding to work in your deployed Pages application is to define a separate Worker project, define your DO there and deploy the worker. Only then, you will be able to bind that DO to your existing Pages project. Again, having your Pages project, practically "borrow" the DurableObject declared/owned by your Worker project |
@gerhardcit sorry for being dense.... but I don't see where this relation between that text and (all I read there is "don't use Workers Sites as it is deprecated, use Pages instead", noting that Pages, especially in advanced mode, can do all that workers sites can) |
You can bind the DO in Pages from the dashboard and it will be already available in the bindings just like KV and D1. See my example above. After you deploy your DO with a blank worker , you can link the DO to Pages under Settings->Functions tab What is also amazing is that you get to debug DO and see the metrics I also disabled the routing for the empty Worker so it is not available. You interact with DO purely from your SvelteKit app . Empty worker is there just so you can publish to Cloudflare network via wrangler since publishing DO with Pages is not supported. |
@gerhardcit if you want Typescript with Intellisense and everything I also tried this repo https://github.com/cloudflare/durable-objects-typescript-rollup-esm Then did And JS version here: https://github.com/cloudflare/durable-objects-rollup-esm |
Thx @clibequilibrium, I'm digging into it. |
As per my message here #13062 (comment) It is going to be in your |
@gerhardcit I've checked with the team and it seems like the general consensus is that something like So for now I think we'll just keep the current implementation and things should get much better whenever convergence allows it... I hope that can work for you? 🙏 (PS: I still owe you an example of how to set up the binding locally, I haven't forgotten, I'll provide one today or in the next few days 🙇) |
Hi ! I made it work without Miniflare entirely via In your pages wrangler.toml file you need to add [[durable_objects.bindings]]
name = "DO_COUNTER"
class_name = "DO_COUNTER"
script_name = "worker" // this is a name of a running worker that your DO runs on. navigate to http://localhost:6284/workers to see the name of the worker with wrangler dev running of DO Worker Then Make sure you are on the latest version of Wrangler on both DO and Pages projects For persistence: const { getPlatformProxy } = await import('wrangler');
platform = await getPlatformProxy({ persist: true }); And run DO via
|
Ok
در تاریخ یکشنبه ۲۵ فوریهٔ ۲۰۲۴، ۰۶:۳۸ Alexander ***@***.***>
نوشت:
… Hi !
I made it work without Miniflare entirely via wrangler pages dev and wrangler
dev full hot reload support and websocket support with persistence!!
In your pages wrangler.toml file you need to add
[[durable_objects.bindings]]name = "DO_COUNTER"class_name = "DO_COUNTER"script_name = "worker" // this is a name of a running worker that your DO runs on. navigate to http://localhost:6284/workers to see the name of the worker with wrangler dev running of DO Worker
Then wrangler dev on the DO worker and wrangler pages dev
--compatibility-date=2023-11-21 --proxy 5173 -- pnpm run dev and voila
full hot reload, persistance and websocket support !
—
Reply to this email directly, view it on GitHub
<#13062 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ANND2JAKP3W3DYMWGCFXEWLYVKTJRAVCNFSM6AAAAABDP3FAAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRSG44TOMJRGU>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
@gerhardcit here's the (very minimal) example I promised 🙂 https://github.com/dario-piotrowicz/sveltekit-durable-object-local-usage-example Please let me know if this helps |
I've created cloudflare/workers-sdk#5164 in the workers-sdk to capture what's been discussed here and maybe get the example moved over there (if we find it useful) Besides that I don't think there's much we can do in the Cloudflare docs (besides maybe linking to the workers-sdk examples somehow?) I'm closing this issue as we can continue the discussion in workers-sdk, I hope that works for you @gerhardcit 🙂 |
@dario-piotrowicz Thank you for the helpful repository. This issue would be even more useful if you could provide more details about the comment below. Specifically, we are looking for a clearer understanding of how to make SvelteKit work with Durable Objects in the actual Cloudflare environment. I eventually understood (it's took 4 hours, at least) but it seems undocumented and the future readers must want to know.
|
@sundaycrafts thanks for your comment 🙂 I'm sorry for the inconvenience the missing documentation caused you 🙇 I'll see what I can do regarding adding the info somewhere in the docs Also as I mentioned earlier in this thread things should hopefully be much simpler/easier after convergence 🤞 (not a huge relief right now but still 😅) |
Thank you, Dario, I've had some time, so I'd like to share an implementation image of a session store that is probably ideal as a real-world example. First, deploy a worker like the following (please note that this is a code snippet, so for the full implementation, refer to Dario's code or the official documentation). It's important to note that if you want to solely use it with Pages, the handler of this worker (export default > fetch func) is actually never used, so it should be closed. // ./sessionstore-worker/src/index.ts
import { Namespace } from './namespace';
import { HTTPResponseCode } from './HTTPResponseCode';
import { getHandler } from './handlers/getHandler';
import { putHandler } from './handlers/putHandler';
import { defaultHandler } from './handlers/defaultHandler';
export interface Env {
SESSION_KV: KVNamespace;
SESSION_DO: DurableObjectNamespace;
}
/** This Durable Object and its logic will be shared with your Sveltekit app **/
export class SessionStore {
private readonly serializedSessions: Record<string, string> = {};
constructor(
private readonly state: DurableObjectState,
private readonly env: Env,
) {}
async fetch(request: Request): Promise<Response> {
const maybeNs = Namespace.tryFromUrl(new URL(request.url));
if (!maybeNs.success) return new Response(maybeNs.error.message, { status: HTTPResponseCode.BAD_REQUEST });
console.log(`[${request.method}] ${maybeNs.data.toString()}`);
switch (request.method.toUpperCase()) {
case 'GET':
return getHandler({
namespace: maybeNs.data,
serializedSessions: this.serializedSessions,
kv: this.env.SESSION_KV,
});
case 'PUT':
return putHandler(request, {
namespace: maybeNs.data,
state: this.state,
serializedSessions: this.serializedSessions,
kv: this.env.SESSION_KV,
});
default:
return defaultHandler();
}
}
}
export default {
async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
// This stub worker reject all requests
return new Response('', { status: HTTPResponseCode.FORBIDDEN });
},
}; // sessionstore-worker/src/handlers/putHandler.ts
import { Namespace } from '../namespace';
export async function putHandler(
request: Request,
context: {
namespace: Namespace;
state: Pick<DurableObjectState, 'waitUntil'>;
serializedSessions: Record<string, string>;
kv: KVNamespace;
},
): Promise<Response> {
const newValue = await request.text();
if (context.serializedSessions[context.namespace.toString()] !== newValue) {
context.serializedSessions[context.namespace.toString()] = newValue;
context.state.waitUntil(context.kv.put(context.namespace.toString(), newValue));
}
return new Response('ok');
} After deployment the worker, you will be able to bind the Durable Object defined in this Worker to Pages. Therefore, bind the DO from the Pages settings screen. After that, you just need to add the DO interface to platform.env as mentioned in Cloudflare's documentation. When a fetch is performed from a DO object bound within Pages, the fetch defined in the worker for that DO (in this example, The specific way to call the bounded DO from SvelteKit side should be clear by looking at Dario's code. For local execution, you should refer to the repository shared by Dario. Specifically, the following part is key. // src/hooks.server.ts
import { dev } from '$app/environment';
/*
When developing, this hook will add proxy objects to the `platform` object which
will emulate any bindings defined in `wrangler.toml`.
*/
let platform: App.Platform;
if (dev) {
const { getPlatformProxy } = await import('wrangler');
platform = await getPlatformProxy();
}
export const handle = async ({ event, resolve }) => {
if (platform) {
event.platform = {
...event.platform,
...platform
};
}
return resolve(event);
}; Another point to note is that Dario's code includes a It might seem a bit inelegant to have to define a placeholder for the worker endpoint for deploying the sessionStore, but I think it makes sense for the sessionStore module and the frontend module to be separated, and it's a good level of granularity. ...so, this is the meaning of the "borrow". I hope this comment will be helpful to future readers. === Edit === I noticed that this comment pointed exactly same thing. |
This seemed to work but I was not able to use WebSockets. The problem seems to be that export const fallback: RequestHandler = async ({
request,
params,
platform
}) => {
console.log('here');
const { room } = params;
const upgradeHeader = request.headers.get('Upgrade');
if (!upgradeHeader || upgradeHeader !== 'websocket') {
return new Response('Durable Object expected Upgrade: websocket', {
status: 426
});
}
let id = platform.env.WEBSOCKET_HIBERNATION_SERVER.idFromName(room);
let stub = platform.env.WEBSOCKET_HIBERNATION_SERVER.get(id);
console.log('here2');
return stub.fetch(request);
}; |
Hi unfortunately there is only 1 blog post: https://joshisa.ninja/2022/05/18/sveltekit-cloudflare-durable-object-websockets.html about it but I wasn't able to get it working; see this discussion with the author: JoshAshby/joshashby.github.io#22 (comment) What I did instead is deploy a worker to upgrade to websocket connection. |
Which Cloudflare product(s) does this pertain to?
Durable Objects
Subject Matter
The way a durableobject class is exported in context of a sveltekit project.
Content Location
https://developers.cloudflare.com/pages/framework-guides/deploy-a-svelte-site/#sveltekit-cloudflare-configuration
Additional information
The latest create-cloudflare project generates this code in hooks.ts when you select a sveltekit project.
This is great. So now in
npm dev
node (using sveltekit dev mode) you can easily use KV and R2 (tested) and probably D1 bindings.However, there is not proper example of how (or where) to export a Durable Object class.. eg:
Given this class:
WHERE do we put this class so that it forms part of the build process.
(It cannot be in a +server.ts or a +page.server.ts file. sveltekit does not like that.
Having this a
wrangler.toml
config of like this:And a
/api/+server.ts
file like this:PLEASE, PLEASE PLEASE create a sveltekit project that works when you do this
npm run build
npx wrangler pages dev .svelte-kit/cloudflare
To prevent this error:
we are soooooo close to getting this in line. But the documentation lacks the full example that provide context of the class.
DurableObjects are easy in a simple cloudflare worker... but NOT in a sveltekit project. Does other frameworks solve this?
The text was updated successfully, but these errors were encountered: