Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Miniflare 3] Support durableObjectsPersist option #517

Merged
merged 4 commits into from
Feb 27, 2023

Conversation

mrbbot
Copy link
Contributor

@mrbbot mrbbot commented Feb 21, 2023

Uses cloudflare/workerd#302 to enable Durable Object persistence.

  • durableObjectsPersist: undefined | false | "memory:" stores "in-memory"
  • durableObjectsPersist: true stores under .mf on the file-system
  • durableObjectsPersist: "(file://)?<path>" stores under <path> on the file-system

Note Miniflare 2 persisted in-memory data between options reloads. In Miniflare 3, workerd restarts on every reload, so if we used workerd's in-memory storage, we'd lose data every time options changed. To maintain Miniflare 2's behaviour, "in-memory" storage isn't actually in-memory. Instead, we write to a temporary directory and clean this up on dispose()/exit. 🙈

Also fixes a bug if multiple services bound to the same Durable Object class. This would result in duplicate entries in durableObjectClassNames. This has been changed from a Map of string[]s to a Map of Set<string>s to enforce the uniqueness constraint.

Closes cloudflare/workers-sdk#2403
Closes cloudflare/workers-sdk#2458

Internal ticket: DEVX-219

@mrbbot mrbbot requested a review from a team February 21, 2023 14:44
@changeset-bot
Copy link

changeset-bot bot commented Feb 21, 2023

⚠️ No Changeset found

Latest commit: 8eda364

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@mrbbot mrbbot added the tre Relating to Miniflare 3 label Feb 21, 2023
@mrbbot mrbbot changed the title Miniflare 3: Support durableObjectsPersist option [Miniflare 3] Support durableObjectsPersist option Feb 23, 2023
packages/tre/src/index.ts Outdated Show resolved Hide resolved
packages/tre/src/index.ts Show resolved Hide resolved
packages/tre/src/index.ts Show resolved Hide resolved
packages/tre/src/index.ts Show resolved Hide resolved
packages/tre/src/index.ts Show resolved Hide resolved
packages/tre/src/plugins/core/index.ts Show resolved Hide resolved
packages/tre/src/plugins/core/index.ts Show resolved Hide resolved
packages/tre/src/runtime/config/workerd.capnp Show resolved Hide resolved
packages/tre/test/plugins/do/index.spec.ts Show resolved Hide resolved
packages/tre/test/plugins/do/index.spec.ts Show resolved Hide resolved
@mrbbot mrbbot requested a review from penalosa February 27, 2023 11:02
Copy link
Contributor

@penalosa penalosa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🎉

Uses cloudflare/workerd#302 to enable Durable Object persistence.

- `durableObjectsPersist: undefined | false | "memory:"` stores
  "in-memory"
- `durableObjectsPersist: true` stores under `.mf` on the file-system
- `durableObjectsPersist: "(file://)?<path>"` stores under `<path>`
  on the file-system

Note Miniflare 2 persisted in-memory data between options reloads. In
Miniflare 3, `workerd` restarts on every reload, so if we used
`workerd`'s in-memory storage, we'd lose data every time options
changed. To maintain Miniflare 2's behaviour, "in-memory" storage
isn't actually in-memory. Instead, we write to a temporary directory
and clean this up on `dispose()`/exit. 🙈

Also fixes a bug if multiple services bound to the same Durable
Object class. This would result in duplicate entries in
`durableObjectClassNames`. This has been changed from a `Map` of
`string[]`s to a `Map` of `Set<string>`s to enforce the uniqueness
constraint.

Closes cloudflare/workers-sdk#2403
Closes cloudflare/workers-sdk#2458

Internal ticket: DEVX-219
@mrbbot mrbbot force-pushed the tre-durable-objects-persist branch from e8b0ae0 to 8eda364 Compare February 27, 2023 14:24
@mrbbot mrbbot merged commit 2246359 into tre Feb 27, 2023
@mrbbot mrbbot deleted the tre-durable-objects-persist branch February 27, 2023 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tre Relating to Miniflare 3
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants