RFC - Mutation Primitive

[ryansolid]

Summary

In this RFC I'm looking for feedback on the potential of a core Mutation primitive. While nothing being proposed could not be done in user-land, there is a good argument for establishing a simple pattern as sort of the complement of createResource. Resources do async calls, but are all about the data reads. Mutations are innately writes and actually don't really fit the pattern.

Background

Libraries like React Query or Apollo have useMutation hooks in React to along with their useQuery. These solutions are much more involved where Solid's createResource is more of a building block. However they establish a good pattern to imitate. One observation is there returned data is generally the last successful results.

However, another influence is Remix's forms. They expose your in-flight request as state as a way to allow for optimistic updates for the life of the request. This seems much more useful.

Proposal

Solid will have a new mutation primitive createMutation.

createMutation

The primitive will roughly carry this signature

export function createMutation<T, U>(
  mutation: (variables: T) => Promise<U>
  options: {
    onSuccess: (data: U, done: () => void) => void
    onError: (error: unknown, done: () => void) => void
  }
): [() => T | undefined, (variables: T) => Promise<U>]

The mutation is the method that takes variables and returns the promise for the mutation. The options object has callbacks for success and error. Important beyond passing data/error they have a done callback. If the arity of the function is 1 the caller will assume that the whole thing is done immediately, but if 2, done can be called at your discretion to complete the mutation. This is important to keep the optimistic updates in place say when refetching a resource.

The first return value is the infight signal which returns the submitted vars while the mutation is executing and undefined otherwise. This is useful for optimistic UIs. The second is the actual function that triggers the mutation.

Using with Resources

Most of the time mutations will be used with resources. In the success callback you will either take the data and apply the change manually in the browser, or tell the resource/s to refetch. Keep in mind if you don't want Suspense to fallback use Transitions when refetching.

// client side mutation
const [todos, { mutate }] = createResource(fetchTodos);
const [adding, addTodo] = createMutation(sendAddTodo, {
    onSuccess(newTodo) {
      mutate(todos => [...todos, newTodo]); // add the todo in the client
    }
});

// refetching
const [todos, { refetch }] = createResource(fetchTodos);
const [adding, addTodo] = createMutation(sendAddTodo, {
    onSuccess(_, done) {
      startTransition(refetch, done); // keep inflight data until Transition completes
    }
});

Optimistic Updates

Optimistic updates are one of the biggest benefits of this approach. There are lots of things you can do from treating the presence of inflight data as a pending indicator or rendering the update as if it is already there. Here is a rudimentary implementation and example of some of the things that can be done with this approach:

https://codesandbox.io/s/solid-todos-mutation-flmh6?file=/index.tsx

UPDATED: Here is another alternative example that handles multiple inflight:
https://codesandbox.io/s/solid-todos-mutation-v2-llequ?file=/src/index.tsx

Impact

This is a new feature with no proposed deprecations. However, this has the potential to be the foundation for other primitives so it is really important that we get the API right. So please leave your thoughts and feeback.

[atk]

For convenience, we should allow short-circuiting the promise with the value, so the signature would look like:

export function createMutation<T, U>(
  mutation: (variables: T) => Promise<U> | U
  options?: {
    onSuccess?: (data: U, done: () => void) => void
    onError?: (error: unknown, done: () => void) => void
  }
): [() => T | undefined, (variables: T) => Promise<U>]

Also, this is basically an async memo. We should at least document that this can be an intended use apart from mutation for resources.

Lastly, will the use of an onError handler stop errors from propagating to the next boundary and thus options and its content may be optional?

[ryansolid]

Yeah I agree on shortcutting.

The thing is they aren't reads so errors really have nothing to do with Error Boundaries.

In a sense this isn't supposed to be an async memo, Resources are. This is very much a mechanism for optimistic updates. Like the data returned is the temporary state and the only way to push the actual read data is through the callbacks. It's up to whatever you are setting to propagate errors and handle data (like Resources in most cases). That's why there is no data/loading/error states.

[trusktr]

If this is really for optimistic updates, can we include that terminology in the API?

[atk]

Ah, I misunderstood the first part of the return value. In this case the added value is basically more control over the mutation of resources?

[ryansolid]

Yes. It doesn't have to be resources necessarily but generally speaking you shouldn't view the mutation as owning the data. I didn't immediately create this type of primitive because there are variations and it is pretty easy to create in userland. However, every "query" library uses a similar pattern. I listed the popular ones like Apollo, and React Query, but true of SWR, Blitz, URQL, etc.. The thing is you could just run a promise and have it mutate the resource directly, that's generally the pattern in all these libraries, but being setup for optimistic updates after seeing Remix seemed like maybe worth something.

[trusktr]

Optimistic updates would be amazing, yes. @davedbase and I were chatting about this with respect to some primitives for solid-primitives. We were chatting about a generic RPC primitive on which communication over any boundary could be written (HTTP, workers, iframes, wasm, etc) and having the API be declarative and reactive, a la Meteor. Optimistic updates would be sweet to have in the mix.

[trusktr]

I don't have an opinion overall, because generally I've wired my own logic directly to signals and that's about it (i may be missing out and should start trying out these higher level patterns, but things seem to be getting a little complicated this far away from signals).

But, what I noticed is it uses callbacks with async done callbacks passed in (if I understand correctly). Should we strive to make everything as signal-based as possible, rather than using callbacks?

Perhaps @davedbase may have an idea here.

I don't know exactly what problem this primitive solves (can you show an example?), but maybe this shows more of the signals-for-everything idea:

const [adding, newTodo, error] = createMutation(sendAddTodo);

createEffect(() => {
  mutate(todos => [...todos, newTodo()]);
})

The reason for this idea is that callback patterns prevent composability. With signals, we can do this:

const [adding, a, errorA] = createMutation(sendA);

const [adding, b, errorB] = createMutation(sendB);

createEffect(() => {
  if (a() && b())
    mutate(things => [...things, a()+b()])
  else if (errorA() || errorB()) {...}
})

I write all this without understanding what createMutation is for, but with understanding of data being passed as callbacks vs signals. If you could provide a code example of what is difficult and what createMutation would then solve, that'd be helpful.

[trusktr]

I like the idea of optimistic updates (I've always loved this when making Meteor apps), but I'm still not clear on how it works here. Perhaps the word "mutation" also doesn't help express "optimistic".

I think the way the RFC is expressed may be useful only for people who know exactly how the patterns work already.

Can you make a full hypothetical code example?

[ryansolid]

Does the provided example, and explanation in my last message not cover that? This example: https://codesandbox.io/s/solid-todos-mutation-flmh6?file=/index.tsx.

I linked some other libraries using this pattern in the Background section, but I agree it seems that maybe there is lack of familiarity with this pattern. This pattern has really been taking over React the last 2-3 years, changing how people view the need for global state management etc... I'm looking at supporting the pattern in a more generic and extensible manner, but still not so generic that this is some any promise wrapper. Resources are still super important here as is the distinction between data with async dependencies and the ability to mutate those async sources independently of those resources.

[davedbase]

I think this example is succinct and clear and believe I've mostly grasped why it's designed in this manner from a high-level pattern perspective. From my perspective, I have 2-3 suggestions.

1. From a naming perspective I think perhaps createMutation is not technically correct/concise. A mutation is the resulting alteration and might be confusing in the sense that the returns values provide the data post-alteration (that might also be a reach). I propose createMutator as an alternative since the result of the primitive is more of a promise setter. The signal is the mutation status.

2. I'm wondering if it makes sense to put the mutator in first param and status second? I went through a bit of a theoretical phase with Solid Primitives and adapted a rule that the critical output of the primitive should be higher in the response array and less important options later. In this case is status as used/important as the mutator? or is it better as written in the example since it's closer to a signals output and keeps the mental model simpler?

3. Using more signals as you suggest @trusktr for the response or result forces the primitive to act more like a resource. The separation between resource and mutation might become blurred. It does add a level of convenience and simplicity though. Using the callbacks forces you to do work and feels less "automatic". Resources feel so ergonomic and reactive because they don't have callbacks.

I'm curious why this isn't modelled to be as a pairing of a resource. Resouce/Mutator are basically read/write so having similar DX for them might reduce cognitive overhead. Consider this:

const [mutation, mutator] = createMutator(Server.updateTodo);
mutation.loading
mutation.error
mutation(); // output from the promise or maybe it's not necessary storing this long term

Also what if you could have mutator and resource interoperable. So what if the resource stored the data and mutator had the ability to update the resource? ie.

const [todo, { mutate }] = createResource(fetchTodo);
const [result, mutator] = createMutator(Server.updateTodo, mutate, { merge: true }); // merge here identifies if the mutate setter should use reconcile automatically or set the update from the root

Of course this assumes that the returned value is of a shape that it can be set back onto the resource. If not then perhaps the user can supply their own mutate similar to the callback option. It doesn't need to come from a resource but has the same interface for ease.

I could be totally off or missed the point with this but those are my gut reactions. I don't mind the callbacks but they don't feel totally Solidlike. Ultimately this comes to testing out specific use cases.

[davedbase]

Ps. I suppose after writing that it's hitting me a bit more that the designs likely have to be different because a Resource retains data whereas a Mutation is more ephemeral ie. you just want the result to perform another mutation. I suppose I'm vying for something that feels more declarative. The callbacks do feel like a weird deviation but given the desire for optimistic rendering, I see why.

[madhav-p]

@davedbase

I'm wondering if it makes sense to put the mutator in first param and status second

Well, I think it will create confusion, because I always thought the convention is to use [state, action]

[madhav-p]

I like the ideas put here, but I think we can make it more low level to work for any Promise, and use signals for the promise result.

function createPromisePrimitive<T, U>(
  src: (variables: T) => Promise<U>,
  options: {
    onSuccess: (data: U, done: () => void) => void,
    onError: (error: unknown, done: () => void) => void
  }
): [
  () => boolean,    // inflight
  {
    invoke: (variables: T) => Promise<U>,    // invokes the promise when not already inflight
    abort: () => void     // aborts the current promise if inflight
  }
]

[atk]

Promises are not inherently abortable. Only functions returning them can be, usually using AbortSignal. However, it is simple enough to implement abort yourself.

[trusktr]

@madhav-p you were onto the same idea we were chatting about on Discord.

If we can rename this concept to something like createPromiseSignal, signalify, promiseSignal, or something to that effect, and then make something else specifically for API requests (because that is what "mutation" means here) but not call in "mutation", then I think the utilities will be better described, at least in so far the way they are being designed.

[madhav-p]

@atk Yeah, you are right, my idea was to have a cancellable promise,
but now that I think about it, it makes more sense to keep it seperate, to favour composition.

[Brendan-csel]

So how does this work if there are 2 mutations in flight? (for the same mutator)
Like normally in a "mutator" function we can await fetch and handle the response inline - which also allows for the mutating function closure to hold some state about this particular mutation event.
Does that fit in this pattern with separate mutator and done functions?

[Brendan-csel]

Here's an example that is close to the original Ryan posted but...

• supports arrays of mutations in flight
• does local mutation of the resource - so only really using the in-flight arrays to support isPending
• can now press ENTER to add items quickly
• delete issues are fixed in Server by guarding for deleting item not found
• has slightly longer delay to see issues and give time to add/update/delete multiple records.

This is the only way I could get this particular example working reliably.

The main issue is timing. It seems impossible to correctly make use of the in-flight items if there is a separate Resource that requires refetching. That is mostly due to the resource potentially discarding results for any individual mutation if a subsequent refetch has been triggered in the meantime - but it still gets more complicated than that.

The only apparent alternative seems to be to make the user wait for each mutation (and subsequent refetch) to complete.

The linked example seems to be the smoothest way to at least allow the user to keep mutating (eg ticking todos) while keeping the UI 99% consistent.

[ryansolid]

Thanks @Brendan-csel these are good ideas. Here is the updated demo I've made to try to address some of these: https://codesandbox.io/s/solid-todos-mutation-v2-llequ?file=/src/index.tsx. I am looking for more feedback but I will update the OP as this progresses.

[Brendan-csel]

That updated demo looks really slick. Nicely done.

[Feel-ix-343]

Is work still being done one this? I think that the approach in createRouteAction with action.input is pretty good, but it still requires a good deal of logic. If solidstart (or solidjs) did something like https://swr.vercel.app/blog/swr-v2.en-US#optimistic-ui, where createRouteData/createResource could return an action method (similar to the mutate method with optimisticData and rollbackOnError as parameters, the amount of logic and mental overhead would be greatly reduced. (When I was first learning about optimistic UI from the solid start docs, I was surprised at first that the apps I use daily do this much logic to get optimistic updates, and I thought it should work by modifying the actual data hook)

That being said, we would not get the grayed-out pending indicators like in Ryan's demo, so it would be important to keep both options.

I would be happy to work on this if necessary and continue the discussion in the solidstart repo if necessary.

[Feel-ix-343]

I guess this could also be added to https://start.solidjs.com/api/refetchRouteData with some specific keys or something