Alternatives Async/side effect models

[yelouafi]

it's been some time since I'm thinking on Async/Side effects models in Elm architecture. I'd really like to write some article on this to complete the first one. I know it was already discussed in #13 but i'd like to present some other alternatives here for discussion. i´ll motivate the need for alternative models and will illustrate with the standard Counter example. Sorry for the long post.

Currently the standard Elm solution works by turning the signature of update from this

update : (state, action) -> state

to this

update : (state, action) -> (state, [Future Action])

// using Reactive streams
update : (state, action) -> (state, Stream Action)

// using callback with Node CPS style
update : (state, action, dispatch -> () ) -> state

Whatever solution we choose, it allows us to express side effects and fire asynchronous actions. However, there is a thing i dislike here : in the first effect-less version we had a nice pure function (state, action) -> state. It's predictable and can be tested with ease. In contrast, it's not that easy in the effect-full version : we now have to mock the function environment to test the effect-full reactions.

IMO it'd be preferable to keep the update pure and clean of any side effect. Its only purpose should be : calculate a new state giving an existing one and a given action.

So i present here 2 alternatives side effect models for discussion :

1- The first model is taken directly from redux (which was inspired by Elm). Like in Elm architecture, redux has pure functions called reducers with the same signature as update. A pretty standard way to fire asynchronous actions in redux is through thunks : i.e. callback function of the form dispatch -> ().

I'll illustrate with the Counter example and an asynchronous increment action

// Synchronous : () -> Action
function increment() {
  return { type: 'increment' }
}

// Asynchronous : () -> ( dispatch -> () )
function incrementAsync() {
  return dispatch => {
    setTimeout( () => dispatch( increment() ),  1000)
  }
}

const view = ({state, dispatch}) =>
    <div>
      <button on-click={[dispatch, increment()]}>+</button>
      <div>{state}</div>
      <button on-click={[dispatch, incrementAsync()]}>+ (Async)</button>
    </div>;

const init = () => 0;

const update = (state, action) => type === 'increment' ? state + 1 : state;

The idea is : instead of firing a normal action, the component fires an action dispatcher (the thunk). Now our main dispatcher has to be enhanced to know about thunks

function dispatch(action) {
  if(typeof action === 'function')
    action(dispatch);
  else {
    state = Counter.update(action);
    updateUI();
  }
}

The main benefit of the above approach is to keep the update function clean of any effect (this is also a golden rule in redux : a reducer must not do any side effect like api calls...). Another benefit is that you can fire multiples actions from the thunk which is useful to manage a flow of multiple actions.

2- the second alternative was inspired by this interesting discussion (also in the redux site, yeah!), the idea is to separate Effect creation and execution in the same way we separate Action creation and update, the update signature will look quite the same as in the current Elm solution

update : (state, action) -> (state, Effect)

However, the Effect above is no longer a Future or a Promise or whatever, it's a just a data object much like the actions that describes the intent. To run the actual effect we'll provide the component with another method execute which will run the real side effect. So back to the Counter example

// an union type to describe "effect-full" value
const UpdateResult = Type({
  Pure: [T],
  WithEffects: [T, T]
});


const Action = Type({
  Increment      : [],
  IncrementLater : []
})

const Effect = Type({
  IncrementAsync : []
});

const view = ({state, dispatch}) =>
    <div>
      <button on-click={[dispatch, Action.Increment()]}>+</button>
      <div>{state}</div>
      <button on-click={[dispatch, Action.IncrementLater()]}>+ (Async)</button>
    </div>;

// pure and withEffects are 2 helper factories
const init = () => pure(0);

const update = (state, action) => Action.case({
  Increment       : () => pure(state + 1),
  IncrementLater  : () => withEffects(state, Effect.IncrementAsync())
}, action);

const execute = (state, effect, dispatch) => Effect.case({
  IncrementAsync: () => {
    setTimeout(() => dispatch(Action.Increment()), 1000)
  }
}, effect);

export default { view, init, update, Action, execute, Effect };

So the component have now 2 additional properties : Effect which like Action documents side effects carried by the component and execute, which like update, run the actual effect. This may seem more boilerplate but has the advantage of keeping the update function pure : we can now test its return value by just checking the returned state and eventually the side effect data. Another advantage may be that Effects are now documented explicitly in the component.

The main dispatcher will look something like

function updateStatePure(newState) {
  state = newState;
  updateUI();
}

export function dispatch(action) {
  const updateResult = App.update(state, action);
  UpdateResult.case({
    Pure: v => updateStatePure(v),
    WithEffects: (v, effect) => {
      updateStatePure(v);
      Counter.execute(effect, dispatch);
    }
  })
}

Here is an example of the above approach with nested components.

What do you think of those 2 models ? the redux solution seems more simple to me as it doesn't add more boilerplate to the current model. The 2nd solution has the advantage to be more explicit and clearly documents the Effects carried by a component.

I'd be also nice to hear the opinion of @evancz or also @gaearon the creator of redux, knowing his Elm background

[slorber]

@yelouafi I think the Redux option is simpler.

About the 2nd option, I'm not sure you understood what khaledh said about DDD / CQRS / EventSourcing. Also the signature update : (state, action) -> (state, Effect) does not match what I know of event-sourcing neither. There's no function that update the view state and trigger effects at the same time. However it is possible to trigger effets with the Saga pattern

I've answered the discussion with me thoughts and why the backend concepts are not so easily transposable to the frontend because of the different context.

Also if you would like to use the DDD / CQRS / ES approach, you should search how a backend system implemented this way would query an external API: this seems to me the closest thing to what we do with Ajax calls (I don't have the answer, I guess it's an implementation detail).

What think it that in ES the command handler may be able to handle such async operations (saga too but with more boilerplate) and fire the request/success/error events. And the ActionCreator in Flux is the closest to the command handler, but it does not have that possibility in your example.

Going back to 1st solution, I'm not sure it's really needed to use a thunk, as we could easily pass the dispatcher to all actionCreators directly.

For example:

function incrementAsync(dispatch) {
    setTimeout( () => dispatch( increment() ),  1000)
}

<button on-click={[incrementAsync(dispatch)]}>+ (Async)</button>

[yelouafi]

I think the Redux option is simpler.

Sure, i agree. But is simpler always synonym of better ? i wonder. If all we're looking for is simplicity we should just stick to React's setState solution but clearly we'are also seeking other things (predictability, testability, scalability/composability). Actually what bothers me in the Redux solution is that it tends to put too much logic in async action creators. Precisely when we're dispatching multiples actions from a single thunk. I prefer to put as much as i can of my logic inside pure functions.

About the 2nd option, I'm not sure you understood what khaledh said about DDD / CQRS / EventSourcing

i'm not trying to implement DDD / CQRS / EventSourcing and i' wouldn't even pretend it as i'm not that aware of this stuff. khaledh's post just gave me the idea with his distinction between Events/apply and Actions/execute to draw a separation between Effect creation and execution, and this for the same reasons we're separating Action creation and execution: make the maximum of our code composed of pure functions.

In the proposed solution i can test both that the application updates the state correctly and in the same time fire the desired effect (e.g. makes the correct api call) without having to mock anything. But the more important advantage IMO is the ability to compose Effects in a complex nested hierarchy. Which is not trivial in the case of thunks.

As a side note, there is a subtle difference between events - which denote facts happening in the surrounding environment - and Actions in Elm : in Elm an Action is the interface that a component exposes to the external world to act on its state : so things like FilterChanged or UserChanged aren't really actions from a semantical point of view. And seen from this angle, i agree with khaledh's statement that Redux conflates Events with Actions at least in the implemented examples (e.g. the 'ADD_TO_CART' action in the Shopping Cart example is handled by multiples reducers: this seems to me more an Event than an Action; in Elm an Action belongs to exactly on component, but a parent component may intercept or modify a child Action)

EDIT

Going back to 1st solution, I'm not sure it's really needed to use a thunk, as we could easily pass the dispatcher to all actionCreators directly.

Yes you're right, and this is the solution i adopted in my last post

[slorber]

I will have to re-read that again later to better understand what you mean.

Can you find any usecase where you would like the ability to compose effects?
I mean for example you have update : (state, action) -> (state, Effect) then what?
You have to bubble the effects to the top before they are applied?

In my opinion, and this is also what we do on the backend, we should not melt together the code that is used to compute the view state, from the code that is used to orchestrate complex operations (the Saga). So what I mean is that in my opinion you don't need that effect composition system, because the saga does not have to be deeply nested: it is a separate actor on its own, and does not have to be implemented as a reducer used by a Redux store: it can live separately because its state is not intended to be consumed by React components.

[ericgj]

@yelouafi I don't completely understand your motivations, but thanks for putting these ideas out there. I think the fact that related ideas are being discussed in redux and elm indicates there's a limit to the current techniques of dealing with side effects.

in the first effect-less version we had a nice pure function (state, action) -> state. It's predictable and can be tested with ease. In contrast, it's not that easy in the effect-full version : we now have to mock the function environment to test the effect-full reactions

I'm not clear exactly how either of your two solutions addresses this. In the second case, is it that your testing environment would define its own dispatch, which would simply inspect the effect rather than execute it?

My own current preference for dealing with testing effectful actions is to inject future-creating function(s) into the action. See for instance this, tested here.

My preference is to have effects be essentially 'black boxes' within app components -- all that actions are responsible for doing is passing in parameters to them and mapping their results to actions. I suspect this is related to what @slorber is saying about having them 'live separately'. But then the downside of that is ... they are black boxes. I can imagine wanting to inspect and manipulate or combine them in some cases, within the app, before executing.

Have you seen this proposal for 'custom' effects in Elm ? It's a bit over my head but I think is addressing a related problem space - i.e. leaving effects as data for as long as possible so they can be combined/manipulated before execution. There was a recent discussion on the Elm google group list about this too.

[yelouafi]

@slorber

Can you find any usecase where you would like the ability to compose effects?
I mean for example you have update : (state, action) -> (state, Effect) then what?
You have to bubble the effects to the top before they are applied?

Yes this is the main use case but there can be others where a parent component want to do other things: like sequencing the child effects, waiting for all of them or pre-processing them.

In my opinion, and this is also what we do on the backend, we should not melt together the code that is used to compute the view state, from the code that is used to orchestrate complex operations

I understand your SoC concern; But this kind of vertical separation may not always be the best fit for client side UI apps. Where we often need generic and reusable components that embed a whole reusable use case (grids, specialized inputs, Dialogs...). And that's why Redux can't completely replace stateful React components.

Say for example i have a Router that may embed arbitrary components; it can bubble actions and side effects from children without need to know anything about them. Of course this containment lead to some boilerplate of wrapping/unwrapping but i think we can work around this.

So what I mean is that in my opinion you don't need that effect composition system, because the saga does not have to be deeply nested

I think the situation in client side UI apps is a bit simpler than in the backend where we can have long living and complex transactions. But perhaps there can be situations where we can have that kind of use cases (like multi-step dialogs) and i agree here that the Elm pattern is not a natural way to express those things, (although this can be achieved using some kind of state-machine-component wrapper around components that handle each step).

@ericgj

is it that your testing environment would define its own dispatch, which would simply inspect the effect rather than execute it

we don't have to define a special dispatch. if i have

const update = (state, action) =>
  action.type === 'getData' ?  [{pending: true}, {type: 'fetch', url: action.url}] :
  /* action.type === 'data' ? */ [{pending: false, data: action.data}, null]

const execute = ({url, dispatch}) => fetch(url).then(data => dispatch({type: 'data', data})

i can check if my update function triggers the right effect; not if it executes it.

expect.deepEqual( 
     update({}, {type: 'getData', url: 'url'}),  
     [{pending: true}, {type: 'fetch', url: 'url'}]
)

I can imagine wanting to inspect and manipulate or combine them in some cases, within the app, before executing.
Have you seen this proposal for 'custom' effects in Elm ?

Actually the main reason for the proposed separation effect creation/effect execution is to make the update function pure, predictable and easily testable. But from the above this can be another benefit of being able to pre-process effects before executing them (like in the evancz example of optimizing multiple GraphQL queries by producing as few queries as possible)

[tomkis]

I have to agree that Redux solution is as closed to ES/CQRS/DDD as it can be. Action creators are commands, Actions are Events and Reducers are event handlers and yes, if there is a need for 3rd party integration or basically any side effect it's always executed in command handler.

However, this approach (even in Redux) has one major drawback with few consequences. What is potentially a part of your domain and it's the decision that you want to perform some side effect is actually abstracted away from your domain logic. We are not concerned about the actual execution because it belongs to service layer.

That's why I believe Saga like approach is better. Because it does not decouple the actual domain logic and intent to perform some side effect

Give the UC:

As a user I want to do some action

1. Mutate the application state somehow
2. Call 3rd party service to log the action

As you can see 2) is an intent to perform some side effect and we want to keep that consistent and in one place. It's also easy to write test which checks whether both steps are executed (we don't need to test actual API call execution we are fine with the intent).

I was trying to tackle that in Redux for a while and come up with https://github.com/salsita/redux-side-effects this repo. I believe that abusing reducer's reduction for side effects is not ideal solution, especially it's very verbose and does not work well with reducers composition.

In my opinion, generators are perfect way to model the behaviour. You can simply return mutated application state and yield side effects which are just functions. Those functions are not executed right away but after the dispatch loop is finished, meaning that reducers are still pure.

Currying is a great help for deferred side effects:

const sideEffect = param => dispatch => console.log('side effect')

Composition with generators works very well because composing reducers is as easy as using yield* for yielding effects upper to the composition hierarchy.

[tomkis]

I was also re-thinking the relation between Redux and CQRS/ES/DDD which leads me to the idea why reducing side effects inside reducers is evolution of using redux-thunk:

Redux (using redux-thunk) is a form of CQRS/ES/DDD

As mentioned above the direct relation is inevitable because:

1. Command is User's intent e.g. click on the button

2. Command Handler is Action Creator (when using thunk-middleware) because it retrieves corresponding aggregate which is actually the application state and it dispatches actions which are events. In traditional CQRS it's called on aggregate.

3. Reducer is event handler because it's lfold of actions (events) into application state

We don't need aggregates on FE

Or actually we do, but just one and it's the app state itself. We don't need aggregates here because:

1. Transaction boundary is within action handling inside reducer.
2. There is no concurrent access which would require ACID

In other words Application State is Root Aggregate

And because there are no aggregates, there are no command handlers

Action creator (using thunk-middleware) is basically a combination of Command handler + Aggregate method call. Because it's dispatching actions => We can access app state in action creator, which is the root aggregate and we can also dispatch actions which is actually same like emitting domain events.

In ES, you can't deny the event

Because it's something that has already happened, there's no way to deny the fact. You can't have invariants in event handler because it's something that must always pass and never fail.

On FE everything is an event

Yes, there are no commands, only events. When you click the button, it's not command. The command is actually inside your brain and it's the intent to click the button. This is the most important distinction between FE and BE implementation of CQRS/ES/DDD. Clicking the button is event and it's same like API response. Just accept it, FE is event oriented.

Facebook chat is an example: When you send a message yet there's no internet connection at the moment the message is displayed in chat (UI state has been updated), but that's something that happened and you can't take it back. The only way to take it back is dispatching another counter action which means clicking on some button (remove the message).

Therefore there are no invariants when handling actions => Dispatched action always (if necessary) results in state mutation. Even if that means showing an error message.

Conclusion

Because there are no commands and command handlers, action creators should be just action factories. Yes, no more logic inside action creators, no more redux-thunk which gives you access to app state and allows you to emit events because the event has already been emitted and it's the user interaction itself.

This basically means, that we have to reduce the side effects within reducers. We just need to find the best way.

[yelouafi]

@tomkis1 thans for your comments; i'm all new to ES/CQRS/DDD so maybe i'm misunderstanding some concepts.

What is potentially a part of your domain and it's the decision that you want to perform some side effect is actually abstracted away from your domain logic. We are not concerned about the actual execution because it belongs to service layer.

Actually this somewhat how it works in Elm : the return value of a reducer is a tuple (stat, Effect). The Effect here is a wrapper around a Task. But at this point there is no execution yet; the execution takes place later by sending the task into a port. You may see that it somewhat maps to your concept of yield/return.

Because there are no commands and command handlers, action creators should be just action factories. Yes, no more logic inside action creators, no more redux-thunk which gives you access to app state and allows you to emit events because the event has already been emitted and it's the user interaction itself.
This basically means, that we have to reduce the side effects within reducers. We just need to find the best way.

That's what i think also. The difference is that instead of returning a function, we return a data object; just like action creators returns data objects; After the reducer folding phase; will come an effect execution phase that will trigger the actual execution.

This makes the reducer more like a Mealy machine a kind of state machine that returns both a next state and an output. The output can then influence the the next input of the machine (the action/event trigered as a reaction to the effect response).

there is also the possiblity of splitting the reducer into 2 functions : transition which returns the next state and output which returns the next effect, so the reducer logic will be like an interplay between the 2 functions, but it seems more painful to implement async logic this way.

[tomkis]

there is also the possiblity of splitting the reducer into 2 functions : transition which returns the next state and output which returns the next effect, so the reducer logic will be like an interplay between the 2 functions, but it seems more painful to implement async logic this way.

I would be also worried about composition and the "interplay" reducer wouldn't be pure anymore, as long as the effects would get executed right away.

[slorber]

@tomkis1 I am happy to see we share the same vision on how to port backend concepts to the frontend. It took me a while to understand the fact there is no command, and that the UI can be considered as a BoundedContext with a single AggregateRoot.

In the backend the async nature comes from the fact that we should generally ship the command front a frontend to the backend (network). In the frontend the async nature is that when the user decices in his head to click on the button, he has to execute his intent by doing a non-immediate action (like moving the mouse and clicking).

• Command: user intent: in his head
• Command shipping: moving the mouse to execute the intent
• Command validation: no need, because if the command was not valid anymore, the UI would make disappear the button...
• Command handler: transform dom events into Flux Action / events
• Saga: listen to events and trigger side-effects (may maintain state if needed)
• Event handler: compute view state for rendering

In current architecture there's not much distinction between command handler and saga.

I did not know Redux thunk was able to call getState, so yes this somehow permits to emulate a saga (that may take stateful decisions)

I have not much problems with the fact that the effects are returned as a thunk. Actually the thunk is not executed immediatly so what you say is not totally correct. However I do agree that the function is not exactly pure as it returns a new function everytime and it is not easy to test.

Another solution would perhaps be to express the effect returned as a free-monad script or something similar, as it would not perform the side-effect until interpretation, and that script could be tested more easily.

[slorber]

We are mostly discussing http request side-effects here.
For DOM side-effects (like giving focus to an input), I don't think ELM as found a good way to do this yet.

See evancz/elm-architecture-tutorial#49

[yelouafi]

@tomkis1

I would be also worried about composition and the "interplay" reducer wouldn't be pure anymore, as long as the effects would get executed right away.

As @slorber said this is not the case if you return a thunk from the "output" function. a more declarative solution is to return a data object describing the side effect instead of the thunk and execute the effect by a specialized interpreter.

Actually the same approach is implemented in the Redux real world example; the actions creators trigger a plain object with a CALL_API key describing the API call and the actions to fire in response to this effect. Then this effect description is interpreted by the api middleware.

@slorber

In current architecture there's not much distinction between command handler and saga.

what about the Mealy machine solution of above ? wouldn't transition (derive next state) correspond to the event handler and output (trigger side effect) correspond to the saga, as output function also takes (state, action) and thus can make stateful decisions?

However I do agree that the function is not exactly pure as it returns a new function everytime and it is not easy to test.
Another solution would perhaps be to express the effect returned as a free-monad script or something similar, as it would not perform the side-effect until interpretation, and that script could be tested more easily.

Doesn't this join somewhat the solution i proposed of separating effect creation and interpretation ? In Redux for example we can create a store enhancer that performs in 3 phases

• invoke the reducers to derive the new state
• invoke the "output/saga" to collect all effects-as-objects; each object would include
  • the type of effect; this determines what interpreter should exexute the effect (e.g. API_CALL, LOCAL_STORAGE ...)
  • arguments expected by the interpreter; this could also include a "decoder" function to preprocess the returned value from side effect
  • the actions to trigger after the effect gets executed; the action payload would be the value returned from the effect (server response or unit/void ...)
• hand each effect-data to the specialized interpreter for actual execution. Each interpreter would then dispatch the action passed as argument(like in the api middleware of the Redux example)

This way the "output/saga" logic would stay purely declarative; the function will be pure and easily testable

We are mostly discussing http request side-effects here.
For DOM side-effects (like giving focus to an input), I don't think ELM as found a good way to do this yet.

I see; i think this is a real issue in the Elm port mechanism (unless there is a solution i don't know);

I think that DOM side effects should be handled by the underlying virtual dom library; in a sens we're already doing DOM side-effects : we create virtual trees which are data objects that describe the DOM side effect; then the virtual library interprets the side-effect description by patching the real DOM; then returns us the effect response which is the user command. So this sounds pretty much like the process i described above.

With Snabbdom i would typically create a snabbdom module that handles/interprets the focus effect then i will simply define a property like 'focus' or 'focusOnCreate' on the virtual node.

[yelouafi]

FYI, created a proof of concept (not really sure if the saga term is correct)

[yelouafi]

yet switched to generators. which gives more flexibility. The idea is similar to @tomkis1 redux-side-effects; but here the generators/saga yield plain data objects which then are dispatched normally via the middleware pipeline

// an "effect creator"
function callApi(endpoint, payload) {
  return { [API_CALL] : { endpoint, payload } }
}

function* getAllProducts(getState) {

  // yield a side effect description, executed later by the appropriate service
  const products = yield callApi(GET_PRODUCTS)

  // trigger an action with returned response
  yield receiveProducts(products)

}

function* checkout(getState) {...}

export default function* rootsaga(getState, action) {

  switch (action.type) {
    case GET_ALL_PRODUCTS:
      yield* getAllProducts(getState)
      break

    case CHECKOUT_REQUEST:
      yield* checkout(getState)
  }
}

[tomkis]

@yelouafi why did you decide for yielding just plain objects describing side effect instead of thunked side effect? That's what I was doing initially (http://blog.javascripting.com/2015/08/12/reduce-your-side-effects/ and https://github.com/salsita/flux-boilerplate) but after using that approach in large production application I realised that it's quite annoying. Therefore I prefer using thunk functions over Maps describing side-effects.

I know that you wanted to implement strict Saga pattern. But is that really necessary? I believe Saga was meant to be used and designed for C# / Java but JS is "quite" functional language and we could seize the fact.

But the deferred execution is still the goal.

[yelouafi]

@tomkis1 I did it exactly for the reasons you mentioned in the blogpost; data is easier to test than thunks (no mock is necessary). Another benefit is that the logger now logs also triggered side effects, so we have full traceability of the events in our app.

Since you already tried this approach in production; can you elaborate on its disadvantages; did you find that the complications outweigh the benefits ?

[tomkis]

There were two issues:

1. Polluting app state by effects (which you basically solved by using yield within reducer as generator)
2. A lot of boilerplate - you had to write effects and effect handlers, which was pretty annoying, besides a lot of people tended to mix up effects and actions together

I wouldn't necessarily say that testing thunks is more difficult:

const SideEffects = {
  loggingSideEffect: message => () => console.log(message)
};

function* reducer(appState = 0, action) {
  if (action === 'INC') {
    yield SideEffects.loggingSideEffect('incremented');

    return appState + 1;
  } else if (action === 'DEC') {
    if (appState > 0) {
      yield SideEffects.oggingSideEffect('incremented');

      return appState - 1;
    } else {
      return 0;
    }
  } else {
    return appState;
  }
}

it('should increment appstate and yield console.log', () => {
  spy(SideEffects, 'loggingSideEffect');

  const value = iterableToValue(reducer(1, 'INC'));
  assert.equal(value, 2);
  assert(SideEffects.loggingSideEffect.calledWith('incremented'));
});

it('should not decrement appstate and not yield console.log', () => {
  spy(SideEffects, 'loggingSideEffect');

  const value = iterableToValue(reducer(0, 'DEC'));
  assert.equal(value, 0);
  assert(SideEffects.loggingSideEffect.notCalled);
});

Like I said already, it's not important to test actual execution of side effect but the intent.

[yelouafi]

i understand, the thunk testing code can be automated, but there is no way to take out the boilerplate involved by the declarative approach : create effect types, effect creators, effect middlewares ... this could be annoying in larges scale.

Another approach which doesn't involve much boilerplate and more declarative than thunks could be yielding an array [fn, ...args]

yield [api.buyProducts, cart]

But thunks are more flexible, if we want to preprocess the result from the service. I think the middleware should provide both options and let the developer decide of what to yield.

[yelouafi]

But thunks are more flexible, if we want to preprocess the result from the service

Actually this is not quite exact, as we can process the result right inside the generators

const resp = process( yield [api.buyProducts, cart] )

[slorber]

I have used in the past commands / events / command handlers in my app, and I can tell this boilerplate is really really boring to write. There's always nearly a strict mapping and in most case you are just copy-pasting constants in 3 different files while you could just have used a thunk.

Still you need the spy to test but I prefer that than the boilerplate i've experienced.

[yelouafi]

In my opinion, and this is also what we do on the backend, we should not melt together the code that is used to compute the view state, from the code that is used to orchestrate complex operations (the Saga)

It's funny that I ended up implementing the exact solution i was contesting, but I think I finally agree with the above

[slorber]

@yelouafi just to be sure, what are we agreeing upon exactly?

What I think is that the ELM architecture propose an elegant model to bubble up local events to the top and redispatch them in the update function to the component that emitted them. This permits to replace (in spite of more boilerplate) the setState() of React components.

However I do not think that this replaces the need for some kind of global event bus that describes business events that fire on our app. TodoCreated is such an event. I may need later in my app a counter that only increments when the todo text contains 3 letters. This is a very special need and it can be easily created without any refactoring to the existing code. Like just listening to the TodoCreated events in a new reducer. In this case it's not really a local component state but is rather a new business view.

Also we can separate side-effects into 2 cases

• Business side-effects: expressing something like "when a todo is created, I want all the other todos to be automatically marked as done"
• Technical side-effects: expressing something like: "when we click on a todo to edit it, the input field must take focus"

I think the Saga is intended to solve the business side-effect, and not the technical side-effect.
For technical side-effects I think there's still a need to nest effects because you can have a list of todos and must be able to know which input should take focus. That effect is also tightly coupled to your view (at least in this case)

For business side-effects I think there's no such sense to nest effects as you will fire them from business events, and these events are rarely nested. I mean you will probably not "embed them" with Signal forwarding in ELM because this would make the events harder to listen.
And I think this business need does not have to be coupled to your view, that's why there's no need to return this effect in a reducer that computes the view.

That does not mean that the Saga reducer that will handle this business side-effect can't reuse some reducers that are already useful for the view. For example see http://stackoverflow.com/a/33829400/82609

[tomkis]

However I do not think that this replaces the need for some kind of global event bus that describes business events that fire on our app. TodoCreated is such an event. I may need later in my app a counter that only increments when the todo text contains 3 letters. This is a very special need and it can be easily created without any refactoring to the existing code. Like just listening to the TodoCreated events in a new reducer. In this case it's not really a local component state but is rather a new business view.

I am afraid re-dispatching actions (even though via effects) is not ideal as it leads to action cascade chain which is really difficult to reason about. It's forbidden even in original Flux and that's exactly because of the action chaining.

However, it's completely fine to dispatch another action within async callback. Like AJAX or anything from its nature being async but I wouldn't say that about business side effects because these should be always modelled synchronously.

IMO action should always be treated as event, and events simply can't yield another events.

Your particular example can be either solved by reducer composition or handling the action within two reducers, the important fact to realise though is that the state for each reducer should be either strictly separated (combineReducers) or you should use something like "master reducer" concept (http://www.code-experience.com/problems-with-flux/)

[yelouafi]

@yelouafi just to be sure, what are we agreeing upon exactly?
What I think is that the ELM architecture propose an elegant model to bubble up local events to the top and redispatch them in the update function to the component that emitted them. This permits to replace (in spite of more boilerplate) the setState() of React components.

Yes, but i think (now) that having separate functions (state, action) -> state and (state, action) -> effect is better than one function (state, action) -> (state, effect). (But apparently that wasn't what you meant by separating view-state updates from operation orchestration).

However I do not think that this replaces the need for some kind of global event bus that describes business events that fire on our app

I agree with what @tomkis1 said; maybe because of the synchronous nature on the front-end (i mean state updates), i can't think of a valid use case for the 'business side effect' concept, as all what you mentioned can be solved by plugging in another reducer; or using the master reducer for conditional logic.

Maybe i don't have yet my mind clear on the concept of 'business side-effect'; For now, is saving a data entity on the server and taking back an auto Id part of my business logic ? or it'is just the api call that is a side effect and the returned Id a normal input to my pure FP logic ?

For now, to me side effects are about inserting some explicit order (";") into the pure domain model. So things like 'render', 'call-api', 'store-locally', 'navigate ...' are side effects because we need those actions to take place at specific points of time (while in a pure FP model, we don't care about order because the automatic resolution of function dependencies). If all my domain logic is synchronous (like in reducers) then i don't need business side effects; because all my logic can be handled using pure functions.

[tomkis]

@slorber

For DOM side-effects (like giving focus to an input), I don't think ELM as found a good way to do this yet.

Is this still a valid point?

export const keyDown = ({keyCode, ev}) => function*(appState) {
  if (keyCode === UP_ARROW && appState.suggestionIndex > 0) {
    yield () => ev.preventDefault();

    return {...appState, suggestionIndex: appState.suggestionIndex - 1};
  } else if (keyCode === DOWN_ARROW && appState.suggestionIndex < appState.suggestionItems - 1) {
    yield () => ev.preventDefault();

    return {...appState, suggestionIndex: appState.suggestionIndex + 1};
  }
};

I have been writing something like this today and reducing DOM side effects within reducers makes even more sense to me. It used to be like this:

export const keyDown = ({keyCode, ev}) => function*(appState) {
  if (keyCode === UP_ARROW && appState.suggestionIndex > 0) {
    return {...appState, suggestionIndex: appState.suggestionIndex - 1};
  } else if (keyCode === DOWN_ARROW && appState.suggestionIndex < appState.suggestionItems - 1) {
    return {...appState, suggestionIndex: appState.suggestionIndex + 1};
  }
};

onKeyDown={ev => {
  if (keyCode === UP_ARROW || keyCode === DOWN_ARROW) {
    ev.preventDefault();
  }

  this.props.dispatch(keyDown(ev.keyCode))
}}

and as you can see the logic separation and duplication is quite obvious

[slorber]

@yelouafi

Yes, but i think (now) that having separate functions (state, action) -> state and (state, action) -> effect is better than one function (state, action) -> (state, effect). (But apparently that wasn't what you meant by separating view-state updates from operation orchestration).

Sometimes the effect to produce is dependent of the program's state. The Saga is not necessarily stateless. To be autonomous and decoupled then the Saga should be able to compute its own state so the signature (state, action) -> (state, effect) makes sense to me. However I would not recommend using the Saga state to render views because it's not the purpose of the Saga.

@tomkis1

I am afraid re-dispatching actions (even though via effects) is not ideal as it leads to action cascade chain which is really difficult to reason about. It's forbidden even in original Flux and that's exactly because of the action chaining.

I'm not saying to use this pattern in each and every case. It could also lead to infinite loops on the backend word and yet it is used.

IMO action should always be treated as event, and events simply can't yield another events.

In real world, if you transfer money from bank account 1 to bank account 2, the transfer is never immediate. The event fired is TransferRegistered, and then later a Saga orchestrate the that complex transaction into new events like AccountDebited and AccountCredited, also handling potential failures during that operation.

But the frontend user actions are synchronous... so we can have a debit/credit transaction right?

However, it's completely fine to dispatch another action within async callback. Like AJAX or anything from its nature being async but I wouldn't say that about business side effects because these should be always modelled synchronously.

Actually I agree conceptually with that. The fact that in Redux actionCreators can use getState means that somehow it already implements the Saga pattern as it can return thunks and be stateful. However the thing is that a Saga should rather be autonomous. What I don't like in an ActionCreator is that if you do "getState" then you have to understand the whole application state, and which part to use to take decision.

To implement a Saga, you should just look at the event list, pick the ones you want to listen to, compute the saga state, and react to some events. If the component is trully autonomous, you NEVER need to understand how is structured the rest of the view because the Saga actually should not care about the view.

So what I mean is that somehow Redux already ships with a Saga pattern, it's just I don't like the API.

---

To understand what I mean by a "business side-effect", look at this action creator. Support it's on a Todo app, and there is a user onboarding.

function createTodo(todo) {
   return (dispatch, getState) => {
       dispatch({type: "TodoCreated",payload: todo});
       if ( getState().isOnboarding ) {
         dispatch({type: "ShowOnboardingTodoCreateCongratulation"});
       }
   }
}

Somehow, you are expressing: "if the user is onboarding, and the user creates a todo, then congratulate the user".

This is already a Saga :)

Now how would you do if instead of congratulating the user, you just want to say that this onboarding step is validated, and you just want to move to the next one (without necessarily caring about which one is the next one?)

I'll let you give me a solution to that problem. Just I think in that solution, the createTodo action creator should not know anything about what could be the next step, otherwise you will end up with complicated and many state dependencies in your actionCreators.

So, what I just want to say in the end, is that there is an advantage of keeping actionCreators as simple as possible, and plugin the behavior of the onboarding in a separate and autonomous component.

function createTodo(todo) {
   return (dispatch, getState) => {
       dispatch({type: "TodoCreated",payload: todo});
   }
}

// I'm fine with (state,action) -> (state,effects) but choose a simpler implementation for the example
function onboardingSaga(state, action, dispatch) {
  switch (action) {
    case "OnboardingStarted": 
        return {onboarding: true, ...state};
    case "OnboardingStarted": 
        return {onboarding: false, ...state};
    case "TodoCreated": 
        if ( state.onboarding ) dispatch({type: "ShowOnboardingTodoCreateCongratulation"});
        return state;
    default: 
        return state;
  }
}

I'm not saying it is the only possible solution, but it permits to couple less the todo creation from the app onboarding, and it is worth using on complex applications.

[slorber]

@tomkis1

For now, to me side effects are about inserting some explicit order (";") into the pure domain model. So things like 'render', 'call-api', 'store-locally', 'navigate ...' are side effects because we need those actions to take place at specific points of time (while in a pure FP model, we don't care about order because the automatic resolution of function dependencies). If all my domain logic is synchronous (like in reducers) then i don't need business side effects; because all my logic can be handled using pure functions.

Taking the example above, I could also use reducers to know if I shoud display the "todo creation onboarding congratulation":

var defaultState = { isCongratulationDisplayed: false }

function onboardingTodoCreateCongratulationReducer(state = defaultState, action) {
  var isOnboarding = isOnboardingReducer(state.isOnboarding,action);
  switch (action) {
    case "TodoCreated": 
        return {isOnboarding: isOnboarding, isCongratulationDisplayed: isOnboarding}
    default: 
        return {isOnboarding: isOnboarding, isCongratulationDisplayed: false}
  }
}

This time, we are only using reducer composition, and we derive from the event log weither or not we should display the congratulation. There is even no need to dispatch any ShowOnboardingTodoCreateCongratulation action.

However, having tried this composition approach already, I can tell you how messy it becomes in the long run. It is much simpler to reason about when the congratulation only shows up after the ShowOnboardingTodoCreateCongratulation action.

If you have a more elegant solution to solve this exact same problem I would be happy to hear it.

[ccorcos]

I agree that everything should be pure. What if we thought of data fetching declaratively, the same as we do rendering to the DOM? That is, what if the Elm "view" function returned not just a DOM tree, but some kind of HTTP fetch data structure as well?

init    : () -> state
update  : (state, action) -> state
view    : (dispatch, state) -> {html, http}

We can render with the DOM tree, and pass the http requests to some other service that handles the mutation, keeps track of which requests are in flight, and calls the event listeners bound to the data structure. This is essentially the same thing React does, but for HTTP requests.

Here's a working example. I'd love to hear your thoughts.

[carloslfu]

Yes, it's good idea. I'm doing some things in this direction, but, why did you named it view?. In this approach you can name it fetch. Think in an FRP approach in that you take actions and send reactions. This looks like:

init      :  () -> state
update    :  (state, action) -> state
reaction  :  (dispatch, state) -> reactionObject
# common reactions
view      :  (dispatch, state) -> vnode
fetch     :  (dispatch, state) -> fetchObj
socket    :  (dispatch, state) -> msgObj

For each reaction that you support, you need a driver, that driver isolates side effects(see the idea of cyclejs drivers) and should optimize your requests/socketMsgs in the same way that a virtual DOM library does, those drivers are attached only to the main component, and this component handles each subcomponent reactions. With that you can do all your app logic declaratively and all preserves pure.

[ccorcos]

Exactly :)

Sent from my iPhone

On Dec 9, 2015, at 04:54, Carlos Galarza notifications@github.com wrote:

Yes, it's good idea. I'm doing some things in this direction, but,why did you named it view?. In this approach you can name it fetch. Think in an FRP approach in that you take actions and send reactions. This looks like:

init : () -> state
update : (state, action) -> state
reaction : (dispatch, state) -> reactionObject

common reactions

view : (dispatch, state) -> vnode
fetch : (dispatch, state) -> fetchObj
socket : (dispatch, state) -> msgObj
For each reaction that you support, you need a driver, that driver isolates side effects(see the idea of cyclejs drivers) and should optimize your requests/socketMsgs in the same way that a virtual DOM library does. With that you can do all your app logic declaratively and all preserves pure.

—
Reply to this email directly or view it on GitHub.

[yelouafi]

@slorber

If you have a more elegant solution to solve this exact same problem I would be happy to hear it

I'm experimenting with a new approach that supports long running Sagas; the Saga state is managed right inside the generator. the code is in the 'span' branch; here is an example

https://github.com/yelouafi/redux-saga/blob/master/examples/counter/src/sagas/index.js

[slorber]

@yelouafi unfortunatly I've never used generators yet so I'll have to learn a bit more about them :)

[yelouafi]

@slorber in a nutshell they are functions that can be paused and resumed. (a good article here http://www.2ality.com/2015/03/es6-generators.html) . In my example above I pause the sagas on yield statements (waiting for future actions or Services's responses), when the yielded values are resolved I resume the saga back with the resolved values.

Generators offer powerful (yet still underused) capabilities of handling complex async operations

[tomkis]

@slorber
I took some time to deeply think through the strict BE like Saga approach on FE and unfortunately I must say that I don't like the idea.

I absolutely understand that orchestrating complex async long running transactions makes sense on BE on the other hand I believe that it's not that beneficial on FE in fact some of the concepts (event chaining) can harm the overall architecture.

In real world, if you transfer money from bank account 1 to bank account 2, the transfer is never immediate. The event fired is TransferRegistered, and then later a Saga orchestrate the that complex transaction into new events like AccountDebited and AccountCredited, also handling potential failures during that operation.

Absolutely agreed that this is an excellent use case for Saga, yet it's really rare and artificial FE example. Working with UI is a sequence of Events: FOO_CLICKED, BAR_CLICKED, WEB_SOCKET_MESSAGE_RECEIVED, MOUSE_MOVED, BAZ_CLICKED.... and the current state (what is actually rendered on screen) is left fold of these behaviours and that's exactly what Greg Young said about Event Sourcing.

My first Flux project didn't treat UI as sequence of Events... I was using actions like ShowOnboardingTodoCreateCongratulation instead of ADD_TODO_CLICKED and doing the logic in one place (store) and as the application grown in its complexity it was very difficult to reason about and most importantly extend it without huge iterative refactoring.

Therefore speaking about strictly FE architecture:

However, having tried this composition approach already, I can tell you how messy it becomes in the long run. It is much simpler to reason about when the congratulation only shows up after the ShowOnboardingTodoCreateCongratulation action.

This is exactly an opposite to what I have experienced while writing complex FE apps.

Like I said, it's really difficult for me to give up from my perspective the biggest benefit that unidirectional dataflow concept bring us for FE apps. Saga is excellent way for solving long running transactions, which are very rare on FE if you think of your UI as sequence of Events.

PS. In case you are interested I wrote a post advocating that.

[slorber]

However, having tried this composition approach already, I can tell you how messy it becomes in the long run. It is much simpler to reason about when the congratulation only shows up after the ShowOnboardingTodoCreateCongratulation action.

This is exactly an opposite to what I have experienced while writing complex FE apps.

@tomkis1 so you mean that for you it is simpler if the congratulation shows up without even an event that says it should be displayed?

---

Even if we are not on the backend and there are not really "long running transactions", I think the pattern helps decouple some parts of your application, and it is useful for complex apps.

See for example: https://msdn.microsoft.com/en-us/library/jj591569.aspx
I think the following example can still be simplified by the process manager, even if the actions would be synchronous.

Using Sagas does not mean you do not use an event log anymore. Yes if you make an implementation mistake it can create infinite ping-pong loops of events but this should not happen much.

Also there's something that is important to note. I'm not sure to have clear ideas on top of that but in all frontend applications, there is some kind of "translation of events". I mean, in your Flux/Redux event log, you don't append raw events like div clicked but rather something more high level like user profile opened. Both are valid events to add, but one will be much more easy do deal with.

Now consider an infinite scroll scenario. The user scroll to load next pages.

You can fire some actions/events like:

• DivScrolled: low level, we should not always load a page on scroll)
• ContentScrolledNearBottom: "medium level": as we are close to the bottom of the page we know we should load the next page
• LoadNextPage (actionCreator) -> NextPageLoaded (event): very high level, it is the real intent of the user)

Which one would you choose? All 3 are valid things for me.

• DivScrolled -> this is very low level and not very convenient, particularly if you fire this for every div that is not easy to trigger the next page fetches (and where would you do so if not in a Saga?)
• ContentScrolledNearBottom -> more convenient to use. You fire a more high level event that describe something that matters for your business (because you don't care so much of scroll events after all unless the user is reaching the end of the page...)
• LoadNextPage -> if you fire this from the jsx with the onscroll listener, then you have a very high level event that ships the primary intent of the user. But what if finally you want later to introduce something else than a page load when the user scrolls near the bottom? You can't because you only fired a very high-level event.

So another benefit of the Saga approach is that you can easily add a layer of translation to your app. Is the JSX the appropriate place to translate the low-level dom event to an user intention? Or should the UI just tell us what has happened, and the interpretation be done somewhere else?

It is my own opinion but I like the UI simply describing what happens, and then interpret what happens as an user intention into another piece of software called Saga.

If a div is scrolled near bottom, then you just fire "divScrolledNearBottom" and let another component being able to understand that and trigger a next page loading if needed.

[tomkis]

@slorber Thanks, your feedback is much appreciated and it seems to me that you are really helping the community!

@tomkis1 so you mean that for you it is simpler if the congratulation shows up without even an event that says it should be displayed?

Yes, from my experience.

Also there's something that is important to note. I'm not sure to have clear ideas on top of that but in all frontend applications, there is some kind of "translation of events". I mean, in your Flux/Redux event log, you don't append raw events like div clicked but rather something more high level like user profile opened. Both are valid events to add, but one will be much more easy do deal with.

This is a perfectly valid point. However, as I explained above. From my experience the lower level the event is the more extendable the code is. I have to agree though that DIV_CLICKED is a way too much.

Which one would you choose? All 3 are valid things for me.
DivScrolled -> this is very low level and not very convenient, particularly if you fire this for every div that is not easy to trigger the next page fetches (and where would you do so if not in a Saga?)
ContentScrolledNearBottom -> more convenient to use. You fire a more high level event that describe something that matters for your business (because you don't care so much of scroll events after all unless the user is reaching the end of the page...)
LoadNextPage -> if you fire this from the jsx with the onscroll listener, then you have a very high level event that ships the primary intent of the user. But what if finally you want later to introduce something else than a page load when the user scrolls near the bottom? You can't because you only fired a very high-level event.

The event translation should in my opinion happen in the View layer. There has been countless discussion about whether all our state should be within single atom and all components must be stateless, I don't necessarily follow the rule. I am trying to keep my view specific logic & state in react components and it works, pretty well I would say. Though rule of thumb: no business rules in views.

Therefore ContentScrolledNearBottom seems a perfect choice here.

It is my own opinion but I like the UI simply describing what happens, and then interpret what happens as an user intention into another piece of software called Saga.

I wish it was so simple, It was my impression too, but then I quickly realised that stateless views are kind of utopia. There is still so much DOM interaction and view specific logic in the real world that keeping everything in global app state is not a realistic option. (cc @sebmarkbage)

I believe that Saga is wonderful concept which will definitely find its use on FE (yes there are still some long running transactions) yet I don't think that it's the ultimate answer for business side effects because it suffers with the same pain points like redux-thunk:

Unit testing is not possible because domain logic processing is not single unit which at the end of the day results in more tangled code.

[slorber]

@tomkis1 thanks I'm trying to do my best haha but I'm not even an expert in backend Sagas btw...

ContentScrolledNearBottom would also be my choice, but then something should still trigger the fetching of the next page. In this case the saga pattern seems nice to do it.

I don't think the saga should necessarily be used everytime an effect has to be done. What I like about this pattern applied to the frontend is the ability to make the implicit explicit. I mean it is easier to understand that a page is loading because there is a "PageLoaded" event than because there is a "ScrolledNearBottom" event.