Questions about side effect and Elixir interop

[codec-abc]

Hi,

Being a total newbie in Elm and Elixir, and I have a few questions about side effect in elchemy:

Quoting the README:

Interop with Elixir - 90% - All of the interop is mature and type-safe-verified based on specs
Elchemy-effects - 15% - You can't and shouldn't write anything with side-effects in Elchemy yet. We're working on finding the best solution for effects that would fit both Elm's and Elixir's community
Elchemy-core for Erlang VM - 5% (Everything for os related tasks like filesystem, OTP goodies etc has to be done)

I'm not sure how to interpret this. Do you mean Interop (which is 90% ready) should only be used for pure computation (only crushing numbers, this kind of things)? Also, side effects and core (filesystem, OTP, etc..) are 5% which seems to indicate you have already some thoughts on it. I'm probably too curious but I would like to know your early thought on this topic if possible. Especially, since side effect and such like tasks related to filesystem and stuff can often fail. Moreover since Elm has no exceptions, it appears at first glance that there is no easy way to reconcile those 2 worlds.

Thanks

[wende]

@codec-abc

That's precisely correct. Also the side effects is the hardest part of Elchemy to get right, hence I'm against any rushed solutions.

So to sum it up:

• Classic interop (i.e ffi) is by design only meant to be used for pure computations. Right now it's only an agreement and there is no safety measures to ensure that, but it's a subject to change in future releases (Implement purity tests for FFI based on property testing #162).
• Side effects will be handled by an another construct used especially for that. It won't ever yield regular values but rather IO operations wrapped in a Task (most likely) type.

Why ffi only for pure computations?

There is many gains to separating IO from your data model. These are the most obvious gains:

• Free concurrency (concurrent computations might be as easy as a single function decorator)
• Free memoization (same as with concurrency you can memoize results without worrying about possible side effects)
• Much easier debugging. If you know your model works properly, then you can reduce your scope to only the functions returning Tasks (or Cmds) which makes finding the cause of error much easier.

But concurrency and memoization are impure concepts!

Yes. But Ffi will only require a perceived purity. What it means is that it will be possible to call impure functions using ffi as long as their result is pure. Elm actually already does it with one function, which is Debug.log which is an impure function with a perceived purity (you can't make Debug.log return something else than what it received)

So why is it so difficult to implement side-effects in Elchemy?

Mostly because we want to be as compatible with Elm's existing libraries as possible. And although surprisingly little libraries use Cmd or Task in their code (Except for libraries under elm-lang but obviously those will have to be rewritten due to use of Native/Elm.Kernel and JS code anyway) we still would like to keep a coherent interface. (Imagine if you used Elchemy on backend and Elm on front-end and using HTTP client libraries would look drastically different. Not really a desirable experience)

So isn't that just all about copying what Elm (TEA) has?

Unfortunately no. There are 3 characteristics of TEA that would make it a terrible choice for backend programming.

1. In Elm, your application's 'process' is a singleton.

   • Why does it work for Elm?
     On the client side your application is always single-core. There is virtually no benefits of separating the logic into many actors. It makes message routing and type safety of messages incomparably easier to manage
   • Why wouldn't it work for Elchemy?
     Erlang VM was designed to make concurrency work seamlessly as long as your logic is expressed with actors. Limiting ourselves to mimicking TEA would basically mean we're sacrificing the entire benefit of running on BEAM. In other words we might have as well just used Elm on Node.js all along 😉

2. Implementation of effect modules requires absurdly deep level of understanding of entire TEA.

   • Why it works for Elm?
     I'll use a very good quote for that:
     An extremely tiny portion of library authors should ever write effect managers. Fundamentally, Elm needs maybe 10 of them total.
     Amount of side effects on front-end is drastically lower than on back-end

   • Why wouldn't it work for Elchemy?
     Although some might argue that the amount of side-effects on the back-end aren't that much different, there is definitely a limitation to that approach. Let's say we implement all effect managers for most popular protocols (HTTP, TCP, UDP, Websocket, File system). Now would you like to reimplement all of the database protocols from scratch? Yeah, I don't think so.
     So we need to assume we want to make implementation of side effects easier so that every developer (let's say with at most intermediate understanding of Elchemy) is able to implement their own interface to any database / protocol / effectful library.

3. Elm's messages aren't type safe

   • Why does it work for Elm?
     As ridiculous as it might sound in Elm all of the messages aren't typesafe. It works because of both point 1 and 2. These two points give the effect module developer a guarantee that
     a) There is only one message type
     b) Unless you're a core member you probably won't be touch unsafe messages anyway.

   • Why wouldn't it work for Elchemy?
     Also mostly because points 1 and 2. The moment we allow a regular developer to touch side effects is the moment we need to make sure it is in any way type checked.

So what can we do?

This a question we were trying to answer for last 6 months and there was a lot of smart people that spent their time to help me come up with the best solution.
There is a roadmap markdown material I'm working on to sum up the efforts and conclusions drawn.
When it's ready I'll link it up here. Until then I hope this rather long answer will be enough to be used as a guideline of whats and whys.

[codec-abc]

Wow, Thanks! I didn't expect such a detailed answer. I think it's worth to be link in the Readme.

[wende]

Addressed in README 👍

[ShalokShalom]

This is so clear and on point, congratulations ^-^