fpdart v2.0.0 #147

SandroMaglione · 2024-03-18T06:57:13Z

This PR is an open discussion on what's the best strategy to release fpdart v2 to minimize friction for users of the library

👇 Please comment below with any idea, comment, critique or opinion 👇

Problems with v1

Too many classes (IO, IOOption, IOEither, Task, TaskOption, TaskEither, Reader, State, ReaderTaskEither):

Similar implementation with different generic parameters = A lot of duplicated code (both core and tests)

/// [IO] 👇
abstract final class _IOHKT {}

final class IO<A> extends HKT<_IOHKT, A>
    with Functor<_IOHKT, A>, Applicative<_IOHKT, A>, Monad<_IOHKT, A> {
  final A Function() _run;
}

/// [Task] 👇
abstract final class _TaskHKT {}

final class Task<A> extends HKT<_TaskHKT, A>
    with Functor<_TaskHKT, A>, Applicative<_TaskHKT, A>, Monad<_TaskHKT, A> {
  final Future<A> Function() _run; /// 👈 Difference: [Future] here
}

Code duplication = Hard to maintain, more lines of code, more code to read (and understand) for contributors
Requires conversion between classes (from*, to*, e.g. toTask, toTaskEither)
Requires having a different Do constructor for each class, making the do notation harder to use
Hard to understand for newcomers, hard to reason with and explain for veterans (and verbose)
More complex code, less contributors

Too much jargon: methods and classes are using terms from pure functional programming (math), less clear and hard to understand (e.g. pure, Reader, chainFirst, traverse, Endo).

Typeclasses do not work well with dart and they cause a lot of overhead to maintain and understand. In fact, they are not necessary to implement the core of fpdart (they can be removed 💁🏼‍♂️).

Too many "utility functions" that may be considered outside of the scope of fpdart (e.g. predicate_extension.dart).

fpdart v2 solution: `Effect`

A single Effect class that contains the API of all other classes in v1 (similar to ReaderTaskEither).

All Effect-classes derive from the same interface IEffect:

abstract interface class IEffect<E, L, R> {
  const IEffect();
  Effect<E, L, R> get asEffect;
}

Benefits

A lot less code: easier to maintain, contribute, test, understand (a single effect.dart)
No need of conversion methods (a lot less code)
A single Do notation (implemented as a factory constructor factory Effect.gen): the do notation also includes Option and Either (since both extend IEffect)
No more jargon: easy to understand method names instead of fp jargon (e.g. succeed instead of pure)
Removed all typeclasses and unnecessary utility methods
Easier to explain and understand (focus on learning a single Effect and how it works)
Smaller API that allows all the same functionalities as before
More resources to focus on better documentation, tests, and examples

Important: Effect comes from the effect library (typescript), which itself was inspired from ZIO.

The Effect class and methods in fpdart are based on effect from typescript (similar API and methods names).

Huge thanks also to @tim-smart for his initial zio.dart implementation.

Downsides

⚠️ Huge breaking change ⚠️
Nearly all tests need to be rewritten
Documentation and examples to redo completely

outdated for now with new `Effect`

SandroMaglione · 2024-04-03T21:33:31Z

New release: fpdart v2.0.0-dev.3:

Added Scope
const constructor for None

packages/fpdart/lib/src/effect.dart

tim-smart · 2024-04-03T22:07:14Z

packages/fpdart/lib/src/effect.dart

+        return deferred.wait<E>().__unsafeRun(context).then(
+              (exit) => signal
+                  .failCause<E, L>(const Interrupted())
+                  .__unsafeRun(context.withoutSignal)


Looks like there is a bug, when the parent signal is interrupted it isn't propagated to the child signal.

packages/fpdart/lib/src/effect.dart

tim-smart · 2024-04-03T22:11:38Z

packages/fpdart/lib/src/effect.dart

+  Effect<Null, L, R> provide(Context<E> context) {
+    final env = context.env;
+    final effect = env is ScopeMixin && !env.scopeClosable
+        ? alwaysIgnore(env.closeScope())


Hmm this could be an issue, as if there was a defect when try to close the scope, it will be swallowed / unhandled.

I added some code to handle defects when closing the scope, anything you think is missing?

fpdart/packages/fpdart/lib/src/effect.dart

Lines 289 to 305 in f691c1e

Effect<E, L, R> _provideEnvCloseScope(E env) =>

env is ScopeMixin && !env.scopeClosable

? Effect<E, L, R>.from(

(context) => _unsafeRun(context).then(

(exit) => switch (exit) {

Left(value: final value) => Left(value),

Right(value: final value) =>

env.closeScope<E, L>()._unsafeRun(context).then(

(exit) => switch (exit) {

Left(value: final value) => Left(value),

Right() => Right(value),

},

),

},

),

)

: this;

You need to close the scope regardless of success or failure.

packages/fpdart/lib/src/effect.dart

tim-smart · 2024-04-03T22:15:52Z

packages/fpdart/lib/src/effect.dart

+  Effect<Scope<E>, L, R> acquireRelease(
+    Effect<Null, Never, Unit> Function(R r) release,
+  ) =>
+      withScope.tapEnv(


Technically you should make the resource acquisition uninterruptible here, would require some thought about how to make interruption toggle-able / regional.

ryanhanks-bestow · 2024-04-03T23:34:11Z

Hey Sandro.

Looking good, I've been playing around.

Given

typedef ProduceEffect<State, ProductionError, Event>
    = Effect<State, ProductionError, Iterable<Event>>;
typedef HandleEffect<State, HandlerError, Event>
    = Effect<(State, Event), HandlerError, State>;
typedef ApplyEffect<State,Error,Event> = Effect<State, Error, (Iterable<Event>, State)>

, how would you do a fold across the iterable result of a ProduceEffect using a HandleEffect to result in an Effect of type ApplyEffect?

Feel free to suggest changes in the shapes of those definitions if you have a better starting approach.

Thanks

ryanhanks-bestow · 2024-04-03T23:47:25Z

Hey Sandro.

Looking good, I've been playing around.

Given
typedef ProduceEffect<State, ProductionError, Event>
    = Effect<State, ProductionError, Iterable<Event>>;
typedef HandleEffect<State, HandlerError, Event>
    = Effect<(State, Event), HandlerError, State>;
typedef ApplyEffect<State,Error,Event> = Effect<State, Error, (Iterable<Event>, State)>
, how would you do a fold across the iterable result of a ProduceEffect using a HandleEffect to result in an Effect of type ApplyEffect?

Feel free to suggest changes in the shapes of those definitions if you have a better starting approach.

Thanks

Also, curious if you have any general suggestions in naming type alias of an effect. Those names aren't the names I'm using verbatim, but I have been applying a naming convention where I prefix an alias with a verb matching the general action within the Effect that maps an Env value to the result and/or error it could produce. So for example, in the actual code I'm working on, I've implemented typedefs as such:

typedef DispatchEffect<State, DispatchError, Event>
    = Effect<State, DispatchError, Iterable<Event>>;

typedef StateEventHandlerEffect<State, HandlerError, Event>
    = Effect<(State,Event), HandlerError, State>;

Again, not 100% sure I'm shaping things here, maybe with this you'll be able to let me know if I'm applying the general use correctly.

Thanks

ryanhanks-bestow · 2024-04-04T06:20:57Z

@SandroMaglione what editor do you use? In intelliJ, when I'm creating an Effect using Effect.gen, I lose type-awareness inside the call to sync (even if I explicitly supply the type parameter A).

Wondering if you might consider this alternative definition for EffectGen:

class EffectGen<E, L> {
  EffectGen({
    required FutureOr<A> Function<A>(IEffect<E, L, A>) async,
    required A Function<A>(IEffect<E, L, A>) sync,
  })
      : this.sync = sync,
        this.async = async;

  FutureOr<A> async<A>(IEffect<E, L, A>effect) => this.async(effect);

  A sync<A>(IEffect<E, L, A>effect) => this.sync(effect);

}

SandroMaglione · 2024-04-04T06:44:33Z

packages/fpdart/lib/src/effect.dart

+  static Effect<E, L, void> sleep<E, L>(Duration duration) => Effect.from(
+        (_) => Future.delayed(
+          duration,
+          () => const Right(null),
+        ),
+      );


It is possible to use Timer+Completer.

@tim-smart How would interruption work for sleep? How can sleep detect an interrupt and block timer?

Suggested change

static Effect<E, L, void> sleep<E, L>(Duration duration) => Effect.from(

(_) => Future.delayed(

duration,

() => const Right(null),

),

);

static Effect<E, L, void> sleep<E, L>(Duration duration) => Effect.from(

(_) {

final completer = Completer<Exit<L, void>>();

Timer(duration, () {

completer.complete(const Right(null));

});

return completer.future;

},

);

First you will want to add a Effect.async/asyncInterrupt for callback apis, then use Effect.asyncInterrupt with the Timer api. Then hook up timer.cancel.

@tim-smart I added async, but I am having some issues understanding the mental model for Deferred and interruption.

Could you give me an example of how to manually interrupt an Effect with code? Does this required Deferred or is it about Context? Are fibers needed?

Sleep should be:

static Effect<E, L, void> sleep<E, L>(Duration duration) => Effect.asyncInterrupt( (resume) { final timer = Timer(duration, () { resume.succeed(null); }); return Effect.succeedLazy(() { timer.cancel(); }); }, );

As for interrupting an Effect, here is some pseudo code:

final cancel = Effect.never.runCallback(); // some time later... cancel();

https://effect-ts.github.io/effect/effect/Effect.ts.html#runcallback

In dart maybe something like:

void Function() runCallback({ void Function(Exit<E, A> exit)? onExit })

Is runCallback the only way to manually trigger an interruption?

How would you test interruption with sleep (or also asyncInterrupt)?

In Effect (typescript) you can pass an abort signal to Effect.runPromise, use the cancel callback from Effect.runCallback or call interrupt on fiber returned from Effect.runFork.

dart doesn't have a built in "cancellation" primitive, so you could use a Future potentially.

Do you think is it possible to use Deferred (or Context) in fpdart to give access to interruption?

Not too sure what you mean, might just have to try it out!

SandroMaglione · 2024-04-04T08:14:48Z

@ryanhanks-bestow could you provide a screenshot of IntelliJ that shows the type not inferred? Do you know of any limitation of IntelliJ regarding Records?

lishaduck · 2024-07-24T01:59:47Z

Hello! Is there anything I could do to help make some progress on this? I'd really like to switch some code to FPDart, but I don't want to if it'll soon need to be rewritten. In addition, I really like Effect-TS and would love to see a Dart equivalent.
I'm certainly happy to help with tests and docs, and would be willing to work through porting implementations.

SandroMaglione · 2024-07-24T14:28:04Z

Hello! Is there anything I could do to help make some progress on this? I'd really like to switch some code to FPDart, but I don't want to if it'll soon need to be rewritten. In addition, I really like Effect-TS and would love to see a Dart equivalent. I'm certainly happy to help with tests and docs, and would be willing to work through porting implementations.

The initial implementation is there. The main issue now is with composition. The dart language doesn’t support union types and type inference is limited. This means that you are required to specific manually all types, which is not ideal from a developer UX perspective. So much so that it doesn’t yet justify using fpdart's Effect over usual OOP dart patterns.

This may be solved by static metaprogramming, so I am actually waiting for some progress on the language.

lishaduck · 2024-07-25T01:06:33Z

The initial implementation is there. The main issue now is with composition. … This may be solved by static metaprogramming, so I am actually waiting for some progress on the language.

Ok then. Thank you for the explanation, it's very helpful for understanding the current progress.
Would you expect fpdart 2 to be stable enough to use then, just with the expectation that it'll get more concise in the future, or do you anticipate macros to entirely subsume the current API?

SandroMaglione · 2024-07-31T17:42:32Z

The initial implementation is there. The main issue now is with composition. … This may be solved by static metaprogramming, so I am actually waiting for some progress on the language.

Ok then. Thank you for the explanation, it's very helpful for understanding the current progress. Would you expect fpdart 2 to be stable enough to use then, just with the expectation that it'll get more concise in the future, or do you anticipate macros to entirely subsume the current API?

I don't expect the Effect encoding to change. Nonetheless, static metaprogramming has the potential to disrupt many common dart patterns. I would advise against using fpdart v2 in production yet.

That being said, the API as it is now works and hopefully won't need to change too much.

lishaduck · 2024-07-31T18:23:05Z

Static metaprogramming has the potential to disrupt many common dart patterns. I would advise against using fpdart v2 in production yet.

It's primarily for a personal project, so there's not really a "production".

I don't expect the Effect encoding to change ... That being said, the API as it is now works and hopefully won't need to change too much.

Ok, thanks! I guess I'll give it a go and report back with any feedback. Thanks again for fpdart!

lishaduck · 2024-08-10T21:30:53Z

Finally got some time to work on migrating some code to fpdart.

Feedback:

I miss the default tryPromise overload.
~~I haven't used Cause much, but I'm wanting it now.~~ I found cause right after I wrote my own impl 😀. ~~Could it get its own file? (moved comment)~~
Is a schema package planned? Dart doesn't really have a "Zod".
Dart's lack of row polymorphism for records is annoying. I thought it'd be a good way to type context, but it intentionally doesn't work.

P.S.: I'll update this comment if I have anything else to add instead of notifying everyone.

lishaduck

I left some comments, feel free to disregard.

lishaduck · 2024-08-11T01:41:01Z

packages/fpdart/lib/src/exit.dart

+
+typedef Exit<L, R> = Either<Cause<L>, R>;
+
+sealed class Cause<L> implements Error {


Is there a reason to use Error over Exception? Errors shouldn't be caught, but Failures should.

lishaduck · 2024-08-11T01:47:13Z

packages/fpdart/lib/src/async_context.dart

@@ -0,0 +1,12 @@
+part of "effect.dart";


Is there a reason to use parts? Circular dependencies?
Just curious.

lishaduck · 2024-08-11T01:48:14Z

packages/fpdart/lib/src/order.dart

@@ -0,0 +1,61 @@
+import 'ordering.dart';
+
+class Order<T> {


Yay! I love Order from Effect, looks similar.

packages/fpdart/lib/src/either.dart

-        Foldable2<_EitherHKT, L, R>,
-        Alt2<_EitherHKT, L, R>,
-        Extend2<_EitherHKT, L, R> {
+sealed class Either<L, R> extends IEffect<Never, L, R> {


lishaduck · 2024-08-11T02:20:50Z

packages/fpdart/lib/src/option.dart

-final class _OptionThrow {
-  const _OptionThrow();
-}
+part of "effect.dart";


Would a micro module be possible, like Effect's, but even lighter?
In my head, you could keep IEffect as base, but keep Option, Either, etc from directly interacting with Effect, only IEffect. That would make migrating from v1 and integrating it into existing projects easier, by not requiring users to use the Effect system (it's obviously recommended to use the Effect portion), and comes with the side-effect of a (slightly) smaller learning curve, as you can use it in chunks.

I thought there were some APIs here that called Effect. Doesn't look like anything really, so I think it should be documented that you can still use Option, Either, etc, without needing to adopt the effect system.

lishaduck · 2024-08-11T02:24:34Z

packages/fpdart/lib/src/exit.dart

+
+typedef Exit<L, R> = Either<Cause<L>, R>;
+
+sealed class Cause<L> implements Error {


Oh, also, could this get its own module? I get that it's an internal detail, but Exit feels different than Cause. Maybe rename it rather than splitting it given the size of Exit?

lishaduck · 2024-08-11T02:39:01Z

packages/fpdart/lib/src/effect.dart

+      );
+
+  /// {@category folding}
+  Effect<E, Never, C> match<C>({


This has a semi-non-trivial definition, so I'm hesitant to suggest this, but maybe it'd be possible to use native pattern matching? i.e.

final val = switch ($.syncEither(effect)) { Left() => ..., Right() => ..., }

rather than

final val = effect.match( () => ..., () => ..., )

Yes, it's more verbose, but the native pattern matching feels much more "right" to me. The function feels like it's working around an issue from JS that isn't in Dart.

Been writing some more code with fpdart, and I definitely think this could be simpler. There're plenty of functions that could just be pattern-matching, which reduces complexity/learning curve, and is more flexible.

airychkov · 2024-11-05T13:59:17Z

Hello! My Second try to use Effect.

test code

abstract interface class FileLoader {
Effect<ConvertFromJson, DataError, IList> loadSource(final String source);
}

final class JsonFileLoader implements FileLoader {
@OverRide
loadSource(final String source) => Effect.gen(
(final $) async {
final fileData = await $.async(Effect.tryCatch(
execute: () => rootBundle.loadString(source, cache: true),
onError: (final error, final stackTrace) => FileError(error: error, stackTrace: stackTrace)));

      return const IList.empty();
    },
  );

}

call Effect
final c = await b.loadSource('asset/file.yaml').provideEnv(File.fromJson).runFutureExit();

get exception

Exception has occurred.
FlutterError (Unable to load asset: "asset/file.yaml".
The asset does not exist or has empty data.

I expect Either.Left If I right undertund examples.
Most of examples is outdated now

lishaduck · 2024-11-05T17:55:42Z

Hello! My Second try to use Effect.

test code

abstract interface class FileLoader { Effect<ConvertFromJson, DataError, IList> loadSource(final String source); }

final class JsonFileLoader implements FileLoader { @OverRide loadSource(final String source) => Effect.gen( (final $) async { final fileData = await $.async(Effect.tryCatch( execute: () => rootBundle.loadString(source, cache: true), onError: (final error, final stackTrace) => FileError(error: error, stackTrace: stackTrace)));

      return const IList.empty();
    },
  );

}

call Effect final c = await b.loadSource('asset/file.yaml').provideEnv(File.fromJson).runFutureExit();

get exception

Exception has occurred. FlutterError (Unable to load asset: "asset/file.yaml". The asset does not exist or has empty data.

I expect Either.Left If I right undertund examples. Most of examples is outdated now

I'm unsure as well, the code should catch it, but if you look at the v2 examples, you'll see that they call catchError. I think that'd work work around it.
Given that it's a FlutterError, perhaps it's got something to do with the frameworks' weird error handling behaviors? Otherwise, perhaps the Effect doesn't handle Errors, just Exceptions, though it looks like it does to me.

Disclaimer: #155 discouraged me enough to wait for macros, so I haven't written much with fpdart in a few months.

SandroMaglione added 30 commits February 16, 2024 13:04

experiment with shared Effect class

f9ece2c

IEffect shared interface

194db27

Either as IEffect

d44181c

Async from IEffect

3861235

call run

0fec429

no Sync/Async/Either, just Effect

289085a

Effect and Either env

8980c86

Option as IEffect

f9c32f0

mapLeft

0c23b78

more Effect methods

4d2612e

asEffect for IEffect interface

e0c2dfa

Effect api with categories

5f945c4

file system example with Effect

bddc3b9

refactor with abstract env

a5af9a8

test with provide in Effect

8e3aa09

catchError

9d42ff6

refactor internals

ae2a3ef

remove all examples

30e643f

outdated for now with new `Effect`

initial (new) tests on Effect

5163c52

import alias

51a2a62

Effect sync execute

b5b4c4c

collect as Iterable

c2e9349

expand Effect API

7fc0a24

orDie API

f70327f

Option API

e66dd02

Either API

15b5634

Exit with Cause

cb69054

tryCatch API

4a85df3

removed Cause by interrupt

1b8d00e

zipLeft and zipRight

69af383

SandroMaglione added 3 commits April 4, 2024 06:22

Scope with test

3ca43b8

test closable Scope

26180a7

v2.0.0-dev.3

895f80a

SandroMaglione requested a review from tim-smart April 3, 2024 21:47

tim-smart reviewed Apr 3, 2024

View reviewed changes

SandroMaglione added 3 commits April 4, 2024 15:27

refactor and remove __unsafeRun

8e34681

removed tryCatch onCancel

43f0592

renamed succeedLazy and failLazy

2f80240

SandroMaglione commented Apr 4, 2024

View reviewed changes

SandroMaglione added 2 commits April 4, 2024 17:10

scope methods in Effect

f691c1e

fix withScope issue

5efd738

SandroMaglione added 4 commits April 6, 2024 10:36

async, asyncInterrupt, sleep

1118bd6

rename constructors sync/future

2fe20e1

example read/write file

4039939

sleep

29331c8

SandroMaglione mentioned this pull request May 21, 2024

Test Helpers #70

Open

lishaduck reviewed Aug 11, 2024

View reviewed changes

	Effect<E, L, R> _provideEnvCloseScope(E env) =>
	env is ScopeMixin && !env.scopeClosable
	? Effect<E, L, R>.from(
	(context) => _unsafeRun(context).then(
	(exit) => switch (exit) {
	Left(value: final value) => Left(value),
	Right(value: final value) =>
	env.closeScope<E, L>()._unsafeRun(context).then(
	(exit) => switch (exit) {
	Left(value: final value) => Left(value),
	Right() => Right(value),
	},
	),
	},
	),
	)
	: this;


		typedef Exit<L, R> = Either<Cause<L>, R>;

		sealed class Cause<L> implements Error {

fpdart v2.0.0 #147

Are you sure you want to change the base?

fpdart v2.0.0 #147

Conversation

SandroMaglione commented Mar 18, 2024

This PR is an open discussion on what's the best strategy to release fpdart v2 to minimize friction for users of the library

👇 Please comment below with any idea, comment, critique or opinion 👇

Problems with v1

fpdart v2 solution: Effect

Benefits

Downsides

SandroMaglione commented Apr 3, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ryanhanks-bestow commented Apr 3, 2024 • edited Loading

ryanhanks-bestow commented Apr 3, 2024 • edited Loading

ryanhanks-bestow commented Apr 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SandroMaglione Apr 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SandroMaglione commented Apr 4, 2024

lishaduck commented Jul 24, 2024

SandroMaglione commented Jul 24, 2024

lishaduck commented Jul 25, 2024 • edited Loading

SandroMaglione commented Jul 31, 2024

lishaduck commented Jul 31, 2024 • edited Loading

lishaduck commented Aug 10, 2024 • edited Loading

lishaduck left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as resolved.

This comment was marked as resolved.

lishaduck Aug 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lishaduck Aug 11, 2024 • edited Loading

Choose a reason for hiding this comment

airychkov commented Nov 5, 2024

lishaduck commented Nov 5, 2024

fpdart v2 solution: `Effect`

ryanhanks-bestow commented Apr 3, 2024 •

edited

Loading

ryanhanks-bestow commented Apr 3, 2024 •

edited

Loading

SandroMaglione Apr 6, 2024 •

edited

Loading

lishaduck commented Jul 25, 2024 •

edited

Loading

lishaduck commented Jul 31, 2024 •

edited

Loading

lishaduck commented Aug 10, 2024 •

edited

Loading

lishaduck Aug 11, 2024 •

edited

Loading

lishaduck Aug 11, 2024 •

edited

Loading