Unify FreeC and Algebra types.

[diesalbla]

Continuation of #1598, which was automatically closed by Github due to the deletion of the series/1.1 branch. In what follows is the original message, and some items from the conversation:

---

This unifies the two ADT in the internal stream implementation: FreeC[F[_], R], the Free monad with catch and interruption; and Algebra[F[_], O, R], for the scope-handling and output instructions. These are combined using the approximate type aliae of PullF, O, R] as FreeC[Algebra[F, O, ?], R]. Notably, an object of type Algebra[F, O, R] can only appear in a FreeC through the FreeC.Eval class. This allows us to unify these two ADTs by doing the following:

• Add an output parameter O (stream output) to the FreeC ADT, and its subclasses; as well as to the ViewL trait (for unrolling).
• Turn FreeC.Eval into an abstract class that replaces the previous Algebra trait, so that all subclasses of Algebra become subclasses of FreeC.Eval. Because FreeC and Algebra ADTs are in separate files, we unseal it.
• We replace the ViewL subclasses from containing a F[R] to containing an Eval[F, R].

As a result, a Pull is no longer an alias for FreeC[Algebra[F, O, ?], Unit], but for FreeC[F, O, R] instead.

With the previous 2-layered representation, there was a need to have in the Algebra object a few methods to lift simple data and fill the type parameters. With the new representations, those methods are not needed and we can remove them in favour of the object they build. That is the case of the private methods openScope and step.

Translate to mapOutput

Before this change, the implementation of Stream[F, A].map[B](f: A => B) had to work at two levels: one being a FunctionK between two type-instances of Algebra, which would differ on the output type. To apply this transformation through the Stream, the FreeC ADT had a translate method. After the changes, now we do not need that FunctionK wrapper, and we can just replace the translate method of FreeC with a mapOutput method.

Modularity

The new design keeps some of the modularity of the previous one: the FreeC main classes and operations are built without any reference to the Algebra classes. The major control-flow operations, such as the FreeC methods and the loop unrolling of ViewL, are also implemented without reference to the Algebra object. So, the separation of concerns still exists between these files, except that now it is connected through sub-classing rather than through a parametric type.

Readability

The new representation has no nested types, no kind-projection ? symbol, fewer "smart constructors", and implements stream output map without using a FunctionK. Also, now the F[_] parameter refers everywhere to the effect in which the stream is run. Against those upseides, there are some downsides in adding an extra type-parameter O, and the use of unsealed traits.

Benchmarking

As an optimistic assesment, since this PR reduces the number of objects neeeded to represent each Algebra operation (by no longer needed the FreeC.Eval wrapper), this may significantly reduce memory consumption. An early benchmarking, as mentioned in the initial PR, suggested a reduction of 2.5% memory consumption.

After-work: unify FreeC and Pull

Now that FreeC has two covariant parameters, the only major difference with the definition of Stream and Pull is that the F[_] parameter is covariant in those but invariant in FreeC. I am experimenting with how to add that covariance to FreeC, but at present it is causing not only some compile errors, but also compiler crashes. This PR, however, seems to be stable enough and carries enough changes to be dealt with separately.

[diesalbla]

@mpilquist For now, the branch has only unfolded/removed the factory methods that were private to this object. However, since the whole Algebra is private to fs2, would it make sense to open the visibility of the Algebra subclasses, and remove this method, as well as eval and output above, and getScope below?

[diesalbla]

The attached file contains (compressed) a spreadsheet with the results of some more benchmarking results.

I used the suite StreamBenchmarks with some changes: I reduced the generated range @GenerateN from 1, 10, 100, 1000, 10000, 100000 to just 10, 1000, and 100000. Here is a patch file with the changes. I run these benchmarks, both in the master branch and in this branch, using the command:

jmh:run -f 1 -i 10 -wi 5 .*StreamBenchmark*. -prof gc

The results of these runs were tabulated into this spreadsheet, in which I also add the cells to highlight the relative improvement (the master measures to the left, this PR to the right).

The results indicate, across most benchmarks, a reduction in memory consumption (as measured by the gc.alloc.rate.norm) of 2.5% (the eval benchmark), 3% (leftAssocConcat), 5% (leftAssocFlatmap), 3.25% (rightAssocConcat), and 5% (rightAssocFlatmap). The one micro-benchmark for which no improvement is observed is in toVector, but that is understandable given that most allocations there go in creating the Vector itself.

For the environment:

$ java -version
openjdk version "12.0.1" 2019-04-16
OpenJDK Runtime Environment AdoptOpenJDK (build 12.0.1+12)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 12.0.1+12, mixed mode, sharing)

[pchlupacek]

@diesalbla seems promissing, thanks for this

[mpilquist]

Looks good to me. Looking forward to unifying FreeC and Pull in to a single type too.