Release CE 3 as separate library

[Odomontois]

Currently cats effect 3 has same namespaces and artifact prefixes as cats effect 2 despite being heavily incompatible.
This creates a major pain for libraries depending on cats-effect 2, since they must prepare a major release replacing CE 2 to 3 and every user will be forced to choose between CE2 and CE3 ecosystems.
CE3 also brings some innovative enhancements, some of them on a very low level, which could be dangerous for reliable systems before those changes will become mature and production-tested. This could slow CE3 adoption.
To make CE3 adoption more swift and safe I propose to change namespace and artifact names for CE3 to
cats.effect3... and "org.typelevel" %% "cats-effect3-$module" correspondingly.
That would also help some libraries such as tofu, that have cats effect dependency in its core, to prepare safe integration packages for the upcoming release.

[djspiewak]

This concept has been discussed a few times over the years. I could dig up some issue references, but I believe no fewer than two others. In general, I'm unconvinced.

The idea of putting major versions into groupIds certainly isn't new. Rich Hickey most notably advocated for it strongly in a talk a few years ago. Specs2 used to actually be Specs. Notably, Specs2 now has major version 4.x, which… probably says something about how this scheme is going.

This kind of name mangling is worthwhile when the expectation is for the two versions to live in parallel for a long, long time. In the case of REST APIs, the parallel existence of /v1/ and /v2/ APIs tends to be on the order of years. I find it very unlikely that CE2 and CE3 will live in parallel as coequal options for that length of time. In part because they aren't as heavily incompatible as they seem (most downstream codebases will be able to migrate simply by swapping their versions and applying the scalafix).

The other aspect of this which needs to be considered is tooling. Both CE2 and CE3 publish their semver contracts as part of the POM metadata, which sbt now respects. In the event that someone, accidentally or intentionally, engineers a dependency diamond, the results will be a build error. We still do have the problem of the linear classpath, but at least the manner in which we construct that classpath is safe.

This also brings me back to my main premise on all of these things: name mangling will never be able to encode the expressive power that a properly layered runtime can. The JVM is now capable of this, though it hasn't really been leveraged too extensively in the ecosystem, and it requires >JDK 8. Despite the sluggish nature of uptake though, it's clear where the wind is blowing. The linear classpath is on its way out, and unchecked dependency eviction is already out, which suggests to me that name mangling is a solution in search of a problem in the future. This isn't HTTP where we have absolutely no control over things other than naming. It also isn't Clojure where it's impossible to build tooling like MiMa.

Which ultimately brings me back to the original premise: names. Do names represent immutable concepts, or are they merely mutable pointers to immutable things? I very strongly believe in the latter, philosophically. This is how Git works. It's how the physical world works (think about postal addresses). The examples go on and on. This is also the semantic that software packages have converged to over the years. The added caveat is that we tack on a version qualifier to many (though not all) name references in order to give us the ability to immutably address the underlying state.

Which again comes back to tooling. Pushing the version redundantly into the name only affects the elements of the software linkage stack which are unguarded by tooling that understands versioning, which is a shrinking space that will continue to get narrower and narrower as some of these long-standing problems are addressed.

All in all, I really don't think this kind of thing carries its weight, and it ultimately saddles the ecosystem with an artificial name mangling that sticks around well beyond the point at which CE2 ceases to be relevant (as with the original Specs), not to mention the fact that the ecosystem is evolving to a point where versioning is handled in a more first-class way, and where naming tricks like this actually work against the infrastructure. It's better to take the modest pain of migration juggling in the short term rather than accepting perpetual pain of "otherness" in the unbounded long term.

[Odomontois]

Can you propose a solution or technique for the library that depends on cats-effect 2, and will be for some time due to compatibility guarantees, still wants to provide a cats-effect-3 integration module, that is easily usable by the majority of library users?

[djspiewak]

What most of the ecosystem has been doing is parallel release branches. Look at cats-effect-testing for a trivial example. We have a series/0.x branch which is for CE2, and a series/1.x branch for CE3. Http4s and Fs2 are doing something similar. Actually, Cats Effect itself is also doing something similar.

munit-cats-effect and weaver both demonstrate a different semantic, which is that they're trying to do things in a single branch with a build matrix. I wouldn't recommend it.

This whole thing is a pain, granted, and we knew it was going to be a pain 2 years ago when we started this journey. I'm sorry. :-( But the good news is that the pain is not long-lived, and the more we can get the ecosystem on board with marching forward in unison, the shorter lived it becomes.

[kubukoz]

@Odomontois we're going to try and make the migration as smooth as possible, and CE2 will still be maintained for some time in case a critical fix is needed. We're looking at public projects and their efforts to migrate, there's a Scalafix migration coming which should take care of most changes, and a migration guide in works.

I know this will be a pain to migrate for some, but in this case, and with the tooling we have, I believe ripping the band-aid off will be better than letting this last for years.

btw. thanks for reminding about tofu, I'll add it to the tracker (#1330)