While doing convertion of Strings to Text, relude is tempting me

[Anton-Latukha]

Because text managed not to ship the show analog (tshow).

And that is why now during migration instead of show now I need to do Text.pack (show a) just to have type cast. And code would need to do the marshaling. Or we would receive more of weird go-around-old-bugs dependencies and imports.

There is a number of preludes, people used to recommend switching to the most drop-in basic basic-prelude, but there were things like: snoyberg/basic-prelude#67.

At least, tshow is there now.

---

But the more I think about it - the more I want to go to the clean polished solution, that is to the relude.

---

relude is seen as "new" prelude, but it is in fact is one of the plder preludes that had "a do-over again properly" anew several times.

1. It is a well-known protolude developed by Stephen Diehl.

2. Then Serokell took protolude and forked & reimplemented it for internal production use, the result is universum.

   In 2017 the RIO prelude was released.

   Then the main developer of toolin in Serokell, one that migrated protolude and created & maintained universum created own company kowainik

3. Then kowainik took the universum and again, forked it and reimplemented the prelude the 3-rd time, into the relude.

   Kowainik overall produced a number of high-quality tooling: GitHub profile Main page, which have pretty wide use.

   Their quality of the processes, the code is high, and the level of documentation to the projects is unprecedented.

   Particularly the relude documentation is what is called "full documentation": For example random module: Lifted.Concurrent or Lifted.IORef

   Snoyman is known to be opinionated, and RIO is of course came-out pretty opinionated, which can be observed that creator of prelude starts to "recommend" a long list of extensions for people to "use with the prelude", and then apologizes that some of them are unsafe and controversial. And Snoyman himself maintains the RIO, and we know how many things he juggles. I have some reports opened that are still waiting for him after a year or more.
   Vince of the foundation is also very opinionated, and became unresponsive these days (a year or more), which created difficulties that were mentioned in a neighbor project Support GHC 9.0.1 (at least in some packages) dhall-lang/dhall-haskell#2154 (comment).
   Dima (I need to be honest, I happen to know him personally, so the nepotia can be involved here) - he had a years-long process of developing the tooling inside Serokell, then as Kowainik, and he is only one of several people that had the ability to do-over the several redos of the prelude, and maintain them afterward. At the time of working with him I known Dima as very humble and chill person, in the social psychology storm that was Serokell, he was calm water, he is a very pleasant to do work with & balanced guy that devoted his life to building useful Haskell tooling.
   And so the 4-th generation of the lean prelude came-out very well. And when I do String <-> Text conversions in HNix, I know that the question of Text, Bytestring and String is fully solved there, show is polymorphic over the return type requested, main use is Text, Bytestring is provided, String is optional. For example, a lot of the things we have in Utils are in the relude.

   And the Prelude dependency graph includes only what newly done prelude should include and nothing more:

   dependency graph

   

   Solved text data type question and solved list question, and safe versions of functions with them, some basic data containers, some basic GHC features as Nats, MTL and that is all.

   That is what packed into the prelude. And those dependencies can be reduced from the imports, so no longer importing Data.Bool ( bool ), Data.Text Text, Data.Text as Text, Data.Maybe (fromMaybe), Control.Applicative (*>), Data.Functor ($>), Control.Monad (<=<) and so on and so on - all those basics are thought about many years ago in protolude and was polished afterwards.

   Since I what to migrate the project from String to Text, and from List to NonEmpty List - it is much cleaner to do the migration to the relude that does prelude properly. show anyway already had type class overhead, so why not use it for its purpose, let type class polymorphism do its work, it would save 100 fold by freeing from the data marshaling.

   They were in the "beta" for ~2.5 years (again, it is after it was stabilized as Protolude, then as Universum, now as Relude), and now they recently proclaimed final stabilization by naming it the 1.0.0.0.

   I recommend to check-out the further description in the: https://github.com/kowainik/relude, because reading more deeply into it I found that what I did recently in HNix - a number of those things were already in the Relude. And many of functions form our Utils are there, so we can reduce and kind-of stop building our own internal analog for prelude. As also there are the Bool <-> Maybe <-> Either casts, which I frequently want to use, functions to work with tuples (first,second,both), and other such basic stuff.

   For me it is easier to migrate to relude with provided by them guardrails for migration once, than to anyway migrate String to Text, but by importing and qualifying and do marshaling back'n'forth the types (which defeats the purpose of doing String -> Text in default Prelude, if to not ship a lot of deps and more imports into every module) and then to do circles around grooming all that and further beyond to the flaws of the Prelude and core packages just to preserve all of the old bugs because they are well-known. Better simply relax and know that relude figured-out proper prelude after going through 4 generations of making prelude, and then ask to relax the incoming people.

   As an additional effect, it would clean-up our code even more and would increase the quality and readability of the code we have in the project.

[Ericson2314]

Is this stuff going to work with GHCJS? I am very skeptical of batteries-included preludes for being antithetical to a fine-grained library ecosystem emphasizing "depend on only what you use" and maximizing portability. Batteries-included design in fundamentally non-compositional as not everything can include all the goodies because some things need to be used to define the goodies, and making an artificial "below the nice stuff, above the nice stuff" split is needlessly frictive.

[Anton-Latukha]

HNix for 1 year does not compile into JS. It all was done gracefully and in the open.

It was broken for some time when we showed-up. And fixed the build.

We removed GHCJS build, since after we fixed it - broke again, and was such for ~3 months - and I specifically made CI to show that broken status repeatedly, it was spamming the master branch, and all PRs & nobody showed any concern and not showed up or was interested in maintaining it.

There are WebAssembly compilers. I think compiling Haskell through GHC into WASM is a much simpler task (a more robust system and tenfold more performant result).

Than metaprogramming untyped JS by Haskell, GHCJS did not give any Haskell-strong guarantees, since JS compiled result is untyped and JS has a lot of different weird type casting and operation artifacts. In Babylon tales, it was a colossus of gold and iron standing on the "Feet of clay".

It also required a loot of jumping through hoops for a project. As for example you mention GHCJS requirements constrained to what can be used as a dependency for the whole project for all platforms. And indeed, guys made some adjustments to less performant libs, in part also to make GHCJS happy.

And with HNix evaluating closures for minutes on GHC 64 Bit x86 multithreaded highly optimized machine code, and we got people complaining that it is too slow, and it is, and it takes Giga Bytes upon Giga Bytes of RAM, and it is. And imagine GHCJS, the HNix on GHCJS was a dream that'd worked much slower and ate even more resources, it would've been like looking the grass grow, and watch how internal browser process killers killing the thread because it ate all browser quotas again and again - so much stability for a system package manager there.

GHCJS development went away ~1.5 years ago.

So let us start to forget about GHCJS.

And would look at tweag/asterius when it would be needed.

As far as I understand, for compilation into WASM the prelude or packages used (if not FFI) - are completely irrelevant, as asterius gets the intermediate products from the stock default GHC to produce WASM.

[Ericson2314]

There is a bunch of discussion over in https://discourse.haskell.org/ about overhauling base, prelude, etc. I strongly encourage waiting for that stuff to settle over the next year or so.

[Anton-Latukha]

Well, if we are not moving to a better prelude now, - I simply would not do the String to Text migration - since the otherwise process is a hassle and would have mediocre result, code result would be yuk and a lot of back'n`forth marshalling.

Since relude is so compatible with base prelude - the new base seems to be multipackage, and a lot of changes, so the move from old base to new base seems more complex then doing it gradually now.

If we would wait for new prelude to show up - 1. - it may never show up, 2 - may show up late 3. we would keep waiting for it. In the meantime we would need to do or would want to do the nothings about the code cleanup.

I am not interested keeping the project and things dormant for a year just to wait for new base.

No sense in according refactors in old prelude, since or in relude or in new base they would be easier, and because of that definitely, no package splitting this year, since if project splits into sub packages - that would be 4-5x more breakages and breaking versions and making all those releases for them, and people just switched or jumped on the splitter infra - and we change it right under their legs after they landed.

If there is things I want to do this year - is to clean-up the project, in large it is already done, upgrade the types to more efficient, and clean interfaces in preparation for the split, and then do the split.

If I managed to prelude -> relude in 1 day. We would migrate, and I would clean-up the code and do refactors enabled by the new prelude. And still would have a full year ahead of time. And then, when new base shows-up, I would migrate the projects, and so far I always kept my promises. If the prelude -> relude is 1 day. And it is with migrating String, List prelude to Text, NonEmpty prelude in internal code without changing the API. So the move relude -> new base would be one day flat, since the code would already be clean, and all type system would be in line for easy migration.

And overall, relude would just as well switch to the new base.
And I start to think that there are 2 models of prelude: 1 - which is pretty static and keeps a lot of backward compatibilities, and 2-nd - which is always developing futuristic one but requires updating to new versions.

With HLS such migrations became trivial, to migrate to new base from relude what would be needed - is to delete the mutex in .cabal and fd -t f -e hs | xargs rg -i '/import .*Relude/d' (find .hs files & then remove import*Relude*$ from them), and right then GHC gives a guide tour and person picks imports for the functions. After since the code is already clean and migrated to current mainline production types - there is almost nothing to do.

I love to move things into one direction and then into the opposite, it is Hegel's Dialectics working on the code design: the argument formed the code (old prelude), then the move into the counter-argument ("underground" counter-culture) (relude), and only after that - that refactoring process makes possible the synthesis of best parts of both. If it was possible to swing from the argument into a synthesis - that wouldn't be Dialectics. The reason to move prelude -> relude -> new prelude - is that we would have a better code of the project as a result, since some things would be simplified and found by migrating and using relude (synthesis happens) and then that already better synthesis result becomes argument and counter-argument is the newly shipped and formed prelude, and so project migrates and gets the 2-nd synthesis, and as a result picked and refactored parts in both parts of the world, what was obvious to do in relude - happened there, and then after new prelude - other things became obvious in other places - so the synthesis. That is why I even talk about it, new prelude news are great - it means we can get 2 syntheses roughly this year.

I do not want to break the stuff after it works, I want to design it better before it turns on.
My main agenda is to make the code as clean, optimized and obvious to people as possible - that when remained puzzle pieces are placed - we'd not needed to worry about any technical debt we sitted on or structural debt, or having the code to complex for people.

This method (argument - counter argument software design swing) (and it works just as when programmers recommend learning more languages to become a better programmer) is a great self-forming design method and the nature of the project is strong to do it - we have self-forming design support from all sides, same paradigms go across everything, both languages are alike, products keep the properties we can rely on to reform a design. But this Hegel's Dialectics method is applicable only when the project refactoring is more important than the backward-compatibility of use (is not in wide use, like currently), so as I see it - we must take the opportunity of refactoring the code by this method. And currently I myself can give a hand to migrate the downstream projects, since there are 2-3 of them and they use small portion of our API.

Lets when new base comes - then we would decide if new base is better than the relude. Currently, any prelude is better than one that does not exist, threads activity on developing it started this week or yesterday. When it comes - lets decide then, in a year - ok. Switch takes ~1 day. That is why I can say, I would migrate the project (or split packages already, I hope) to a new base prelude personally. Because anyway, somewhere around, probably after that timeframe we would roughly arrive around the possibility of MVP versions.

[Ericson2314]

Well, if we are not moving to a better prelude now, - I simply would not do the String to Text migration - since the otherwise process is a hassle and would have mediocre result, code result would be yuk and a lot of back'n`forth marshalling.

That's fine with me.

I am not so concerned with the breaking of old stuff, and so if you want to do this and then switch again I won't complain about churn.

[Anton-Latukha]

I would wait several more days before finishing the work to get people more time for input, and would take time to learn Existential and Skolem, since the migration uncovered the only difficulty , 1 place where the type boundaries were completely lifted after creation, in project there should be places that relate to language creation type boundary programming theorem, but since core and new languages are of same paradigm - the type boundaries skewing we would get should be very minor, it is probably RankNTypes use and this place in the Parser. Overall I am interested to learn more about that theorem and see it better in our design and if it is - put the finger on it to show it.

---

Yep, returned to say that it can be seen as I'm being stubborn (and have that property) and standing ground, but want to underline that in the discussions we manage to evolve the processes.

Since you brought legitimate useful information we changed context and the discussion and the future would change a bit, that is a synthesis. During work I would know that I would not hammer by nails things and that we would migrate the code to a new base when it comes.

I am stubborn, but also during discussion I move a lot 🤣.

I see new prelude superb news, and it only can help our processes and future to everybody. And here - the more we work on the code - the more it cleans up and gets more and more plasticity. In the process, I practically tried the amount of work to do, and estimated it as ~1 day to travel to the new prelude of a new base, and promise to myself migrate the project to the new prelude.

So that is the current synthesis.

[layus]

I think we can go with relude, and roll back it we ever need it. HNix is a big piece of software, and depends on many things. A custom prelude is not much more compared on all our dependencies at the moment.

I guess we may rely more and more on it, but that is fine with me.

[Anton-Latukha]

It is not a foundation that tried to reinvent a lot.

relude is very compatible with the base and passes-through the interfaces of the packages it depends on (mtl and such).

The only main thing it does - it gives the total functions, and takes away the unsafe toys form the programmer for them to not fall in them without awareness, it makes to explicitly ask for unsafe things or import them explicitly from the unsafe data types that code still uses, so the programmers and the code start to track where the unsafe data structures and functions are used.

That is also why picked it to migrate to new total data types and functions, because relude would not stop rubbing our noses to migrate stuff to the NonEmpty List, NonEmpty sets, hashmaps and so on.

And ease it gives on migration to Text is also a huge thing.

And as I said, it expells unsafe functions, and all total parts of the interfaces of packages are transparent, so the imports to underlying packages work transparently.

I actually got used to it in a day or two, it packs basic things nicely, the only downsides:

1. In time it is easy to forget the mental overhead of imports prior to it. And it easy to fall into a habit using some helper functions that it ships. Since new base is coming, it should be a weighted approach to not lean to hard on all the helper functions in prelude, but only on occasional most useful.
2. During Hoogle searches going to prelude Haddocks a lot. And relude has superb docs and interfaces in that regard.

[Anton-Latukha]

Would note that text datatype conversion map is: https://hackage.haskell.org/package/relude-1.0.0.1/docs/Relude-String.html

The show is "1 function to rule them all", it essentially replaces the majority of use cases.

But show does a -> String -> b (does the text data type marshaling two times), so it is a very costly operation.

Would migrate to show, simplify the code, and then would have a 2-nd pass specializing it to a map.