extremely slow evaluation speed

[poscat0x04]

Evaluating "${(import <nixpkgs> {}).hello}" in hnix takes around 15 second whereas it's near instant in nix repl. This is wayyyy to slow.

[Anton-Latukha]

And takes a lot of memory. There are already #200 and #508 opened about this.

Let us work on the accuracy of the evaluation, and basic feature parity, there is no sense in any evaluation speed performance if it does not evaluate properly or if it can evaluate gibberish fast. Project needs to 100% bug2bug cover the Nix language quirks, and test coverage for them, have flakes support etc.. There are miles of things to be concerned with things that need to work, then to be preoccupied with a question of the speed. Also to put things upfront, HNix would probably never be as fast as reference Nix or RNix, for obvious reasons and for reasons that the projects have a different aims.

[Anton-Latukha]

import also works in Nix not as it is expected in a lazy language. It does not bind it lazily to the value until requested, but actually evaluates the <nixpkgs> at point ASAP before and imports the derivation result - that may be the difference in the behavior.

HNix needs to at least finish the builtins before looking at Nixpkgs evaluations. HNix already evaluates the lib, which is promising. There is already derivationStrict on the merge, what is left is fetch*ers functions.

And for me hnix --repl evaluation of "${(import <nixpkgs> {}).hello}" throws:

file 'nix/fetchurl.nix' was not found in the Nix search path

[Anton-Latukha]

There are a big number of things that dampen the performance currently.

For example, HNix currently uses the https://hackage.haskell.org/package/hashing library that is extremely slow. Like 100 times slower than low level hashing libs. When the Nix processes rely on hashing pretty hard. That was done for the convenience of pure Haskell development before we arrive to speeding-up tasks or solidifying hashing.

Currently, the hnix-store needs to migrate to cryptonite, after that maybe we would switch to the cryptonite in HNix.

Also, the merge of strictDerivation should make derivation production faster. However, I can be a fan of lazy - lazy evaluation of data can be quite slow in some cases.

The project is vast, interesting, and challenging, with a lot of advanced Haskell use. There are always many ways to join in.

If something - my personal request would be less λP stuff only where it very required - λP features make Haskell code rigid, which makes it not refactorable. And we need to polish and refactor A LOT, the core components are essentially quite ready&formed but they need refactoring, constraining the higher rank types where they were used initially just to make the code work, porting the custom type system and code to the recursion-schemes lib etc - that should make the compilation optimizations much more efficient.

[Anton-Latukha]

Closing this in favor of the main thread: #200.