making hint faster; pre-loading some modules at compiled time? Is there a straightforward way to do that?

[archywillhe]

looks like loadModules only takes place at run time.

Is there an easy way to pre-load modules into the interpreter at compile time?

[archywillhe]

will the packages included in unsafeRunInterpreterWithArgs be built at compile or runtime? I'm tinkering around to port hint into wasm with Persistent so just figuring ways to make it faster; thanks!

[archywillhe]

or does GHC handle the "pre-loading" at compile time already? (So the one of the only downsides of interpreting (in hint vs having the code built into a module) is that I don't get use to use Template Haskell? Any sources where I can read more on it? Thanks! )

[gelisam]

First of all, since you're concerned about performance, I recommend reading the GHCi documentation on loading compiled code. GHCi and hint are both using the ghc library, a lot of the ghci documentation should apply to hint as well.

[archywillhe]

Aw I see! Thanks!

[gelisam]

To clarify: with the above approach,

1. your program is compiled and optimized into a binary which only contains your program
2. a library is compiled and optimized
3. at runtime, the ghc library (via hint) loads the optimized version of the library
4. at runtime, the ghc library (via hint) interprets a piece of code which makes calls to functions from that library, and those calls run very fast

I don't think this is what you mean by "pre-loading".

[archywillhe]

Thanks for laying things out so nicely! That's what I suspected; so it doesn't look like pre-loading can be done (unless I fork hint and rebuild hint with some of libraries I want to "pre-load" I guess)

[gelisam]

I'm tinkering around to port hint into wasm

Oh boy, good luck with that!

I know that ghc can produce wasm code, but is it possible to compile ghc itself into wasm yet? it sounds much harder.

hint uses the ghc library, which holds most of the code for the ghc executable. so if it's not possible to compile ghc itself into wasm, it's probably not possible to compile the ghc library into wasm either.

In particular, can wasm code load files from the filesystem? The ghc library relies on a ton of files being installed on the target machine, many of which are installed on your machine when you install the ghc executable. Here is the full list of files I had to copy to a blank linux container in order to run a tiny hint program without installing ghc on that container. This includes some .so files, which probably have a very different representation in a machine-independent representation like wasm as they do on a linux machine. And ghc's runtime happens to use a hand-written linker, which would thus need to be updated in order to read whatever is the wasm alternative to .so.

so... yeah, I would be very excited if I could run hint in the browser, but I think there's still a lot of work to be done until we can get there.

[archywillhe]

Aw thanks for the pointers! Yeah, I was thinking of porting out a subset of ghc which hint uses (and see what I can do with them in the bridging into wasm). I wasn't aware ghc's linker relies so much on the structure of a filesystem. Looks like that would require a lot of rewriting then

[archywillhe]

and oh boy deploy-hint is looking really nice! I will give it a try in production

[gelisam]

So the one of the only downsides of interpreting (in hint vs having the code built into a module) is that I don't get use to use Template Haskell?

What do you mean? hint supports TemplateHaskell just fine:

import Language.Haskell.Interpreter

-- |
-- >>> main
-- Right "Hello, world!"
main :: IO ()
main = do
  r <- runInterpreter $ do
    set [languageExtensions := [TemplateHaskell]]
    setImports ["Prelude", "Language.Haskell.TH"]
    interpret "$(litE (stringL \"Hello, world!\"))" (as :: String)
  print r

[archywillhe]

oh I see; that's amazing! I thought Stage 1 code is off-limit; it is real fantastic this works!

[archywillhe]

(Think I got confused with another library that doesn't support some aspects of Template Haskell; think it was ghc-hotswap or something )

[gelisam]

will the packages included in unsafeRunInterpreterWithArgs be built at compile or runtime?

Are you saying that you are writing unsafeRunInterpreterWithArgs ["-package acme-dont"] to "include" packages, like this?

import Language.Haskell.Interpreter
import Language.Haskell.Interpreter.Unsafe

-- |
-- >>> main
-- "hello"
-- Right ()
main :: IO ()
main = do
  r <- unsafeRunInterpreterWithArgs ["-package acme-dont"] $ do
    setImports ["Prelude", "Acme.Dont"]
    ioAction1 <- interpret "do (putStrLn \"hello\")" (as :: IO ())
    ioAction2 <- interpret "don't (putStrLn \"hello\")" (as :: IO ())
    liftIO ioAction1  -- prints "hello"
    liftIO ioAction2  -- doesn't
  print r

Note that the -package flag is only affecting the visibility of the packages, it does not affect when the package is loaded. In particular, unsafeRunInterpreterWithArgs is an IO action, and IO actions run at runtime, not at compile-time.

The only code which ghc runs at compile time is:

• TemplateHaskell splices
• some pure functions, if ghc chooses to inline their definition and then simplify

[gelisam]

Is there an easy way to pre-load modules into the interpreter at compile time?

Yes, there is! In the following program, don't is compiled into the executable, it is not loaded at runtime.

import Acme.Dont
import Language.Haskell.Interpreter

type Don't = IO () -> IO ()

-- |
-- >>> main
-- "hello"
-- Right ()
main :: IO ()
main = do
  r <- runInterpreter $ do
    setImports ["Prelude"]
    ioAction1 <- interpret "do (putStrLn \"hello\")" (as :: IO ())
    mkIoAction2 <- interpret "\\don't -> don't (putStrLn \"hello\")" (as :: Don't -> IO ())
    let ioAction2 = mkIoAction2 don't
    liftIO ioAction1  -- prints "hello"
    liftIO ioAction2  -- doesn't
  print r

When you compile the above program, ghc loads the previously-compiled acme-dont package, finds the .a file which contains the .o file which contains the implementation of don't (and similarly for the hint and ghc packages), compiles your program into another .o file, and links all of the relevant .o files into a single binary.

When you run the above program, the ghc library still reads a bunch of files, and loads the (compiled version of the) Prelude at runtime. And it interprets the program \don't -> don't (putStrLn "hello") at runtime. But the result of this program is a function which takes an implementation of don't as an argument, and that's a big deal, because we don't have to load the acme-dont package at runtime in order to obtain such an implementation, we already have one baked into the executable!

[archywillhe]

aw I see! This is super smart! Gotcha! Thanks again!