synthesize functions

[KiaraGrouwstra]

- [ ] generating datasets with fixed test set

add-on

• supervise by types: basically include similar to the i/o features using a string based encoder like for i/o as well for now
  • split io_feats and taskType based on param type instance combo
  • supervise with i/o types
    • make this optional to allow still using the old model as well. that may be tricky tho since they imply different numbers of params in the function type signatures...? note that the behavior of mkCharMap got updated to register characters used in types as well.
  • supervise with hole types
    • get the type signature by exprType on the function
    • using the expected type for the expression, compare with the function signature to potentially fill in (part of) its type variables
    • for the function's parameter slot, assign the substituted input type for the ExprTypeSig
  • supervise with node types: types of symbols/blocks
  • supervise with sub-tree types: types of rule variants
• zero-mask non-compiling expansions - mask branch

benchmark

neuro-symbolic program synthesis

implement:

• backprop by supervision
  • combine Encoder/R3NN for backprop
• implement test loop
• ensure nnets are reused across a generation
• ensure symbols reflect more than just LHS
• CE loss
• test loss
  - [ ] add additional encoders - out of scope

test:

• [ ] ensure I split the encoding tensor right -- well, it's learning 🤷‍♀️

experiments go at #10.

[KiaraGrouwstra]

aside from my issues on the remote machine (#24), trying to run tensorflow/haskell locally:

• Cabal: ?
• Stack fails
• GPU Docker fails
• consume their Docker from Stack; stack --docker --docker-image=tensorflow/haskell:v0 setup -- to make it find Tensorflow C lib use the extra-lib-dirs and extra-include-dirs from their stack.yaml.
• build upon their Docker image running Stack from inside that: docker run -v $PWD:/app -ti synthesis-mount stack --allow-different-user build runs, but I'll need to either cram the deps in base images, mount (below) or face rebuilding from scratch each runs.
• Docker volume mounting, see reproducibility #16.
• by Stack+Nix using their shell.nix (vars) and maybe bits from their Stack yamls.
• Nix: ?

potential alternative: hasktorch:

• Cabal: progress on cabal-hasktorch branch, now errors with Constraint refers to an unknown package: libtorch-ffi ==1.4.0.0
• Stack: use pip3 install --user pyyaml typing + source setenv + build HaskTorch using with-gcc: /usr/bin/gcc-8 over Arch's 9, then stack build. then hasktorch-ffi-th errors with Missing (or bad) C library: ATen, implying something is up with my Torch binaries. makes sense if Torch binaries are a dependency, but yay pytorch did not fix. HaskTorch itself manages fine, seemingly due to its stack.yaml's extra-lib-dirs: - deps/libtorch/lib. somehow even using an absolute path /run/media/kiara/meltan/downloads/repos/hasktorch/deps/libtorch/lib that doesn't seem to do it from my repo tho?
  fish-compatible source setenv alternative:

set HASKTORCH_LIB_PATH "/run/media/kiara/meltan/downloads/repos/hasktorch/deps/libtorch/lib/:/run/media/kiara/meltan/downloads/repos/hasktorch/deps/mklml/lib/"
set LD_LIBRARY_PATH "$HASKTORCH_LIB_PATH:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH

as dep:

Unsupported tarball from https://github.com/hasktorch/hasktorch/archive/99545bb638430b321c03dcf79328ea27cd8a282a.tar.gz: Symbolic link dest not found from hasktorch-99545bb638430b321c03dcf79328ea27cd8a282a/spec/Declarations.yaml to ../deps/pytorch/build/aten/src/ATen/Declarations.yaml, looking for hasktorch-99545bb638430b321c03dcf79328ea27cd8a282a/deps/pytorch/build/aten/src/ATen/Declarations.yaml.
This may indicate that the source is a git archive which uses git-annex.
See https://github.com/commercialhaskell/stack/issues/4579 for further information.

that seems to match (1, 2) (unresolved). Stack does handle submodules fwiw.

• Stack + Nix: ?
• Nix + Cabal: see nix files, instructions in readme.

• Nix + Stack: ?
• Docker: any of the above from Docker. I doubt their current Dockerfile was intended to consume as a library tho as it's Cabal-based...

[tscholak]

after a pip3 install --user pyyaml typing installation works using gcc 8 over Arch's 9. 🎉 🎉

I'm glad! You can also rely on nix setting you up. I'm using it on my NixOS box at home on a daily basis.