LLVM ExecutionEngine no longer supported by rustc

[murarth]

The API was removed: rust-lang/rust#35174

[akavel]

Are there any possible venues for going forward given this situation?

[murarth]

Interpreting MIR output may be an option. There's a project that does that: https://github.com/solson/miri

I don't know how well that project functions or how effective that method is.

[murarth]

I've taken a quick look at miri and it looks fairly complete. MIR interpretation could be the way to implementing some of the features people have come to expect from REPLs (and are currently lacking in rusti). This looks very promising.

[akavel]

I've also taken a look; I've tried to focus on known limitations, which seem enumerated quite thoroughly in the miri-report.pdf (having which is already quite a good sign for the project's quality). Then I tried to skim over the commits since ~Apr 15 (date of the report) to see what was done since then. Summarizing my understanding of what's currently missing:

• no FFI support [that's actually most unfortunate news for me personally, as I hoped to use a Rust interpreter primarily as a sandbox for interacting with WinAPI];
• char and float types — seems they're already implemented starting with rust-lang/miri@2dbd30f and rust-lang/miri@4b83156
• pointers to "trait objects" [I don't know Rust well enough to understand exact meaning here]
• #repr(..) annotations for adjusting structs/enums
• limited support for cross-crate calls [? I'm not sure how much that would matter for rusti's usecase]
• one edge case in calls to closures
• function pointers — seems done starting with rust-lang/miri@384623d
• compiler [LLVM?] intrinsics are implemented on case-by-case basis, not all are supported yet
• user-defined destructors [! quite notable omission if I understand this correctly; though arguably not critical for rusti usecase] — however "it has most of the machinery in place to do so already"
• complex constants and statics — seems done starting with rust-lang/miri@cc1ca73 and rust-lang/miri@ddfbb65 respectively; unless I'm misunderstanding something (esp. not sure about constants)
• accounting for target machine primitive type alignment and endianness — seems done starting with rust-lang/miri@0288486, unless I misunderstand
• tail calls optimization? [does Rust support TCO? I didn't even know!]
• various TODOs and FIXMEs left in the code

From what I noticed, the following places can also be used to see what's still known as not working correctly:

• tests/compile-fail/bugs/* [edit: seems this dir is not present in the repo anymore]
• tests/run-pass/ -> see commented out lines in files (e.g. [1], [2])
• src/interpreter/mod.rs -> there is stuff marked as unimplemented!(), but I'm not quite sure what it means there; lot of this seems claimed as implemented in the report, so I think I'm misunderstanding something here.

With all that said... notably, apart from tons of stuff that reportedly is working correctly:

• miri seems seriously considered for official use in rustc for evaluating compile-time constant expressions (?) - or at least as a first prototype for that (?)
• another developer seems to have jumped in quite seriously at some point, improving miri for use in his https://github.com/oli-obk/priroda project ("A graphical debugger for Rust MIR", running in a browser IIUC)
• a stepwise MIR interpreter seems to have been added at one point

[murarth]

Yes, there are limitations. ExecutionEngine has some limitations, too.

FFI and intrinsics struck me as two of the biggest hurdles for MIR interpretation. C FFI could be done with existing external libraries. Rust FFI (should one wish to load a compiled crate) would be a more difficult task.

[murarth]

There are libffi bindings for Rust: https://github.com/tov/libffi-rs

low may have what rusti needs.

[willcrichton]

I updated my own mini-REPL to work with the latest nightly: https://github.com/willcrichton/lia/blob/master/lia-jit/src/lib.rs

It uses the ExecutionEngine from llvm-rs.

[hastebrot]

The API was removed

Does the ExecutionEngine in exec.rs replace the missing one from src/librustc_llvm/lib.rs (in rust-lang/rust)?

[murarth]

@hastebrot: No. It is merely a high-level API on top of the now removed LLVM API.

[Type1J]

@murarth What's the status of making rusti work with current stable or nightly? Is it being pursued, or is this effectively dead?

[bedax]

you may be interested in: https://github.com/google/evcxr

I'm not sure if rusti is looking at that for inspiration, would be nice if it did. As good as evcxr is, rusti's goals are still valid

[murarth]

@Type1J Not actively being pursued by me. This project is effectively dead, for the foreseeable future.

Building a new method of in-memory executing maybe be possible (perhaps using libffi and rustc MIR/MIRI), but it would be a significant undertaking. I'm currently working on other projects and, unfortunately, this isn't a priority for me right now.

[dmix]

@murarth it might be good to link to this thread in the README. It's not clear it's a dead project besides the 2016 flag.