Beyond FEVM: Additional Runtimes & Native Actors

[Stebalien]

Proposal: Design a light-weight process where users/contributors can propose and deploy WASM smart contracts via network upgrades.

FEVM (EVM on FVM) added one runtime to the Filecoin network, but the goal is to support multiple runtimes allowing users to port existing ecosystems to the Filecoin network.

For example, Fluence would like to deploy AquaVM to bring decentralized compute to the Filecoin network. However, the AquaVM is written in rust and can't easily (or reasonably) be ported to the EVM (it's VM for AIR script and running a VM inside the EVM inside the FVM is just a bit too much).

Ideally, users would be able to directly deploy new runtimes and other WASM actors to the Filecoin network (along with custom addressing schemes, custom account types, new VMs, etc.). Unfortunately, we're not quite there yet. Allowing arbitrary user-deployed WASM actors requires:

1. Hardening & polishing our WASM runtime environment so we don't bake-in any warts.
2. Some additional hardening of the FVM's gas model with respect to potentially malicious WASM.
3. A WASM to native compiler with a predictable runtime (wasmtime is working on one, wasmer already has one).
4. Confidence in all the above.

The FVM devs are hard at work on that first step: hardening & polishing. However, the last three steps will take some time.

This leaves us with two problems:

1. It's difficult to gain that confidence without users actively building & deploying real WASM actors.
2. Neither we nor our users want to wait. We want to start shipping ASAP.

Which brings us back to the proposal at hand. The FVM team would like a light-weight process where:

1. Users (e.g., Fluence) can submit new webassembly actors for consideration by the core developers and wider community. The bar for inclusion would be:
   1. The actor is "generally useful".
   2. The actor is not malicious and/or won't "break the network" (although the core implementers won't make any guarantees that the actor is secure and/or correct).
   3. The actor could otherwise not be implemented as an EVM smart contract. E.g., it needs some capability not available to the EVM and/or it needs performance not achievable in an EVM smart contract.
2. If accepted, the new actor would be included in the next network upgrade.

Such smart contracts would not (IMO) live in the "builtin actors" repo but would live in a separate "contributed actors" repo. I'd expect that the multisig and payment channel actors would, at a minimum, be moved to this separate repo as well (the EVM to follow but that'll take a bit longer).

This is effectively a "soft launch". It allows us to test and harden the FVM ASAP without jumping directly into the deep end.

[ZenGround0]

I agree with everything except the phrasing of this:

The actor is not obviously malicious and/or won't obviously "break the network".

I think we want strong confidence that these actors won't break the network and this phrasing implies we only need weak confidence. I think the FVM + lack of integration of these actors into the builtin actor system probably provides almost all of this strong confidence.

A nice-to-have piece of tooling for this effort would be some kind of deterministic build. I don't think this is strictly necessary -- we can rely on trust instead. But it would be good to point to some public build process that can verify the link between a hash of readable code and the WASM module. This way the FIP process makes it easy to review the exact code that ends up in the contributed actors bundle and verify the exact compiler/other build tools used to make create the bundle. If we had this tool we could then use the same thing for changing builtin actors bundles during network upgrades.

[Stebalien]

You're right. The point I wanted to make is that, e.g., security bugs in the contract are out of scope. I'll update it.

[anorth]

Many people would like to deploy many native actors to Filecoin, if it were possible. I don't think whether or not one is a "runtime" is relevant. At a high level this proposal strikes me as a distraction from just doing the work necessary to support user actors. The existence of a process by which some people could get some subset of actors eventually deployed will remove much of the motivation to do the job properly, and I predict will push out the eventual deployment of that capability.

The proposal adds more responsibility for governance-type things to core devs. They would become even more of a committee deciding who can do what on Filecoin, perceived as limiting innovation and/or picking favourites if their decision was to have any teeth. This is a very slippery slope. And given the low frequency of network upgrades, additional complexity introduced to upgrades, additional governance burden, and process burden on community developers, this proposal is unlikely to birth to any significant innovations. I don't think it's worth the tradeoffs.

While I do not favour this proposal overall, the following would make it less objectionable:

• Remove all reference to any specific parties, or types of actors (e.g. "runtimes"). Motivate it simply as a mechanism to support the deployment of some actors before it can be permissionless, with a mandate to support as many as possible.
• Remove criteria (i) "generally useful". This is very hard to judge and highly subjective (c.f. "useful data"). In many cases the utility or lack thereof could only be known in hindsight. This criteria is a recipe for favouritism and bias. What if it's buggy? What if it's illegal in some jurisdiction? What if it's not illegal, but kinda scammy? Or just misleading? Or just hard to figure out? Or lucrative with a big first-mover advantage? What if I'm being bribed to accept it? Instead, propose that any actor should by default expect to be included unless specific technical concerns cause it to be rejected (and even this is dragon territory).
• Remove criteria (iii) "can't be EVM". This is a flavour of (i) directly biased against people who can't/don't build for EVM. EVM implementers are already advantaged on Filecoin today, while others have no ability to deploy something. This criteria propagates that bias unnecessarily by appealing to a hypothetical implementer who could write EVM code instead of an actual human who writes for WASM (e.g. me). But almost any new actor targeting Filecoin would be better native anyway.
• Remove core devs as having any responsibility, replaced with a group composed of FVM developers (c.f. FIL+ governance). Align the potential governance pain with those who can do the work to remove it, and the network security risk with those most responsible for the attack surface. But still somehow prevent them going slow and/or picking favourites.

If it's not to be a narrow picking of favourites, the other end of the spectrum boils down to an audit committee for user actors, specifically their interactions with the VM. But if "the core implementers won't make any guarantees", who will? We need the FVM to make those guarantees, and when it does everyone can deploy actors.

[Stebalien]

I get where you're coming from, but I disagree strongly:

• I have all the constraints around "generally useful", "not EVM", etc. to reduce the load on core devs. I want to be able to quickly reject proposed actors that could already be implemented with the existing EVM support. I'm using runtimes as an example of where the EVM won't cut it. I'm happy to reword these, but I share your concern that this process could become a major burden if not setup correctly.
• This is not a slippery slope. Anyone can deploy an EVM actor and EVM actors are sufficient for most users. I'm talking about capabilities here, not personal preference; I get not liking solidity.
• While I know you're looking for better performance, there's a very good chance that that better performance will always require some form of human consensus. Native actors get their current performance because:
  • We rely on an optimizing compiler (wasmtime's cranelift) to compile WASM down to native. Optimizing compilers have unpredictable compile times as a general rule. For untrusted user-supplied code, we'd either need to interpret (skip this step) it or compile it with a single-pass compiler. Both of these options will yield significantly worse performance.
  • We rely on the fact that the produced native code executes efficiently. We've tuned our gas model based on the average execution time of our actors, not based on the worst possible program. The worst possible program would cost ~80gas for every instruction that could randomly access memory (20x what we usually charge for instructions) simply because we wouldn't be able to assume caches. Unfortunately, that's most instructions unless we can somehow statically reason about memory locality.
• Regardless of what we do, we need more product feedback. This is a way to collect product feedback while we can still break things.

[anorth]

The narrow/conservative/low effort extreme then is to require a FIP to deploy new actors. Which is a process we already have. I would still avoid documenting an inclusion bar, but this time to admit the core devs authority here (with the mandate to consult and give weight to community input). A common law approach. So could you alternatively motivate this against the existing FIP process?

[Stebalien]

I'm thinking of this as (at least at the beginning) a special case of the FIP process where the FIP author can avoid specifying the actual behavior of the new actor/runtime in-depth beyond general documentation, a link to the code, and a reproducible build path.

The current FIP process requires contributors to describe the change with enough detail such that someone else could, in theory, implement it (although I'll admit that that doesn't always happen). This is important for core components where the core implementers need to review the design in-detail, but less important in this case.

[arajasek]

I'm fairly torn over this discussion, and not especially favourably disposed towards it. Restating what you said, it sounds like our primary motivations are:

• this will help us get more confidence in the FVM's support of native actors over time
• we want support for some such runtimes to start shipping NOW

Re: confidence, I'm only slightly persuaded by this. While this would be a somewhat lower bar than "the deep end", it still feels like any thorns could have fairly serious impact. I defer to you here, but it feels like we should be able to get that increased confidence through other (though likely slower) means.

Re: timing, I'm more open to this argument, though I do think we need to have some process that determines what's "urgent" enough to warrant this special treatment. In this matter I disagree with @anorth fairly strongly, and would want to only include select actors that we agree has some compelling use-case. This is a somewhat tricky position to place the Core Devs in, but I would argue we are already in that position. We prioritize certain changes over others based on our perceived importance all the time, and this follows that pattern (until we get to full support for native actors).

So overall, I'm weakly opposed to this, but will go along with it if we decide it's necessary.

As for more concrete details of the proposal, I am concerned about the integration work being a bit more than expected, because we historically underestimate the work needed for face-level "simple" changes like this one. I'd also like to know more about the long-term strategy for these actors. Would we be eventually deprecating them and migrating them into "real" native actors?

[Stebalien]

Re: confidence, I'm only slightly persuaded by this. While this would be a somewhat lower bar than "the deep end", it still feels like any thorns could have fairly serious impact. I defer to you here, but it feels like we should be able to get that increased confidence through other (though likely slower) means.

This is more about confidence in our actor runtime API choices. E.g.:

1. Right now, we use numbers to reference open IPLD blocks, we may need to use wasm "references" (supports garbage collection).
2. We may need to look harder at the wasm component model.

On the other hand, these are things we likely could work through ourselves. We'll just get better product feedback if we have other users testing things out and pointing out issues. So maybe it's not much of an issue? It's just hard to gain confidence that your product is "correct" if you only have one user.

[boneyard93501]

Just to add a little background, belatedly. Fluence, the decentralized serverless compute protocol, is pushing toward minimal viable mainnet (MVM) on FVM for Q4/23. As part of the MVM milestone, Fluence intends to onboard several Filecoin miners to provide high-quality compute capacity to the Fluence peer-to-peer compute network. Aqua, Fluence's distributed, p2p choreography and composition engine, plays a critical part not only in the execution of deployed compute services but also in the validation, verification and reward distribution to capacity providers.

Specifically, the Aqua Virtual Machine (Aqua VM), which is deployed to every Fluence compute peer to manage peer-based workflow execution, also plays an integral part on-chain in the aforementioned validation, verification and eventual reward distribution of (executed) compute jobs. As such, AVM, as a on-chain Wasm module, is an integral aspect fo the Fluence solution and currently implemented on Near as part of our testnet.

As Fluence is actively looking toward migrating to FVM for a Q4 launch, it critical to begin testing as soon as possible against a feasible, well-known milestone (runtime) target. Hence, @Stebalien 's proposal closely reflects our, and possibly other projects', needs to prepare for successful FVM launches in the very near future and the Fluence community greatly appreciates your support.

[Stebalien]

Could you speak to the requirements of AquaVM and alternatives you've considered? E.g.:

• Why isn't the EVM sufficient for your use-case?
• What performance requirements do you have?
• Could you leverage an L2?

[boneyard93501]

Of course. AquaVM is developed in Rust and compiles to Wasm. While itself a "simple" Wasm module, i.e., no WASM IT or wasi considerations at play, it is still sufficiently complex that a port to EVM is not feasible. That includes but is not limited to AquaVM's reliance on ed25519 which in itself continues to be a bit of an EVM killer.

From a Wasm runtime perspective, we have been successfully using both Wasmer and currently Wasmtime, where I believe Near, see below, uses Wasmer. Moreover, AquaVM is comprised of pure functions only and plays quite nicely within the Wasm sandbox.

As mentioned earlier, AquaVM is currently deployed on Near as part of our testnet, see the contract. Of course, one of the driving factors for Fluence to target and embrace FVM as our mainnet, scheduled to go live Q4 '23, was the prospect of a (L1 native) Wasm runtime.

With respect to alternative on-chain deployment options, we have just very recently started to look at IPC and that may be a possibility. Since the on-chain use of AquaVM drives payment settlements between capacity consumers and providers, i.e. SPs, having AquaVM on L1 does carry some extra merit.

Regarding performance requirements, i don't have the numbers right now but @mikevoronov should be able to add them in short order.

[mikevoronov]

Regarding performance, atm AIR script and supplied data are unbounded, near Q3/Q4 we'll limit them, and I guess that the total time will be up to 50-100ms. But now on average it's about 10-25 ms, we have a benchmark system, and the full report is here.

Basically, AquaVM execution is a graph traversal and the heaviest operation is signatures verifying.

[raulk]

Thanks for opening this discussion, @Stebalien.

I do regard runtimes as a distinct class of actors (from an organizational perspective), in that they summon new developer ecosystems to Filecoin and have multiplicative effects on network activity by offering new deployment hosts for programs that were previously unable to run on the network. That impact alone makes introducing a runtime very different to introducing, say, a hypothetical new market implementation, or a reputation system as built-in actors.

Granted, once we're at the end of the user-deployed Wasm actors roadmap, deploying new runtimes will be permissionless, that will make them fungible with other actors "classes" (technically speaking). But even in that situation, new runtimes are prone to require new syscalls and platform capabilities (e.g. crypto calls, randomness schemes, environmental data, addressing schemes). In other words, I predict these runtimes will be preceded a FIP or series of FIPs anyway to enable building blocks for the runtime. Those FIPs will clearly identify the introduction of the runtime as the change motivator. And this will summon community discussion around runtime fundamentals and its benefits/impact on Filecoin ("why should we do this work if it appears to only enable X runtime?").

In those discussions, the community will likely assess the utility, projections, priority, and in general the fitness of the Filecoin network to host the runtime and the eventual workloads deployed on it (gas, competition, explorer readiness, etc.)

I imagine this event will act as a soft checkpoint to identify network risks (scalability, system message pricing, security, reputational, etc.), preempt projects that may be in the backburner (e.g. gas planes, crypto upgrades, etc.), and it will implicitly act as a gating event.

Given all of this, I think @Stebalien is right in that we should start sharing ideas on how to approach thhe introduction of new runtimes sooner than later.

And part of that discussion is how we can enable runtimes to be deployed ahead of time, in a more permissioned manner during a transition period, in order to keep the innovation cycle short, and to facilitate sandboxed testing.

Some factors I'd consider to make the decision (specifically for AquaVM) -- leaving code audits and other more technical elements aside for a second:

• Is there a clearly articulated promise of value to the Filecoin network?
• Is the runtime's domain synergistic with Filecoin somehow (e.g. compute)? Is this a case where 1+1=3?
• Is there a clear commitment by the sponsors to support, monitor, and evolve the runtime?
• Is there a clear commitment by the sponsors to the Filecoin network and community?
• Do plan to continue upgrading/migrating the runtime as the syscall API changes?
• How is the success of their platform intertwined with the deployment of the runtime on Filecoin? (skin in the game)
• Is the risk profile acceptable for core devs?
• Will the sponsors join Core Devs as observers and potentially as full members in the future?

[raulk]

Note: @Stebalien and I discussed adding an experimental Wasm interpreter runtime as a built-in actor. Precedents already exist for built-in runtimes (EVM runtime). This would enable devs to permissionlessly deploy Wasm actors albeit in a virtualized environment (i.e. not "native"), which should be sufficient to unblock experimental use cases. @Stebalien plans to follow-up with a discussion.

[BlocksOnAChain]

I love this idea, which is already tested in other Web3 ecosystems and has a lot of significant use cases across the space.
An example I have and know that people are using heavily: is https://github.com/paritytech/wasmi, so we should probably look into it... If we decide to explore this route.

[Stebalien]

Forked to #779.

[expede]

@Stebalien We had chatted a bit about this stuff in Lisbon last year, but I just wanted to pipe up and say that IPVM is still interested in integrating :) We're pretty sure that we don't need anything special deployed in order to make this work, but our team is interested in looking at the netlayer & calling convention APIs to see if we can reuse each other's work!

It also may be interesting to explore an efficient UCAN actor to help bridge systems like web3storage directly to/from Filecoin. We had played with this in Cosmos at one stage, and mainly it could be more fuel-efficient than implementing it in e.g. EVM bytecode.

[Stebalien]

See #779 for a discussion on how we're thinking about enabling "native" wasm. For wasm runtimes like the IPVM, we may need to go one step further and provide a way to "link" multiple wasm modules together at runtime.

Are there any IPVM specs for how IPLD access works from Wasm? The FVM does does this by effectively exposing a "blockstore" to the user.

Our "data model" is described here:

• Data model: https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0030.md#ipld-memory-model
• Wasm API: https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0030.md#namespace-ipld
• (new) Rules for what data can/cannot be accessed: https://github.com/filecoin-project/FIPs/blob/5d5bfeb9d8c0e595f7d87e106a51587d112b49f2/FIPS/fip-XXXX.md

[boneyard93501]

@Stebalien @raulk et al.
Thanks for putting this out and the various opportunities to discuss. Very much appreciated.
As a result of the various discussions and faster progression on our ZKP work, we no longer anticipate running AquaVM on (L1) FVM Wasm. Instead, we'll combine a zk proof with L1 FEVM or IPC, if need be.
Again, very much appreciate the opportunity to make our case and looking forward to seeing the evolution of custom actors and IPC.