From a894263342caf0843547ee9d3ad811accada6431 Mon Sep 17 00:00:00 2001 From: HuangYi Date: Thu, 16 Jun 2022 17:56:33 +0800 Subject: [PATCH 1/4] draft ADR for stateful precompiled contracts ref: #1116 --- docs/architecture/README.md | 1 + .../adr-003-stateful-precompiles.md | 87 +++++++++++++++++++ 2 files changed, 88 insertions(+) create mode 100644 docs/architecture/adr-003-stateful-precompiles.md diff --git a/docs/architecture/README.md b/docs/architecture/README.md index a4ad1404fd..a6de649322 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -36,3 +36,4 @@ Please add a entry below in your Pull Request for an ADR. - [ADR 001: State](adr-001-state.md) - [ADR 002: EVM Hooks](adr-002-evm-hooks.md) +- [ADR 003: Stateful Precompiled Contracts](adr-003-stateful-precompiles.md) diff --git a/docs/architecture/adr-003-stateful-precompiles.md b/docs/architecture/adr-003-stateful-precompiles.md new file mode 100644 index 0000000000..4896e0dd1d --- /dev/null +++ b/docs/architecture/adr-003-stateful-precompiles.md @@ -0,0 +1,87 @@ +# ADR 003: Stateful Precompiled Contracts + +## Changelog + +- 2022-06-16: first draft + +## Status + +DRAFT + +## Abstract + +Support stateful precompiled contracts to improve inter-operabilities between EVM smart contracts and native functionalities. + +## Context + +We need inter-operabilities to allow EVM smart contracts to access native functionalities, like manage native tokens through bank module, send/receive IBC messages through IBC modules. + +The EVM hooks solution is not ideal, because the message processing is asynchronously, so the caller can't get the return value. + +Precompiled contract can provide a much better interface for smart contract developers. But the default precompiled contract implementation in go-ethereum is stateless, we need to do some patches to support stateful ones properly. + +> +> + +## Decision + +### Interface Changes + +Change the `PrecompiledContract` interface like this (need to patch `go-ethereum`): + +``` + type PrecompiledContract interface { + RequiredGas(input []byte) uint64 +- Run(input []byte) ([]byte, error) ++ Run(input []byte, caller common.Address, value *big.Int, readonly bool) ([]byte, error) + } +``` + +There are extra parameters passed to the precompiled contract: + +- `caller`: the address of caller, aka. `msg.sender`. +- `value`: the value of current call, aka. `msg.value`. +- `readonly`: it's set to `true` for both `staticcall` and `delegatecall`, in these call types, the callee contract is not supposed to modify states. A stateful precompiled contract normally should just fail if it's `true`. + +### Snapshot and Revert + +To implement a stateful precompiled contract, one should be aware of the semantics of `StateDB` itself, basically it keeps all the state writes in memory and maintains a list of journal logs for the write operations, and it supports snapshot and revert by undoing the journal logs backward to a certain point in history. + +So the precompiled contract must not write to cosmos-sdk storage directly, because the side effects won't be reverted by `StateDB` when exception happens in smart contract. You should always maintain the dirty states in memory, and append a journal entry to `StateDB` for each modification operation which can undo it when called. + +When reading from cosmos-sdk storage, you are actually reading the committed states, you need to read the in memory caches for the dirty states, for example the accounts and EVM contract storage are cached in `StateDB` itself, and different precompiled contracts may cache different native states. + +### Example + +TODO + +## Consequences + +> This section describes the resulting context, after applying the decision. All consequences should be listed here, not just the "positive" ones. A particular decision may have positive, negative, and neutral consequences, but all of them affect the team and project in the future. + +### Backwards Compatibility + +- State machine breaking +- Need to patch `go-ethereum` + +### Positive + +- Better interface for interaction between EVM contract and native functionalities. + +### Negative + +- Need to patch `go-ethereum`. + +### Neutral + +- Precompiled contract implementation need to be careful with the in memory dirty states. + +## Further Discussions + +## Test Cases [optional] + +Test cases for an implementation are mandatory for ADRs that are affecting consensus changes. Other ADRs can choose to include links to test cases if applicable. + +## References + +- [ADR-001 EVM Hooks](adr-002-evm-hooks.md) From 5439680eeca38334f9fa1189bbac43aed62af21e Mon Sep 17 00:00:00 2001 From: HuangYi Date: Thu, 16 Jun 2022 18:05:55 +0800 Subject: [PATCH 2/4] fixes --- .../adr-003-stateful-precompiles.md | 23 +++++++++---------- 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/docs/architecture/adr-003-stateful-precompiles.md b/docs/architecture/adr-003-stateful-precompiles.md index 4896e0dd1d..8023cd3785 100644 --- a/docs/architecture/adr-003-stateful-precompiles.md +++ b/docs/architecture/adr-003-stateful-precompiles.md @@ -20,37 +20,36 @@ The EVM hooks solution is not ideal, because the message processing is asynchron Precompiled contract can provide a much better interface for smart contract developers. But the default precompiled contract implementation in go-ethereum is stateless, we need to do some patches to support stateful ones properly. -> -> - ## Decision ### Interface Changes -Change the `PrecompiledContract` interface like this (need to patch `go-ethereum`): +Change the `PrecompiledContract` interface like this (need to patch `go-ethereum`): ``` type PrecompiledContract interface { - RequiredGas(input []byte) uint64 -- Run(input []byte) ([]byte, error) -+ Run(input []byte, caller common.Address, value *big.Int, readonly bool) ([]byte, error) + RequiredGas(input []byte) uint64 +- Run(input []byte) ([]byte, error) ++ Run(input []byte, caller common.Address, value *big.Int, readonly bool) ([]byte, error) } ``` There are extra parameters passed to the precompiled contract: -- `caller`: the address of caller, aka. `msg.sender`. -- `value`: the value of current call, aka. `msg.value`. -- `readonly`: it's set to `true` for both `staticcall` and `delegatecall`, in these call types, the callee contract is not supposed to modify states. A stateful precompiled contract normally should just fail if it's `true`. +- `caller`: aka. `msg.sender`. +- `value`: aka. `msg.value`. +- `readonly`: it's set to `true` for both `staticcall` and `delegatecall`, in these call types, the callee contract is not supposed to modify states. A stateful contract normally should just fail if it's `true`. ### Snapshot and Revert -To implement a stateful precompiled contract, one should be aware of the semantics of `StateDB` itself, basically it keeps all the state writes in memory and maintains a list of journal logs for the write operations, and it supports snapshot and revert by undoing the journal logs backward to a certain point in history. +To implement a stateful precompiled contract, one should be aware of the semantics of `StateDB` itself, basically it keeps all the state writes in memory and maintains a list of journal logs for the write operations, and it supports snapshot and revert by undoing the journal logs backward to a certain point in history. The dirty states are written into cosmos-sdk storage when commit at the end of the tx execution. -So the precompiled contract must not write to cosmos-sdk storage directly, because the side effects won't be reverted by `StateDB` when exception happens in smart contract. You should always maintain the dirty states in memory, and append a journal entry to `StateDB` for each modification operation which can undo it when called. +The precompiled contract must not write to cosmos-sdk storage directly, because the side effects can't be reverted by `StateDB` when exception happens. You should always maintain the dirty states in memory, and append a journal entry to `StateDB` for each modification which can undo it when called. When reading from cosmos-sdk storage, you are actually reading the committed states, you need to read the in memory caches for the dirty states, for example the accounts and EVM contract storage are cached in `StateDB` itself, and different precompiled contracts may cache different native states. +It's also tricky if not impossible to let two precompiled contracts to write to the same piece of native states, because their in-memory states would be in conflicts. + ### Example TODO From 57b6ad31799a4c8c4efe6564f106347e0d931f87 Mon Sep 17 00:00:00 2001 From: HuangYi Date: Wed, 6 Jul 2022 11:28:21 +0800 Subject: [PATCH 3/4] add example --- .../adr-003-stateful-precompiles.md | 115 +++++++++++++++++- 1 file changed, 110 insertions(+), 5 deletions(-) diff --git a/docs/architecture/adr-003-stateful-precompiles.md b/docs/architecture/adr-003-stateful-precompiles.md index 8023cd3785..c30a752a0b 100644 --- a/docs/architecture/adr-003-stateful-precompiles.md +++ b/docs/architecture/adr-003-stateful-precompiles.md @@ -30,7 +30,7 @@ Change the `PrecompiledContract` interface like this (need to patch `go-ethereum type PrecompiledContract interface { RequiredGas(input []byte) uint64 - Run(input []byte) ([]byte, error) -+ Run(input []byte, caller common.Address, value *big.Int, readonly bool) ([]byte, error) ++ Run(evm *vm.EVM, input []byte, caller common.Address, value *big.Int, readonly bool) ([]byte, error) } ``` @@ -52,7 +52,109 @@ It's also tricky if not impossible to let two precompiled contracts to write to ### Example -TODO +```golang +// ExtState manage in memory dirty states which are committed together with StateDB +type ExtState interface { + Commit(sdk.Context) error +} + +// ExtStateDB expose `AppendJournalEntry` api on top of `vm.StateDB` interface +type ExtStateDB interface { + AppendJournalEntry(statedb.JournalEntry) +} + +// BankContract expose native token functionalities to EVM smart contract +type BankContract struct { + ctx sdk.Context + bankKeeper types.BankKeeper + balances map[common.Address]map[common.Address]*Balance +} + +func (bc *BankContract) RequiredGas(input []byte) uint64 { + // TODO estimate required gas + return 0 +} + +func (bc *BankContract) Run(evm *vm.EVM, input []byte, caller common.Address, value *big.Int, readonly bool) ([]byte, error) { + stateDB, ok := evm.StateDB.(ExtStateDB) + if !ok { + return nil, errors.New("not run in ethermint") + } + + // parse input + methodID := input[:4] + if bytes.Equal(methodID, MintMethod.ID) { + if readonly { + return nil, errors.New("the method is not readonly") + } + args, err := MintMethod.Inputs.Unpack(input[4:]) + if err != nil { + return nil, errors.New("fail to unpack input arguments") + } + recipient := args[0].(common.Address) + amount := args[1].(*big.Int) + if amount.Sign() <= 0 { + return nil, errors.New("invalid amount") + } + + if _, ok := bc.balances[caller]; !ok { + bc.balances[caller] = make(map[common.Address]*Balance) + } + balances := bc.balances[caller] + if balance, ok := balances[recipient]; ok { + balance.DirtyAmount = new(big.Int).Add(balance.DirtyAmount, amount) + } else { + // query original amount + addr := sdk.AccAddress(recipient.Bytes()) + originAmount := bc.bankKeeper.GetBalance(bc.ctx, addr, EVMDenom(caller)).Amount.BigInt() + dirtyAmount := new(big.Int).Add(originAmount, amount) + balances[recipient] = &Balance{ + OriginAmount: originAmount, + DirtyAmount: dirtyAmount, + } + } + stateDB.AppendJournalEntry(bankMintChange{bc: bc, caller: caller, recipient: recipient, amount: amount}) + } else if bytes.Equal(methodID, BalanceOfMethod.ID) { + args, err := BalanceOfMethod.Inputs.Unpack(input[4:]) + if err != nil { + return nil, errors.New("fail to unpack input arguments") + } + token := args[0].(common.Address) + addr := args[1].(common.Address) + if balances, ok := bc.balances[token]; ok { + if balance, ok := balances[addr]; ok { + return BalanceOfMethod.Outputs.Pack(balance.DirtyAmount) + } + } + // query from storage + amount := bc.bankKeeper.GetBalance(bc.ctx, sdk.AccAddress(addr.Bytes()), EVMDenom(token)).Amount.BigInt() + return BalanceOfMethod.Outputs.Pack(amount) + } else { + return nil, errors.New("unknown method") + } + return nil, nil +} + +func (bc *BankContract) Commit(ctx sdk.Context) error { + // Write the dirty balances through bc.bankKeeper +} + +type bankMintChange struct { + bc *BankContract + caller common.Address + recipient common.Address + amount *big.Int +} + +func (ch bankMintChange) Revert(*statedb.StateDB) { + balance := ch.bc.balances[ch.caller][ch.recipient] + balance.DirtyAmount = new(big.Int).Sub(balance.DirtyAmount, ch.amount) +} + +func (ch bankMintChange) Dirtied() *common.Address { + return nil +} +``` ## Consequences @@ -73,13 +175,16 @@ TODO ### Neutral -- Precompiled contract implementation need to be careful with the in memory dirty states. +- Precompiled contract implementation need to be careful with the in memory dirty states to maintain the invariants. ## Further Discussions -## Test Cases [optional] +## Test Cases -Test cases for an implementation are mandatory for ADRs that are affecting consensus changes. Other ADRs can choose to include links to test cases if applicable. +- Check the state is persisted after tx committed. +- Check exception revert works. +- Check multiple precompiled contracts don't intervene each other. +- Check static call and delegate call don't mutate states. ## References From 84abc6005934cd39a5022585e1d82cfcffa70c3c Mon Sep 17 00:00:00 2001 From: yihuang Date: Thu, 1 Sep 2022 10:20:00 +0800 Subject: [PATCH 4/4] Update docs/architecture/adr-003-stateful-precompiles.md --- docs/architecture/adr-003-stateful-precompiles.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/architecture/adr-003-stateful-precompiles.md b/docs/architecture/adr-003-stateful-precompiles.md index c30a752a0b..37d723a188 100644 --- a/docs/architecture/adr-003-stateful-precompiles.md +++ b/docs/architecture/adr-003-stateful-precompiles.md @@ -172,6 +172,7 @@ func (ch bankMintChange) Dirtied() *common.Address { ### Negative - Need to patch `go-ethereum`. +- Don't work well with external EVM implementations if we'll support them through evmc interface, unless we patch them all somehow. ### Neutral