Enable simulated back-end to be used via JSON RPC

[Pet3ris]

Hi there,

This is to continue the discussion in the following thread: #20212. The original thread proposed moving ethapi out of internal but was closed as the problem was solved a different way.

There's another good reason to consider moving ethapi out of internal:

---

I'm trying to use go-ethereum's RPC API over a simulated back-end object as a library. I'm using a simulated back-end for smart contracts from another programming language and would like to have some fallback access to the simulated back-end state through the RPC API rather than having to implement each end-point one by one.

In order to set up the RPC API I'm using NewServer() and using a bi-directional pipe as the codec. The place where this approach fails is that APIs() are only available to full or light ethereum clients, but not the simulated back-end. The simulated back-end exposes GetAPIs() but this function is only available as part of internal/ethapi.

As a result I'm trying to effectively copy & paste the whole internal/api folder in my codebase but running into another related issue that I've filed.

Would love to also hear your guidance on how to achieve this and hope this helps support the overall thinking around the ethapi!

---

To summarize - from my perspective there should be a way to use APIs or GetAPIs with the simulated back-end, I wouldn't actually need the whole ethapi package to be exposed.

[MariusVanDerWijden]

I spent some time thinking about this issue, but I don't think that we should implement it. The SimulatedBackend should imo only be used from go code and I don't think setting up a RPC API on it should be a priority. The sim backend is mainly useful for unit testing in go, for other use cases the --dev mode of ethereum is better suited.

For the simulated backend to be able to provide all necessary functions to create a meaningful RPC API, it needs to implement the ethapi.Backend interface

go-ethereum/internal/ethapi/backend.go

Line 41 in 45cb1a5

type Backend interface {

. This would mean adding a lot more functionality into it, that is not needed by most users, so I would prefer not to do it.

[trajan0x]

Massive +1 on this for e2e testing.

Alternatively, is there a way to run with --dev from go?

[MariusVanDerWijden]

You can start it similar to every other command

// start geth in one thread
go func() {
cmd := exec.Command("geth", "dev", "http.port=8545")
err := cmd.Start()
if err != nil {
	log.Fatal(err)
}
err = cmd.Wait()
}

and then connect to it using the ethclient in another thread

cl, err := ethclient.Dial("127.0.0.1:8545")

(Untested code)

[fjl]

I think what @0xNero was asking is whether there is a way to spin up an RPC server that behaves like geth --dev does though a Go API function. For Go apps and libraries, it would be much more convenient to just import the dev RPC server instead of requiring geth to be installed as a CLI tool.

[nettrino]

Came across this issue as I was trying to see if its possible to run https://github.com/crytic/etheno to capture calls made during tests using a simulated backend, so +1 to be able to capture JSON RPC sequences. Will give @MariusVanDerWijden 's snippet a try but definitely +1 on the feature overall.

[kalverra]

Plus one on this.

I spent some time thinking about this issue, but I don't think that we should implement it. The SimulatedBackend should imo only be used from go code and I don't think setting up a RPC API on it should be a priority. The sim backend is mainly useful for unit testing in go, for other use cases the --dev mode of ethereum is better suited.

For the simulated backend to be able to provide all necessary functions to create a meaningful RPC API, it needs to implement the ethapi.Backend interface

go-ethereum/internal/ethapi/backend.go

Line 41 in 45cb1a5

type Backend interface {

. This would mean adding a lot more functionality into it, that is not needed by most users, so I would prefer not to do it.

I often run into a situation of wrapping eth client methods like so

func WaitForTransactionConfirmed(ctxt context.Context, ethClient *ethclient.Client, txHash common.Hash) (bool, error)

with no real way to unit test them without a --dev node. I could work on it depending on the scope. Not sure if things have changed drastically since you made this comment @MariusVanDerWijden

[MariusVanDerWijden]

You can also start a normal geth node from code like this:

func StartGethNode(filename string) *GethNode {
	// import genesis
	genesis := new(core.Genesis)
	file, err := os.Open(filename)
	if err != nil {
		panic(err)
	}
	if err := json.NewDecoder(file).Decode(genesis); err != nil {
		panic(err)
	}
	// Create node
	n, err := node.New(&node.Config{HTTPPort: 1234, AuthPort: 1235})
	if err != nil {
		panic(err)
	}

	ethcfg := &ethconfig.Config{Genesis: genesis, TrieTimeout: time.Minute, TrieDirtyCache: 256, TrieCleanCache: 256, Miner: miner.Config{GasCeil: 30_000_000}}
	ethservice, err := eth.New(n, ethcfg)
	if err != nil {
		panic(err)
	}
	if err := n.Start(); err != nil {
		panic(err)
	}
	return &GethNode{
		Eth: ethservice,
		api: *catalyst.NewConsensusAPI(ethservice),
	}
}

(You need to provide a genesis file to this method

[Pet3ris]

Better late than never!

[fjl]

I'm actually not sure if the merged PR is entirely what you wanted originally though. @MariusVanDerWijden tagged this issue in his PR, but it doesn't fully match the description here.