A simple Starknet RPC benchmarking tool

Table of content

• How it works
• Dependencies
• Running
• Benchmarks

How it works

MADARA bench runs various Starknet RPC nodes in isolated, resource-constrained containers for testing. These nodes are automatically set up and have their RPC endpoints opened up for you to test using an online api (served with FastAPI).

Dependencies

Tip

If you are using Nixos or the nix package manager, you can skip to running (you will still need to specify secrets).

MADARA bench currently only supports running on linux and requires docker and docker-compose to be installed on the host system.

Step 1: installing build-essential

This is needed by certain python packages

sudo apt update
sudo apt install build-essential

Step 2: installing python 3.12

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install -y python3.12 python3.12-venv

Step 3: installing poetry (python package manager)

official instructions

curl -sSL https://install.python-poetry.org | python3 -

Step 4: specifying secrets

The following are required for MADARA bench to start:

echo abc > secrets/gateway_key.secret
echo https://eth-sepolia.g.alchemy.com/abc > secrets/rpc_api.secret
echo wss://eth-sepolia.g.alchemy.com/abc > secrets/rpc_api_ws.secret

You can get an RPC api key frm Alchemy, Infura or BlastAPI, amongst others. gateway_key is a special key which has been given out to node development teams in the ecosystem and is used to bypass sequencer rate limits for testing purposes. It is not available publicly.

Important

By default, MADARA bench runs on starknet testnet, and you will need your RPC api keys to point to ethereum sepolia for this to work.

Running

To start MADARA bench, run the following command:

make start

This will automatically build docker images for each RPC node, create individual volumes for each databases, start the nodes and serve a FastAPI endpoint at 0.0.0.0:8000/docs.

Warning

If this is the first time you are running MADARA bench, sit back and grab a cup of coffee as building the node images for the first time can take a while.

To stop MADARA bench, run:

make stop

You can also get a list of all available commands by running:

make help

---

Nix

If you are using Nixos or the nix package manager, you do not need to install any dependencies and can instead just run:

nix develop --extra-experimental-features "nix-command flakes" .#start

This will download every dependency into a development shell, independent of the rest of your system and start MADARA bench. This is the preferred way of running MADARA bench and will also handle auto-closing docker containers for you.

Benchmarks

Once you have started MADARA bench, start by heading to your FastAPI endpoint. There you will see multiple sections:

• bench: run system and RPC benchmarks
• system: display system metrics (CPU, memory and storage usage)
• read: query individual RPC methods on each node
• trace: run tracing RPC calls on each node
• debug: display useful extra information

RPC benchmarks are procedural, that is to say inputs are generated automatically as the chain keeps making progress. This way, you do not need to worry about passing valid or up-to-date parameters, you can just focus on benchmarking.

Note

When needed, RPC method inputs are generated by sampling from a random point in the last 1000 blocks of the chain. For a more concrete example of how this works, check generators.py.

Currently, only a few RPC methods are supported for benchmarking, with more to be added in the future. You can request coverage of new RPC methods in an issue.