Skip to content

Commit ab1cd0c

Browse files
cloudgrayAlex | Interchain Labs
cloudgray
and
Alex | Interchain Labs
authored
test(jsonrpc): add jsonrpc compatibility test (cosmos#419)
* test(rpc-compat): add test simulator * refactor(tests): jsonrpc test * test(jsonprc): refactor test suite * test(jsonrpc): refactor test suite * test(jsonrpc): refactor test suite * test(jsonrpc): add network start/stop scripts of evmd and geth * test(jsonrpc): fix api counting in report * test(jsonrpc): refactor result format * test(jsonrpc): refactor * test(jsonrpc): add state generator * test(jsonrpc): refactor simulator * test(jsonrpc): refactor * test(jsonrpc): setup geth dev mode * test(jsonrpc): clean unused code * test(jsonrpc): setup initial state * test(jsonrpc): add tx metadata for test * WIP: test(jsonrpc): fix some failing tests of eth_ apis * test(jsonrpc): fix eth test * WIP: test(jsonrpc): fix debug api test cases * test(jsonrpc): set pruning config as nothing for testing * test(jsonrpc): enhance debug testing * test(jsonrpc): fix net test case * chore: update working history * test(jsonrpc): skip personal apis * test(jsonrpc): add websocket api test * jsonrpc(test): fix personal apis * test(jsonrpc): add missing debug apis * test(jsonrpc): update working history * test(jsonrpc): fix eth_sign / eth_signTransaction test * test(jsonrpc): refactor simulator * test(jsonrpc): refactor simulator * test(jsonrpc): refactor simulator * test(jsonrpc): refactor simulator * test(jsonrpc): refactor simulator * test(jsonrpc): refactor simulator * test(jsonrpc): refactor simulator * test(jsonrpc): update working history * WIP: add type check between responses from evmd and geth * WIP: add type check between responses from evmd and geth * WIP: test(jsonrpc): add type check between responses from evmd and geth * test(jsonrpc): modify test * test(jsonrpc): add debug apis * test(jsonrpc): fix failing tests * test(jsonrpc): fix failing test * test(jsonrpc): fix test * test(jsonrpc): fix test * test(jsonrpc): remove unnecessary files * chore: add compatibility test github ci job * chore: update github-ci job * chore: fix jsonrpc-compatibility test githib-ci job * test(jsonrpc): refactor test suite * test(jsonrpc): refactor test suite * test(jsonrpc): refactor test suite * test(jsonrpc): refactor * test(jsonrpc): refactor test suite * test(jsonrpc): refactor test suite * test(jsonrpc): fix github-ci * test(jsonrpc): refactor test suite * fix github-ci * fix github-ci * fix github-ci * fix: github-ci * fix: github-ci * test(jsonrpc): add eth tests * WIP: chore: debugging github-ci failure * fix: github-ci * test(jsonrpc): add missing debug tests * fix github-ci * test(jsonrpc): add eth_pendingtransaction test * fix: github-ci * test(jsonrpc): add missing debug tests * Revert "test(jsonrpc): add missing debug tests" This reverts commit 790cce7. * Revert "fix: github-ci" This reverts commit b3a777a. * fix: github-ci * fix: github-ci * WIP: fix: github-ci * WIP: fix github0-ci * test(jsonrpc): refactor test suite * fix: github-ci * chore: add debug code for github-ci * chore: add debug code for github-ci * fix: github-ci * fix: github-ci * test(jsonrpc): refactor test suite * test(jsonrpc): fix github-ci * fix: github-ci * fix: github-ci * fix: github-ci * fix: github-ci * fix: github-ci * fix: github-ci * fix github-ci * test(jsonrpc): fix eth_getProof api * fix eth_getProof api * fix: github-ci * fix: github-ci * fix: github-ci * test(jsonrpc): fix api test * fix: github-ci * fix: github-ci * fix: github-ci * fix: github-ci * fix: github-ci * fix: github-ci * test(jsonrpc): modify test script * fix: github-ci * test(jsonrpc): refactor test suite * test(jsonrpc): refactor test suite * test(jsonrpc): fix websocket api tests * chore: update README.md for jsonrpc test * chore: remove binary file --------- Co-authored-by: Alex | Interchain Labs <alex@interchainlabs.io>
1 parent b9bdbe6 commit ab1cd0c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+8295
-2
lines changed

‎.github/workflows/jsonrpc-compatibility.yml‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
name: JSON-RPC Compatibility Test
2+
on:
3+
pull_request:
4+
paths:
5+
- '.github/workflows/jsonrpc-compatibility.yml'
6+
- 'tests/jsonrpc/**'
7+
- 'rpc/**'
8+
- 'evmd/**'
9+
- '**/**.go'
10+
- 'go.mod'
11+
- 'go.sum'
12+
- 'evmd/go.mod'
13+
- 'evmd/go.sum'
14+
merge_group:
15+
push:
16+
branches:
17+
- main
18+
- release/**
19+
permissions: read-all
20+
21+
jobs:
22+
cleanup-runs:
23+
runs-on: ubuntu-latest
24+
steps:
25+
- uses: rokroskar/workflow-run-cleanup-action@master
26+
env:
27+
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
28+
if: ${{ !startsWith(github.ref, 'refs/tags/') && github.ref != 'refs/heads/main' }}
29+
30+
jsonrpc-compatibility-test:
31+
runs-on: depot-ubuntu-22.04-8
32+
timeout-minutes: 45
33+
steps:
34+
- uses: actions/setup-go@v5
35+
with:
36+
go-version: "1.22"
37+
check-latest: true
38+
39+
- uses: actions/checkout@v4
40+
41+
- uses: technote-space/get-diff-action@v6.1.2
42+
with:
43+
PATTERNS: |
44+
.github/workflows/jsonrpc-compatibility.yml
45+
tests/jsonrpc/**
46+
rpc/**
47+
evmd/**
48+
**/**.go
49+
go.mod
50+
go.sum
51+
evmd/go.mod
52+
evmd/go.sum
53+
54+
- name: Set up Docker
55+
uses: docker/setup-buildx-action@v3
56+
if: ${{ env.GIT_DIFF }}
57+
58+
- name: Run JSON-RPC compatibility tests
59+
run: |
60+
echo "🧪 Running JSON-RPC compatibility tests using containerized environment..."
61+
62+
# Run the containerized test suite
63+
make test-rpc-compat > test_output.log 2>&1 || {
64+
echo "⚠️ Test exited with non-zero code, checking results..."
65+
}
66+
67+
# Display the test output
68+
cat test_output.log
69+
70+
# Check for failure criterion: count of "Failed" tests > 0
71+
failed_count=$(grep -o "Failed:.*[0-9]" test_output.log | grep -o "[0-9]\+" | head -1 || echo "0")
72+
73+
echo "📊 Test Results Summary:"
74+
echo " Failed tests: $failed_count"
75+
76+
if [ "$failed_count" -gt 0 ]; then
77+
echo "❌ CI FAILURE: Found $failed_count failed tests"
78+
echo " Criterion: CI fails when any test has status 'Failed' (not 'Not Implemented', 'Legacy', or 'Skipped')"
79+
exit 1
80+
else
81+
echo "✅ CI PASS: No failed tests found"
82+
echo " Note: 'Not Implemented', 'Legacy', and 'Skipped' tests are acceptable and don't cause CI failure"
83+
fi
84+
if: ${{ env.GIT_DIFF }}

‎Makefile‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -370,7 +370,13 @@ localnet-stop:
370370
localnet-start: localnet-stop localnet-build-env localnet-build-nodes
371371

372372

373-
.PHONY: localnet-start localnet-stop localnet-build-env localnet-build-nodes
373+
test-rpc-compat:
374+
@./tests/jsonrpc/scripts/run-compat-test.sh
375+
376+
test-rpc-compat-stop:
377+
cd tests/jsonrpc && docker compose down
378+
379+
.PHONY: localnet-start localnet-stop localnet-build-env localnet-build-nodes test-rpc-compat test-rpc-compat-stop
374380

375381
test-system: build
376382
ulimit -n 1300

‎contrib/images/evmd-env/Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,4 +39,4 @@ RUN chown -R nonroot:nonroot /evmd
3939
USER nonroot
4040

4141
ENTRYPOINT ["/usr/bin/wrapper.sh"]
42-
CMD ["start", "--log_format", "plain", "--minimum-gas-prices", "0.0001atest", "--json-rpc.api", "eth,txpool,personal,net,debug,web3", "--chain-id", "local-4221"]
42+
CMD ["start", "--log_format", "plain", "--minimum-gas-prices", "0.0001atest", "--json-rpc.enable", "true", "--json-rpc.api", "eth,txpool,personal,net,debug,web3", "--chain-id", "local-4221"]

‎tests/jsonrpc/.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
.evmd/
2+
.geth-data/

‎tests/jsonrpc/Dockerfile‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Dockerfile for JSON-RPC Compatibility Test Simulator
2+
FROM golang:1.24-alpine AS builder
3+
4+
# Install dependencies
5+
RUN apk add --no-cache git
6+
7+
# Set working directory
8+
WORKDIR /workspace
9+
10+
# Copy go mod files
11+
COPY go.mod go.sum ./
12+
RUN go mod download
13+
14+
# Copy the entire source code
15+
COPY . .
16+
17+
# Build the simulator
18+
WORKDIR /workspace/tests/jsonrpc/simulator
19+
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o simulator .
20+
21+
# Final stage
22+
FROM alpine:latest
23+
24+
# Install ca-certificates and curl for debugging
25+
RUN apk --no-cache add ca-certificates curl
26+
27+
# Set working directory
28+
WORKDIR /app
29+
30+
# Copy the binary
31+
COPY --from=builder /workspace/tests/jsonrpc/simulator/simulator .
32+
33+
# Copy contract files needed at runtime
34+
COPY --from=builder /workspace/tests/jsonrpc/simulator/contracts ./contracts
35+
36+
# Make it executable
37+
RUN chmod +x simulator
38+
39+
# Default command
40+
CMD ["./simulator"]

‎tests/jsonrpc/README.md‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
# JSON-RPC Compatibility Testing
2+
3+
This directory contains tools and scripts for testing JSON-RPC API compatibility between Cosmos EVM and Ethereum clients.
4+
5+
## Quick Start
6+
7+
```bash
8+
# From project root
9+
make test-rpc-compat
10+
```
11+
12+
## Test Guide
13+
14+
### 1. Build EVMD Docker Image
15+
16+
```bash
17+
# From project root
18+
make localnet-build-env
19+
```
20+
21+
### 2. Start Nodes
22+
23+
```bash
24+
# Start evmd with JSON-RPC enabled
25+
./tests/jsonrpc/scripts/evmd/start-evmd.sh
26+
27+
# Start geth for comparison
28+
./tests/jsonrpc/scripts/geth/start-geth.sh
29+
30+
# Or start both at once
31+
./tests/jsonrpc/scripts/start-networks.sh
32+
```
33+
34+
### 3. Run Compatibility Tests
35+
36+
```bash
37+
# Use the simulator for comprehensive testing
38+
cd tests/jsonrpc/simulator
39+
go build .
40+
./simulator
41+
```
42+
43+
### 4. Stop Nodes
44+
45+
```bash
46+
# Stop evmd
47+
./tests/jsonrpc/scripts/evmd/stop-evmd.sh
48+
49+
# Stop geth
50+
./tests/jsonrpc/scripts/geth/stop-geth.sh
51+
52+
# Or stop both at once
53+
./tests/jsonrpc/scripts/stop-networks.sh
54+
```
55+
56+
## Available Endpoints
57+
58+
### evmd Endpoints
59+
60+
- **JSON-RPC**: http://localhost:8545
61+
- **WebSocket**: ws://localhost:8546
62+
- **Cosmos REST**: http://localhost:1317
63+
- **Tendermint RPC**: http://localhost:26657
64+
- **gRPC**: localhost:9090
65+
66+
### geth Endpoints
67+
68+
- **JSON-RPC**: http://localhost:8547
69+
- **WebSocket**: ws://localhost:8548
70+
71+
## Scripts Structure
72+
73+
### `scripts/evmd/`
74+
75+
- `start-evmd.sh` - Initialize and start single-node evmd for testing
76+
- `stop-evmd.sh` - Stop the evmd testing node
77+
78+
### `scripts/geth/`
79+
80+
- `start-geth.sh` - Start geth node using ethereum/client-go:v1.15.10
81+
- `stop-geth.sh` - Stop the geth testing node
82+
83+
### `scripts/`
84+
85+
- `start-both.sh` - Start both evmd and geth nodes
86+
- `stop-both.sh` - Stop both nodes
87+
88+
## Testing with Simulator
89+
90+
The simulator in `./simulator/` is the primary tool for comprehensive compatibility testing:
91+
92+
```bash
93+
cd tests/jsonrpc/simulator
94+
go build .
95+
./simulator
96+
```
97+
98+
## Configuration
99+
100+
The scripts use the following defaults:
101+
102+
### evmd Configuration
103+
104+
- Container name: `evmd-jsonrpc-test`
105+
- Chain ID: `local-4221`
106+
- Validator count: 1
107+
- Data directory: `tests/jsonrpc/.evmd`
108+
109+
### geth Configuration
110+
111+
- Container name: `geth-jsonrpc-test`
112+
- Chain ID: 1337 (dev mode)
113+
- Data directory: `tests/jsonrpc/.geth-data`
114+
115+
## Troubleshooting
116+
117+
### Container fails to start
118+
119+
- Check if the Docker image was built: `docker images | grep cosmos/evmd`
120+
- Check container logs: `docker logs evmd-jsonrpc-test`
121+
122+
### JSON-RPC not responding
123+
124+
- Verify the container is running: `docker ps | grep evmd-jsonrpc-test`
125+
- Check if ports are bound: `docker port evmd-jsonrpc-test`
126+
- Test with curl: `curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}' http://localhost:8545`

‎tests/jsonrpc/docker-compose.yml‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
version: "3"
2+
3+
services:
4+
evmd:
5+
container_name: evmd-compat-test
6+
image: "cosmos/evmd"
7+
privileged: true
8+
user: root
9+
environment:
10+
- DEBUG=0
11+
- ID=0
12+
- LOG=${LOG:-evmd.log}
13+
- EVMDHOME=/data/node0/evmd
14+
cap_add:
15+
- SYS_PTRACE
16+
security_opt:
17+
- seccomp:unconfined
18+
ports:
19+
- "8545:8545"
20+
- "8546:8546"
21+
volumes:
22+
- ./.evmd:/data:Z
23+
- ./scripts/evmd/container-start-evmd.sh:/container-start-evmd.sh:ro
24+
networks:
25+
- jsonrpc-test
26+
entrypoint: ["/bin/sh", "-c"]
27+
command: ["/container-start-evmd.sh"]
28+
healthcheck:
29+
test: ["CMD", "sh", "-c", "wget --no-verbose --tries=1 --timeout=5 --post-data='{\"jsonrpc\":\"2.0\",\"method\":\"eth_chainId\",\"params\":[],\"id\":1}' --header='Content-Type: application/json' -O- http://localhost:8545 | grep -q result || exit 1"]
30+
interval: 10s
31+
timeout: 10s
32+
retries: 8
33+
start_period: 45s
34+
35+
geth:
36+
container_name: geth-compat-test
37+
image: "ethereum/client-go:latest"
38+
privileged: true
39+
ports:
40+
- "8547:8545"
41+
- "8548:8546"
42+
networks:
43+
- jsonrpc-test
44+
command: >
45+
--dev
46+
--dev.period 1
47+
--http
48+
--http.addr 0.0.0.0
49+
--http.port 8545
50+
--http.api eth,net,web3,personal,admin,debug,miner,txpool
51+
--http.corsdomain "*"
52+
--http.vhosts "*"
53+
--ws
54+
--ws.addr 0.0.0.0
55+
--ws.port 8546
56+
--ws.api eth,net,web3,personal,admin,debug,miner,txpool
57+
--ws.origins "*"
58+
--allow-insecure-unlock
59+
--rpc.allow-unprotected-txs
60+
--nodiscover
61+
healthcheck:
62+
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:8545"]
63+
interval: 10s
64+
timeout: 5s
65+
retries: 5
66+
start_period: 20s
67+
68+
simulator:
69+
container_name: simulator-compat-test
70+
privileged: true
71+
build:
72+
context: ../../
73+
dockerfile: tests/jsonrpc/Dockerfile
74+
depends_on:
75+
evmd:
76+
condition: service_healthy
77+
geth:
78+
condition: service_healthy
79+
restart: on-failure
80+
networks:
81+
- jsonrpc-test
82+
environment:
83+
- EVMD_URL=http://evmd:8545
84+
- EVMD_WS_URL=ws://evmd:8546
85+
- GETH_URL=http://geth:8545
86+
- GETH_WS_URL=ws://geth:8546
87+
volumes:
88+
- ./simulator:/app/results:Z
89+
command: ["sh", "-c", "sleep 5 && ./simulator"]
90+
91+
networks:
92+
jsonrpc-test:
93+
driver: bridge

0 commit comments

Comments
 (0)