Aurora Relayer v2

Web3-compatible relayer server for Aurora, implements;

• a JSON-RPC server compatible with Ethereum's Web3 API for Aurora Engine instances deployed on the NEAR Protocol.
• an indexer which continuously reads JSON files generated by Aurora Refiner and populate local embedded storage

This document is useful for developers, contributors and those who would like to build, run and experiment Aurora Relayer v2 natively. For packaged releases please see the distribution project Standalone Aurora Relayer and Refiner which containerizes and packs Aurora Relayer v2 and Aurora Refiner for end users

Prerequisites and Dependencies

• requires Go and Make to build from source
• requires a NEAR account and signing key to start relayer, see Configuration
• requires JSON files, generated by Aurora Refiner, to index blocks
• depends on Aurora Relayer v2 Base library which provides common functionality for public and private Aurora Relayer implementations

How To Build

git clone https://github.com/aurora-is-near/relayer2-public.git
cd relayer2-public
make

build artifacts are placed in out directory

• relayer executable
• config directory
• build.info file

How To Use

Relayer requires a set of configuration files to start (see Configuration for details), and default configuration files for mainnet and testnet are provided under config directory after build process. If config file path is not specified, Relayer starts with config/testnet.yaml by default.

Configuring a signing key

To be able to call eth_sendRawTransaction, you must have a NEAR account and signing key on the network you are relaying to, and said NEAR account must have a sufficient Ⓝ balance to be able to send transactions.

For those who prefer not to use personal accounts, a new signing key can be generated.

Setting the Signer Key Location

Ensure the signer key location is defined in the configuration file: Ex. :

endpoint:
  ...
  engine:
    ...
    signerKey: config/relayer.json

Generating the Signer Key

Execute the following command to generate a new signer key:

./relayer generate-key

Note: The default configuration file used is config/testnet.yaml.

Upon successful execution, a relayer.json file will be generated. An example of its structure is as follows:

{
  "account_id": "703f78e5355ae6fedf6384257a18e4cfb55bada321f6e7a35b1e21a3803b03e0",
  "public_key": "ed25519:8ZAoFCzzj7FwADapMyRCctau295MpvKT5Y1z6cPySCko",
  "secret_key": "ed25519:3j8Rxcnx6BcVvphxJpxKMJGWDfjTrSZTvDx7EdWm7L223dwkK8ZgebXieiadAZ3v5Xfg9AKx4XYsaPPcfmncFNo1"
}

Update the configuration file's signer field with the account_id value from the generated relayer.json file:

endpoint:
  ...
  engine:
    ...
    signer: 703f78e5355ae6fedf6384257a18e4cfb55bada321f6e7a35b1e21a3803b03e0
    signerKey: config/relayer.json

Activating implicit account

In the NEAR Protocol, implicit accounts require activation. This is achieved when a contract-based account or an externally-owned account transfers funds to the implicit account. This step is crucial as the signer account balance is utilized to cover the costs for eth_sendRawTransaction. Utilize any NEAR-compatible wallet or the near send command. Below is an example that transfers 0.5 NEAR to the generated account:

near send myaccount.near 703f78e5355ae6fedf6384257a18e4cfb55bada321f6e7a35b1e21a3803b03e0 0.5

Handling Multiple Access Keys

When dispatching multiple transactions concurrently, each signed by a different EOA, you may encounter the ERR_INCORRECT_NONCE error. The solution is for the relayer to use multiple access keys, signing each transaction with a distinct key.

Generating Multiple Keys

After configuring and activating the access key, execute the following:

./relayer add-keys --config config/mainnet.yaml -n 100

This command generates 100 keys, storing them in the same directory as the config/relayer.json file. Upon restarting, the relayer will leverage these access keys for transaction signing.

Removing Keys

To delete these keys from the account, run:

./relayer delete-keys --config config/mainnet.yaml

You can also use ENV variables to configure the signing account and private key:

Mainnet

export AURORA_RELAYER_ENDPOINT_ENGINE_SIGNER=<signer> # or set in config/mainnet.yaml
export AURORA_RELAYER_ENDPOINT_ENGINE_SIGNERKEY=<path to Near Credentials file> # or set in config/mainnet.yaml
./relayer start --config config/mainnet.yaml

Testnet

export AURORA_RELAYER_ENDPOINT_ENGINE_SIGNER=<signer> # or set in config/testnet.yaml
export AURORA_RELAYER_ENDPOINT_ENGINE_SIGNERKEY=<path to Near Credentials file> # or set in config/testnet.yaml
./relayer start # default config file is config/testnet.yaml

If you're using the NEAR CLI, you will find your signing keys stored as JSON key files under your $HOME/.near-credentials/ directory.

Commands available

command	description
start	starts relayer with default config
version	prints relayer version info
help	prints usage

for detailed explanation and flags of each command see help

./relayer <command> --help

How To Configure

Config Name	Environment Variable Name (prefix AURORA_RELAYER_)	Default	Runtime Change	Description
logger.level	LOGGER_LEVEL	info	✅	7 levels from highest to lowest (panic, fatal, error, warn, info, debug, trace)
logger.filePath	LOGGER_FILEPATH	/tmp/relayer/log/relayer.log	❌	path to log file(s)
logger.logToConsole	LOGGER_LOGTOCONSOLE	false	❌	logs are sent to stdout
logger.logToFile	LOGGER_LOGTOFILE	true	❌	logs are sent to log file, see logger.filePath
db.badger.core.gcIntervalSeconds	BADGER_CORE_GCINTERVALSECONDS	10	❌	BadgerDB garbage collector trigger period
db.badger.core.scanRangeThreshold	BADGER_CORE_SCANRANGETHRESHOLD	2000	❌	max allowed block range for log requests
db.badger.core.maxScanIterators	BADGER_CORE_MAXSCANITERATORS	10000	❌	max logs returned if the scanRangeThreshold exceeded
db.badger.core.filterTtlMinutes	BADGER_CORE_FILTERTTLMINUTES	15	❌	retention time for stored filters (not used)
db.badger.core.options				see BadgerDB for more options
db.badger.core.options.Dir	BADGER_CORE_OPTIONS_DIR	/tmp/relayer/data	❌	see BadgerDB for details
db.badger.core.options.ValueDir	BADGER_CORE_OPTIONS_VALUEDIR	/tmp/relayer/data	❌	see BadgerDB for details
db.badger.core.options.InMemory	BADGER_CORE_OPTIONS_INMEMORY	false	❌	see BadgerDB for details
db.badger.core.options.DetectConflicts	BADGER_CORE_OPTIONS_DETECTCONFLICTS	false	❌	see BadgerDB for details
endpoint.filterFilePath	ENDPOINT_FILTERFILEPATH	config/filter.yaml	❌	path to filter configuration file that specifies whitelist/blacklist based on IP, EOA, CA
endpoint.chainId	ENDPOINT_CHAINID	user should specify ❗	❌	e.g.: mainnet:1313161554, testnet:1313161555
endpoint.engine.nearNetworkID	ENDPOINT_ENGINE_NEARNETWORKID	user should specify ❗	❌	e.g.: mainnet, testnet
endpoint.engine.nearNodeURL	ENDPOINT_ENGINE_NEARNODEURL	user should specify ❗	❌	URL of a Near Node to communicate
endpoint.engine.signer	ENDPOINT_ENGINE_SIGNER	user should specify ❗	❌	to be able to communicate with Near Node, user must have a Near account
endpoint.engine.signerKey	ENDPOINT_ENGINE_SIGNERKEY	user should specify ❗	❌	path to JSON file containing the Near Credentials. To be able to communicate with Near Node, user must have a Near signer key associated to account
endpoint.engine.functionKeyPrefixPattern	ENDPOINT_ENGINE_FUNCTIONKEYPREFIXPATTERN	fk*.	❌	prefix pattern to search for key pair files of Function Call Keys
endpoint.engine.functionKeyMapper	ENDPOINT_ENGINE_FUNCTIONKEYMAPPER	CRC32	❌	hashing algorithm used to map Eth senders to Function Call Keys
endpoint.engine.asyncSendRawTxs	ENDPOINT_ENGINE_ASYNCSENDRAWTXS	true	❌	if true, transaction calls to Near Node are made in async. fashion
endpoint.engine.minGasPrice	ENDPOINT_ENGINE_MINGASPRICE	0	❌
endpoint.engine.minGasLimit	ENDPOINT_ENGINE_MINGASLIMIT	21000	❌
endpoint.engine.gasForNearTxsCall	ENDPOINT_ENGINE_GASFORNEARTXSCALL	300000000000000	❌
endpoint.engine.depositForNearTxsCall	ENDPOINT_ENGINE_DEPOSITEFORNEARTXSCALL	0	❌
endpoint.engine.retryWaitTimeMsForNearTxsCall	ENDPOINT_ENGINE_RETRYWAITTIMEMSFORNEXTTXSCALL	3000	❌
endpoint.engine.retryNumberForNearTxsCall	ENDPOINT_ENGINE_RETRYNUMBERFORNEXTNEARTXSCALL	3	❌
endpoint.eth.protocolVersion	ENDPOINT_ETH_PROTOCOLVERSION	0x41	❌
endpoint.eth.hashrate	ENDPOINT_ETH_HASHRATE	0	❌
endpoint.eth.gasEstimate	ENDPOINT_ETH_GASESTIMATE	0x6691b7	❌
endpoint.eth.gasPrice	ENDPOINT_ETH_GASPRICE	0x42c1d80	❌
endpoint.eth.proxyEndpoints.url	ENDPOINT_ETH_PROXYENDPOINTS_URL	https://testnet.aurora.dev:443	❌
endpoint.eth.proxyEndpoints.endpoints	ENDPOINT_ETH_PROXYENDPOINTS_ENDPOINTS	empty list	✅	e.g.: eth_estimateGas, debug_traceTransaction
endpoint.eth.proxyEndpoints.disabledEndpoints	ENDPOINT_ETH_DISABLEDENDPOINTS	empty list	✅	e.g.: debug_traceTransaction
rpcNode.httpHost	RPCNODE_HTTPHOST	localhost	❌
rpcNode.httpPort	RPCNODE_HTTPPORT	8545	❌
rpcNode.httpPathPrefix	RPCNODE_HTTPPATHPREFIX	"*"	❌	"*" allows any path, please provide specific prefix starting with "/" to limit request URL
rpcNode.httpCors	RPCNODE_HTTPCORS	""	❌
rpcNode.httpCompress	RPCNODE_HTTPCOMPRESS	true	❌
rpcNode.httpTimeout	RPCNODE_HTTPTIMEOUT	300	❌	in seconds
rpcNode.wsHost	RPCNODE_WSHOST	localhost	❌
rpcNode.wsPort	RPCNODE_WSPORT	8545	❌
rpcNode.wsPathPrefix	RPCNODE_WSPATHPREFIX	"*"	❌	"*" allows any path, please provide specific prefix starting with "/" to limit request URL
rpcNode.wsHandshakeTimeout	RPCNODE__WSHANDSHAKETIMEOUT	10	❌	in seconds
rpcNode.maxBatchRequest	RPCNODE_MAXBATCHREQUEST	10000	❌
indexer.sourceFolder	INDEXER_SOURCE_FOLDER	/tmp/relayer/json/	❌
indexer.subFolderBatchSize	INDEXER_SUBFOLDERBATCHSIZE	10000	❌
indexer.keepFiles	INDEXER_KEEPFILES	true	❌	deletes the json files after indexing if set to false
indexer.genesisBlock	INDEXER_GENESISBLOCK	9820210	❌
indexer.fromBlock	INDEXER_FROMBLOCK	9820210	❌
indexer.toBlock	INDEXER_TOBLOCK	0	❌
indexer.retryCountOnFailure	INDEXER_RETRYCOUNTONFAILURE	10	❌
indexer.waitForBlockMs	INDEXER_WAITFORBLOCKMS	500	❌
indexer.forceReindex	INDEXER_FORCEREINDEX	false	❌	re-index any block range defined by fromBlock to toBlock. Upon completion of re-indexing, indexer will continue indexing new blocks
backupIndexer.indexFromBackup	BACKUPINDEXER_INDEXFROMBACKUP	false	❌	enables indexing from backup files for faster indexing if set to true
backupIndexer.backupDir	BACKUPINDEXER_BACKUPDIR		❌	path to the backup files
backupIndexer.backupNamePrefix	BACKUPINDEXER_BACKUPNAMEPREFIX		❌
backupIndexer.from	BACKUPINDEXER_FROM		❌	sets the start block of the backup indexer, starts from begining of the backup if left empty or set to zero
backupIndexer.to	BACKUPINDEXER_TO		❌	sets the end block of the backup indexer, stops at the end of bakcups if left empty or set to zero
prehistoryIndexer.indexFromPrehistory	PREHISTORYINDEXER_INDEXFROMPREHISTORY	false	❌	enables indexing blocks from 0 to the cretion of the Aurora Contract if set to true
prehistoryIndexer.prehistoryHeight	PREHISTORYINDEXER_PREHISTORYHEIGHT		❌	start of the Aurora Contract
prehistoryIndexer.from	PREHISTORYINDEXER_FROM	0	❌	sets the start block of the prehistory indexer
prehistoryIndexer.to	PREHISTORYINDEXER_TO		❌	sets the end block of the prehistory indexer
prehistoryIndexer.batchSize	PREHISTORYINDEXER_BATCHSIZE	10000	❌	controls the read batch size for optimum operation
prehistoryIndexer.archiveURL	PREHISTORYINDEXER_ARCHIVEURL		❌	the Near archive postgres DB info to retrieve the prehistory data, e.g.: postgres://public_readonly:nearprotocol@104.199.89.51/mainnet_explorer

JSON-RPC Methods

Method	Status	Notes
web3_clientVersion	✅
web3_sha3	✅
net_listening	✅
net_peerCount	✅
net_version	✅
eth_accounts	✅
eth_blockNumber	✅
eth_call	✅
eth_chainId	✅
eth_coinbase	✅
eth_compileLLL	❌	Unsupported
eth_compileSerpent	❌	Unsupported
eth_compileSolidity	❌	Unsupported
eth_estimateGas	✅
eth_gasPrice	✅
eth_getBalance	✅
eth_getBlockByHash	✅
eth_getBlockByNumber	✅
eth_getBlockTransactionCountByHash	✅
eth_getBlockTransactionCountByNumber	✅
eth_getCode	✅
eth_getCompilers	✅
eth_getFilterChanges	✅
eth_getFilterLogs	✅
eth_getLogs	✅
eth_getProof	❌	EIP-1186
eth_getStorageAt	✅
eth_getTransactionByBlockHashAndIndex	✅
eth_getTransactionByBlockNumberAndIndex	✅
eth_getTransactionByHash	✅
eth_getTransactionCount	✅
eth_getTransactionReceipt	✅
eth_getUncleByBlockHashAndIndex	✅
eth_getUncleByBlockNumberAndIndex	✅
eth_getUncleCountByBlockHash	✅
eth_getUncleCountByBlockNumber	✅
eth_getWork	❌	Unsupported
eth_hashrate	✅
eth_mining	✅
eth_newBlockFilter	✅
eth_newFilter	✅
eth_newPendingTransactionFilter	✅
eth_pendingTransactions	✅	Undocumented
eth_protocolVersion	✅
eth_sendRawTransaction	✅
eth_sendTransaction	❌	Unsupported
eth_sign	❌	Unsupported
eth_signTransaction	❌	Unsupported
eth_signTypedData	❌	Unsupported
eth_submitHashrate	❌	Unsupported
eth_submitWork	❌	Unsupported
eth_syncing	✅
eth_uninstallFilter	✅
db_getHex	❌	Deprecated
db_getString	❌	Deprecated
db_putHex	❌	Deprecated
db_putString	❌	Deprecated
shh_addToGroup	❌	Discontinued
shh_getFilterChanges	❌	Discontinued
shh_getMessages	❌	Discontinued
shh_hasIdentity	❌	Discontinued
shh_newFilter	❌	Discontinued
shh_newGroup	❌	Discontinued
shh_newIdentity	❌	Discontinued
shh_post	❌	Discontinued
shh_uninstallFilter	❌	Discontinued
shh_version	❌	Discontinued
txpool_content	❌	Unsupported
txpool_inspect	❌	Unsupported
txpool_status	❌	Unsupported
parity_pendingTransactions	✅	Parity extension

Legend: ❌ = not supported. 🚧 = work in progress. ✅ = supported.

Versioning

• 2.0.0 is the initial version of Aurora Relayer v2
• Aurora Relayer v1 is being deprecated, and not source code compatible with this version.
• We are following Go lang's practices and applying semantic versioning vMAJOR[.MINOR[.PATCH[-PRERELEASE][+BUILD]]]
  • MAJOR indicates backward incompatibility
  • MINOR backward compatible and stable release with new features and/or bug fixes
  • PATCH backward compatible and stable release for hot-fixes published out of planned release cycle
  • PRERELEASE does not guarantees stability, mostly used for QA cycles, e.g.: alpha, beta with optional build number

Migration

from v2.0.x to v2.1.x

Relayer2-Public v2.1.0 introduces a new JSON-RPC server. Therefore, for those who already have a native Relayer2-Public with version v2.0.x should update the configuration .yaml file accordingly.

sample v2.0.x config.yaml

rpcNode:
  geth:                    # -> deprecated
    HTTPHost: localhost
    HTTPPort: 8545
    HTTPPathPrefix: ""
    HTTPCors: 
      - "*"
    HTTPVirtualHosts:      # -> deprecated
      - "*"
    HTTPModules:           # -> deprecated
      - "net"
      - "web3"
      - "eth"
      - "parity"
    WSHost: localhost
    WSPort: 8545
    WSPathPrefix: ""
    WSModules:             # -> deprecated
      - "net"
      - "web3"
      - "eth"
      - "parity"

sample v2.1. config.yaml

rpcNode:
  httpHost: localhost
  httpPort: 8545
  httpPathPrefix: "*"
  httpCors: 
    - "*"
  httpCompress: true      # -> new subfield
  httpTimeout: 300        # -> new subfield
  wsHost: localhost
  wsPort: 8545
  wsPathPrefix: "*"
  wsHandshakeTimeout: 10  # -> new subfield
  maxBatchRequests: 1000  # -> new subfield

IMPORTANT Please note that configuration update step is automatically done if you are using distribution project Standalone Aurora Relayer and Refiner.

Contributing