finish ethereum browser documentation and updated setup

docs/docs/ledger-browser/plugin-apps/ethereum-browser.md

@@ -0,0 +1,61 @@
+# Ethereum Browser App
+
+Application for browsing ledger state stored in a database by the Cacti Ethereum Persistence Plugin.
+
+## Features
+- Browse ledger blocks and transactions.
+- View account transaction history.
+- See ERC20 and ERC721 tokens owned by a specified account.
+
+## Setup
+
+### Persistence Plugin
+
+#### Supabase
+
+The persistence plugin requires a Supabase instance to save ledger data. You can use the same Supabase instance as for the GUI (but in a separate schema), or create a separate instance specifically for this plugin.
+
+To set up the GUI app, you'll need a `Supabase URL`, `API key`, and the `Schema` under which the data resides in the database.
+
+Additionally, you'll need a `PostgreSQL connection string` to start the persistence plugin.
+
+#### Ethereum Ledger (Optional)
+
+This step is optional as you can use any running Ethereum ledger. However, for testing purposes, you may use our [geth-all-in-one](../../../../tools/docker/geth-all-in-one/README.md). To start it, execute the following commands from the root of your project:
+
+```shell
+# Build
+docker build ./tools/docker/geth-all-in-one/ -t cactus_geth_all_in_one
+
+# Run
+docker run --rm --name geth_aio_testnet --detach -p 8545:8545 -p 8546:8546 cactus_geth_all_in_one
+```
+
+### Persistence Plugin
+
+Follow the instructions in the [plugin README file](../../../../packages/cactus-plugin-persistence-ethereum/README.md).
+
+To quickly set up the plugin for your Ethereum ledger, run the sample setup script:
+
+```shell
+# Replace the environment variables with JSON-RPC WS url to your ledger and postgresql connection string to your database instance.
+ETHEREUM_RPC_WS_HOST=ws://127.0.0.1:8546 SUPABASE_CONNECTION_STRING=postgresql://postgres:your-super-secret-and-long-postgres-password@127.0.0.1:5432/postgres npm run sample-setup
+```
+
+## Configuration
+- `supabaseUrl`: URL of your Supabase instance.
+- `supabaseKey`: Supabase API key.
+- `supabaseSchema`: Database schema under which Ethereum persistence tables were created.
+
+### Sample Configuration
+
+Uses a localhost `supabase-all-in-one` instance with data stored in the `ethereum` schema.
+
+```json
+{
+ supabaseUrl: "http://localhost:8000",
+ supabaseKey:
+ "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyAgCiAgICAicm9sZSI6ICJhbm9uIiwKICAgICJpc3MiOiAic3VwYWJhc2UtZGVtbyIsCiAgICAiaWF0IjogMTY0MTc2OTIwMCwKICAgICJleHAiOiAxNzk5NTM1NjAwCn0.dc_X5iR_VP_qT0zsiyj_I_OZ2T9FtRU2BBNWN8Bu4GE",
+ supabaseSchema: "ethereum",
+}
+```

packages/cactus-plugin-persistence-ethereum/README.md

@@ -26,46 +26,75 @@ Clone the git repository on your local machine. Follow these instructions that w
 
 ### Prerequisites
 
+#### Build
+
 In the root of the project, execute the command to install and build the dependencies. It will also build this persistence plugin:
 
 ```sh
 yarn run configure
 ```
 
-### Usage
+#### Ethereum Ledger and Connector
 
-Instantiate a new `PluginPersistenceEthereum` instance:
+This plugin requires a running Ethereum ledger that you want to persist to a database. For testing purposes, you can use our [test geth-all-in-one Docker image](../../tools/docker/geth-all-in-one/README.md). Make sure you have the JSON-RPC WS address ready.
 
-```typescript
-import { PluginPersistenceEthereum } from "@hyperledger/cactus-plugin-persistence-ethereum";
-import { v4 as uuidv4 } from "uuid";
+Once you have an Ethereum ledger ready, you need to start the [Ethereum Cacti Connector](../cactus-plugin-ledger-connector-ethereum/README.md). We recommend running the connector on the same ApiServer instance as the persistence plugin for better performance and reduced network overhead. See the connector package README for more instructions, or check out the [setup sample scripts](./src/test/typescript/manual).
 
-const persistencePlugin = new PluginPersistenceEthereum({
- instanceId: uuidv4(),
- apiClient: new SocketIOApiClient(apiConfigOptions),
- logLevel: "info",
- connectionString:
- "postgresql://postgres:your-super-secret-and-long-postgres-password@localhost:5432/postgres",
-});
+#### Supabase Instance
 
-// Initialize the connection to the DB
-await persistencePlugin.onPluginInit();
+You need a running Supabase instance to serve as a database backend for this plugin. See the ['Setup Supabase' section of the ledger browser documentation](../../docs/docs/ledger-browser/setup.md) for detailed instructions. Make sure you have the PostgreSQL connection string ready.
+
+### Setup Tutorials
+
+We've created some sample scripts to help you get started quickly. All the steps have detailed comments on it so you can quickly understand the code.
+
+#### Sample Setup
+
+Location: [./src/test/typescript/manual/sample-setup.ts](./src/test/typescript/manual/sample-setup.ts)
+
+This sample script can be used to set up `ApiServer` with the Ethereum connector and persistence plugins to monitor and store ledger data in a database. You need to have a ledger running before executing this script. You can add custom code (e.g., to specify tokens to be monitored) after the comment `CUSTOM CODE GOES HERE !!!!` in the script file.
+
+By default, the script will try to use a localhost Ethereum ledger (`ws://127.0.0.1:8546`) and our `supabase-all-in-one` instance running on localhost.
+
+```shell
+npm run sample-setup
+```
+
+Custom ledger and supabase can be set with environment variables `ETHEREUM_RPC_WS_HOST` and `SUPABASE_CONNECTION_STRING`:
+
+```shell
+ETHEREUM_RPC_WS_HOST=ws://127.0.0.1:8546 SUPABASE_CONNECTION_STRING=postgresql://postgres:your-super-secret-and-long-postgres-password@127.0.0.1:5432/postgres npm run sample-setup
+```
+
+#### Complete Sample Scenario
+
+Location: [./src/test/typescript/manual/common-setup-methods](./src/test/typescript/manual/common-setup-methods)
+
+This script starts the test Ethereum ledger for you, deploys a sample ERC721 contract, and mints some tokens. Then it synchronizes everything to a database and monitors for all new blocks. This script can also be used for manual, end-to-end tests of a plugin.
+
+By default, the script will try to use our `supabase-all-in-one` instance running on localhost.
+
+```shell
+npm run complete-sample-scenario
 ```
 
-Alternatively, import `PluginFactoryLedgerPersistence` from the plugin package and use it to create a plugin.
+Custom supabase can be set with environment variable `SUPABASE_CONNECTION_STRING`:
+
+```shell
+SUPABASE_CONNECTION_STRING=postgresql://postgres:your-super-secret-and-long-postgres-password@127.0.0.1:5432/postgres npm run complete-sample-scenario
+```
+
+### Usage
+
+Instantiate a new `PluginPersistenceEthereum` instance:
 
 ```typescript
-import { PluginFactoryLedgerPersistence } from "@hyperledger/cactus-plugin-persistence-ethereum";
-import { PluginImportType } from "@hyperledger/cactus-core-api";
+import { PluginPersistenceEthereum } from "@hyperledger/cactus-plugin-persistence-ethereum";
 import { v4 as uuidv4 } from "uuid";
 
-const factory = new PluginFactoryLedgerPersistence({
- pluginImportType: PluginImportType.Local,
-});
-
-const persistencePlugin = await factory.create({
+const persistencePlugin = new PluginPersistenceEthereum({
  instanceId: uuidv4(),
- apiClient: new SocketIOApiClient(apiConfigOptions),
+ apiClient: new EthereumApiClient(apiConfigOptions),
  logLevel: "info",
  connectionString:
  "postgresql://postgres:your-super-secret-and-long-postgres-password@localhost:5432/postgres",

packages/cactus-plugin-persistence-ethereum/package.json

@@ -54,7 +54,9 @@
  "codegen:openapi": "npm run generate-sdk",
  "copy-sql": "mkdir -p ./dist/lib/main/ && cp -Rfp ./src/main/sql ./dist/lib/main/",
  "copy-yarn-lock": "mkdir -p ./dist/lib/ && cp -rfp ../../yarn.lock ./dist/yarn.lock",
- "generate-sdk": "openapi-generator-cli generate -i ./src/main/json/openapi.json -g typescript-axios -o ./src/main/typescript/generated/openapi/typescript-axios/ --reserved-words-mappings protected=protected --ignore-file-override ../../openapi-generator-ignore"
+ "generate-sdk": "openapi-generator-cli generate -i ./src/main/json/openapi.json -g typescript-axios -o ./src/main/typescript/generated/openapi/typescript-axios/ --reserved-words-mappings protected=protected --ignore-file-override ../../openapi-generator-ignore",
+ "complete-sample-scenario": "npm run build && node ./dist/lib/test/typescript/manual/complete-sample-scenario.js",
+ "sample-setup": "npm run build && node ./dist/lib/test/typescript/manual/sample-setup.js"
  },
  "dependencies": {
  "@ethersproject/abi": "5.7.0",

packages/cactus-plugin-persistence-ethereum/src/test/typescript/manual/common-setup-methods.ts

@@ -29,13 +29,9 @@ import { PluginPersistenceEthereum } from "../../../main/typescript/plugin-persi
 // Constants
 //////////////////////////////////
 
-const SUPABASE_CONNECTION_STRING = process.env.SUPABASE_CONNECTION_STRING ?? "";
-if (!SUPABASE_CONNECTION_STRING) {
- console.error(
- "Please set SUPABASE_CONNECTION_STRING environment variable before running this script",
- );
- exit(1);
-}
+const SUPABASE_CONNECTION_STRING =
+ process.env.SUPABASE_CONNECTION_STRING ??
+ "postgresql://postgres:your-super-secret-and-long-postgres-password@127.0.0.1:5432/postgres";
 
 const testLogLevel: LogLevelDesc = "info";
 const sutLogLevel: LogLevelDesc = "info";
@@ -60,14 +56,9 @@ let apiServer: ApiServer;
  * and Ethereum Persistence plugin (for storing data read from ledger to the database).
  *
  * @param port Port under which an ApiServer will be started. Can't be 0.
- * @param rpcApiHttpHost Ledger RPC HTTP URL
  * @param rpcApiWsHost Ledger RPC WS URL
  */
-export async function setupApiServer(
- port: number,
- rpcApiHttpHost: string,
- rpcApiWsHost: string,
-) {
+export async function setupApiServer(port: number, rpcApiWsHost: string) {
  // PluginLedgerConnectorEthereum requires a keychain plugin to operate correctly, ensuring secure data storage.
  // For testing and debugging purposes, we use PluginKeychainMemory, which stores all secrets in memory (remember: this is not secure!).
  const keychainPlugin = new PluginKeychainMemory({
@@ -80,7 +71,6 @@ export async function setupApiServer(
  // We create ethereum connector instance. It will connect to the ledger through RPC endpoints rpcApiHttpHost and rpcApiWsHost.
  const connector = new PluginLedgerConnectorEthereum({
  instanceId: uuidV4(),
- rpcApiHttpHost,
  rpcApiWsHost,
  logLevel: sutLogLevel,
  pluginRegistry: new PluginRegistry({ plugins: [keychainPlugin] }),
@@ -136,7 +126,6 @@ export async function cleanupApiServer() {
  log.info("cleanupApiServer called.");
 
  if (apiServer) {
- log.info("Shutdown the server...");
  await apiServer.shutdown();
  }
 }

packages/cactus-plugin-persistence-ethereum/src/test/typescript/manual/complete-sample-scenario.ts

@@ -17,7 +17,6 @@ import {
  GethTestLedger,
  WHALE_ACCOUNT_PRIVATE_KEY,
 } from "@hyperledger/cactus-test-geth-ledger";
-import process from "process";
 import Web3, { ContractAbi, TransactionReceipt } from "web3";
 import { Web3Account } from "web3-eth-accounts";
 import TestERC721ContractJson from "../../solidity/TestERC721.json";
@@ -54,17 +53,16 @@ const containerImageVersion = "2023-07-27-2a8c48ed6";
  *
  * @returns `[rpcApiHttpHost, rpcApiWsHost]`
  */
-export async function setupTestLedger(): Promise<[string, string]> {
+async function setupTestLedger(): Promise<string> {
  log.info(`Start Ledger ${containerImageName}:${containerImageVersion}...`);
  ledger = new GethTestLedger({
  containerImageName,
  containerImageVersion,
  });
  await ledger.start();
- const rpcApiHttpHost = await ledger.getRpcApiHttpHost();
  const rpcApiWsHost = await ledger.getRpcApiWebSocketHost();
- log.info(`Ledger started, RPC: ${rpcApiHttpHost} WS: ${rpcApiWsHost}`);
- return [rpcApiHttpHost, rpcApiWsHost];
+ log.info(`Ledger started, WS RPC: ${rpcApiWsHost}`);
+ return rpcApiWsHost;
 }
 
 /**
@@ -193,13 +191,13 @@ async function deployAndMintTokens() {
 
 async function main() {
  // Start the test ethereum ledger which we'll monitor and run some sample operations.
- const [rpcApiHttpHost, rpcApiWsHost] = await setupTestLedger();
+ const rpcApiWsHost = await setupTestLedger();
 
  // Create test account
  constTestAcc = await ledger.createEthTestAccount(constTestAccBalance);
 
  // Create Web3 provider that will be used by other methods.
- web3 = new Web3(rpcApiHttpHost);
+ web3 = new Web3(rpcApiWsHost);
  const account = web3.eth.accounts.privateKeyToAccount(
  "0x" + WHALE_ACCOUNT_PRIVATE_KEY,
  );
@@ -209,7 +207,7 @@ async function main() {
 
  // Set up the ApiServer with Ethereum Connector and Ethereum Persistence plugins.
  // It returns the persistence plugin, which we can use to run monitoring operations.
- const persistence = await setupApiServer(9782, rpcApiHttpHost, rpcApiWsHost);
+ const persistence = await setupApiServer(9782, rpcApiWsHost);
  console.log("Environment is running...");
 
  // Deploy an ERC721 contract to our test ledger and mint some tokens,
@@ -230,9 +228,14 @@ async function main() {
  });
 }
 
-// Cleanup environment on exit
-process.on("exit", cleanupEnvironment);
-process.on("SIGINT", cleanupEnvironment);
-process.on("SIGTERM", cleanupEnvironment);
+process.once("uncaughtException", async () => {
+ await cleanupEnvironment();
+ process.exit();
+});
+
+process.once("SIGINT", () => {
+ console.log("SIGINT received...");
+ throw new Error();
+});
 
 main();

packages/cactus-plugin-persistence-ethereum/src/test/typescript/manual/sample-setup.ts

@@ -1,35 +1,18 @@
-import { exit } from "node:process";
 import { cleanupApiServer, setupApiServer } from "./common-setup-methods";
 
-const ETHEREUM_RPC_HTTP_HOST = process.env.ETHEREUM_RPC_HTTP_HOST ?? "";
-if (!ETHEREUM_RPC_HTTP_HOST) {
- console.error(
- "Please set ETHEREUM_RPC_HTTP_HOST environment variable before running this script",
- );
- exit(1);
-}
-
-const ETHEREUM_RPC_WS_HOST = process.env.ETHEREUM_RPC_WS_HOST ?? "";
-if (!ETHEREUM_RPC_WS_HOST) {
- console.error(
- "Please set ETHEREUM_RPC_WS_HOST environment variable before running this script",
- );
- exit(1);
-}
+const ETHEREUM_RPC_WS_HOST =
+ process.env.ETHEREUM_RPC_WS_HOST ?? "ws://127.0.0.1:8546";
 
 async function main() {
  // Set up the ApiServer with Ethereum Connector and Ethereum Persistence plugins.
  // It returns the persistence plugin, which we can use to run monitoring operations.
- const persistence = await setupApiServer(
- 9781,
- ETHEREUM_RPC_HTTP_HOST,
- ETHEREUM_RPC_WS_HOST,
- );
+ const persistence = await setupApiServer(9781, ETHEREUM_RPC_WS_HOST);
  console.log("Environment is running...");
 
+ // CUSTOM CODE GOES HERE !!!!
  // Inform our persistence plugin about the deployed contract.
  // From now on, the persistence plugin will monitor any token operations on this contract.
- await persistence.addTokenERC721("0x123");
+ // await persistence.addTokenERC721("0x123");
 
  // Start monitoring for ledger state changes.
  // Any updates will be pushed to the database, and all errors will be printed to the console.
@@ -39,9 +22,14 @@ async function main() {
  });
 }
 
-// Cleanup environment on exit
-process.on("exit", cleanupApiServer);
-process.on("SIGINT", cleanupApiServer);
-process.on("SIGTERM", cleanupApiServer);
+process.once("uncaughtException", async () => {
+ await cleanupApiServer();
+ process.exit();
+});
+
+process.once("SIGINT", () => {
+ console.log("SIGINT received...");
+ throw new Error();
+});
 
 main();