The Autonomi Network

Autonomi.com | Documentation

Critical infrastructure of the next web. Assembled from everyday devices and owned by us all.

Autonomi is a fully autonomous data and communications network that provides:

• Lifetime storage with one-time payment
• Private by design with multilayered encryption
• Blockchainless data architecture
• Decentralized infrastructure built from everyday devices
• Content-addressable storage using Kademlia and libp2p

Table of Contents

• For Users
• For Developers
• For the Technical
• Using a Local Network
• Metrics Dashboard

For Users

• CLI The client command line interface that enables users to interact with the network from their terminal.
• Node The backbone of the Autonomi network. Nodes can run on commodity hardware and provide storage space and validate transactions on the network.
• Web App: Coming Soon!

Building the Node from Source

If you wish to build a version of antnode from source, some special consideration must be given if you want it to connect to the current beta network.

You should build from the stable branch, as follows:

git checkout stable
cargo build --release --bin antnode

Running the Node

To run a node and receive rewards, you need to specify your Ethereum address as a parameter. Rewards are paid to the specified address.

cargo run --release --bin antnode -- --rewards-address <YOUR_ETHEREUM_ADDRESS_TO_RECEIVE_REWARDS>

More options about EVM Network below.

For Developers

Main Crates

• Autonomi API The client APIs allowing use of the Autonomi network to users and developers.
• Autonomi CLI The client command line interface that enables users to interact with the network from their terminal.
• Node The backbone of the Autonomi network. Nodes can be run on commodity hardware and connect to the network.
• Node Manager Use to create a local network for development and testing.
• Node RPC The RPC server used by the nodes to expose API calls to the outside world.

Transport Protocols and Architectures

The Autonomi network uses quic as the default transport protocol.

For the Technical

• Logging The generalised logging crate used by the autonomi network (backed by the tracing crate).
• Metrics The metrics crate used by the autonomi network.
• Networking The networking layer, built atop libp2p which allows nodes and clients to communicate.
• Protocol The protocol used by the autonomi network.
• Bootstrap The network bootstrap cache or: how the network layer discovers bootstrap peers.
• Build Info Small helper used to get the build/commit versioning info for debug purposes.

Using a Local Network

We can explore the network's features by using multiple node processes to form a local network. We also need to run a local EVM network for our nodes and client to connect to.

Follow these steps to create a local network:

1. Prerequisites

The latest version of Rust should be installed. If you already have an installation, use rustup update to get the latest version.

Run all the commands from the root of this repository.

If you haven't already, install Foundry. We need to have access to Anvil, which is packaged with Foundry, to run an EVM node: https://book.getfoundry.sh/getting-started/installation

To collect rewards for you nodes, you will need an EVM address, you can create one using metamask.

2. Run a local EVM node

cargo run --bin evm-testnet

This creates a CSV file with the EVM network params in your data directory.

3. Create the test network and pass the EVM params

--rewards-address is the address where you will receive your node earnings on.

cargo run --bin antctl -- local run --build --clean --rewards-address <YOUR_ETHEREUM_ADDRESS>

The EVM Network parameters are loaded from the CSV file in your data directory automatically when the local mode is enabled.

4. Verify node status

cargo run --bin antctl -- status

The Antctl run command starts the node processes. The status command should show twenty-five running nodes.

5. Uploading and Downloading Data

To upload a file or a directory, you need to set the SECRET_KEY environment variable to your EVM secret key:

When running a local network, you can use the SECRET_KEY printed by the evm-testnet command step 2 as it has all the money.

SECRET_KEY=<YOUR_EVM_SECRET_KEY> cargo run --bin ant -- --local file upload <path>

The output will print out the address at which the content was uploaded.

Now to download the files again:

cargo run --bin ant -- --local file download <addr> <dest_path>

RPC

The node manager launches each node process with a remote procedure call (RPC) service. The workspace has a client binary that can be used to run commands against these services.

Run the status command with the --details flag to get the RPC port for each node:

$ cargo run --bin antctl -- status --details
...
===================================
antctl-local25 - RUNNING
===================================
Version: 0.103.21
Peer ID: 12D3KooWJ4Yp8CjrbuUyeLDsAgMfCb3GAYMoBvJCRp1axjHr9cf8
Port: 38835
RPC Port: 34416
Multiaddr: /ip4/127.0.0.1/udp/38835/quic-v1/p2p/12D3KooWJ4Yp8CjrbuUyeLDsAgMfCb3GAYMoBvJCRp1axjHr9cf8
PID: 62369
Data path: /home/<<user_directory>>/.local/share/autonomi/node/12D3KooWJ4Yp8CjrbuUyeLDsAgMfCb3GAYMoBvJCRp1axjHr9cf8
Log path: /home/<<user_directory>>/.local/share/autonomi/node/12D3KooWJ4Yp8CjrbuUyeLDsAgMfCb3GAYMoBvJCRp1axjHr9cf8/logs
Bin path: target/release/antnode
Connected peers: 24

Now you can run RPC commands against any node.

The info command will retrieve basic information about the node:

$ cargo run --bin antnode_rpc_client -- 127.0.0.1:34416 info
Node info:
==========
RPC endpoint: https://127.0.0.1:34416
Peer Id: 12D3KooWJ4Yp8CjrbuUyeLDsAgMfCb3GAYMoBvJCRp1axjHr9cf8
Logs dir: /home/<<user_directory>>/.local/share/autonomi/node/12D3KooWJ4Yp8CjrbuUyeLDsAgMfCb3GAYMoBvJCRp1axjHr9cf8/logs
PID: 62369
Binary version: 0.103.21
Time since last restart: 1614s

The netinfo command will return connected peers and listeners:

$ cargo run --bin antnode_rpc_client -- 127.0.0.1:34416 netinfo
Node's connections to the Network:

Connected peers:
Peer: 12D3KooWJkD2pB2WdczBJWt4ZSAWfFFMa8FHe6w9sKvH2mZ6RKdm
Peer: 12D3KooWRNCqFYX8dJKcSTAgxcy5CLMcEoM87ZSzeF43kCVCCFnc
Peer: 12D3KooWLDUFPR2jCZ88pyYCNMZNa4PruweMsZDJXUvVeg1sSMtN
Peer: 12D3KooWC8GR5NQeJwTsvn9SKChRZqJU8XS8ZzKPwwgBi63FHdUQ
Peer: 12D3KooWJGERJnGd5N814V295zq1CioxUUWKgNZy4zJmBLodAPEj
Peer: 12D3KooWJ9KHPwwiRpgxwhwsjCiHecvkr2w3JsUQ1MF8q9gzWV6U
Peer: 12D3KooWSBafke1pzz3KUXbH875GYcMLVqVht5aaXNSRtbie6G9g
Peer: 12D3KooWJtKc4C7SRkei3VURDpnsegLUuQuyKxzRpCtsJGhakYfX
Peer: 12D3KooWKg8HsTQ2XmBVCeGxk7jHTxuyv4wWCWE2pLPkrhFHkwXQ
Peer: 12D3KooWQshef5sJy4rEhrtq2cHGagdNLCvcvMn9VXwMiLnqjPFA
Peer: 12D3KooWLfXHapVy4VV1DxWndCt3PmqkSRjFAigsSAaEnKzrtukD

Node's listeners:
Listener: /ip4/127.0.0.1/udp/38835/quic-v1
Listener: /ip4/192.168.1.86/udp/38835/quic-v1
Listener: /ip4/172.17.0.1/udp/38835/quic-v1
Listener: /ip4/172.18.0.1/udp/38835/quic-v1
Listener: /ip4/172.20.0.1/udp/38835/quic-v1

Node control commands:

$ cargo run --bin antnode_rpc_client -- 127.0.0.1:34416 restart 5000
Node successfully received the request to restart in 5s

$ cargo run --bin antnode_rpc_client -- 127.0.0.1:34416 stop 6000
Node successfully received the request to stop in 6s

$ cargo run --bin antnode_rpc_client -- 127.0.0.1:34416 update 7000
Node successfully received the request to try to update in 7s

NOTE: it is preferable to use the node manager to control the node rather than RPC commands.

Tear Down

When you're finished experimenting, tear down the network:

cargo run --bin antctl -- local kill

Metrics Dashboard

Use the open-metrics feature flag on the node / client to start an OpenMetrics exporter. The metrics are served via a webserver started at a random port. Check the log file / stdout to find the webserver URL, Metrics server on http://127.0.0.1:xxxx/metrics

The metrics can then be collected using a collector (for e.g. Prometheus) and the data can then be imported into any visualization tool (for e.g., Grafana) to be further analyzed. Refer to this Guide to easily setup a dockerized Grafana dashboard to visualize the metrics.

Testing

The root of the repository provides a test.sh script that can run either unit tests, integration tests, or a full suite of end to end tests. The integration tests do not require a running local network, but the end to end tests do.

By default, test.sh will run the unit test suite.

See the sections below for further guidance.

Test Levels

The test suite provides three distinct testing levels:

1. Unit Tests (./test.sh unit)

Fast, isolated tests that don't require network connectivity:

• Tests individual functions and modules
• No network dependencies or EVM integration required
• Typically completes in under 30 seconds
• Tests core logic in isolation

./test.sh unit

Example output:

🧪 Unit Tests Summary
================================================
✅ ant-bootstrap: 8 tests passed
✅ ant-logging: 12 tests passed  
✅ autonomi (lib): 45 tests passed
✅ ant-networking: 23 tests passed
✅ ant-protocol: 15 tests passed
✅ node-launchpad: 7 tests passed

Total: 110 unit tests passed in 28.3s

2. Integration Tests (./test.sh integration)

Network-based tests using a local test network:

• Requires local EVM testnet and Autonomi network
• Tests basic network connectivity and operations
• Moderate test duration (2-5 minutes)
• Validates network integration without comprehensive data operations

./test.sh integration

3. Full Network Tests (./test.sh full)

Comprehensive end-to-end testing with complete logging infrastructure:

• Tests all data types: Chunks, Files, Registers, Vaults, GraphEntries, ScratchPads, Pointers
• Validates upload/download operations for every supported data structure
• Tests eventual consistency and network resilience
• Comprehensive logging with per-node log capture
• Automatic failure analysis and log interrogation
• Typically takes 10-15 minutes

./test.sh full

Enhanced Logging and Analysis

The test runner includes sophisticated logging capabilities for debugging and failure analysis:

Command Line Options

# Enable debug logging and save all logs
./test.sh full --debug

# Save logs without debug verbosity  
./test.sh full --save-logs

# Enable automatic failure analysis
./test.sh full --analyze-failures

# Combine options
./test.sh full --debug --analyze-failures

Log Structure

When logging is enabled, tests create a timestamped directory structure:

/tmp/autonomi_test_run_2025-07-06_20-21-55/
├── logs/
│   ├── network/
│   │   ├── antctl.log           # Network startup logs
│   │   ├── evm-testnet.log      # EVM blockchain logs  
│   │   └── status-checks.log    # Network health monitoring
│   ├── tests/
│   │   ├── address.log          # Address/chunk tests
│   │   ├── files.log            # File upload/download tests
│   │   ├── vault.log            # Vault operations tests
│   │   ├── registers.log        # Register tests
│   │   ├── graph.log            # Graph entry tests
│   │   └── [other test logs]
│   ├── nodes/
│   │   ├── 12D3KooW.../logs/    # Individual node logs
│   │   └── [25 node directories]
│   └── summary/
│       ├── final-summary.txt    # Complete test summary
│       ├── test-runner.log      # Main execution log
│       └── failure-analysis.txt # Automatic failure analysis

Automatic Failure Analysis

When tests fail, the system automatically:

1. Captures comprehensive logs from all network components
2. Analyzes failure patterns in node logs and test output
3. Provides diagnostic summaries with specific error locations
4. Suggests remediation steps based on common failure patterns

Example failure analysis:

🔍 FAILURE ANALYSIS
==================
Test: large.dat download failed
Location: /tmp/autonomi_test_run_2025-07-06_20-21-55/logs/tests/files.log:127

Error Pattern: "Connection timeout"
Node Logs: 3 nodes reported connectivity issues
Recommendation: Check network startup sequence and peer discovery

Detailed Logs:
- Network: /tmp/autonomi_test_run_2025-07-06_20-21-55/logs/network/
- Tests: /tmp/autonomi_test_run_2025-07-06_20-21-55/logs/tests/  
- Nodes: /tmp/autonomi_test_run_2025-07-06_20-21-55/logs/nodes/

API vs CLI Test Distinction

The full test suite clearly separates:

API Integration Tests (Core Functionality)

Tests the core Rust client API and network integration:

• ✅ Address/chunk operations
• ✅ File upload/download
• ✅ Vault operations
• ✅ Register management
• ✅ Graph entries
• ✅ Wallet functionality
• ✅ Payment processing

When API tests pass → Core network functionality is healthy

CLI Interface Tests (Optional/Diagnostic)

Tests the command-line interface wrapper:

• Command parsing and validation
• File I/O operations
• User interface functionality

CLI test failures typically indicate interface issues rather than core network problems.

Retrospective Analysis

Analyze previous test runs using saved logs:

# Analyze a specific test run
./test.sh analyze /tmp/autonomi_test_run_2025-07-06_20-21-55

# View summary of recent test runs
./test.sh analyze --recent

Prerequisites for Testing

Before running network tests, ensure:

1. Foundry/Anvil installed for EVM testing:

   curl -L https://foundry.paradigm.xyz | bash
   foundryup

2. Rust toolchain up to date:

   rustup update

3. Network ports available (EVM testnet and node ports)

Troubleshooting Common Issues

"Anvil not found"

• Install Foundry: curl -L https://foundry.paradigm.xyz | bash && foundryup
• Ensure ~/.foundry/bin is in your PATH

"Network startup timeout"

• Check available memory (network requires ~2GB)
• Verify no conflicting processes on required ports
• Review network logs in test output directory

"Token insufficient errors"

• EVM testnet may not be ready - wait 10-15 seconds after startup
• Check EVM testnet logs for deployment issues

"Node connection failures"

• Network discovery can take 60-90 seconds for full connectivity
• Check individual node logs for connectivity issues
• Verify firewall isn't blocking local connections

Running Tests in CI/CD

For automated testing environments:

# Run with comprehensive logging for CI analysis
./test.sh full --save-logs --analyze-failures

# Parse exit codes
if ./test.sh unit; then
    echo "Unit tests passed"
else  
    echo "Unit tests failed - check compilation issues"
    exit 1
fi

Contributing

Feel free to clone and modify this project. Pull requests are welcome.

Community

Join our community for support and discussions:

• Discord: https://discord.gg/autonomi
• Forum: https://forum.autonomi.community/
• X (Twitter): https://x.com/WithAutonomi
• Reddit: https://www.reddit.com/r/autonomi/
• LinkedIn: https://uk.linkedin.com/company/withautonomi

For detailed contribution guidelines, see CONTRIBUTING.md.

Pull Request Process

1. Please direct all pull requests to the main branch.
2. Ensure that your commit messages clearly describe the changes you have made and use the Conventional Commits specification.
3. All PRs must pass automated CI tests and peer review before being merged.
4. PRs should be <= 200 lines changed (lines added + lines deleted).
5. PRs should clearly reference an issue when applicable using GitHub keywords.

License

This Autonomi Network repository is licensed under the General Public License (GPL), version 3 (LICENSE).