Skip to content

Revamp the toplevel and the docker README #5979

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 9, 2025
Merged

Revamp the toplevel and the docker README #5979

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0843165
resources: Remove unused file
lutter Mar 17, 2025
44934aa
Remove unused Travis config
lutter Mar 17, 2025
1e2c7aa
Revamp the toplevel and the docker README
lutter Mar 17, 2025
a595a0a
docs: Remove getting-started.md
lutter Apr 30, 2025
86c61b4
README.md: Incorporate review suggestions
lutter May 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 0 additions & 87 deletions .travis.yml
View file
Open in desktop

This file was deleted.

215 changes: 69 additions & 146 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,193 +3,116 @@
[![Build Status](https://github.com/graphprotocol/graph-node/actions/workflows/ci.yml/badge.svg)](https://github.com/graphprotocol/graph-node/actions/workflows/ci.yml?query=branch%3Amaster)
[![Getting Started Docs](https://img.shields.io/badge/docs-getting--started-brightgreen.svg)](docs/getting-started.md)

[The Graph](https://thegraph.com/) is a protocol for building decentralized applications (dApps) quickly on Ethereum and IPFS using GraphQL.
## Overview

Graph Node is an open source Rust implementation that event sources the Ethereum blockchain to deterministically update a data store that can be queried via the GraphQL endpoint.
[The Graph](https://thegraph.com/) is decentralized protocol that organizes and distributes blockchain data across the leading Web3 networks. A key component of The Graph's tech stack is Graph Node.

For detailed instructions and more context, check out the [Getting Started Guide](docs/getting-started.md).
Before using `graph-node,` it is highly recommended that you read the [official Graph documentation](https://thegraph.com/docs/en/subgraphs/quick-start/) to understand Subgraphs, which are the central mechanism for extracting and organizing blockchain data.

## Quick Start
This guide is for:

### Prerequisites

To build and run this project you need to have the following installed on your system:

- Rust (latest stable) – [How to install Rust](https://www.rust-lang.org/en-US/install.html)
- Note that `rustfmt`, which is part of the default Rust installation, is a build-time requirement.
- PostgreSQL – [PostgreSQL Downloads](https://www.postgresql.org/download/)
- IPFS – [Installing IPFS](https://docs.ipfs.io/install/)
- Protobuf Compiler - [Installing Protobuf](https://grpc.io/docs/protoc-installation/)

For Ethereum network data, you can either run your own Ethereum node or use an Ethereum node provider of your choice.

**Minimum Hardware Requirements:**
1. Subgraph developers who want to run `graph-node` locally to test their Subgraphs during development
2. Contributors who want to add features or fix bugs to `graph-node` itself

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Local Set Up

- To build graph-node with `cargo`, 8GB RAM are required.
## Running `graph-node` from Docker images

### Docker
For subgraph developers, it is highly recommended to use prebuilt Docker
images to set up a local `graph-node` environment. Please read [these
instructions](./docker/README.md) to learn how to do that.

The easiest way to run a Graph Node is to use the official Docker compose setup. This will start a Postgres database, IPFS node, and Graph Node.
[Follow the instructions here](./docker/README.md).
## Running `graph-node` from source

### Running a Local Graph Node
This is usually only needed for developers who want to contribute to `graph-node`.

This is a quick example to show a working Graph Node. It is a [subgraph for Gravatars](https://github.com/graphprotocol/example-subgraph).
### Prerequisites

1. Install IPFS and run `ipfs init` followed by `ipfs daemon`.
2. Install PostgreSQL and run `initdb -D .postgres -E UTF8 --locale=C` followed by `pg_ctl -D .postgres -l logfile start` and `createdb graph-node`.
3. If using Ubuntu, you may need to install additional packages:
- `sudo apt-get install -y clang libpq-dev libssl-dev pkg-config`
4. In the terminal, clone https://github.com/graphprotocol/example-subgraph, and install dependencies and generate types for contract ABIs:
To build and run this project, you need to have the following installed on your system:

```
yarn
yarn codegen
```
- Rust (latest stable): Follow [How to install
Rust](https://www.rust-lang.org/en-US/install.html). Run `rustup install
stable` in _this directory_ to make sure all required components are
installed. The `graph-node` code assumes that the latest available
`stable` compiler is used.
- PostgreSQL: [PostgreSQL Downloads](https://www.postgresql.org/download/) lists
downloads for almost all operating systems.
- For OSX: We highly recommend [Postgres.app](https://postgresapp.com/).
- For Linux: Use the Postgres version that comes with the distribution.
- IPFS: [Installing IPFS](https://docs.ipfs.io/install/)
- Protobuf Compiler: [Installing Protobuf](https://grpc.io/docs/protoc-installation/)

5. In the terminal, clone https://github.com/graphprotocol/graph-node, and run `cargo build`.
For Ethereum network data, you can either run your own Ethereum node or use an Ethereum node provider of your choice.

Once you have all the dependencies set up, you can run the following:
### Create a database

```
cargo run -p graph-node --release -- \
--postgres-url postgresql://USERNAME[:PASSWORD]@localhost:5432/graph-node \
--ethereum-rpc NETWORK_NAME:[CAPABILITIES]:URL \
--ipfs 127.0.0.1:5001
```
Once Postgres is running, you need to issue the following commands to create a database
and configure it for use with `graph-node`.

Try your OS username as `USERNAME` and `PASSWORD`. For details on setting
the connection string, check the [Postgres
documentation](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING).
`graph-node` uses a few Postgres extensions. If the Postgres user with which
you run `graph-node` is a superuser, `graph-node` will enable these
extensions when it initializes the database. If the Postgres user is not a
superuser, you will need to create the extensions manually since only
superusers are allowed to do that. To create them you need to connect as a
superuser, which in many installations is the `postgres` user:
The name of the `SUPERUSER` depends on your installation, but is usually `postgres` or your username.

```bash
psql -q -X -U <SUPERUSER> graph-node <<EOF
psql -U <SUPERUSER> <<EOF
create user graph with password '<password>';
create database "graph-node" with owner=graph template=template0 encoding='UTF8' locale='C';
create extension pg_trgm;
create extension pg_stat_statements;
create extension btree_gist;
create extension postgres_fdw;
grant usage on foreign data wrapper postgres_fdw to <USERNAME>;
grant usage on foreign data wrapper postgres_fdw to graph;
EOF

```

This will also spin up a GraphiQL interface at `http://127.0.0.1:8000/`.
For convenience, set the connection string to the database in an environment
variable, and save it, e.g., in `~/.bashrc`:

6. With this Gravatar example, to get the subgraph working locally run:

```
yarn create-local
```bash
export POSTGRES_URL=postgresql://graph:<password>@localhost:5432/graph-node
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
export POSTGRES_URL=postgresql://graph:<password>@localhost:5432/graph-node
Export the connection:
export POSTGRES_URL=postgresql://graph:<password>@localhost:5432/graph-node

```

Then you can deploy the subgraph:

```
yarn deploy-local
```
Use the `POSTGRES_URL` from above to have `graph-node` connect to the
database. If you ever need to manually inspect the contents of your
database, you can do that by running `psql $POSTGRES_URL`. Running this
command is also a convenient way to check that the database is up and
running and that the connection string is correct.

This will build and deploy the subgraph to the Graph Node. It should start indexing the subgraph immediately.
### Build and Run `graph-node`

### Command-Line Interface
Clone this repository and run this command at the root of the repository:

```bash
export GRAPH_LOG=debug
cargo run -p graph-node --release -- \
--postgres-url $POSTGRES_URL \
--ethereum-rpc NETWORK_NAME:[CAPABILITIES]:URL \
--ipfs 127.0.0.1:5001
```
USAGE:
graph-node [FLAGS] [OPTIONS] --ethereum-ipc <NETWORK_NAME:FILE> --ethereum-rpc <NETWORK_NAME:URL> --ethereum-ws <NETWORK_NAME:URL> --ipfs <HOST:PORT> --postgres-url <URL>

FLAGS:
--debug Enable debug logging
-h, --help Prints help information
-V, --version Prints version information

OPTIONS:
--admin-port <PORT> Port for the JSON-RPC admin server [default: 8020]
--elasticsearch-password <PASSWORD>
Password to use for Elasticsearch logging [env: ELASTICSEARCH_PASSWORD]

--elasticsearch-url <URL>
Elasticsearch service to write subgraph logs to [env: ELASTICSEARCH_URL=]
The argument for `--ethereum-rpc` contains a network name (e.g. `mainnet`) and
a list of provider capabilities (e.g. `archive,traces`). The URL is the address
of the Ethereum node you want to connect to, usually a `https` URL, so that the
entire argument might be `mainnet:archive,traces:https://provider.io/some/path`.

--elasticsearch-user <USER> User to use for Elasticsearch logging [env: ELASTICSEARCH_USER=]
--ethereum-ipc <NETWORK_NAME:[CAPABILITIES]:FILE>
Ethereum network name (e.g. 'mainnet'), optional comma-separated capabilities (eg full,archive), and an Ethereum IPC pipe, separated by a ':'
When `graph-node` starts, it prints the various ports that it is listening on.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When `graph-node` starts, it prints the various ports that it is listening on.
When `graph-node` starts, it will expose:

The most important of these is the GraphQL HTTP server, which by default
is at `http://localhost:8000`. You can use routes like `/subgraphs/name/<subgraph-name>`
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
is at `http://localhost:8000`. You can use routes like `/subgraphs/name/<subgraph-name>`
at `http://localhost:8000`.
You can use routes like `/subgraphs/name/<subgraph-name>`

and `/subgraphs/id/<IPFS hash>` to query subgraphs once you have deployed them.

--ethereum-polling-interval <MILLISECONDS>
How often to poll the Ethereum node for new blocks [env: ETHEREUM_POLLING_INTERVAL=] [default: 500]
### Deploying a Subgraph

--ethereum-rpc <NETWORK_NAME:[CAPABILITIES]:URL>
Ethereum network name (e.g. 'mainnet'), optional comma-separated capabilities (eg 'full,archive'), and an Ethereum RPC URL, separated by a ':'

--ethereum-ws <NETWORK_NAME:[CAPABILITIES]:URL>
Ethereum network name (e.g. 'mainnet'), optional comma-separated capabilities (eg `full,archive), and an Ethereum WebSocket URL, separated by a ':'

--node-id <NODE_ID>
A unique identifier for this node instance. Should have the same value between consecutive node restarts [default: default]

--http-port <PORT> Port for the GraphQL HTTP server [default: 8000]
--ipfs <HOST:PORT> HTTP address of an IPFS node
--postgres-url <URL> Location of the Postgres database used for storing entities
--subgraph <[NAME:]IPFS_HASH> Name and IPFS hash of the subgraph manifest
--ws-port <PORT> Port for the GraphQL WebSocket server [default: 8001]
```
Follow the [Subgraph deployment
guide](https://thegraph.com/docs/en/subgraphs/developing/introduction/).
After setting up `graph-cli` as described, you can deploy a Subgraph to your
local Graph Node instance.

### Advanced Configuration

The command line arguments generally are all that is needed to run a
`graph-node` instance. For advanced uses, various aspects of `graph-node`
can further be configured through [environment
variables](https://github.com/graphprotocol/graph-node/blob/master/docs/environment-variables.md). Very
large `graph-node` instances can also split the work of querying and
indexing across [multiple databases](./docs/config.md).

## Project Layout

- `node` — A local Graph Node.
- `graph` — A library providing traits for system components and types for
common data.
- `core` — A library providing implementations for core components, used by all
nodes.
- `chain/ethereum` — A library with components for obtaining data from
Ethereum.
- `graphql` — A GraphQL implementation with API schema generation,
introspection, and more.
- `mock` — A library providing mock implementations for all system components.
- `runtime/wasm` — A library for running WASM data-extraction scripts.
- `server/http` — A library providing a GraphQL server over HTTP.
- `store/postgres` — A Postgres store with a GraphQL-friendly interface
and audit logs.

## Roadmap

🔨 = In Progress

🛠 = Feature complete. Additional testing required.

✅ = Feature complete


| Feature | Status |
| ------- | :------: |
| **Ethereum** | |
| Indexing smart contract events | ✅ |
| Handle chain reorganizations | ✅ |
| **Mappings** | |
| WASM-based mappings| ✅ |
| TypeScript-to-WASM toolchain | ✅ |
| Autogenerated TypeScript types | ✅ |
| **GraphQL** | |
| Query entities by ID | ✅ |
| Query entity collections | ✅ |
| Pagination | ✅ |
| Filtering | ✅ |
| Block-based Filtering | ✅ |
| Entity relationships | ✅ |
| Subscriptions | ✅ |
variables](https://github.com/graphprotocol/graph-node/blob/master/docs/environment-variables.md).

Very large `graph-node` instances can also be configured using a
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Very large `graph-node` instances can also be configured using a
> Please note: Very large `graph-node` instances can also be configured using a

[configuration file](./docs/config.md) That is usually only necessary when
the `graph-node` needs to connect to multiple chains or if the work of
indexing and querying needs to be split across [multiple databases](./docs/config.md).

## Contributing

Expand Down
Loading
Loading