Exposes a multi-tenanted DWN (aka Decentralized Web Node) through a JSON-RPC API over http:
and ws:
- LevelDB ✔️
- SQLite ✔️
- MySQL ✔️
- PostgreSQL ✔️
See more in Storage Options
Interested in contributing instantly? You can make your updates directly without cloning in the running CodeSandbox environment.
npm install @web5/dwn-server
import { DwnServer } from '@web5/dwn-server';
const server = new DwnServer();
server.start();
docker run -p 3000:3000 -v myvolume:/dwn-server/data ghcr.io/tbd54566975/dwn-server:main
This can run on services like AWS, GCP, VPS, home server (with ngrok or cloudflare), fly.io, render.com etc. Ideally the volume is persistent so that data is kept (or has to be synced back from another DWN instance).
Running the command above will run the latest version at the time the image is pulled. If you need to run a specific version (and in many cases this is recommended) you can see the list published images here
To run a specific image:
docker pull ghcr.io/tbd54566975/dwn-server@sha256:870e0f0f12016e6607060a81ea31458443f7439522fab2688d7a6706ab366c58
git clone https://github.com/TBD54566975/dwn-server.git
cd dwn-server
npm install
npm run server
A docker image is continuously published from this repository, but if you want to build it locally run:
docker build -t dwn-server .
JSON-RPC is a lightweight remote procedure call (RPC) protocol that uses JSON as a data format for exchanging information between a client and a server over a network. JSON-RPC is language-independent and transport-agnostic which makes it usable in a variety of contexts (e.g. browser, server-side)
With JSON-RPC, a client sends a request message to a server over a network, and the server responds with a response message.
The request message consists of:
- a method name (
method
) - a set of parameters (
params
) - an identifier (
id
).
The response message contains:
- the same identifier that was sent with the request message (
id
) - the result of the method invocation (
result
) - an error message if the method invocation failed (
error
)
Used to send DWeb Messages to the server.
Property | Required (Y/N) | Description |
---|---|---|
target |
Y | The DID that the message is intended for |
message |
Y | The DWeb Message |
encodedData |
N | Data associated to the message (e.g. data associated to a RecordsWrite ) |
{
"jsonrpc": "2.0",
"id": "b23f9e31-4966-4972-8048-af3eed43cb41",
"method": "dwn.processMessage",
"params": {
"message": {
"recordId": "bafyreidtix6ghjmsbg7eitexsmwzvjxc7aelagsqasybmql7zrms34ju6i",
"descriptor": {
"interface": "Records",
"method": "Write",
"dataCid": "bafkreidnfo6aux5qbg3wwzy5hvwexnoyhk3q3v47znka2afa6mf2rffkbi",
"dataSize": 32,
"dateCreated": "2023-04-30T22:49:37.713976Z",
"dateModified": "2023-04-30T22:49:37.713976Z",
"dataFormat": "application/json"
},
"authorization": {
"payload": "eyJyZWNvcmRJZCI6ImJhZnlyZWlkdGl4Nmdoam1zYmc3ZWl0ZXhzbXd6dmp4YzdhZWxhZ3NxYXN5Ym1xbDd6cm1zMzRqdTZpIiwiZGVzY3JpcHRvckNpZCI6ImJhZnlyZWlheTVwNWZ1bzJhc2hqZXRvbzR1M3p1b282dW02cGlzNHl5NnUzaHE1emxsdmZhN2ZubXY0In0",
"signatures": [
{
"protected": "eyJhbGciOiJFZERTQSIsImtpZCI6ImRpZDprZXk6ejZNa3UxaDRMZGtoWFczSG5uQktBTnhnVWFRMTYyY3ZXbVJ1emNiZDJZZThWc3RaI2RpZDprZXk6ejZNa3UxaDRMZGtoWFczSG5uQktBTnhnVWFRMTYyY3ZXbVJ1emNiZDJZZThWc3RaI3o2TWt1MWg0TGRraFhXM0hubkJLQU54Z1VhUTE2MmN2V21SdXpjYmQyWWU4VnN0WiJ9",
"signature": "cy_RtWjjVK2mmKkI_35qiv54_1Pp_f7SjAx0z75PBL4th-fgfjuLZmF-V3czCWwFYMMnN0W4zl3LJ2jEf_t9DQ"
}
]
}
},
"target": "did:key:z6Mku1h4LdkhXW3HnnBKANxgUaQ162cvWmRuzcbd2Ye8VstZ",
"encodedData": "ub3-FwUsSs4GgZWqt5eXSH41RKlwCx41y3dgio9Di74"
}
}
{
"jsonrpc": "2.0",
"id": "18eb421f-4750-4e31-a062-412b71139546",
"result": {
"reply": {
"status": {
"code": 202,
"detail": "Accepted"
}
}
}
}
{
"jsonrpc": "2.0",
"id": "1c7f6ed8-eaaf-447c-aaf3-b9e61f3f59af",
"error": {
"code": -50400,
"message": "Unexpected token ';', \";;;;@!#@!$$#!@%\" is not valid JSON"
}
}
RecordsWrite
data can be of any size. If needed, large amounts of data can be streamed to the server over http by:
- including the JSON-RPC request message in a
dwn-request
request header - setting the
content-type
request header toapplication/octet-stream
- sending binary data in the request body.
💡 Examples can be found in the
examples
directory.
RecordsWrite
data can be of any size. RecordsWrite
messages returned as the result of a RecordsQuery
will include encodedData
if the RecordsWrite
data is under 9.77KB
. Data larger than this will need to be fetched using RecordsRead
which can be done over http. The response to a RecordsRead
includes:
- The JSON-RPC response message in a
dwn-response
header - The associated data as binary in the response body.
Examples can be found in the examples
directory.
💡 TODO: Add examples in
examples
directory
By default, when you call web5.connect()
there will be some bootstrap DWN nodes included which allow people to reach you via your DID.
You may want to run a DWN server just for you, or as a public service for you and your friends and family. DWNs can be as simple as a docker image or a node process running somewhere.
DWN-servers can run anywhere you can run node.js or docker. http and websocket need to be available to the DWN server. See below for some suggestions.
You can run an instance on the render.com service:
- Create a render.com account
- Fork this repo
- Upgrade your render.com account to a paid account
- Create a new "Web service" type application
- Choose the forked repo to run (or you can point to the main repo)
- Choose the "starter" size instance
- Create a 1GB (or larger) disk, and mount it on /dwn-server/data
You can run a DWN-server on your local machine or home server and expose it to the internet using ngrok.
First, install ngrok
Then run:
docker run -p 3000:3000 -v myvolume:/dwn-server/data ghcr.io/tbd54566975/dwn-server:main
## in another terminal:
ngrok http 3000
Note the resulting publicly addressable https url for your DWN instance.
Cloudflare has a tunnel service that you can use to expose your DWN server to the internet, if you run it on a server at home. With cloudflared installed, run the following commands:
git clone https://github.com/TBD54566975/dwn-server.git
cd dwn-server
npm install
npm run server
## in another terminal:
cloudflared tunnel --url http://localhost:3000
... check back soon ... (enterprising people I am sure can work it out)
Script | Description |
---|---|
npm run build:esm |
compiles typescript into ESM JS |
npm run build:cjs |
compiles typescript into CommonJS |
npm run build |
compiles typescript into ESM JS & CommonJS |
npm run clean |
deletes compiled JS |
npm run lint |
runs linter |
npm run lint:fix |
runs linter and fixes auto-fixable problems |
npm run test |
runs tests |
npm run server |
starts server |
npm run prepare |
prepares husky for pre-commit hooks (auto-runs with npm install ) |
Configuration can be set using environment variables
Env Var | Description | Default |
---|---|---|
DS_PORT |
Port that the server listens on | 3000 |
DS_MAX_RECORD_DATA_SIZE |
Maximum size for RecordsWrite data. use b , kb , mb , gb for value |
1gb |
DS_WEBSOCKET_SERVER |
Whether to enable listening over ws: . values: on ,off |
on |
DWN_BASE_URL |
Base external URL of this DWN. Used to construct URL paths such as the Request URI for the Web5 Connect flow. |
http://localhost |
DWN_EVENT_STREAM_PLUGIN_PATH |
Path to DWN Event Stream plugin to use. Default single-node implementation will be used if left empty. | unset |
DWN_REGISTRATION_STORE_URL |
URL to use for storage of registered DIDs. Leave unset to if DWN does not require registration (ie. open for all) | unset |
DWN_REGISTRATION_PROOF_OF_WORK_SEED |
Optional seed to generate the challenge nonce from, this allows all DWN instances in a cluster to generate the same challenge. | unset |
DWN_REGISTRATION_PROOF_OF_WORK_ENABLED |
Require new users to complete a proof-of-work challenge | false |
DWN_REGISTRATION_PROOF_OF_WORK_INITIAL_MAX_HASH |
Initial maximum allowed hash in 64 char HEX string. The more leading zeros (smaller number) the higher the difficulty. | 000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF |
DWN_STORAGE |
URL to use for storage by default. See Storage Options for details | level://data |
DWN_STORAGE_MESSAGES |
Connection URL or file path to custom plugin to use for the message store. | value of DWN_STORAGE |
DWN_STORAGE_DATA |
Connection URL or file path to custom plugin to use for the data store. | value of DWN_STORAGE |
DWN_STORAGE_RESUMABLE_TASKS |
Connection URL or file path to custom plugin to use for the resumable task store. | value of DWN_STORAGE |
DWN_STORAGE_EVENTS |
Connection URL or file path to custom plugin to use for the event store. | value of DWN_STORAGE |
DWN_TERMS_OF_SERVICE_FILE_PATH |
Required terms of service agreement if set. Value is path to the terms of service file. | unset |
DWN_TTL_CACHE_URL |
URL of the TTL cache used by the DWN. Currently only supports SQL databases. | sqlite:// |
Several built storage options are supported, and may be configured with the DWN_STORAGE_*
environment variables:
Database | Example | Notes |
---|---|---|
LevelDB | level://data |
use three slashes for absolute paths, two for relative. Example shown uses directory data in the current working directory |
Sqlite | sqlite://dwn.db |
use three slashes for absolute paths, two for relative. Example shown creates a file dwn.db in the current working directory |
MySQL | mysql://user:pass@host/db?debug=true&timezone=-0700 |
all URL options documented here |
PostgreSQL | postgres:///dwn |
any options other than the URL scheme (postgres:// ) may also be specified via standard environment variables |
In some scenarios, you may want to provide a custom implementation of a pluggable module for the DWN Server. The following interfaces defined in dwn-sdk-js
package are supported:
DataStore
MessageStore
ResumableDataStore
EventLog
EventStream
To load your custom plugin, specify the absolute path to the .js
file of your custom implementation using the corresponding environment variable. For instance, use DWN_STORAGE_DATA
for a custom DWN Data Store.
Refer to the tests/plugins/*.ts
files for examples of plugin implementations. In summary, you need to:
- Implement the corresponding interface from the
dwn-sdk-js
package. For example, implement theDataStore
interface for a DWN Data Store. - Ensure that the built
.js
file that will be referenced by the DWN Server config environment variable contains a class that:- Is a default export. This is how DWN Server locates the correct class for instantiation.
- Has a public constructor that does not take any arguments. This is how DWN Server instantiates the plugin.
There are multiple optional registration gates, each of which can be enabled (all are disabled by default). Tenants (DIDs) must comply with whatever
requirements are enabled before they are allowed to use the server. Tenants that have not completed the registration requirements will be met with a 401. Note that registration is tracked in a database, and only SQL-based databases are supported (LevelDB is not supported). Current registration
requirements are available at the /info
endpoint.
- Proof of Work (
DWN_REGISTRATION_PROOF_OF_WORK_ENABLED=true
) - new tenants must GET/registration/proof-of-work
for a challenge, then generate a nonce that produces a string that has a sha256 hex sum starting with the specified (complexity
) number of zeros (0
) when added to the end of the challenge (sha256(challenge + nonce)
). This nonce should be POSTed to/registration/proof-of-work
with a JSON body including thechallenge
, the nonce in fieldresponse
anddid
. Challenges expire after 5 minutes, and complexity will increase based on the number of successful proof-of-work registrations that have been completed within the last hour. This registration requirement is listed in/info
asproof-of-work-sha256-v0
. - Terms of Service (
DWN_TERMS_OF_SERVICE_FILE_PATH=/path/to/terms-of-service.txt
) - new tenants must GET/registration/terms-of-service
to fetch the terms. These terms must be displayed to the human end-user, who must actively accept them. When the user accepts the terms, send the sha256 hash of the accepted terms and the user's did via POST/registration/terms-of-service
. The JSON body should have fieldstermsOfServiceHash
anddid
. To change the terms, update the file and restart the server. Users that accepted the old terms will be blocked until they accept the new terms. This registration requirement is listed in/info
asterms-of-service
.
the server exposes information about itself via the /info
endpoint, which returns data in the following format:
{
"server": "@web5/dwn-server",
"maxFileSize": 1073741824,
"registrationRequirements": ["proof-of-work-sha256-v0", "terms-of-service"],
"version": "0.1.5",
"sdkVersion": "0.2.6",
"webSocketSupport": "true"
}
server
is read from theprocess.env.npm_package_name
variable thatnpm
provides. If that does not exist, it will check for aDWN_SERVER_PACKAGE_NAME
environment variable set by the user, or otherwise it will default to@web5/dwn-server
.version
andsdkVersion
are read from thepackage.json
file. It will locate the file's path either from theprocess.env.npm_package_json
variable thatnpm
provides. If that does not exist, it will check for aDWN_SERVER_PACKAGE_JSON
environment variable set by the user, or otherwise it will default to/dwn-server/package.json
which is the path within the default Docker container build.