Skip to content

Commit

Permalink
BM-102: Single BTC node in docker producing blocks (#95)
Browse files Browse the repository at this point in the history 
* Tried with a setup/run combo.

* Perform the setup at each startup.

* Added to docker-compose.yml

* Try different ports.

* Fix typo.

* What if no ports are mapped?

* Example of command to remove with docker-compose

* Use default ports again.

* Fix typo in image name.

* Set an IP address.

* Rename to bitcoinsim. Remove curl and jq
  • Loading branch information
@aakoshh
aakoshh authored Aug 17, 2022
1 parent 30b45f4 commit 5b77d2f
Show file tree
Hide file tree
Showing 6 changed files with 345 additions and 2 deletions.
11 changes: 9 additions & 2 deletions contrib/images/Makefile
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
all: babylond-env
all: babylond-env bitcoinsim

babylond-env: babylond-rmi
docker build --tag babylonchain/babylond -f babylond-env/Dockerfile \
Expand All @@ -11,4 +11,11 @@ babylond-dlv: babylond-rmi
babylond-rmi:
docker rmi babylonchain/babylond 2>/dev/null; true

.PHONY: all babylond-env babylond-dlv babylond-rmi
bitcoinsim:
docker build --tag babylonchain/bitcoinsim -f bitcoinsim/Dockerfile \
$(shell git rev-parse --show-toplevel)

bitcoinsim-rmi:
docker rmi babylonchain/bitcoinsim 2>/dev/null; true

.PHONY: all babylond-env babylond-dlv babylond-rmi bitcoinsim bitcoinsim-rmi
33 changes: 33 additions & 0 deletions contrib/images/bitcoinsim/Dockerfile
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
FROM golang:1.18-alpine AS build

RUN apk add --no-cache openssh-client git make build-base linux-headers libc-dev

WORKDIR /work

ENV GO111MODULE=on

RUN git clone https://github.com/btcsuite/btcd.git
RUN cd btcd && go install -v . ./cmd/... && cd -

RUN git clone https://github.com/btcsuite/btcwallet.git
RUN cd btcwallet && go install -v . ./cmd/... && cd -


FROM alpine:3.14 AS run
RUN apk add bash expect expect-dev
WORKDIR /bitcoinsim

COPY --from=build /go/bin/* /usr/bin/
COPY contrib/images/bitcoinsim/wrapper.sh /bitcoinsim/wrapper.sh
COPY contrib/images/bitcoinsim/btcwallet_create.exp /bitcoinsim/btcwallet_create.exp

ENV RPCUSER=rpcuser
ENV RPCPASS=rpcpass
ENV PASSPHRASE=pass
ENV GENERATE_INTERVAL_SECS=30

EXPOSE 18554 18555 18556

ENTRYPOINT ["/bitcoinsim/wrapper.sh"]
CMD []
STOPSIGNAL SIGTERM
162 changes: 162 additions & 0 deletions contrib/images/bitcoinsim/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# bitcoinsim

`bitcoinsim` is a Docker image we build to be able to run a single Bitcoin node in the local devnet that produces blocks at regular intervals without consuming CPU.

To achieve this we use [btcd](https://github.com/btcsuite/btcd) in `--simnet` mode and make calls to the [generate](https://github.com/btcsuite/btcd/blob/v0.23.1/rpcserver.go#L886) API endpoint.

The image also runs a [btcwallet](https://github.com/btcsuite/btcwallet) process which can be used to make transfers. To talk to `btcd` programmatically we can use the [rpcclient](https://github.com/btcsuite/btcd/blob/v0.23.1/rpcclient/mining.go#L54-L62), and for the wallet the [chain](https://github.com/btcsuite/btcwallet/tree/master/chain) client.

See more at:
* https://gist.github.com/davecgh/2992ed85d41307e794f6
* http://piotrpasich.com/how-to-setup-own-bitcoin-simulation-network/
* https://blog.krybot.com/a?ID=00950-ef39d506-48ea-45df-81c5-2115e2f4a0f6


## Build

You can build the image with the following command:

```bash
make bitcoinsim
```

## Test

One way to see if it works is to run the container interactively:

```bash
docker run -it --rm --name bitcoinsim babylonchain/bitcoinsim
```

The logs should show that a wallet is created with and mining is started:

```console
$ docker run -it --rm babylonchain/bitcoinsim
Starting btcd...
Creating a wallet...
spawn btcwallet --simnet -u rpcuser -P rpcpass --create
2022-08-16 23:39:33.091 [INF] BTCD: Version 0.23.1-beta
2022-08-16 23:39:33.091 [INF] BTCD: Loading block database from '/root/.btcd/data/simnet/blocks_ffldb'
Enter the private passphrase for your new wallet:
Confirm passphrase:
Do you want to add an additional layer of encryption for public data? (n/no/y/yes) [no]: n
Do you have an existing wallet seed you want to use? (n/no/y/yes) [no]: n
Your wallet generation seed is:
b46944f9dd96792481ec71c4d945a3e79959e429c5b77fcda14e587cabebc672
IMPORTANT: Keep the seed in a safe place as you
will NOT be able to restore your wallet without it.
Please keep in mind that anyone who has access
to the seed can also restore your wallet thereby
giving them access to all your funds, so it is
imperative that you keep it in a secure location.
Once you have stored the seed in a safe and secure location, enter "OK" to continue: OK
Creating the wallet...
2022-08-16 23:39:33.114 [INF] BTCD: Block database loaded
2022-08-16 23:39:33.116 [INF] INDX: Committed filter index is enabled
2022-08-16 23:39:33.116 [INF] INDX: Catching up indexes from height -1 to 0
2022-08-16 23:39:33.116 [INF] INDX: Indexes caught up to height 0
2022-08-16 23:39:33.116 [INF] CHAN: Chain state (height 0, hash 683e86bd5c6d110d91b94b97137ba6bfe02dbbdb8e3dff722a669b5d69d77af6, totaltx 1, work 2)
2022-08-16 23:39:33.117 [INF] RPCS: Generating TLS certificates...
2022-08-16 23:39:33.136 [INF] RPCS: Done generating TLS certificates
2022-08-16 23:39:33.141 [INF] AMGR: Loaded 0 addresses from file '/root/.btcd/data/simnet/peers.json'
2022-08-16 23:39:33.141 [INF] RPCS: RPC server listening on 0.0.0.0:18556
2022-08-16 23:39:33.141 [INF] CMGR: Server listening on 0.0.0.0:18555
2022-08-16 23:39:35.814 [INF] WLLT: Opened wallet
The wallet has been created successfully.
Starting btcwallet...
2022-08-16 23:39:35.822 [INF] BTCW: Version 0.15.1-alpha
2022-08-16 23:39:35.822 [INF] BTCW: Generating TLS certificates...
2022-08-16 23:39:35.846 [INF] BTCW: Done generating TLS certificates
2022-08-16 23:39:35.846 [INF] RPCS: Listening on 0.0.0.0:18554
2022-08-16 23:39:35.846 [INF] BTCW: Attempting RPC client connection to localhost:18556
2022-08-16 23:39:35.872 [INF] RPCS: New websocket client 127.0.0.1:49538
2022-08-16 23:39:35.872 [INF] CHNS: Established connection to RPC server localhost:18556
2022-08-16 23:39:36.768 [INF] WLLT: Opened wallet
2022-08-16 23:39:36.771 [INF] WLLT: RECOVERY MODE ENABLED -- rescanning for used addresses with recovery_window=250
2022-08-16 23:39:36.790 [INF] WLLT: Started rescan from block 683e86bd5c6d110d91b94b97137ba6bfe02dbbdb8e3dff722a669b5d69d77af6 (height 0) for 0 addresses
2022-08-16 23:39:36.791 [INF] RPCS: Beginning rescan for 0 addresses
2022-08-16 23:39:36.791 [INF] RPCS: Skipping rescan as client has no addrs/utxos
2022-08-16 23:39:36.791 [INF] RPCS: Finished rescan
2022-08-16 23:39:36.791 [INF] WLLT: Catching up block hashes to height 0, this might take a while
2022-08-16 23:39:36.792 [INF] WLLT: Done catching up block hashes
2022-08-16 23:39:36.792 [INF] WLLT: Finished rescan for 0 addresses (synced to block 683e86bd5c6d110d91b94b97137ba6bfe02dbbdb8e3dff722a669b5d69d77af6, height 0)
Creating miner address...
Restarting btcd with mining address Sjfg9uCaTqSR7UvGRuuMRJtUtWRhoBA7YX...
2022-08-16 23:39:40.861 [ERR] CHNS: Websocket receive error from localhost:18556: websocket: close 1006 unexpected EOF
2022-08-16 23:39:40.862 [INF] CHNS: Failed to connect to localhost:18556: dial tcp 127.0.0.1:18556: connect: connection refused
2022-08-16 23:39:40.862 [INF] CHNS: Retrying connection to localhost:18556 in 5s
2022-08-16 23:39:40.873 [INF] BTCD: Version 0.23.1-beta
2022-08-16 23:39:40.873 [INF] BTCD: Loading block database from '/root/.btcd/data/simnet/blocks_ffldb'
2022-08-16 23:39:40.897 [INF] BCDB: Detected unclean shutdown - Repairing...
2022-08-16 23:39:40.900 [INF] BCDB: Database sync complete
2022-08-16 23:39:40.901 [INF] BTCD: Block database loaded
2022-08-16 23:39:40.910 [INF] INDX: Committed filter index is enabled
2022-08-16 23:39:40.913 [INF] INDX: Catching up indexes from height -1 to 0
2022-08-16 23:39:40.913 [INF] INDX: Indexes caught up to height 0
2022-08-16 23:39:40.913 [INF] CHAN: Chain state (height 0, hash 683e86bd5c6d110d91b94b97137ba6bfe02dbbdb8e3dff722a669b5d69d77af6, totaltx 1, work 2)
2022-08-16 23:39:40.927 [INF] AMGR: Loaded 0 addresses from file '/root/.btcd/data/simnet/peers.json'
2022-08-16 23:39:40.927 [INF] RPCS: RPC server listening on 0.0.0.0:18556
2022-08-16 23:39:40.927 [INF] CMGR: Server listening on 0.0.0.0:18555
Generating enought blocks for the first coinbase to mature...
2022-08-16 23:39:45.890 [INF] RPCS: New websocket client 127.0.0.1:49552
2022-08-16 23:39:45.890 [INF] CHNS: Reestablished connection to RPC server localhost:18556
2022-08-16 23:39:45.890 [INF] WLLT: RECOVERY MODE ENABLED -- rescanning for used addresses with recovery_window=250
2022-08-16 23:39:45.891 [WRN] CHNS: Received unexpected reply: {"hash":"683e86bd5c6d110d91b94b97137ba6bfe02dbbdb8e3dff722a669b5d69d77af6","height":0} (id 16)
2022-08-16 23:39:45.896 [INF] MINR: Block submitted via CPU miner accepted (hash 3650f1363cc7fc4c3f01d1d904caa9f9687ef08927498658b6c50b585edb7872, amount 50 BTC)
...
2022-08-16 23:39:45.920 [INF] MINR: Block submitted via CPU miner accepted (hash 7a6dc189c7ffc38db73304e2993dacb241a932fc3980ca840da466b58e2d26d8, amount 50 BTC)
[
"3650f1363cc7fc4c3f01d1d904caa9f9687ef08927498658b6c50b585edb7872",
"243d889cc77cbabcb4d28a3c5224dc3c0ef6970d0ea9f7e143e1216abb87c0d4",
"1577ee934812dace99bbab757ce6ccbed443da0863baab3b8ba7efe1b4453917",
...
"63cb51a672732ec1175b1ed288d20cd8f1cb8653fdd7dc31f90fbdb9844c7998",
"7a6dc189c7ffc38db73304e2993dacb241a932fc3980ca840da466b58e2d26d8"
]
2022-08-16 23:39:46.148 [INF] WLLT: Catching up block hashes to height 32, this might take a while
2022-08-16 23:39:46.151 [INF] WLLT: Done catching up block hashes
2022-08-16 23:39:46.151 [INF] WLLT: Finished rescan for 1 address (synced to block 42ad6a44786a2c6edcfcca7098b3d18cc13c668a004c494ca87566c404f9ae34, height 32)
Checking balance...
50
Generating a block every 30 seconds.
Press [CTRL+C] to stop...
2022-08-16 23:39:50.975 [INF] MINR: Block submitted via CPU miner accepted (hash 1e4931b5b590e297f736da282ab1367f1704ff5a21943c3883161d4f0a60dda2, amount 50 BTC)
[
"1e4931b5b590e297f736da282ab1367f1704ff5a21943c3883161d4f0a60dda2"
]

```

Then we can connect to this from another container to query the balance:

```console
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
93e58681637d babylonchain/bitcoinsim "/bitcoinsim/wrapper.sh" 5 minutes ago Up 5 minutes 18554-18556/tcp bitcoinsim
$ docker exec -it bitcoinsim sh
/bitcoinsim # btcctl --simnet --wallet -u $RPCUSER -P $RPCPASS getbalance
600
/bitcoinsim #

```

The balance will go up as more blocks mature. The default coinbase maturity is 100 blocks.

## Use in docker-compose

The image has been added to the main `docker-compose.yml` file. It will build it if it's not built already and start running it as part of the local testnet. The other containers can use the default ports to connect to it, which are also exposed on the host.

It can be started on its own like so:

```bash
docker-compose up -d bitcoinsim
```

Currently the image doesn't support restarting, the container has to be completely removed and recreated if it's stopped. It can be removed with the following command:

```bash
docker-compose stop bitcoinsim
docker-compose rm bitcoinsim
```

The ports are mapped to the same default ports that `btcd` and `btcwallet` would use, so if the host already runs these services they might clash.
73 changes: 73 additions & 0 deletions contrib/images/bitcoinsim/btcwallet_create.exp
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
#!/usr/bin/expect -f
#
# This Expect script was generated by autoexpect on Tue Aug 16 22:44:36 2022
# Expect and autoexpect were both written by Don Libes, NIST.
#
# Note that autoexpect does not guarantee a working script. It
# necessarily has to guess about certain things. Two reasons a script
# might fail are:
#
# 1) timing - A surprising number of programs (rn, ksh, zsh, telnet,
# etc.) and devices discard or ignore keystrokes that arrive "too
# quickly" after prompts. If you find your new script hanging up at
# one spot, try adding a short sleep just before the previous send.
# Setting "force_conservative" to 1 (see below) makes Expect do this
# automatically - pausing briefly before sending each character. This
# pacifies every program I know of. The -c flag makes the script do
# this in the first place. The -C flag allows you to define a
# character to toggle this mode off and on.

set force_conservative 0 ;# set to 1 to force conservative mode even if
;# script wasn't run conservatively originally
if {$force_conservative} {
set send_slow {1 .1}
proc send {ignore arg} {
sleep .1
exp_send -s -- $arg
}
}

#
# 2) differing output - Some programs produce different output each time
# they run. The "date" command is an obvious example. Another is
# ftp, if it produces throughput statistics at the end of a file
# transfer. If this causes a problem, delete these patterns or replace
# them with wildcards. An alternative is to use the -p flag (for
# "prompt") which makes Expect only look for the last line of output
# (i.e., the prompt). The -P flag allows you to define a character to
# toggle this mode off and on.
#
# Read the man page for more info.
#
# -Don

set RPCUSER [lindex $argv 0]
set RPCPASS [lindex $argv 1]
set PASSPHRASE [lindex $argv 2]

set timeout -1
spawn btcwallet --simnet -u $RPCUSER -P $RPCPASS --create
match_max 100000
expect -exact "Enter the private passphrase for your new wallet: "
send -- "$PASSPHRASE\r"
expect -exact "\r
Confirm passphrase: "
send -- "$PASSPHRASE\r"
expect -exact "\r
Do you want to add an additional layer of encryption for public data? (n/no/y/yes) \[no\]: "
send -- "n\r"
expect -exact "n\r
Do you have an existing wallet seed you want to use? (n/no/y/yes) \[no\]: "
send -- "n\r"
expect "n\r
Your wallet generation seed is:\r
*\r
IMPORTANT: Keep the seed in a safe place as you\r
will NOT be able to restore your wallet without it.\r
Please keep in mind that anyone who has access\r
to the seed can also restore your wallet thereby\r
giving them access to all your funds, so it is\r
imperative that you keep it in a secure location.\r
Once you have stored the seed in a safe and secure location, enter \"OK\" to continue: "
send -- "OK\r"
expect eof
56 changes: 56 additions & 0 deletions contrib/images/bitcoinsim/wrapper.sh
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
#!/usr/bin/env sh
set -euo pipefail
#set -x

# btcctl will be looking for this file, but the wallet doesn't create it.
mkdir -p /root/.btcwallet && touch /root/.btcwallet/btcwallet.conf
mkdir -p /root/.btcd && touch /root/.btcd/btcd.conf

# Create a wallet with and a miner address, then mine enough blocks for the miner to have some initial balance.

echo "Starting btcd..."
btcd --simnet -u $RPCUSER -P $RPCPASS --rpclisten=0.0.0.0:18556 --listen=0.0.0.0:18555 2>&1 &
BTCD_PID=$!

echo "Creating a wallet..."
# Used autoexpect to create the wallet in the first instance.
# https://stackoverflow.com/questions/4857702/how-to-provide-password-to-a-command-that-prompts-for-one-in-bash
expect btcwallet_create.exp $RPCUSER $RPCPASS $PASSPHRASE

echo "Starting btcwallet..."
btcwallet --simnet -u $RPCUSER -P $RPCPASS --rpclisten=0.0.0.0:18554 2>&1 &
BTCWALLET_PID=$!

# Allow some time for the wallet to start
sleep 5

echo "Creating miner address..."
MINING_ADDR=$(btcctl --simnet --wallet -u $RPCUSER -P $RPCPASS getnewaddress)
echo $MINING_ADDR > mining.addr

echo "Restarting btcd with mining address $MINING_ADDR..."
kill -9 $BTCD_PID
sleep 1
btcd --simnet -u $RPCUSER -P $RPCPASS --rpclisten=0.0.0.0:18556 --listen=0.0.0.0:18555 --miningaddr=$MINING_ADDR 2>&1 &
BTCD_PID=$!

# Allow btcd to start
sleep 5

echo "Generating enough blocks for the first coinbase to mature..."
btcctl --simnet -u $RPCUSER -P $RPCPASS generate 100

# Allow some time for the wallet to catch up.
sleep 5

echo "Checking balance..."
btcctl --simnet --wallet -u $RPCUSER -P $RPCPASS getbalance

echo "Generating a block every ${GENERATE_INTERVAL_SECS} seconds."
echo "Press [CTRL+C] to stop..."
while true
do
btcctl --simnet -u $RPCUSER -P $RPCPASS generate 1

sleep ${GENERATE_INTERVAL_SECS}
done
12 changes: 12 additions & 0 deletions docker-compose.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,18 @@ services:
localnet:
ipv4_address: 192.168.10.5

bitcoinsim:
build:
context: ./
dockerfile: ./contrib/images/bitcoinsim/Dockerfile
image: babylonchain/bitcoinsim:latest
container_name: bitcoinsim
ports:
- 18554-18556:18554-18556
networks:
localnet:
ipv4_address: 192.168.10.6

networks:
localnet:
driver: bridge
Expand Down

0 comments on commit 5b77d2f

Please sign in to comment.