AddressBook: Nil Address

[alexanderbez]

Tendermint version:

v0.32.0-dev0

ABCI app

Gaia

What happened:

A 4 node local network of gaia hangs during start:

I[2019-06-20|16:31:22.880] Starting ABCI with Tendermint                module=main
I[2019-06-20|16:31:23.028] Starting multiAppConn                        module=proxy impl=multiAppConn
I[2019-06-20|16:31:23.029] Starting localClient                         module=abci-client connection=query impl=localClient
I[2019-06-20|16:31:23.029] Starting localClient                         module=abci-client connection=mempool impl=localClient
I[2019-06-20|16:31:23.029] Starting localClient                         module=abci-client connection=consensus impl=localClient
I[2019-06-20|16:31:23.029] Starting EventBus                            module=events impl=EventBus
I[2019-06-20|16:31:23.029] Starting PubSub                              module=pubsub impl=PubSub
I[2019-06-20|16:31:23.066] Starting IndexerService                      module=txindex impl=IndexerService
I[2019-06-20|16:31:23.066] ABCI Handshake App Info                      module=consensus height=0 hash= software-version= protocol-version=0
I[2019-06-20|16:31:23.066] ABCI Replay Blocks                           module=consensus appHeight=0 storeHeight=0 stateHeight=0
I[2019-06-20|16:31:23.071] asserted all invariants                      module=x/crisis duration=13.3µs height=0
I[2019-06-20|16:31:23.089] Completed ABCI Handshake - Tendermint and App are synced module=consensus appHeight=0 appHash=
I[2019-06-20|16:31:23.089] Version info                                 module=node software=0.31.7 block=10 p2p=7
I[2019-06-20|16:31:23.089] This node is a validator                     module=consensus addr=060B95D2B8C399D715B0EE6A3CBC828224B3F62C pubKey=PubKeyEd25519{AFE053776F3CF570A91D39118EAFEAFB39572962201C6A67A048FA63239978AA}
I[2019-06-20|16:31:23.169] P2P Node ID                                  module=p2p ID=a206f3fd5e8f1ba3f6817141df9c909aba0819e3 file=/gaiad/node0/gaiad/config/node_key.json
I[2019-06-20|16:31:23.169] Adding persistent peers                      module=p2p addrs="[a2130493db7d51045df5547bcbe04d3d32f98216@192.168.10.5:26656 a3fb0c1bbe7dbbf8b0756374adf02d1ccac4c413@192.168.10.4:26656 ee6c6e397cec825795422622d5f3f867ce2ddd55@192.168.10.3:26656]"
I[2019-06-20|16:31:23.169] Add our address to book                      module=p2p book=/gaiad/node0/gaiad/config/addrbook.json addr=<nil>:0

The problem seems to be:

I[2019-06-20|16:31:23.169] Add our address to book                      module=p2p book=/gaiad/node0/gaiad/config/addrbook.json addr=<nil>:0

To Replicate:

1. Clone gaia
2. Check out branch bez/update-sdk-events
3. Modify networks/local/gaiadnode/Dockerfile as follows:

-CMD ["start"]
+CMD ["start", "--log_level=*:debug"]

4. Run: rm -fr ./build/**; make clean install build-linux localnet-start
5. Tail one of the container logs: docker logs -f gaiadnode0

ref: #3716

/cc @melekes

[alexanderbez]

#3740 closes this apparently.

[melekes]

#3740 should fix this.

[alexanderbez]

@melekes, the nodes still stall:

I[2019-06-21|12:05:51.022] Starting ABCI with Tendermint                module=main
I[2019-06-21|12:05:51.186] Starting multiAppConn                        module=proxy impl=multiAppConn
I[2019-06-21|12:05:51.186] Starting localClient                         module=abci-client connection=query impl=localClient
I[2019-06-21|12:05:51.186] Starting localClient                         module=abci-client connection=mempool impl=localClient
I[2019-06-21|12:05:51.186] Starting localClient                         module=abci-client connection=consensus impl=localClient
I[2019-06-21|12:05:51.186] Starting EventBus                            module=events impl=EventBus
I[2019-06-21|12:05:51.186] Starting PubSub                              module=pubsub impl=PubSub
I[2019-06-21|12:05:51.230] Starting IndexerService                      module=txindex impl=IndexerService
I[2019-06-21|12:05:51.231] ABCI Handshake App Info                      module=consensus height=0 hash= software-version= protocol-version=0
I[2019-06-21|12:05:51.231] ABCI Replay Blocks                           module=consensus appHeight=0 storeHeight=0 stateHeight=0
I[2019-06-21|12:05:51.234] asserted all invariants                      module=x/crisis duration=11µs height=0
I[2019-06-21|12:05:51.245] Completed ABCI Handshake - Tendermint and App are synced module=consensus appHeight=0 appHash=
I[2019-06-21|12:05:51.245] Version info                                 module=node software=0.31.7 block=10 p2p=7
I[2019-06-21|12:05:51.245] This node is a validator                     module=consensus addr=4F9A4DF162A0E8E26770E6470A59CAD8684E5A7E pubKey=PubKeyEd25519{A899146397C44D55584ED3DEA338821498DB43535F8622F461864CF3FE6DACE8}
I[2019-06-21|12:05:51.295] P2P Node ID                                  module=p2p ID=a363d98879564e59e8a88006b7b642bec7fa7b68 file=/gaiad/node1/gaiad/config/node_key.json
I[2019-06-21|12:05:51.296] Adding persistent peers                      module=p2p addrs="[01fb1529989a39e65d1735f161480e184e676692@192.168.10.5:26656 56f0551e5172552ff0552aaf9b8fbf9f38250404@192.168.10.4:26656 8e3109249f3b32f8343058e0b1f1b9b33741e3ea@192.168.10.2:26656]"
I[2019-06-21|12:05:51.296] Add our address to book                      module=p2p book=/gaiad/node1/gaiad/config/addrbook.json addr=a363d98879564e59e8a88006b7b642bec7fa7b68@0.0.0.0:26656

[alexanderbez]

output from aborting the process: https://pastebin.com/CqxW0ASi

[melekes]

Fixed in #3743.