Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

amber/flint validator loses peer to sentry after "Invalid justification provided by $SENTRY_IDENTITY for #..." #259

Closed
AGx10k opened this issue Jun 28, 2020 · 6 comments
Closed

amber/flint validator loses peer to sentry after "Invalid justification provided by $SENTRY_IDENTITY for #..." #259

AGx10k opened this issue Jun 28, 2020 · 6 comments

Comments

@AGx10k
Copy link

AGx10k commented Jun 28, 2020

seen it several times on Amber and Flint networks. looks like peer is lost after invalid justification by sentry:

2020-06-28 00:29:55 Idle (1 peers), best: #1355404 (0x6075…0f45), finalized #1340999 (0x95e1…8484), ⬇ 4.8kiB/s ⬆ 4.1kiB/s
2020-06-28 00:29:56 Invalid justification provided by QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb for #0xe6a1…7a87
2020-06-28 00:29:56 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:00 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:00 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:00 Idle (0 peers), best: #1355404 (0x6075…0f45), finalized #1340999 (0x95e1…8484), ⬇ 18.0kiB/s ⬆ 11.8kiB/s

full logs for tonight's issue on Amber (it was handled by my script https://github.com/AGx10k/substrate-restart-stalled-blocks):

amber-validator (QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb is sentry):

2020-06-28 00:28:54 Starting consensus session on top of parent 0x4825444ef8ff3baf2bc57a49770fdfff64481feb94243b7a85534d43bcdf0691
2020-06-28 00:28:54 Prepared block for proposing at 1355394 [hash: 0x35d68f2e44dd87110e6034c01878b50b22895d9ec87e23fc2ad94559ae353158; parent_hash: 0x4825…0691; extrinsics (3): [0xa0f3…8a30, 0x3f60…fda3, 0x78e8…efdc]]
2020-06-28 00:28:54 Pre-sealed block for proposal at 1355394. Hash now 0x4153f7c2bac71e33f1c3c5d4858750a2294b2ce7b1ef856929af9933e744af58, previously 0x35d68f2e44dd87110e6034c01878b50b22895d9ec87e23fc2ad94559ae353158.
2020-06-28 00:28:54 Imported #1355394 (0x4153…af58)
2020-06-28 00:28:55 Idle (1 peers), best: #1355394 (0x4153…af58), finalized #1340999 (0x95e1…8484), ⬇ 5.2kiB/s ⬆ 4.8kiB/s
2020-06-28 00:29:00 Imported #1355395 (0x178d…38b0)
2020-06-28 00:29:00 Idle (1 peers), best: #1355395 (0x178d…38b0), finalized #1340999 (0x95e1…8484), ⬇ 4.2kiB/s ⬆ 4.0kiB/s
2020-06-28 00:29:05 Idle (1 peers), best: #1355395 (0x178d…38b0), finalized #1340999 (0x95e1…8484), ⬇ 3.7kiB/s ⬆ 3.5kiB/s
2020-06-28 00:29:06 Imported #1355396 (0x3349…2610)
2020-06-28 00:29:10 Idle (1 peers), best: #1355396 (0x3349…2610), finalized #1340999 (0x95e1…8484), ⬇ 5.1kiB/s ⬆ 4.5kiB/s
2020-06-28 00:29:12 Starting consensus session on top of parent 0x3349ec313e0b517c3bf9a67600059c1700c5468f0aadc403dfc06a45eb152610
2020-06-28 00:29:12 Prepared block for proposing at 1355397 [hash: 0xc15f9613bd0cb03ff0830388a5ba5bdbee59a4a97e7e14b4962ef9b37e1e35f2; parent_hash: 0x3349…2610; extrinsics (2): [0x14c3…bbb2, 0x78e8…efdc]]
2020-06-28 00:29:12 Pre-sealed block for proposal at 1355397. Hash now 0xbfb1d30e852c1119dad909d239ac477f27289e1e3c46c85c9293c7dfa9007d22, previously 0xc15f9613bd0cb03ff0830388a5ba5bdbee59a4a97e7e14b4962ef9b37e1e35f2.
2020-06-28 00:29:12 Imported #1355397 (0xbfb1…7d22)
2020-06-28 00:29:15 Idle (1 peers), best: #1355397 (0xbfb1…7d22), finalized #1340999 (0x95e1…8484), ⬇ 4.7kiB/s ⬆ 4.3kiB/s
2020-06-28 00:29:18 Imported #1355398 (0xd126…6efd)
2020-06-28 00:29:20 Idle (1 peers), best: #1355398 (0xd126…6efd), finalized #1340999 (0x95e1…8484), ⬇ 3.8kiB/s ⬆ 3.6kiB/s
2020-06-28 00:29:24 Imported #1355399 (0x1bb5…3139)
2020-06-28 00:29:25 Idle (1 peers), best: #1355399 (0x1bb5…3139), finalized #1340999 (0x95e1…8484), ⬇ 5.0kiB/s ⬆ 4.4kiB/s
2020-06-28 00:29:30 Imported #1355400 (0x7cc4…b801)
2020-06-28 00:29:30 Idle (1 peers), best: #1355400 (0x7cc4…b801), finalized #1340999 (0x95e1…8484), ⬇ 4.6kiB/s ⬆ 4.1kiB/s
2020-06-28 00:29:35 Idle (1 peers), best: #1355400 (0x7cc4…b801), finalized #1340999 (0x95e1…8484), ⬇ 3.7kiB/s ⬆ 3.5kiB/s
2020-06-28 00:29:36 New epoch 381 launching at block 0xe6a1…7a87 (block slot 265550696 >= start slot 265550696).
2020-06-28 00:29:36 Next epoch starts at slot 265554296
2020-06-28 00:29:36 Imported #1355401 (0xe6a1…7a87)
2020-06-28 00:29:40 Idle (1 peers), best: #1355401 (0xe6a1…7a87), finalized #1340999 (0x95e1…8484), ⬇ 5.1kiB/s ⬆ 4.5kiB/s
2020-06-28 00:29:42 Imported #1355402 (0xe09c…5e3a)
2020-06-28 00:29:42 Imported #1355402 (0x3452…0634)
2020-06-28 00:29:45 Idle (1 peers), best: #1355402 (0xe09c…5e3a), finalized #1340999 (0x95e1…8484), ⬇ 4.8kiB/s ⬆ 4.0kiB/s
2020-06-28 00:29:48 Starting consensus session on top of parent 0xe09c41fbd15280c1076104e4554d8c6dfa270ade02a957c2ce5dcb3a1c255e3a
2020-06-28 00:29:48 Prepared block for proposing at 1355403 [hash: 0xcb94eebb0390aa26d96aafca132ef0b4564786c64a05d2ec18c635abb436f12c; parent_hash: 0xe09c…5e3a; extrinsics (3): [0x4a13…e5ec, 0xfde5…0b45, 0x78e8…efdc]]
2020-06-28 00:29:48 Pre-sealed block for proposal at 1355403. Hash now 0x92dc07422321aec3c6be85c0cac88dabf72e1c8a2b6216dda5f7c02a45f00cae, previously 0xcb94eebb0390aa26d96aafca132ef0b4564786c64a05d2ec18c635abb436f12c.
2020-06-28 00:29:48 Imported #1355403 (0x92dc…0cae)
2020-06-28 00:29:48 Imported #1355403 (0xc0db…e2be)
2020-06-28 00:29:48 Imported #1355403 (0x99b7…a682)
2020-06-28 00:29:50 Idle (1 peers), best: #1355403 (0x92dc…0cae), finalized #1340999 (0x95e1…8484), ⬇ 4.8kiB/s ⬆ 4.5kiB/s
2020-06-28 00:29:54 Reorg from #1355403,0x92dc…0cae to #1355404,0x6075…0f45, common ancestor #1355402,0xe09c…5e3a
2020-06-28 00:29:54 Imported #1355404 (0x6075…0f45)
2020-06-28 00:29:55 Idle (1 peers), best: #1355404 (0x6075…0f45), finalized #1340999 (0x95e1…8484), ⬇ 4.8kiB/s ⬆ 4.1kiB/s
2020-06-28 00:29:56 Invalid justification provided by QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb for #0xe6a1…7a87
2020-06-28 00:29:56 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:00 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:00 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:00 Idle (0 peers), best: #1355404 (0x6075…0f45), finalized #1340999 (0x95e1…8484), ⬇ 18.0kiB/s ⬆ 11.8kiB/s
2020-06-28 00:30:01 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:01 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:02 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:02 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:04 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:04 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:05 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:05 Idle (0 peers), best: #1355404 (0x6075…0f45), finalized #1340999 (0x95e1…8484), ⬇ 37.4kiB/s ⬆ 24.7kiB/s
2020-06-28 00:30:06 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:06 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:08 Invalid justification provided by QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb for #0xe6a1…7a87
2020-06-28 00:30:08 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:09 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:30:11 Idle (0 peers), best: #1355406 (0x6d02…ac96), finalized #1340999 (0x95e1…8484), ⬇ 24.2kiB/s ⬆ 16.3kiB/s
2020-06-28 00:30:16 Idle (0 peers), best: #1355406 (0x6d02…ac96), finalized #1340999 (0x95e1…8484), ⬇ 23.8kiB/s ⬆ 18.4kiB/s

////// skip repeats

2020-06-28 00:36:41 Idle (0 peers), best: #1355406 (0x6d02…ac96), finalized #1340999 (0x95e1…8484), ⬇ 5.1kiB/s ⬆ 4.9kiB/s
2020-06-28 00:36:46 Idle (0 peers), best: #1355406 (0x6d02…ac96), finalized #1340999 (0x95e1…8484), ⬇ 4.4kiB/s ⬆ 4.3kiB/s

////// validator is restarted by script because best height is stalled

2020-06-28 00:36:51 It isn't safe to expose RPC publicly without a proxy server that filters available set of RPC methods.
2020-06-28 00:36:51 It isn't safe to expose RPC publicly without a proxy server that filters available set of RPC methods.
2020-06-28 00:36:51 Centrifuge Chain Node
2020-06-28 00:36:51   version 2.0.0-908086f-x86_64-linux-gnu
2020-06-28 00:36:51   by Centrifuge, 2019-2020
2020-06-28 00:36:51 Chain specification: Centrifuge Testnet Amber CC2
2020-06-28 00:36:51 Node name: AGx1 🔶
2020-06-28 00:36:51 Roles: AUTHORITY
2020-06-28 00:36:59 Highest known block at #1355406
2020-06-28 00:36:59 Local node identity is: QmXHW5FKi9JdKZBYKpjCcjzyKEUgfhusGbBbwUWdNkjRDL
2020-06-28 00:36:59 Starting BABE Authorship worker
2020-06-28 00:36:59 Prometheus server started at 0.0.0.0:9616
2020-06-28 00:36:59 Libp2p listener ListenerId(1) closed
2020-06-28 00:37:00 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:37:01 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:37:03 Reserved peer QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb disconnected
2020-06-28 00:37:04 Idle (0 peers), best: #1355406 (0x6d02…ac96), finalized #1340999 (0x95e1…8484), ⬇ 0.7kiB/s ⬆ 0.7kiB/s
2020-06-28 00:37:07 Applying authority set change scheduled at block #1341002
2020-06-28 00:37:08 Applying GRANDPA set change to new set [(Public(52237c29a7cb263163daca10670bc94629836fb1f52542e87c9fe45ad6b75dad (5DvQKoQV...)), 1), (Public(b7e87f546c26514f73c3dcbedc634748acdc12bc152ac63c0051674d6cc70b5e (5GDqgkfV...)), 1), (Public(b5fe42f275370d783ac2e8bccf7ce8696cddeb4738764ae2e9ce62d08b3ef59b (5GBL4Bkn...)), 1), (Public(424d9d04d03ca22c27498232c3e1a37c658224dcfd0b6759f3e2e90ec822caef (5DZe5Dcr...)), 1), (Public(7c74b9d92480283265c8fbb63ba0c20c9aa74620841b9e4eb0d9cbef473f7f99 (5EstUCPa...)), 1), (Public(e1a2d79bef77946d8210739f87f099c578b375a5aab9e10964bea8c05cd3f4bb (5HAZ1Gde...)), 1), (Public(4792315cc18f23432816dbff0ea6f8dc80e9a16ce2057582bccbabec8b19119a (5DgYghea...)), 1), (Public(09c4956b18b884d0ff69e31ddd0836af26f804c0a10691ad83cc6123ffb39db0 (5CHWgiqx...)), 1), (Public(2f765c5a4a4f7c2b19dd058066c2f1b2c527db973c3f34545fb68b53c22b15b6 (5D8wGWaQ...)), 1), (Public(a630c92eaa71e5b5d3666444da89705b8f9c08b9adc9523fe2446b30fcab612b (5FpcJGF6...)), 1), (Public(55075b215a98b93cb3f33ffe60a1de36a2e64520251060bd6e1c5b45c53ac91c (5DzC7Uqy...)), 1), (Public(fcf8d1d4d5eb0cc7fa48fa9c00e3d249bae3bb0f6a220bd1e5f7c049f6f5c1c2 (5HnPqcbA...)), 1), (Public(e2ff5155a4f4a70f880f11d46cb2f6ed863585e1e7e2282e7cb7a14c03bd8184 (5HCLXN2C...)), 1), (Public(6ac95555b0e42914fdf91ffb7259a652477e8cace43a1bad1bdac90e0dfa7af0 (5EUijxSC...)), 1), (Public(4f340fda5009d5db62f23e3e4f86250f777e7e457680ab2f31d8a9b5a75a079d (5DrZ75jx...)), 1)]
2020-06-28 00:37:08 Imported justification for block #1341002 that triggers command Changing authorities, signaling voter.
2020-06-28 00:37:08 Applying authority set change scheduled at block #1355401
2020-06-28 00:37:08 Applying GRANDPA set change to new set [(Public(52237c29a7cb263163daca10670bc94629836fb1f52542e87c9fe45ad6b75dad (5DvQKoQV...)), 1), (Public(b7e87f546c26514f73c3dcbedc634748acdc12bc152ac63c0051674d6cc70b5e (5GDqgkfV...)), 1), (Public(b5fe42f275370d783ac2e8bccf7ce8696cddeb4738764ae2e9ce62d08b3ef59b (5GBL4Bkn...)), 1), (Public(424d9d04d03ca22c27498232c3e1a37c658224dcfd0b6759f3e2e90ec822caef (5DZe5Dcr...)), 1), (Public(e1a2d79bef77946d8210739f87f099c578b375a5aab9e10964bea8c05cd3f4bb (5HAZ1Gde...)), 1), (Public(7c74b9d92480283265c8fbb63ba0c20c9aa74620841b9e4eb0d9cbef473f7f99 (5EstUCPa...)), 1), (Public(4792315cc18f23432816dbff0ea6f8dc80e9a16ce2057582bccbabec8b19119a (5DgYghea...)), 1), (Public(09c4956b18b884d0ff69e31ddd0836af26f804c0a10691ad83cc6123ffb39db0 (5CHWgiqx...)), 1), (Public(2f765c5a4a4f7c2b19dd058066c2f1b2c527db973c3f34545fb68b53c22b15b6 (5D8wGWaQ...)), 1), (Public(a630c92eaa71e5b5d3666444da89705b8f9c08b9adc9523fe2446b30fcab612b (5FpcJGF6...)), 1), (Public(55075b215a98b93cb3f33ffe60a1de36a2e64520251060bd6e1c5b45c53ac91c (5DzC7Uqy...)), 1), (Public(fcf8d1d4d5eb0cc7fa48fa9c00e3d249bae3bb0f6a220bd1e5f7c049f6f5c1c2 (5HnPqcbA...)), 1), (Public(e2ff5155a4f4a70f880f11d46cb2f6ed863585e1e7e2282e7cb7a14c03bd8184 (5HCLXN2C...)), 1), (Public(6ac95555b0e42914fdf91ffb7259a652477e8cace43a1bad1bdac90e0dfa7af0 (5EUijxSC...)), 1), (Public(4f340fda5009d5db62f23e3e4f86250f777e7e457680ab2f31d8a9b5a75a079d (5DrZ75jx...)), 1), (Public(db7c98cbb7ba933482d0e40a5c72c95e3594c52f78f1dbb96729114ada6fda2b (5H2VMetr...)), 1)]
2020-06-28 00:37:08 Imported justification for block #1355401 that triggers command Changing authorities, signaling voter.
2020-06-28 00:37:08 Could not cast prevote: previously known block 0xe58bb18c2504d91a87d8d9f8248b0ec4f47fce73004db9dafde0eaf6e3001852 has disappeared
2020-06-28 00:37:09 Idle (1 peers), best: #1355474 (0x82cb…36b0), finalized #1355471 (0x8ea9…b487), ⬇ 8.5kiB/s ⬆ 1.4kiB/s
2020-06-28 00:37:12 Imported #1355475 (0xc3c5…1b95)
2020-06-28 00:37:14 Idle (1 peers), best: #1355475 (0xc3c5…1b95), finalized #1355473 (0xc3be…6670), ⬇ 3.0kiB/s ⬆ 2.1kiB/s
2020-06-28 00:37:18 Imported #1355476 (0x639a…cfb5)
2020-06-28 00:37:19 Idle (1 peers), best: #1355476 (0x639a…cfb5), finalized #1355473 (0xc3be…6670), ⬇ 3.4kiB/s ⬆ 3.6kiB/s
2020-06-28 00:37:24 Idle (1 peers), best: #1355476 (0x639a…cfb5), finalized #1355474 (0x82cb…36b0), ⬇ 3.6kiB/s ⬆ 3.6kiB/s
2020-06-28 00:37:24 Imported #1355477 (0x0b6c…6165)
2020-06-28 00:37:29 Idle (1 peers), best: #1355477 (0x0b6c…6165), finalized #1355475 (0xc3c5…1b95), ⬇ 3.3kiB/s ⬆ 3.2kiB/s
2020-06-28 00:37:29 Libp2p => Failed to put record: QuorumFailed { key: Key(b"\x12 \xe8\xfcw\x0cV\xd4\xed\n\xc6\x9a\x1bWQ]N~\xed\x81_^l\x96R\xad\xdf\xd0X\x8b+\x1e\xb4S"), num_results: 0, quorum: 20 }
2020-06-28 00:37:29 Failed to put hash 'Key(b"\x12 \xe8\xfcw\x0cV\xd4\xed\n\xc6\x9a\x1bWQ]N~\xed\x81_^l\x96R\xad\xdf\xd0X\x8b+\x1e\xb4S")' on Dht.
2020-06-28 00:37:30 Imported #1355478 (0x74ac…ad43)
2020-06-28 00:37:34 Idle (1 peers), best: #1355478 (0x74ac…ad43), finalized #1355476 (0x639a…cfb5), ⬇ 4.0kiB/s ⬆ 4.0kiB/s
2020-06-28 00:37:36 Imported #1355479 (0x9b23…35ef)
2020-06-28 00:37:39 Idle (1 peers), best: #1355479 (0x9b23…35ef), finalized #1355477 (0x0b6c…6165), ⬇ 3.5kiB/s ⬆ 3.4kiB/s
2020-06-28 00:37:42 Imported #1355480 (0x13e6…27a8)
2020-06-28 00:37:44 Idle (1 peers), best: #1355480 (0x13e6…27a8), finalized #1355478 (0x74ac…ad43), ⬇ 3.1kiB/s ⬆ 2.2kiB/s
2020-06-28 00:37:48 Imported #1355481 (0x79bc…fd77)
2020-06-28 00:37:48 Reorg from #1355481,0x79bc…fd77 to #1355481,0x0eda…3b9b, common ancestor #1355480,0x13e6…27a8
2020-06-28 00:37:48 Imported #1355481 (0x0eda…3b9b)
2020-06-28 00:37:49 Idle (1 peers), best: #1355481 (0x0eda…3b9b), finalized #1355478 (0x74ac…ad43), ⬇ 1.2kiB/s ⬆ 1.1kiB/s
2020-06-28 00:37:54 Idle (1 peers), best: #1355481 (0x0eda…3b9b), finalized #1355479 (0x9b23…35ef), ⬇ 4.8kiB/s ⬆ 3.4kiB/s
2020-06-28 00:37:54 Imported #1355482 (0xf29e…bb92)
2020-06-28 00:37:59 Idle (1 peers), best: #1355482 (0xf29e…bb92), finalized #1355480 (0x13e6…27a8), ⬇ 2.3kiB/s ⬆ 2.4kiB/s
2020-06-28 00:38:00 Imported #1355483 (0x5253…2407)
2020-06-28 00:38:04 Idle (1 peers), best: #1355483 (0x5253…2407), finalized #1355481 (0x0eda…3b9b), ⬇ 3.3kiB/s ⬆ 3.3kiB/s
2020-06-28 00:38:06 Imported #1355484 (0x3c0e…6b41)
2020-06-28 00:38:06 Imported #1355484 (0x4e02…33b5)
2020-06-28 00:38:06 Imported #1355484 (0x3718…261b)
2020-06-28 00:38:09 Idle (1 peers), best: #1355484 (0x3c0e…6b41), finalized #1355481 (0x0eda…3b9b), ⬇ 3.5kiB/s ⬆ 2.3kiB/s
2020-06-28 00:38:12 Imported #1355485 (0x0150…1722)
2020-06-28 00:38:14 Idle (1 peers), best: #1355485 (0x0150…1722), finalized #1355482 (0xf29e…bb92), ⬇ 2.1kiB/s ⬆ 2.1kiB/s
2020-06-28 00:38:18 Imported #1355486 (0x7e01…5524)
2020-06-28 00:38:19 Idle (1 peers), best: #1355486 (0x7e01…5524), finalized #1355483 (0x5253…2407), ⬇ 4.3kiB/s ⬆ 3.4kiB/s
2020-06-28 00:38:24 Imported #1355487 (0x2a00…6323)
2020-06-28 00:38:24 Idle (1 peers), best: #1355487 (0x2a00…6323), finalized #1355484 (0x3c0e…6b41), ⬇ 2.6kiB/s ⬆ 2.6kiB/s
2020-06-28 00:38:29 Idle (1 peers), best: #1355487 (0x2a00…6323), finalized #1355485 (0x0150…1722), ⬇ 3.2kiB/s ⬆ 2.4kiB/s
2020-06-28 00:38:30 Imported #1355488 (0x1009…4305)
2020-06-28 00:38:34 Idle (1 peers), best: #1355488 (0x1009…4305), finalized #1355486 (0x7e01…5524), ⬇ 4.2kiB/s ⬆ 3.4kiB/s
2020-06-28 00:38:36 Imported #1355489 (0x6c23…c23d)
2020-06-28 00:38:39 Idle (1 peers), best: #1355489 (0x6c23…c23d), finalized #1355486 (0x7e01…5524), ⬇ 2.7kiB/s ⬆ 1.8kiB/s
2020-06-28 00:38:42 Imported #1355490 (0x8254…0a49)
2020-06-28 00:38:44 Idle (1 peers), best: #1355490 (0x8254…0a49), finalized #1355487 (0x2a00…6323), ⬇ 1.8kiB/s ⬆ 1.9kiB/s
2020-06-28 00:38:48 Imported #1355491 (0xea50…2ba8)

amber-sentry for the same time period:

2020-06-28 00:28:54 Imported #1355394 (0x4153…af58)
2020-06-28 00:28:54 Idle (21 peers), best: #1355394 (0x4153…af58), finalized #1355390 (0xfb1b…a7fb), ⬇ 16.3kiB/s ⬆ 18.6kiB/s
2020-06-28 00:28:59 Idle (21 peers), best: #1355394 (0x4153…af58), finalized #1355391 (0xa0de…4f7c), ⬇ 18.4kiB/s ⬆ 14.6kiB/s
2020-06-28 00:29:00 Imported #1355395 (0x178d…38b0)
2020-06-28 00:29:04 Idle (21 peers), best: #1355395 (0x178d…38b0), finalized #1355391 (0xa0de…4f7c), ⬇ 11.8kiB/s ⬆ 5.8kiB/s
2020-06-28 00:29:06 Imported #1355396 (0x3349…2610)
2020-06-28 00:29:07 Failed to push message to queue, dropped it
2020-06-28 00:29:07 Failed to push message to queue, dropped it
2020-06-28 00:29:07 Failed to push message to queue, dropped it
2020-06-28 00:29:07 Failed to push message to queue, dropped it
2020-06-28 00:29:07 Failed to push message to queue, dropped it
2020-06-28 00:29:07 Failed to push message to queue, dropped it
2020-06-28 00:29:07 Failed to push message to queue, dropped it
2020-06-28 00:29:09 Idle (21 peers), best: #1355396 (0x3349…2610), finalized #1355392 (0xb986…e006), ⬇ 31.5kiB/s ⬆ 38.0kiB/s
2020-06-28 00:29:12 Imported #1355397 (0xbfb1…7d22)
2020-06-28 00:29:14 Idle (21 peers), best: #1355397 (0xbfb1…7d22), finalized #1355394 (0x4153…af58), ⬇ 17.0kiB/s ⬆ 22.9kiB/s
2020-06-28 00:29:18 Imported #1355398 (0xd126…6efd)
2020-06-28 00:29:19 Idle (21 peers), best: #1355398 (0xd126…6efd), finalized #1355395 (0x178d…38b0), ⬇ 20.5kiB/s ⬆ 18.7kiB/s
2020-06-28 00:29:24 Imported #1355399 (0x1bb5…3139)
2020-06-28 00:29:24 Idle (21 peers), best: #1355399 (0x1bb5…3139), finalized #1355396 (0x3349…2610), ⬇ 31.3kiB/s ⬆ 28.6kiB/s
2020-06-28 00:29:29 Idle (21 peers), best: #1355399 (0x1bb5…3139), finalized #1355396 (0x3349…2610), ⬇ 10.8kiB/s ⬆ 9.1kiB/s
2020-06-28 00:29:30 Imported #1355400 (0x7cc4…b801)
2020-06-28 00:29:34 Idle (21 peers), best: #1355400 (0x7cc4…b801), finalized #1355398 (0xd126…6efd), ⬇ 39.6kiB/s ⬆ 27.5kiB/s
2020-06-28 00:29:36 New epoch 381 launching at block 0xe6a1…7a87 (block slot 265550696 >= start slot 265550696).
2020-06-28 00:29:36 Next epoch starts at slot 265554296
2020-06-28 00:29:36 Imported #1355401 (0xe6a1…7a87)
2020-06-28 00:29:40 Idle (21 peers), best: #1355401 (0xe6a1…7a87), finalized #1355398 (0xd126…6efd), ⬇ 13.8kiB/s ⬆ 13.8kiB/s
2020-06-28 00:29:42 Imported #1355402 (0xe09c…5e3a)
2020-06-28 00:29:42 Imported #1355402 (0x3452…0634)
2020-06-28 00:29:45 Idle (21 peers), best: #1355402 (0xe09c…5e3a), finalized #1355398 (0xd126…6efd), ⬇ 8.4kiB/s ⬆ 32.3kiB/s
2020-06-28 00:29:48 Imported #1355403 (0x92dc…0cae)
2020-06-28 00:29:48 Imported #1355403 (0xc0db…e2be)
2020-06-28 00:29:48 Imported #1355403 (0x99b7…a682)
2020-06-28 00:29:50 Idle (21 peers), best: #1355403 (0x92dc…0cae), finalized #1355398 (0xd126…6efd), ⬇ 11.6kiB/s ⬆ 12.2kiB/s
2020-06-28 00:29:51 Applying authority set change scheduled at block #1355401
2020-06-28 00:29:51 Applying GRANDPA set change to new set [(Public(52237c29a7cb263163daca10670bc94629836fb1f52542e87c9fe45ad6b75dad (5DvQKoQV...)), 1), (Public(b7e87f546c26514f73c3dcbedc634748acdc12bc152ac63c0051674d6cc70b5e (5GDqgkfV...)), 1), (Public(b5fe42f275370d783ac2e8bccf7ce8696cddeb4738764ae2e9ce62d08b3ef59b (5GBL4Bkn...)), 1), (Public(424d9d04d03ca22c27498232c3e1a37c658224dcfd0b6759f3e2e90ec822caef (5DZe5Dcr...)), 1), (Public(e1a2d79bef77946d8210739f87f099c578b375a5aab9e10964bea8c05cd3f4bb (5HAZ1Gde...)), 1), (Public(7c74b9d92480283265c8fbb63ba0c20c9aa74620841b9e4eb0d9cbef473f7f99 (5EstUCPa...)), 1), (Public(4792315cc18f23432816dbff0ea6f8dc80e9a16ce2057582bccbabec8b19119a (5DgYghea...)), 1), (Public(09c4956b18b884d0ff69e31ddd0836af26f804c0a10691ad83cc6123ffb39db0 (5CHWgiqx...)), 1), (Public(2f765c5a4a4f7c2b19dd058066c2f1b2c527db973c3f34545fb68b53c22b15b6 (5D8wGWaQ...)), 1), (Public(a630c92eaa71e5b5d3666444da89705b8f9c08b9adc9523fe2446b30fcab612b (5FpcJGF6...)), 1), (Public(55075b215a98b93cb3f33ffe60a1de36a2e64520251060bd6e1c5b45c53ac91c (5DzC7Uqy...)), 1), (Public(fcf8d1d4d5eb0cc7fa48fa9c00e3d249bae3bb0f6a220bd1e5f7c049f6f5c1c2 (5HnPqcbA...)), 1), (Public(e2ff5155a4f4a70f880f11d46cb2f6ed863585e1e7e2282e7cb7a14c03bd8184 (5HCLXN2C...)), 1), (Public(6ac95555b0e42914fdf91ffb7259a652477e8cace43a1bad1bdac90e0dfa7af0 (5EUijxSC...)), 1), (Public(4f340fda5009d5db62f23e3e4f86250f777e7e457680ab2f31d8a9b5a75a079d (5DrZ75jx...)), 1), (Public(db7c98cbb7ba933482d0e40a5c72c95e3594c52f78f1dbb96729114ada6fda2b (5H2VMetr...)), 1)]
2020-06-28 00:29:54 Failed to push message to queue, dropped it
2020-06-28 00:29:54 Failed to push message to queue, dropped it
2020-06-28 00:29:54 Failed to push message to queue, dropped it
2020-06-28 00:29:54 Failed to push message to queue, dropped it
2020-06-28 00:29:54 Reorg from #1355403,0x92dc…0cae to #1355404,0x6075…0f45, common ancestor #1355402,0xe09c…5e3a
2020-06-28 00:29:54 Imported #1355404 (0x6075…0f45)
2020-06-28 00:29:55 Idle (21 peers), best: #1355404 (0x6075…0f45), finalized #1355401 (0xe6a1…7a87), ⬇ 43.7kiB/s ⬆ 24.5kiB/s
2020-06-28 00:30:00 Idle (20 peers), best: #1355404 (0x6075…0f45), finalized #1355402 (0xe09c…5e3a), ⬇ 32.2kiB/s ⬆ 27.3kiB/s
2020-06-28 00:30:00 Imported #1355405 (0x4df3…6e1a)
2020-06-28 00:30:00 Failed to push message to queue, dropped it
2020-06-28 00:30:00 Failed to push message to queue, dropped it
2020-06-28 00:30:00 Failed to push message to queue, dropped it
2020-06-28 00:30:00 Failed to push message to queue, dropped it
2020-06-28 00:30:00 Failed to push message to queue, dropped it
2020-06-28 00:30:00 Failed to push message to queue, dropped it
2020-06-28 00:30:03 Discovered new external address for our node: /ip4/10.156.1.54/tcp/30334/p2p/QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb
2020-06-28 00:30:05 Idle (20 peers), best: #1355405 (0x4df3…6e1a), finalized #1355403 (0xc0db…e2be), ⬇ 36.2kiB/s ⬆ 35.9kiB/s
2020-06-28 00:30:05 Discovered new external address for our node: /ip4/10.0.0.3/tcp/30334/p2p/QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb
2020-06-28 00:30:06 Imported #1355406 (0x6d02…ac96)
2020-06-28 00:30:10 Idle (20 peers), best: #1355406 (0x6d02…ac96), finalized #1355404 (0x6075…0f45), ⬇ 35.1kiB/s ⬆ 29.5kiB/s
2020-06-28 00:30:11 Failed to push message to queue, dropped it
2020-06-28 00:30:11 Failed to push message to queue, dropped it
2020-06-28 00:30:12 Imported #1355407 (0x8f81…2897)
2020-06-28 00:30:14 Handler initialization process is too long with PeerId("QmXgMtfQ5WhumBnVNZQbQFnQfLCoAWNp8c5rkC3ShuzeKL")
2020-06-28 00:30:15 Idle (20 peers), best: #1355407 (0x8f81…2897), finalized #1355405 (0x4df3…6e1a), ⬇ 26.7kiB/s ⬆ 23.1kiB/s
2020-06-28 00:30:19 Imported #1355408 (0xa2db…c9d3)
2020-06-28 00:30:20 Idle (20 peers), best: #1355408 (0xa2db…c9d3), finalized #1355405 (0x4df3…6e1a), ⬇ 38.5kiB/s ⬆ 30.6kiB/s
2020-06-28 00:30:24 Imported #1355409 (0xd5fc…fe9d)
2020-06-28 00:30:25 Idle (20 peers), best: #1355409 (0xd5fc…fe9d), finalized #1355406 (0x6d02…ac96), ⬇ 31.5kiB/s ⬆ 29.8kiB/s
2020-06-28 00:30:27 Failed to push message to queue, dropped it
2020-06-28 00:30:27 Failed to push message to queue, dropped it
2020-06-28 00:30:27 Failed to push message to queue, dropped it
2020-06-28 00:30:27 Failed to push message to queue, dropped it
2020-06-28 00:30:27 Failed to push message to queue, dropped it
2020-06-28 00:30:27 Failed to push message to queue, dropped it
2020-06-28 00:30:27 Failed to push message to queue, dropped it
2020-06-28 00:30:27 Failed to push message to queue, dropped it
2020-06-28 00:30:30 Idle (20 peers), best: #1355409 (0xd5fc…fe9d), finalized #1355407 (0x8f81…2897), ⬇ 32.4kiB/s ⬆ 29.0kiB/s
2020-06-28 00:30:30 Imported #1355410 (0x8034…c8d4)
2020-06-28 00:30:35 Idle (20 peers), best: #1355410 (0x8034…c8d4), finalized #1355408 (0xa2db…c9d3), ⬇ 30.5kiB/s ⬆ 31.5kiB/s
2020-06-28 00:30:36 Imported #1355411 (0xf5a1…e638)
2020-06-28 00:30:36 Imported #1355411 (0xcf03…349f)
2020-06-28 00:30:40 Idle (20 peers), best: #1355411 (0xf5a1…e638), finalized #1355409 (0xd5fc…fe9d), ⬇ 34.2kiB/s ⬆ 32.1kiB/s
2020-06-28 00:30:42 Handler initialization process is too long with PeerId("QmXgMtfQ5WhumBnVNZQbQFnQfLCoAWNp8c5rkC3ShuzeKL")
2020-06-28 00:30:42 Imported #1355412 (0x6a95…77a3)
2020-06-28 00:30:45 Idle (20 peers), best: #1355412 (0x6a95…77a3), finalized #1355410 (0x8034…c8d4), ⬇ 26.0kiB/s ⬆ 20.7kiB/s
2020-06-28 00:30:48 Imported #1355413 (0x0860…44de)
2020-06-28 00:30:50 Idle (20 peers), best: #1355413 (0x0860…44de), finalized #1355411 (0xf5a1…e638), ⬇ 37.5kiB/s ⬆ 32.6kiB/s
2020-06-28 00:30:54 Imported #1355414 (0xc810…4260)
2020-06-28 00:30:55 Idle (20 peers), best: #1355414 (0xc810…4260), finalized #1355411 (0xf5a1…e638), ⬇ 33.4kiB/s ⬆ 26.7kiB/s
2020-06-28 00:31:00 Imported #1355415 (0xd2f9…2160)
2020-06-28 00:31:00 Idle (20 peers), best: #1355415 (0xd2f9…2160), finalized #1355412 (0x6a95…77a3), ⬇ 34.7kiB/s ⬆ 31.8kiB/s
2020-06-28 00:31:00 Imported #1355415 (0x531e…8a4e)
2020-06-28 00:31:00 Imported #1355415 (0xd73a…3c02)
2020-06-28 00:31:00 Imported #1355415 (0x73ce…c301)
2020-06-28 00:31:00 Imported #1355415 (0x6d27…7c53)
2020-06-28 00:31:05 Idle (20 peers), best: #1355415 (0xd2f9…2160), finalized #1355413 (0x0860…44de), ⬇ 31.7kiB/s ⬆ 30.2kiB/s
2020-06-28 00:31:06 Imported #1355416 (0xd6db…b122)
2020-06-28 00:31:07 Failed to push message to queue, dropped it
2020-06-28 00:31:07 Failed to push message to queue, dropped it
2020-06-28 00:31:07 Failed to push message to queue, dropped it
2020-06-28 00:31:07 Failed to push message to queue, dropped it
2020-06-28 00:31:07 Failed to push message to queue, dropped it
2020-06-28 00:31:07 Failed to push message to queue, dropped it
2020-06-28 00:31:07 Failed to push message to queue, dropped it
2020-06-28 00:31:07 Failed to push message to queue, dropped it
2020-06-28 00:31:07 Failed to push message to queue, dropped it
2020-06-28 00:31:07 Failed to push message to queue, dropped it
2020-06-28 00:31:08 Failed to push message to queue, dropped it
2020-06-28 00:31:08 Failed to push message to queue, dropped it
2020-06-28 00:31:10 Idle (20 peers), best: #1355416 (0xd6db…b122), finalized #1355414 (0xc810…4260), ⬇ 37.4kiB/s ⬆ 31.9kiB/s
2020-06-28 00:31:11 Failed to push message to queue, dropped it
2020-06-28 00:31:11 Failed to push message to queue, dropped it
2020-06-28 00:31:11 Failed to push message to queue, dropped it
2020-06-28 00:31:11 Failed to push message to queue, dropped it
2020-06-28 00:31:11 Failed to push message to queue, dropped it
2020-06-28 00:31:11 Failed to push message to queue, dropped it
2020-06-28 00:31:11 Failed to push message to queue, dropped it
2020-06-28 00:31:11 Failed to push message to queue, dropped it
2020-06-28 00:31:11 Failed to push message to queue, dropped it
2020-06-28 00:31:11 Failed to push message to queue, dropped it
2020-06-28 00:31:11 Failed to push message to queue, dropped it
2020-06-28 00:31:11 Failed to push message to queue, dropped it
2020-06-28 00:31:12 Imported #1355417 (0x85d1…4005)
2020-06-28 00:31:15 Failed to push message to queue, dropped it
2020-06-28 00:31:15 Failed to push message to queue, dropped it
2020-06-28 00:31:15 Idle (20 peers), best: #1355417 (0x85d1…4005), finalized #1355415 (0xd2f9…2160), ⬇ 33.8kiB/s ⬆ 29.3kiB/s
2020-06-28 00:31:18 Imported #1355418 (0x14f9…79af)
2020-06-28 00:31:20 Idle (20 peers), best: #1355418 (0x14f9…79af), finalized #1355416 (0xd6db…b122), ⬇ 30.2kiB/s ⬆ 22.8kiB/s
2020-06-28 00:31:24 Imported #1355419 (0x8a3a…daff)
2020-06-28 00:31:25 Idle (20 peers), best: #1355419 (0x8a3a…daff), finalized #1355416 (0xd6db…b122), ⬇ 33.2kiB/s ⬆ 33.3kiB/s
2020-06-28 00:31:30 Idle (20 peers), best: #1355419 (0x8a3a…daff), finalized #1355417 (0x85d1…4005), ⬇ 31.8kiB/s ⬆ 29.1kiB/s
2020-06-28 00:31:30 Imported #1355420 (0x4e6a…fb7c)
2020-06-28 00:31:35 Idle (20 peers), best: #1355420 (0x4e6a…fb7c), finalized #1355418 (0x14f9…79af), ⬇ 32.3kiB/s ⬆ 29.1kiB/s
2020-06-28 00:31:36 Imported #1355421 (0x0adb…8983)
2020-06-28 00:31:40 Idle (20 peers), best: #1355421 (0x0adb…8983), finalized #1355419 (0x8a3a…daff), ⬇ 30.7kiB/s ⬆ 31.1kiB/s
2020-06-28 00:31:42 Imported #1355422 (0xa253…ba24)
2020-06-28 00:31:45 Idle (20 peers), best: #1355422 (0xa253…ba24), finalized #1355420 (0x4e6a…fb7c), ⬇ 31.2kiB/s ⬆ 28.1kiB/s
2020-06-28 00:31:48 Imported #1355423 (0x4ca3…28b8)
2020-06-28 00:31:50 Idle (20 peers), best: #1355423 (0x4ca3…28b8), finalized #1355421 (0x0adb…8983), ⬇ 26.8kiB/s ⬆ 27.4kiB/s
2020-06-28 00:31:54 Imported #1355424 (0x4956…9edf)
2020-06-28 00:31:54 Reorg from #1355424,0x4956…9edf to #1355424,0x2b3d…18f9, common ancestor #1355423,0x4ca3…28b8
2020-06-28 00:31:54 Imported #1355424 (0x2b3d…18f9)
2020-06-28 00:31:55 Failed to push message to queue, dropped it
2020-06-28 00:31:55 Failed to push message to queue, dropped it
2020-06-28 00:31:55 Failed to push message to queue, dropped it
2020-06-28 00:31:55 Failed to push message to queue, dropped it
2020-06-28 00:31:55 Failed to push message to queue, dropped it
2020-06-28 00:31:55 Failed to push message to queue, dropped it
2020-06-28 00:31:55 Failed to push message to queue, dropped it
2020-06-28 00:31:55 Failed to push message to queue, dropped it
2020-06-28 00:31:55 Failed to push message to queue, dropped it
2020-06-28 00:31:55 Failed to push message to queue, dropped it
2020-06-28 00:31:55 Failed to push message to queue, dropped it
2020-06-28 00:31:55 Failed to push message to queue, dropped it
2020-06-28 00:31:55 Failed to push message to queue, dropped it
2020-06-28 00:31:55 Idle (20 peers), best: #1355424 (0x2b3d…18f9), finalized #1355421 (0x0adb…8983), ⬇ 29.1kiB/s ⬆ 25.9kiB/s
2020-06-28 00:32:00 Imported #1355425 (0x14bc…3051)
2020-06-28 00:32:00 Idle (20 peers), best: #1355425 (0x14bc…3051), finalized #1355422 (0xa253…ba24), ⬇ 28.8kiB/s ⬆ 27.1kiB/s
2020-06-28 00:32:05 Idle (20 peers), best: #1355425 (0x14bc…3051), finalized #1355423 (0x4ca3…28b8), ⬇ 28.4kiB/s ⬆ 23.7kiB/s
2020-06-28 00:32:06 Imported #1355426 (0xb66e…5427)
2020-06-28 00:32:10 Idle (20 peers), best: #1355426 (0xb66e…5427), finalized #1355424 (0x2b3d…18f9), ⬇ 29.4kiB/s ⬆ 27.3kiB/s
2020-06-28 00:32:12 Imported #1355427 (0xeab8…d08a)
2020-06-28 00:32:12 Reorg from #1355427,0xeab8…d08a to #1355427,0x8171…ec3c, common ancestor #1355426,0xb66e…5427
2020-06-28 00:32:12 Imported #1355427 (0x8171…ec3c)
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:12 Failed to push message to queue, dropped it
2020-06-28 00:32:15 Idle (20 peers), best: #1355427 (0x8171…ec3c), finalized #1355425 (0x14bc…3051), ⬇ 31.7kiB/s ⬆ 29.8kiB/s
2020-06-28 00:32:16 Failed to push message to queue, dropped it
2020-06-28 00:32:16 Failed to push message to queue, dropped it
2020-06-28 00:32:18 Imported #1355428 (0xea11…0990)
2020-06-28 00:32:18 Imported #1355428 (0xb3f4…d905)
2020-06-28 00:32:18 Imported #1355428 (0x05fe…ab8d)
2020-06-28 00:32:20 Idle (20 peers), best: #1355428 (0xea11…0990), finalized #1355425 (0x14bc…3051), ⬇ 22.0kiB/s ⬆ 19.3kiB/s
2020-06-28 00:32:24 Imported #1355429 (0x5a4e…98c3)
2020-06-28 00:32:24 Imported #1355429 (0xbc20…dba2)
2020-06-28 00:32:25 Idle (20 peers), best: #1355429 (0x5a4e…98c3), finalized #1355426 (0xb66e…5427), ⬇ 34.7kiB/s ⬆ 29.4kiB/s
2020-06-28 00:32:28 Failed to push message to queue, dropped it
2020-06-28 00:32:28 Failed to push message to queue, dropped it
2020-06-28 00:32:28 Failed to push message to queue, dropped it
2020-06-28 00:32:28 Failed to push message to queue, dropped it
2020-06-28 00:32:28 Failed to push message to queue, dropped it
2020-06-28 00:32:28 Failed to push message to queue, dropped it
2020-06-28 00:32:30 Imported #1355430 (0xcbc1…01ae)
2020-06-28 00:32:30 Idle (20 peers), best: #1355430 (0xcbc1…01ae), finalized #1355427 (0x8171…ec3c), ⬇ 36.3kiB/s ⬆ 27.7kiB/s
2020-06-28 00:32:32 Failed to push message to queue, dropped it
2020-06-28 00:32:32 Failed to push message to queue, dropped it
2020-06-28 00:32:35 Idle (20 peers), best: #1355430 (0xcbc1…01ae), finalized #1355428 (0xea11…0990), ⬇ 27.9kiB/s ⬆ 26.1kiB/s
2020-06-28 00:32:36 Imported #1355431 (0x85ec…8ada)
2020-06-28 00:32:36 Imported #1355431 (0x9744…14c2)
2020-06-28 00:32:40 Idle (20 peers), best: #1355431 (0x85ec…8ada), finalized #1355429 (0x5a4e…98c3), ⬇ 28.3kiB/s ⬆ 29.3kiB/s
2020-06-28 00:32:42 Imported #1355432 (0x2fc3…2a09)
2020-06-28 00:32:45 Idle (20 peers), best: #1355432 (0x2fc3…2a09), finalized #1355430 (0xcbc1…01ae), ⬇ 29.6kiB/s ⬆ 26.9kiB/s
2020-06-28 00:32:48 Imported #1355433 (0xfded…de0c)
2020-06-28 00:32:48 Discovered new external address for our node: /ip4/10.156.1.51/tcp/30334/p2p/QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb
2020-06-28 00:32:49 Discovered new external address for our node: /ip4/10.156.1.50/tcp/30334/p2p/QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb
2020-06-28 00:32:50 Idle (20 peers), best: #1355433 (0xfded…de0c), finalized #1355430 (0xcbc1…01ae), ⬇ 26.4kiB/s ⬆ 23.3kiB/s
2020-06-28 00:32:54 Imported #1355434 (0x37bb…71cf)
2020-06-28 00:32:55 Idle (20 peers), best: #1355434 (0x37bb…71cf), finalized #1355432 (0x2fc3…2a09), ⬇ 42.5kiB/s ⬆ 33.4kiB/s
2020-06-28 00:33:00 Imported #1355435 (0xf3a2…dbe4)
2020-06-28 00:33:00 Idle (20 peers), best: #1355435 (0xf3a2…dbe4), finalized #1355432 (0x2fc3…2a09), ⬇ 25.1kiB/s ⬆ 27.5kiB/s
2020-06-28 00:33:05 Idle (20 peers), best: #1355435 (0xf3a2…dbe4), finalized #1355433 (0xfded…de0c), ⬇ 29.7kiB/s ⬆ 29.2kiB/s
2020-06-28 00:33:06 Imported #1355436 (0x0459…aafb)
2020-06-28 00:33:06 Failed to push message to queue, dropped it
2020-06-28 00:33:06 Failed to push message to queue, dropped it
2020-06-28 00:33:06 Failed to push message to queue, dropped it
2020-06-28 00:33:09 Failed to push message to queue, dropped it
2020-06-28 00:33:10 Idle (20 peers), best: #1355436 (0x0459…aafb), finalized #1355434 (0x37bb…71cf), ⬇ 33.6kiB/s ⬆ 34.4kiB/s
2020-06-28 00:33:11 Failed to push message to queue, dropped it
2020-06-28 00:33:12 Imported #1355437 (0x22e1…daf7)
2020-06-28 00:33:13 Failed to push message to queue, dropped it
2020-06-28 00:33:13 Failed to push message to queue, dropped it
2020-06-28 00:33:15 Idle (20 peers), best: #1355437 (0x22e1…daf7), finalized #1355435 (0xf3a2…dbe4), ⬇ 33.5kiB/s ⬆ 30.8kiB/s
2020-06-28 00:33:18 Imported #1355438 (0x5015…857a)
2020-06-28 00:33:18 Failed to push message to queue, dropped it
2020-06-28 00:33:18 Failed to push message to queue, dropped it
2020-06-28 00:33:18 Failed to push message to queue, dropped it
2020-06-28 00:33:18 Failed to push message to queue, dropped it
2020-06-28 00:33:18 Failed to push message to queue, dropped it
2020-06-28 00:33:18 Failed to push message to queue, dropped it
2020-06-28 00:33:18 Failed to push message to queue, dropped it
2020-06-28 00:33:20 Idle (20 peers), best: #1355438 (0x5015…857a), finalized #1355436 (0x0459…aafb), ⬇ 30.1kiB/s ⬆ 30.5kiB/s
2020-06-28 00:33:22 Handler initialization process is too long with PeerId("QmVKsvydUPUMYLGNtJu8kLinzuC2vb5vXYw3Fs9fJSHWEb")
2020-06-28 00:33:24 Imported #1355439 (0x3565…9fa8)
2020-06-28 00:33:25 Idle (20 peers), best: #1355439 (0x3565…9fa8), finalized #1355436 (0x0459…aafb), ⬇ 29.0kiB/s ⬆ 28.8kiB/s
2020-06-28 00:33:30 Imported #1355440 (0xaa51…18f3)
2020-06-28 00:33:30 Idle (20 peers), best: #1355440 (0xaa51…18f3), finalized #1355437 (0x22e1…daf7), ⬇ 33.5kiB/s ⬆ 33.7kiB/s
2020-06-28 00:33:35 Idle (20 peers), best: #1355440 (0xaa51…18f3), finalized #1355438 (0x5015…857a), ⬇ 26.4kiB/s ⬆ 28.4kiB/s
2020-06-28 00:33:36 Imported #1355441 (0xe71d…df93)
2020-06-28 00:33:37 Failed to push message to queue, dropped it
2020-06-28 00:33:37 Failed to push message to queue, dropped it
2020-06-28 00:33:40 Idle (20 peers), best: #1355441 (0xe71d…df93), finalized #1355439 (0x3565…9fa8), ⬇ 26.0kiB/s ⬆ 28.1kiB/s
2020-06-28 00:33:42 Imported #1355442 (0x92af…e5ba)
2020-06-28 00:33:42 Reorg from #1355442,0x92af…e5ba to #1355442,0x9372…ef23, common ancestor #1355441,0xe71d…df93
2020-06-28 00:33:42 Imported #1355442 (0x9372…ef23)
2020-06-28 00:33:43 Failed to push message to queue, dropped it
2020-06-28 00:33:43 Failed to push message to queue, dropped it
2020-06-28 00:33:43 Failed to push message to queue, dropped it
2020-06-28 00:33:43 Failed to push message to queue, dropped it
2020-06-28 00:33:43 Failed to push message to queue, dropped it
2020-06-28 00:33:43 Failed to push message to queue, dropped it
2020-06-28 00:33:43 Failed to push message to queue, dropped it
2020-06-28 00:33:45 Idle (20 peers), best: #1355442 (0x9372…ef23), finalized #1355440 (0xaa51…18f3), ⬇ 28.4kiB/s ⬆ 27.3kiB/s
2020-06-28 00:33:48 Imported #1355443 (0x16d2…cfbb)
2020-06-28 00:33:50 Idle (20 peers), best: #1355443 (0x16d2…cfbb), finalized #1355441 (0xe71d…df93), ⬇ 31.3kiB/s ⬆ 28.5kiB/s
2020-06-28 00:33:55 Idle (20 peers), best: #1355443 (0x16d2…cfbb), finalized #1355441 (0xe71d…df93), ⬇ 18.5kiB/s ⬆ 21.6kiB/s
2020-06-28 00:33:59 Handler initialization process is too long with PeerId("QmXgMtfQ5WhumBnVNZQbQFnQfLCoAWNp8c5rkC3ShuzeKL")
2020-06-28 00:34:00 Imported #1355444 (0xcff2…a6dc)
2020-06-28 00:34:00 Idle (20 peers), best: #1355444 (0xcff2…a6dc), finalized #1355441 (0xe71d…df93), ⬇ 30.6kiB/s ⬆ 28.5kiB/s
2020-06-28 00:34:05 Idle (20 peers), best: #1355444 (0xcff2…a6dc), finalized #1355442 (0x9372…ef23), ⬇ 33.7kiB/s ⬆ 28.3kiB/s
2020-06-28 00:34:06 Imported #1355445 (0xbe20…b416)
2020-06-28 00:34:11 Idle (20 peers), best: #1355445 (0xbe20…b416), finalized #1355443 (0x16d2…cfbb), ⬇ 31.6kiB/s ⬆ 22.1kiB/s
2020-06-28 00:34:12 Imported #1355446 (0xd6b5…274a)
2020-06-28 00:34:16 Idle (20 peers), best: #1355446 (0xd6b5…274a), finalized #1355444 (0xcff2…a6dc), ⬇ 32.8kiB/s ⬆ 29.3kiB/s
2020-06-28 00:34:21 Idle (20 peers), best: #1355446 (0xd6b5…274a), finalized #1355444 (0xcff2…a6dc), ⬇ 25.5kiB/s ⬆ 22.8kiB/s
2020-06-28 00:34:24 Imported #1355447 (0x2a38…315b)
2020-06-28 00:34:24 Failed to push message to queue, dropped it
2020-06-28 00:34:26 Idle (20 peers), best: #1355447 (0x2a38…315b), finalized #1355444 (0xcff2…a6dc), ⬇ 29.7kiB/s ⬆ 26.7kiB/s
2020-06-28 00:34:30 Imported #1355448 (0xd5e4…68d3)
2020-06-28 00:34:31 Idle (20 peers), best: #1355448 (0xd5e4…68d3), finalized #1355445 (0xbe20…b416), ⬇ 33.1kiB/s ⬆ 28.0kiB/s
2020-06-28 00:34:36 Idle (20 peers), best: #1355448 (0xd5e4…68d3), finalized #1355446 (0xd6b5…274a), ⬇ 23.3kiB/s ⬆ 25.7kiB/s
2020-06-28 00:34:36 Imported #1355449 (0x9526…db06)
2020-06-28 00:34:36 Imported #1355449 (0x98fc…641a)
2020-06-28 00:34:41 Idle (20 peers), best: #1355449 (0x9526…db06), finalized #1355447 (0x2a38…315b), ⬇ 31.1kiB/s ⬆ 28.4kiB/s
2020-06-28 00:34:42 Imported #1355450 (0x4f37…c2ee)
2020-06-28 00:34:46 Idle (20 peers), best: #1355450 (0x4f37…c2ee), finalized #1355448 (0xd5e4…68d3), ⬇ 32.9kiB/s ⬆ 30.2kiB/s
2020-06-28 00:34:48 Discovered new external address for our node: /ip4/10.0.0.3/tcp/30334/p2p/QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb
2020-06-28 00:34:48 Imported #1355451 (0x099d…80da)
2020-06-28 00:34:48 Imported #1355451 (0xfd57…b586)
2020-06-28 00:34:51 Idle (20 peers), best: #1355451 (0x099d…80da), finalized #1355449 (0x9526…db06), ⬇ 32.2kiB/s ⬆ 28.4kiB/s
2020-06-28 00:34:54 Imported #1355452 (0x719e…d0cb)
2020-06-28 00:34:56 Idle (20 peers), best: #1355452 (0x719e…d0cb), finalized #1355450 (0x4f37…c2ee), ⬇ 32.7kiB/s ⬆ 28.7kiB/s
2020-06-28 00:35:00 Imported #1355453 (0x717b…ec90)
2020-06-28 00:35:01 Idle (20 peers), best: #1355453 (0x717b…ec90), finalized #1355450 (0x4f37…c2ee), ⬇ 29.1kiB/s ⬆ 30.7kiB/s
2020-06-28 00:35:03 Failed to push message to queue, dropped it
2020-06-28 00:35:03 Failed to push message to queue, dropped it
2020-06-28 00:35:03 Failed to push message to queue, dropped it
2020-06-28 00:35:03 Failed to push message to queue, dropped it
2020-06-28 00:35:03 Failed to push message to queue, dropped it
2020-06-28 00:35:03 Failed to push message to queue, dropped it
2020-06-28 00:35:03 Failed to push message to queue, dropped it
2020-06-28 00:35:03 Discovered new external address for our node: /ip4/10.156.1.54/tcp/30334/p2p/QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb
2020-06-28 00:35:05 Failed to push message to queue, dropped it
2020-06-28 00:35:05 Failed to push message to queue, dropped it
2020-06-28 00:35:05 Failed to push message to queue, dropped it
2020-06-28 00:35:05 Failed to push message to queue, dropped it
2020-06-28 00:35:05 Failed to push message to queue, dropped it
2020-06-28 00:35:05 Failed to push message to queue, dropped it
2020-06-28 00:35:05 Failed to push message to queue, dropped it
2020-06-28 00:35:06 Idle (20 peers), best: #1355453 (0x717b…ec90), finalized #1355451 (0x099d…80da), ⬇ 32.6kiB/s ⬆ 28.7kiB/s
2020-06-28 00:35:06 Imported #1355454 (0xf1ea…89cd)
2020-06-28 00:35:06 Imported #1355454 (0xdc89…1ead)
2020-06-28 00:35:06 Failed to push message to queue, dropped it
2020-06-28 00:35:06 Failed to push message to queue, dropped it
2020-06-28 00:35:10 Handler initialization process is too long with PeerId("QmXgMtfQ5WhumBnVNZQbQFnQfLCoAWNp8c5rkC3ShuzeKL")
2020-06-28 00:35:11 Idle (20 peers), best: #1355454 (0xf1ea…89cd), finalized #1355452 (0x719e…d0cb), ⬇ 35.4kiB/s ⬆ 28.8kiB/s
2020-06-28 00:35:12 Imported #1355455 (0x89f5…6572)
2020-06-28 00:35:12 Imported #1355455 (0x8dd4…4e8f)
2020-06-28 00:35:16 Idle (20 peers), best: #1355455 (0x89f5…6572), finalized #1355453 (0x717b…ec90), ⬇ 26.1kiB/s ⬆ 32.7kiB/s
2020-06-28 00:35:17 Failed to push message to queue, dropped it
2020-06-28 00:35:17 Failed to push message to queue, dropped it
2020-06-28 00:35:17 Failed to push message to queue, dropped it
2020-06-28 00:35:17 Failed to push message to queue, dropped it
2020-06-28 00:35:17 Failed to push message to queue, dropped it
2020-06-28 00:35:17 Failed to push message to queue, dropped it
2020-06-28 00:35:17 Failed to push message to queue, dropped it
2020-06-28 00:35:18 Imported #1355456 (0xb907…bcaa)
2020-06-28 00:35:21 Idle (20 peers), best: #1355456 (0xb907…bcaa), finalized #1355454 (0xf1ea…89cd), ⬇ 30.9kiB/s ⬆ 27.8kiB/s
2020-06-28 00:35:24 Imported #1355457 (0x5ce8…8d2e)
2020-06-28 00:35:26 Idle (20 peers), best: #1355457 (0x5ce8…8d2e), finalized #1355454 (0xf1ea…89cd), ⬇ 28.7kiB/s ⬆ 25.7kiB/s
2020-06-28 00:35:30 Imported #1355458 (0x5940…33ad)
2020-06-28 00:35:30 Imported #1355458 (0x8e0d…ac36)
2020-06-28 00:35:31 Idle (20 peers), best: #1355458 (0x5940…33ad), finalized #1355455 (0x89f5…6572), ⬇ 40.7kiB/s ⬆ 30.7kiB/s
2020-06-28 00:35:33 Failed to push message to queue, dropped it
2020-06-28 00:35:33 Failed to push message to queue, dropped it
2020-06-28 00:35:33 Failed to push message to queue, dropped it
2020-06-28 00:35:33 Failed to push message to queue, dropped it
2020-06-28 00:35:33 Failed to push message to queue, dropped it
2020-06-28 00:35:33 Failed to push message to queue, dropped it
2020-06-28 00:35:36 Imported #1355459 (0xec42…2ac1)
2020-06-28 00:35:36 Idle (20 peers), best: #1355459 (0xec42…2ac1), finalized #1355456 (0xb907…bcaa), ⬇ 30.5kiB/s ⬆ 28.2kiB/s
2020-06-28 00:35:41 Idle (20 peers), best: #1355459 (0xec42…2ac1), finalized #1355457 (0x5ce8…8d2e), ⬇ 30.2kiB/s ⬆ 33.1kiB/s
2020-06-28 00:35:42 Imported #1355460 (0xf096…fd88)
2020-06-28 00:35:46 Idle (20 peers), best: #1355460 (0xf096…fd88), finalized #1355458 (0x5940…33ad), ⬇ 24.7kiB/s ⬆ 21.8kiB/s
2020-06-28 00:35:48 Imported #1355461 (0xccf7…cfcc)
2020-06-28 00:35:51 Idle (20 peers), best: #1355461 (0xccf7…cfcc), finalized #1355459 (0xec42…2ac1), ⬇ 31.2kiB/s ⬆ 24.1kiB/s
2020-06-28 00:35:54 Imported #1355462 (0x83e0…a493)
2020-06-28 00:35:56 Idle (20 peers), best: #1355462 (0x83e0…a493), finalized #1355460 (0xf096…fd88), ⬇ 33.3kiB/s ⬆ 30.9kiB/s
2020-06-28 00:36:00 Imported #1355463 (0x7f40…13df)
2020-06-28 00:36:01 Idle (20 peers), best: #1355463 (0x7f40…13df), finalized #1355460 (0xf096…fd88), ⬇ 26.3kiB/s ⬆ 24.3kiB/s
2020-06-28 00:36:06 Imported #1355464 (0xc90a…ac79)
2020-06-28 00:36:06 Idle (20 peers), best: #1355464 (0xc90a…ac79), finalized #1355461 (0xccf7…cfcc), ⬇ 32.0kiB/s ⬆ 26.6kiB/s
2020-06-28 00:36:11 Idle (20 peers), best: #1355464 (0xc90a…ac79), finalized #1355462 (0x83e0…a493), ⬇ 36.5kiB/s ⬆ 31.1kiB/s
2020-06-28 00:36:12 Imported #1355465 (0x4ce0…2dfd)
2020-06-28 00:36:12 Reorg from #1355465,0x4ce0…2dfd to #1355465,0xbce4…6f85, common ancestor #1355464,0xc90a…ac79
2020-06-28 00:36:12 Imported #1355465 (0xbce4…6f85)
2020-06-28 00:36:13 Failed to push message to queue, dropped it
2020-06-28 00:36:16 Idle (21 peers), best: #1355465 (0xbce4…6f85), finalized #1355463 (0x7f40…13df), ⬇ 27.7kiB/s ⬆ 28.1kiB/s
2020-06-28 00:36:18 Imported #1355466 (0x0eb2…99f2)
2020-06-28 00:36:18 Reorg from #1355466,0x0eb2…99f2 to #1355466,0x05d4…9fac, common ancestor #1355465,0xbce4…6f85
2020-06-28 00:36:18 Imported #1355466 (0x05d4…9fac)
2020-06-28 00:36:21 Idle (20 peers), best: #1355466 (0x05d4…9fac), finalized #1355464 (0xc90a…ac79), ⬇ 25.5kiB/s ⬆ 30.8kiB/s
2020-06-28 00:36:24 Imported #1355467 (0x8d25…d177)
2020-06-28 00:36:24 Reorg from #1355467,0x8d25…d177 to #1355467,0x2bf1…de3c, common ancestor #1355466,0x05d4…9fac
2020-06-28 00:36:24 Imported #1355467 (0x2bf1…de3c)
2020-06-28 00:36:25 Failed to push message to queue, dropped it
2020-06-28 00:36:25 Failed to push message to queue, dropped it
2020-06-28 00:36:26 Idle (20 peers), best: #1355467 (0x2bf1…de3c), finalized #1355464 (0xc90a…ac79), ⬇ 38.7kiB/s ⬆ 30.1kiB/s
2020-06-28 00:36:30 Imported #1355468 (0xbca6…9691)
2020-06-28 00:36:31 Idle (20 peers), best: #1355468 (0xbca6…9691), finalized #1355465 (0xbce4…6f85), ⬇ 38.5kiB/s ⬆ 43.8kiB/s
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:35 Failed to push message to queue, dropped it
2020-06-28 00:36:36 Imported #1355469 (0x3049…eb3c)
2020-06-28 00:36:36 Idle (20 peers), best: #1355469 (0x3049…eb3c), finalized #1355466 (0x05d4…9fac), ⬇ 32.3kiB/s ⬆ 47.0kiB/s
2020-06-28 00:36:39 Failed to push message to queue, dropped it
2020-06-28 00:36:39 Failed to push message to queue, dropped it
2020-06-28 00:36:39 Failed to push message to queue, dropped it
2020-06-28 00:36:39 Failed to push message to queue, dropped it
2020-06-28 00:36:39 Failed to push message to queue, dropped it
2020-06-28 00:36:39 Failed to push message to queue, dropped it
2020-06-28 00:36:39 Failed to push message to queue, dropped it
2020-06-28 00:36:39 Failed to push message to queue, dropped it
2020-06-28 00:36:39 Failed to push message to queue, dropped it
2020-06-28 00:36:39 Failed to push message to queue, dropped it
2020-06-28 00:36:41 Idle (20 peers), best: #1355469 (0x3049…eb3c), finalized #1355467 (0x2bf1…de3c), ⬇ 28.8kiB/s ⬆ 36.7kiB/s
2020-06-28 00:36:42 Imported #1355470 (0x4db5…6481)
2020-06-28 00:36:42 Imported #1355470 (0x417c…c13b)
2020-06-28 00:36:46 Idle (20 peers), best: #1355470 (0x4db5…6481), finalized #1355468 (0xbca6…9691), ⬇ 32.0kiB/s ⬆ 48.7kiB/s
2020-06-28 00:36:48 Imported #1355471 (0x8ea9…b487)
2020-06-28 00:36:51 Idle (21 peers), best: #1355471 (0x8ea9…b487), finalized #1355468 (0xbca6…9691), ⬇ 8.4kiB/s ⬆ 7.7kiB/s
2020-06-28 00:36:54 Imported #1355472 (0xc27b…512d)
2020-06-28 00:36:56 Idle (20 peers), best: #1355472 (0xc27b…512d), finalized #1355469 (0x3049…eb3c), ⬇ 36.2kiB/s ⬆ 67.5kiB/s
2020-06-28 00:37:00 Imported #1355473 (0xc3be…6670)
2020-06-28 00:37:01 Failed to push message to queue, dropped it
2020-06-28 00:37:01 Failed to push message to queue, dropped it
2020-06-28 00:37:01 Failed to push message to queue, dropped it
2020-06-28 00:37:01 Failed to push message to queue, dropped it
2020-06-28 00:37:01 Failed to push message to queue, dropped it
2020-06-28 00:37:01 Failed to push message to queue, dropped it
2020-06-28 00:37:01 Idle (20 peers), best: #1355473 (0xc3be…6670), finalized #1355470 (0x4db5…6481), ⬇ 36.0kiB/s ⬆ 49.7kiB/s
2020-06-28 00:37:06 Failed to push message to queue, dropped it
2020-06-28 00:37:06 Failed to push message to queue, dropped it
2020-06-28 00:37:06 Failed to push message to queue, dropped it
2020-06-28 00:37:06 Failed to push message to queue, dropped it
2020-06-28 00:37:06 Failed to push message to queue, dropped it
2020-06-28 00:37:06 Failed to push message to queue, dropped it
2020-06-28 00:37:06 Failed to push message to queue, dropped it
2020-06-28 00:37:06 Failed to push message to queue, dropped it
2020-06-28 00:37:06 Failed to push message to queue, dropped it
2020-06-28 00:37:06 Failed to push message to queue, dropped it
2020-06-28 00:37:06 Imported #1355474 (0x82cb…36b0)
2020-06-28 00:37:06 Idle (20 peers), best: #1355474 (0x82cb…36b0), finalized #1355471 (0x8ea9…b487), ⬇ 32.7kiB/s ⬆ 45.7kiB/s
2020-06-28 00:37:07 Discovered new external address for our node: /ip4/10.0.0.3/tcp/30334/p2p/QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb
2020-06-28 00:37:11 Idle (21 peers), best: #1355474 (0x82cb…36b0), finalized #1355472 (0xc27b…512d), ⬇ 32.3kiB/s ⬆ 43.0kiB/s
2020-06-28 00:37:12 Imported #1355475 (0xc3c5…1b95)
2020-06-28 00:37:16 Idle (22 peers), best: #1355475 (0xc3c5…1b95), finalized #1355473 (0xc3be…6670), ⬇ 30.8kiB/s ⬆ 36.1kiB/s
2020-06-28 00:37:18 Imported #1355476 (0x639a…cfb5)
2020-06-28 00:37:21 Idle (21 peers), best: #1355476 (0x639a…cfb5), finalized #1355474 (0x82cb…36b0), ⬇ 37.3kiB/s ⬆ 56.0kiB/s
2020-06-28 00:37:24 Imported #1355477 (0x0b6c…6165)
2020-06-28 00:37:25 Failed to push message to queue, dropped it
2020-06-28 00:37:25 Failed to push message to queue, dropped it
2020-06-28 00:37:25 Failed to push message to queue, dropped it
2020-06-28 00:37:25 Handler initialization process is too long with PeerId("QmXgMtfQ5WhumBnVNZQbQFnQfLCoAWNp8c5rkC3ShuzeKL")
2020-06-28 00:37:26 Idle (21 peers), best: #1355477 (0x0b6c…6165), finalized #1355475 (0xc3c5…1b95), ⬇ 44.6kiB/s ⬆ 28.0kiB/s
2020-06-28 00:37:30 Imported #1355478 (0x74ac…ad43)
2020-06-28 00:37:31 Idle (21 peers), best: #1355478 (0x74ac…ad43), finalized #1355476 (0x639a…cfb5), ⬇ 27.2kiB/s ⬆ 37.2kiB/s
2020-06-28 00:37:36 Imported #1355479 (0x9b23…35ef)
2020-06-28 00:37:36 Idle (20 peers), best: #1355479 (0x9b23…35ef), finalized #1355476 (0x639a…cfb5), ⬇ 33.2kiB/s ⬆ 53.0kiB/s
2020-06-28 00:37:39 Failed to push message to queue, dropped it
2020-06-28 00:37:39 Failed to push message to queue, dropped it
2020-06-28 00:37:39 Failed to push message to queue, dropped it
2020-06-28 00:37:39 Failed to push message to queue, dropped it
2020-06-28 00:37:39 Failed to push message to queue, dropped it
2020-06-28 00:37:39 Failed to push message to queue, dropped it
2020-06-28 00:37:39 Failed to push message to queue, dropped it
2020-06-28 00:37:39 Failed to push message to queue, dropped it
2020-06-28 00:37:39 Failed to push message to queue, dropped it
2020-06-28 00:37:41 Idle (20 peers), best: #1355479 (0x9b23…35ef), finalized #1355477 (0x0b6c…6165), ⬇ 30.4kiB/s ⬆ 28.0kiB/s
2020-06-28 00:37:42 Imported #1355480 (0x13e6…27a8)
2020-06-28 00:37:46 Idle (20 peers), best: #1355480 (0x13e6…27a8), finalized #1355478 (0x74ac…ad43), ⬇ 30.7kiB/s ⬆ 38.6kiB/s
2020-06-28 00:37:48 Imported #1355481 (0x79bc…fd77)
2020-06-28 00:37:48 Reorg from #1355481,0x79bc…fd77 to #1355481,0x0eda…3b9b, common ancestor #1355480,0x13e6…27a8
2020-06-28 00:37:48 Imported #1355481 (0x0eda…3b9b)
2020-06-28 00:37:48 Discovered new external address for our node: /ip4/10.156.1.51/tcp/30334/p2p/QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb
2020-06-28 00:37:49 Discovered new external address for our node: /ip4/10.156.1.50/tcp/30334/p2p/QmPfsj2YrU8aW6A6PJZrcfNgnCxLVB1Ea5Ekt2uf8Bq8Jb
2020-06-28 00:37:50 Failed to push message to queue, dropped it
2020-06-28 00:37:50 Failed to push message to queue, dropped it
2020-06-28 00:37:51 Idle (20 peers), best: #1355481 (0x0eda…3b9b), finalized #1355479 (0x9b23…35ef), ⬇ 34.6kiB/s ⬆ 72.3kiB/s
2020-06-28 00:37:54 Imported #1355482 (0xf29e…bb92)
2020-06-28 00:37:56 Idle (20 peers), best: #1355482 (0xf29e…bb92), finalized #1355480 (0x13e6…27a8), ⬇ 37.2kiB/s ⬆ 37.6kiB/s
2020-06-28 00:38:00 Imported #1355483 (0x5253…2407)
2020-06-28 00:38:01 Idle (20 peers), best: #1355483 (0x5253…2407), finalized #1355480 (0x13e6…27a8), ⬇ 35.1kiB/s ⬆ 43.5kiB/s
2020-06-28 00:38:06 Imported #1355484 (0x3c0e…6b41)
2020-06-28 00:38:06 Imported #1355484 (0x4e02…33b5)
2020-06-28 00:38:06 Imported #1355484 (0x3718…261b)
2020-06-28 00:38:07 Idle (20 peers), best: #1355484 (0x3c0e…6b41), finalized #1355481 (0x0eda…3b9b), ⬇ 37.1kiB/s ⬆ 49.3kiB/s
2020-06-28 00:38:12 Idle (20 peers), best: #1355484 (0x3c0e…6b41), finalized #1355482 (0xf29e…bb92), ⬇ 38.1kiB/s ⬆ 28.9kiB/s
2020-06-28 00:38:12 Imported #1355485 (0x0150…1722)
2020-06-28 00:38:17 Idle (20 peers), best: #1355485 (0x0150…1722), finalized #1355483 (0x5253…2407), ⬇ 32.9kiB/s ⬆ 52.4kiB/s
2020-06-28 00:38:18 Failed to push message to queue, dropped it
2020-06-28 00:38:18 Failed to push message to queue, dropped it
2020-06-28 00:38:18 Imported #1355486 (0x7e01…5524)
2020-06-28 00:38:22 Idle (21 peers), best: #1355486 (0x7e01…5524), finalized #1355484 (0x3c0e…6b41), ⬇ 37.2kiB/s ⬆ 36.2kiB/s
2020-06-28 00:38:24 Imported #1355487 (0x2a00…6323)
2020-06-28 00:38:26 Failed to push message to queue, dropped it
2020-06-28 00:38:26 Failed to push message to queue, dropped it
2020-06-28 00:38:26 Failed to push message to queue, dropped it
2020-06-28 00:38:26 Failed to push message to queue, dropped it
2020-06-28 00:38:26 Failed to push message to queue, dropped it
2020-06-28 00:38:26 Failed to push message to queue, dropped it
2020-06-28 00:38:27 Idle (21 peers), best: #1355487 (0x2a00…6323), finalized #1355485 (0x0150…1722), ⬇ 40.8kiB/s ⬆ 81.4kiB/s
2020-06-28 00:38:30 Imported #1355488 (0x1009…4305)
2020-06-28 00:38:32 Idle (20 peers), best: #1355488 (0x1009…4305), finalized #1355485 (0x0150…1722), ⬇ 35.0kiB/s ⬆ 44.1kiB/s
2020-06-28 00:38:36 Imported #1355489 (0x6c23…c23d)
2020-06-28 00:38:37 Idle (20 peers), best: #1355489 (0x6c23…c23d), finalized #1355486 (0x7e01…5524), ⬇ 37.2kiB/s ⬆ 37.5kiB/s
2020-06-28 00:38:42 Idle (21 peers), best: #1355489 (0x6c23…c23d), finalized #1355487 (0x2a00…6323), ⬇ 35.0kiB/s ⬆ 44.1kiB/s
2020-06-28 00:38:42 Imported #1355490 (0x8254…0a49)
2020-06-28 00:38:45 Failed to push message to queue, dropped it
2020-06-28 00:38:47 Idle (20 peers), best: #1355490 (0x8254…0a49), finalized #1355488 (0x1009…4305), ⬇ 37.3kiB/s ⬆ 38.9kiB/s
2020-06-28 00:38:48 Imported #1355491 (0xea50…2ba8)
@AGx10k
Copy link
Author

AGx10k commented Jun 28, 2020

seen exactly the same on flint couple days ago: invalid justification => Reserved peer disconnected => 0 peers

@AGx10k
Copy link
Author

AGx10k commented Jun 28, 2020

looks related to this paritytech/substrate#4678

@AGx10k
Copy link
Author

AGx10k commented Jun 28, 2020

just noticed that validator was not receiving finalized blocks for some time:

image

@AGx10k
Copy link
Author

AGx10k commented Jun 28, 2020

so possibly this can also be related to this paritytech/substrate#2187

@Mitchell-Grimes
Copy link

Is there any update to this? Looking to see if this was anything to do with file descriptor limitations that were previously seen on polkadot (similar behavior with the node not being seen by sentries: paritytech/polkadot#856)

Looking at TCP connections alone, it doesn't look anything like the same magnitude, but it does showcase the spike, clean connections, repeat. The gaps are when the service is restarted.
image

Will create a script to temporarily monitor this and post logs

@mikiquantum
Copy link
Contributor

Sentry setup has been deprecated by Parity, closing for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mikiquantum @AGx10k @Mitchell-Grimes