bug: Fail to start when passing a high number of shards

[AlbertoSoutullo]

Problem

Passing as parameters 64 shards makes nWaku crash.

Impact

Starting nWaku fails.

To reproduce

Run nWaku with the following parameters

/usr/bin/wakunode --relay=false --rest=true --rest-address=0.0.0.0 --max-connections=1500 --discv5-discovery=true --discv5-enr-auto-update=True --log-level=INFO --metrics-server=True --metrics-server-address=0.0.0.0 --nat=extip:$IP --cluster-id=2 --num-shards-in-network=64 --shard=0 --shard=1 --shard=2 --shard=3 --shard=4 --shard=5 --shard=6 --shard=7 --shard=8 --shard=9 --shard=10 --shard=11 --shard=12 --shard=13 --shard=14 --shard=15 --shard=16 --shard=17 --shard=18 --shard=19 --shard=20 --shard=21 --shard=22 --shard=23 --shard=24 --shard=25 --shard=26 --shard=27 --shard=28 --shard=29 --shard=30 --shard=31 --shard=32 --shard=33 --shard=34 --shard=35 --shard=36 --shard=37 --shard=38 --shard=39 --shard=40 --shard=41 --shard=42 --shard=43 --shard=44 --shard=45 --shard=46 --shard=47 --shard=48 --shard=49 --shard=50 --shard=51 --shard=52 --shard=53 --shard=54 --shard=55 --shard=56 --shard=57 --shard=58 --shard=59 --shard=60 --shard=61 --shard=62 --shard=63

Expected behavior

Waku starts without problems

Screenshots/logs

NTC 2025-01-23 12:05:55.057+00:00 REST service started                       tid=7 file=server.nim:186 address=0.0.0.0:8645
INF 2025-01-23 12:05:55.057+00:00 Starting REST HTTP server                  tid=7 file=builder.nim:105 url=http://0.0.0.0:8645/
INF 2025-01-23 12:05:55.057+00:00 Running nwaku node                         topics="wakunode main" tid=7 file=waku.nim:203 version=v0.34.0-rc.1
INF 2025-01-23 12:05:55.057+00:00 Configuration: Enabled protocols           topics="wakunode waku" tid=7 file=waku.nim:71 relay=false rlnRelay=false store=false filter=false lightpush=false peerExchange=false
INF 2025-01-23 12:05:55.057+00:00 Configuration. Network                     topics="wakunode waku" tid=7 file=waku.nim:79 cluster=2 maxPeers=none(int)
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=0
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=1
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=2
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=3
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=4
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=5
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=6
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=7
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=8
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=9
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=10
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=11
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=12
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=13
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=14
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=15
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=16
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=17
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=18
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=19
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=20
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=21
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=22
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=23
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=24
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=25
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=26
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=27
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=28
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=29
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=30
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=31
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=32
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=33
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=34
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=35
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=36
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=37
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=38
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=39
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=40
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=41
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=42
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=43
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=44
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=45
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=46
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=47
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=48
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=49
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=50
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=51
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=52
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=53
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=54
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=55
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=56
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=57
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=58
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=59
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=60
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=61
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=62
INF 2025-01-23 12:05:55.057+00:00 Configuration. Shards                      topics="wakunode waku" tid=7 file=waku.nim:82 shard=63
INF 2025-01-23 12:05:55.057+00:00 Initializing networking                    tid=7 file=waku_node.nim:131 addrs=@[/ip4/10.42.79.33/tcp/60000]
INF 2025-01-23 12:05:55.057+00:00 Created WakuMetadata protocol              topics="waku node" tid=7 file=protocol.nim:120 clusterId=2 shards="{4, 5, 54, 11, 39, 3, 49, 2, 22, 26, 38, 29, 32, 19, 46, 16, 52, 6, 63, 47, 12, 28, 51, 48, 56, 61, 41, 13, 30, 1, 43, 50, 20, 14, 60, 9, 35, 7, 59, 55, 37, 17, 31, 36, 25, 18, 33, 34, 44, 62, 27, 10, 0, 15, 23, 24, 21, 40, 42, 8, 57, 58, 45, 53}"
INF 2025-01-23 12:05:55.057+00:00 mounting sharding                          topics="waku node" tid=7 file=waku_node.nim:236 clusterId=2 shardCount=64
INF 2025-01-23 12:05:55.057+00:00 mounting libp2p ping protocol              topics="waku node" tid=7 file=waku_node.nim:1197
INF 2025-01-23 12:05:55.057+00:00 mounting store client                      topics="waku node" tid=7 file=waku_node.nim:948
INF 2025-01-23 12:05:55.057+00:00 mounting legacy store client               topics="waku node" tid=7 file=waku_node.nim:815
INF 2025-01-23 12:05:55.057+00:00 Starting Waku node                         topics="waku node" tid=7 file=waku_node.nim:1318 version=v0.34.0-rc.1
INF 2025-01-23 12:05:55.057+00:00 Setting up AutonatService                  topics="libp2p autonatservice" tid=7 file=service.nim:207
INF 2025-01-23 12:05:55.057+00:00 PeerInfo                                   topics="waku node" tid=7 file=waku_node.nim:1283 peerId=16U*WZvMGA addrs=@[/ip4/10.42.79.33/tcp/60000]
INF 2025-01-23 12:05:55.057+00:00 Listening on                               topics="waku node" tid=7 file=waku_node.nim:1306 full=[/ip4/0.0.0.0/tcp/60000/p2p/16Uiu2HAmSX7xQRfMjDXhBxFCt4eZiBA6rXYAeuH84VyCDYWZvMGA] localIp=10.42.79.33 switchAddress=@[/ip4/10.42.79.33/tcp/60000]
INF 2025-01-23 12:05:55.057+00:00 Announcing addresses                       topics="waku node" tid=7 file=waku_node.nim:1308 full=[/ip4/10.42.79.33/tcp/60000/p2p/16Uiu2HAmSX7xQRfMjDXhBxFCt4eZiBA6rXYAeuH84VyCDYWZvMGA]
INF 2025-01-23 12:05:55.057+00:00 DNS: discoverable ENR                      topics="waku node" tid=7 file=waku_node.nim:1309 enr=enr:-QEmuEB11f0rRCGj0llBeqMnjAVNZGr4qgf98TiRYstk2ymE8wv5kZV9orHUCd7iustfPjm9R3-DkWtT_VO_xsZRP_nAAYJpZIJ2NIJpcIQKKk8him11bHRpYWRkcnOAg3JzdriCAAL__________wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIlzZWNwMjU2azGhA83_kepJMd2-PD5sR8WjTGz6uvDVEjJBZEhMGIbcdxxng3RjcILqYIN1ZHCCIyiFd2FrdTIA
INF 2025-01-23 12:05:55.057+00:00 Node started successfully                  topics="waku node" tid=7 file=waku_node.nim:1357
INF 2025-01-23 12:05:55.057+00:00 PeerInfo                                   topics="waku node" tid=7 file=waku_node.nim:1283 peerId=16U*WZvMGA addrs=@[/ip4/10.42.79.33/tcp/60000]
INF 2025-01-23 12:05:55.057+00:00 Listening on                               topics="waku node" tid=7 file=waku_node.nim:1306 full=[/ip4/0.0.0.0/tcp/60000/p2p/16Uiu2HAmSX7xQRfMjDXhBxFCt4eZiBA6rXYAeuH84VyCDYWZvMGA] localIp=10.42.79.33 switchAddress=@[/ip4/10.42.79.33/tcp/60000]
INF 2025-01-23 12:05:55.057+00:00 Announcing addresses                       topics="waku node" tid=7 file=waku_node.nim:1308 full=[/ip4/10.42.79.33/tcp/60000/p2p/16Uiu2HAmSX7xQRfMjDXhBxFCt4eZiBA6rXYAeuH84VyCDYWZvMGA]
INF 2025-01-23 12:05:55.057+00:00 DNS: discoverable ENR                      topics="waku node" tid=7 file=waku_node.nim:1309 enr=enr:-QEmuEB11f0rRCGj0llBeqMnjAVNZGr4qgf98TiRYstk2ymE8wv5kZV9orHUCd7iustfPjm9R3-DkWtT_VO_xsZRP_nAAYJpZIJ2NIJpcIQKKk8him11bHRpYWRkcnOAg3JzdriCAAL__________wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIlzZWNwMjU2azGhA83_kepJMd2-PD5sR8WjTGz6uvDVEjJBZEhMGIbcdxxng3RjcILqYIN1ZHCCIyiFd2FrdTIA
ERR 2025-01-23 12:05:55.057+00:00 Starting waku failed                       topics="wakunode main" tid=7 file=wakunode2.nim:78 error="Error in updateApp: failed to update multiaddress in ENR updateAddressInENR: Record exceeds maximum size"

nwaku version/commit hash

/usr/bin/wakunode --version
version / git commit hash: v0.34.0-rc.1

Additional context

[Ivansete-status]

Thanks for raising this!
Nevertheless, I think we just need to end the execution when the number of shards is greater than certain value.
@AlbertoSoutullo - What is the maximum value supported without crashing?

[AlbertoSoutullo]

Thanks for raising this! Nevertheless, I think we just need to end the execution when the number of shards is greater than certain value. @AlbertoSoutullo - What is the maximum value supported without crashing?

Works fine until 60, included.