Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Identify behavior adds observed address, but contains ephemeral port causing AutoNAT probe failure #4153

Closed
b-zee opened this issue Jul 3, 2023 · 2 comments · Fixed by #4158

Comments

@b-zee
Copy link
Contributor

b-zee commented Jul 3, 2023

Summary

When nodes use the Identify behavior, their observed address gets reported. This seems to be added as a candidate for an external address.

When using AutoNAT, these observed addresses might be dialed by a server, resulting a failed DialError report.

Expected behaviour

AutoNAT to succeed.

Actual behaviour

I have tried to reproduce this in an MVP. Our production code yielded this behavior in, say, about 5% of nodes. I do see that the nodes are asking for a probe-back on their ephemeral ports, but it seems the actual listening address is the first that is dial by the server and we get a successful report.

Code is here: https://github.com/b-zee/rust-libp2p/blob/28c4786d4ae05a30c40dde9113831fe3b63f98cf/examples/safe/src/main.rs

Debug Output

Look for AutoNAT InboundProbeEvent: Request { probe_id: ProbeId(7), peer: PeerId("12D3KooWFAnNTegMwtQhqBejyx1xoMxDULvBKCZYeJrwP1oLr5Wb"), addresses: ["/ip4/127.0.0.1/tcp/10001/p2p/12D3KooWFAnNTegMwtQhqBejyx1xoMxDULvBKCZYeJrwP1oLr5Wb", "/ip4/127.0.0.1/tcp/45072/p2p/12D3KooWFAnNTegMwtQhqBejyx1xoMxDULvBKCZYeJrwP1oLr5Wb"] }

This line contains multiple (ephemeral) addresses of the peer which are probed by AutoNAT.

    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on build directory
    Finished release [optimized] target(s) in 1.54s
     Running `target/release/safe --port=10005`
[2023-07-03T14:16:00.433Z INFO  safe] main
[2023-07-03T14:16:00.433Z INFO  safe] opts: Opt { port: 10005, peers: ["/ip4/127.0.0.1/tcp/10000", "/ip4/127.0.0.1/tcp/10001", "/ip4/127.0.0.1/tcp/10002", "/ip4/127.0.0.1/tcp/10003", "/ip4/127.0.0.1/tcp/10004", "/ip4/127.0.0.1/tcp/10005"] }
[2023-07-03T14:16:00.433Z TRACE safe] handle_event: NewListenAddr { listener_id: ListenerId(1), address: "/ip4/127.0.0.1/tcp/10005" }
[2023-07-03T14:16:00.433Z INFO  safe] dialing bootstrap peer
[2023-07-03T14:16:00.433Z INFO  safe] dialing bootstrap peer
[2023-07-03T14:16:00.433Z INFO  safe] dialing bootstrap peer
[2023-07-03T14:16:00.433Z INFO  safe] dialing bootstrap peer
[2023-07-03T14:16:00.433Z INFO  safe] dialing bootstrap peer
[2023-07-03T14:16:00.433Z INFO  safe] dialing bootstrap peer
[2023-07-03T14:16:00.433Z ERROR safe] dialing bootstrap peer error: Dial error: no addresses for peer.
[2023-07-03T14:16:00.433Z TRACE safe] handle_event: OutgoingConnectionError { connection_id: ConnectionId(1), peer_id: None, error: Transport([("/ip4/127.0.0.1/tcp/10000", Other(Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) }))]) }
[2023-07-03T14:16:00.433Z TRACE safe] handle_event: OutgoingConnectionError { connection_id: ConnectionId(3), peer_id: None, error: Transport([("/ip4/127.0.0.1/tcp/10002", Other(Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) }))]) }
[2023-07-03T14:16:00.434Z TRACE safe] handle_event: OutgoingConnectionError { connection_id: ConnectionId(5), peer_id: None, error: Transport([("/ip4/127.0.0.1/tcp/10004", Other(Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) }))]) }
[2023-07-03T14:16:00.434Z TRACE safe] handle_event: ConnectionEstablished { peer_id: PeerId("12D3KooWFAnNTegMwtQhqBejyx1xoMxDULvBKCZYeJrwP1oLr5Wb"), connection_id: ConnectionId(2), endpoint: Dialer { address: "/ip4/127.0.0.1/tcp/10001", role_override: Dialer }, num_established: 1, concurrent_dial_errors: Some([]), established_in: 817.806µs }
[2023-07-03T14:16:00.434Z TRACE safe] handle_event: ConnectionEstablished { peer_id: PeerId("12D3KooWPmP6JMXMbxyC16RczyuVwF1ZtQCH7bfx2ZfDw6BzAcCZ"), connection_id: ConnectionId(4), endpoint: Dialer { address: "/ip4/127.0.0.1/tcp/10003", role_override: Dialer }, num_established: 1, concurrent_dial_errors: Some([]), established_in: 848.165µs }
[2023-07-03T14:16:00.467Z TRACE safe] handle_event: IncomingConnection { connection_id: ConnectionId(7), local_addr: "/ip4/127.0.0.1/tcp/10005", send_back_addr: "/ip4/127.0.0.1/tcp/38362" }
[2023-07-03T14:16:00.484Z TRACE safe] handle_event: IncomingConnection { connection_id: ConnectionId(8), local_addr: "/ip4/127.0.0.1/tcp/10005", send_back_addr: "/ip4/127.0.0.1/tcp/38376" }
[2023-07-03T14:16:00.500Z TRACE safe] handle_event: IncomingConnection { connection_id: ConnectionId(9), local_addr: "/ip4/127.0.0.1/tcp/10005", send_back_addr: "/ip4/127.0.0.1/tcp/38380" }
[2023-07-03T14:16:00.508Z TRACE safe] handle_event: ConnectionEstablished { peer_id: PeerId("12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs"), connection_id: ConnectionId(7), endpoint: Listener { local_addr: "/ip4/127.0.0.1/tcp/10005", send_back_addr: "/ip4/127.0.0.1/tcp/38362" }, num_established: 1, concurrent_dial_errors: None, established_in: 41.132633ms }
[2023-07-03T14:16:00.508Z TRACE safe] handle_event: Behaviour(Identify(Sent { peer_id: PeerId("12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs") }))
[2023-07-03T14:16:00.516Z TRACE safe] handle_event: Behaviour(Identify(Received { peer_id: PeerId("12D3KooWFAnNTegMwtQhqBejyx1xoMxDULvBKCZYeJrwP1oLr5Wb"), info: Info { public_key: PublicKey { publickey: Ed25519(PublicKey(compressed): 4f856a857456e5e87ba8f5f7f59afdf6a79ad1b59d0ff2082d4764f84c5eadc) }, protocol_version: "/safe/0.1.0", agent_version: "rust-libp2p/0.43.0", listen_addrs: ["/ip4/127.0.0.1/tcp/10001"], protocols: [StreamProtocol { inner: Right("/ipfs/id/1.0.0") }, StreamProtocol { inner: Right("/ipfs/id/push/1.0.0") }, StreamProtocol { inner: Right("/libp2p/autonat/1.0.0") }], observed_addr: "/ip4/127.0.0.1/tcp/49364" } }))
[2023-07-03T14:16:00.516Z INFO  safe] Identify info from PeerId("12D3KooWFAnNTegMwtQhqBejyx1xoMxDULvBKCZYeJrwP1oLr5Wb"): observed address "/ip4/127.0.0.1/tcp/49364"
[2023-07-03T14:16:00.516Z TRACE safe] handle_event: Behaviour(Identify(Received { peer_id: PeerId("12D3KooWPmP6JMXMbxyC16RczyuVwF1ZtQCH7bfx2ZfDw6BzAcCZ"), info: Info { public_key: PublicKey { publickey: Ed25519(PublicKey(compressed): cf3feb297a96a26d5d1354f9932aeaf46e798e83a536aa5ecec78b1712ef12) }, protocol_version: "/safe/0.1.0", agent_version: "rust-libp2p/0.43.0", listen_addrs: ["/ip4/127.0.0.1/tcp/10003"], protocols: [StreamProtocol { inner: Right("/ipfs/id/1.0.0") }, StreamProtocol { inner: Right("/ipfs/id/push/1.0.0") }, StreamProtocol { inner: Right("/libp2p/autonat/1.0.0") }], observed_addr: "/ip4/127.0.0.1/tcp/36728" } }))
[2023-07-03T14:16:00.516Z INFO  safe] Identify info from PeerId("12D3KooWPmP6JMXMbxyC16RczyuVwF1ZtQCH7bfx2ZfDw6BzAcCZ"): observed address "/ip4/127.0.0.1/tcp/36728"
[2023-07-03T14:16:00.516Z TRACE safe] handle_event: Behaviour(Identify(Sent { peer_id: PeerId("12D3KooWFAnNTegMwtQhqBejyx1xoMxDULvBKCZYeJrwP1oLr5Wb") }))
[2023-07-03T14:16:00.516Z TRACE safe] handle_event: Behaviour(Identify(Sent { peer_id: PeerId("12D3KooWPmP6JMXMbxyC16RczyuVwF1ZtQCH7bfx2ZfDw6BzAcCZ") }))
[2023-07-03T14:16:00.527Z TRACE safe] handle_event: ConnectionEstablished { peer_id: PeerId("12D3KooWSCAKKxQprrvkkFzhkFn9fYYDxutLrjUBLU3LvRhn2mRV"), connection_id: ConnectionId(8), endpoint: Listener { local_addr: "/ip4/127.0.0.1/tcp/10005", send_back_addr: "/ip4/127.0.0.1/tcp/38376" }, num_established: 1, concurrent_dial_errors: None, established_in: 42.755691ms }
[2023-07-03T14:16:00.527Z TRACE safe] handle_event: Behaviour(Identify(Sent { peer_id: PeerId("12D3KooWSCAKKxQprrvkkFzhkFn9fYYDxutLrjUBLU3LvRhn2mRV") }))
[2023-07-03T14:16:00.532Z TRACE safe] handle_event: IncomingConnection { connection_id: ConnectionId(10), local_addr: "/ip4/127.0.0.1/tcp/10005", send_back_addr: "/ip4/127.0.0.1/tcp/38384" }
[2023-07-03T14:16:00.542Z TRACE safe] handle_event: ConnectionEstablished { peer_id: PeerId("12D3KooWGqThYDJHcvzRxaoCTZp7rmrC3F1DJTHsM4unvq8PgfCa"), connection_id: ConnectionId(9), endpoint: Listener { local_addr: "/ip4/127.0.0.1/tcp/10005", send_back_addr: "/ip4/127.0.0.1/tcp/38380" }, num_established: 1, concurrent_dial_errors: None, established_in: 41.607772ms }
[2023-07-03T14:16:00.542Z TRACE safe] handle_event: Behaviour(Identify(Sent { peer_id: PeerId("12D3KooWGqThYDJHcvzRxaoCTZp7rmrC3F1DJTHsM4unvq8PgfCa") }))
[2023-07-03T14:16:00.565Z TRACE safe] handle_event: IncomingConnection { connection_id: ConnectionId(11), local_addr: "/ip4/127.0.0.1/tcp/10005", send_back_addr: "/ip4/127.0.0.1/tcp/38386" }
[2023-07-03T14:16:00.573Z TRACE safe] handle_event: ConnectionEstablished { peer_id: PeerId("12D3KooWP8is1eMWg5SdnDBPy5ePd5dkMXSNLcX2qFEXnEAtRxbB"), connection_id: ConnectionId(10), endpoint: Listener { local_addr: "/ip4/127.0.0.1/tcp/10005", send_back_addr: "/ip4/127.0.0.1/tcp/38384" }, num_established: 1, concurrent_dial_errors: None, established_in: 41.064442ms }
[2023-07-03T14:16:00.573Z TRACE safe] handle_event: Behaviour(Identify(Sent { peer_id: PeerId("12D3KooWP8is1eMWg5SdnDBPy5ePd5dkMXSNLcX2qFEXnEAtRxbB") }))
[2023-07-03T14:16:00.590Z TRACE safe] handle_event: Behaviour(Identify(Received { peer_id: PeerId("12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs"), info: Info { public_key: PublicKey { publickey: Ed25519(PublicKey(compressed): cd822ac5b72799937c1acaf9bda545b709e4f9210a1c3de81b5e77ca236e) }, protocol_version: "/safe/0.1.0", agent_version: "rust-libp2p/0.43.0", listen_addrs: ["/ip4/127.0.0.1/tcp/10009"], protocols: [StreamProtocol { inner: Right("/ipfs/id/push/1.0.0") }, StreamProtocol { inner: Right("/libp2p/autonat/1.0.0") }, StreamProtocol { inner: Right("/ipfs/id/1.0.0") }], observed_addr: "/ip4/127.0.0.1/tcp/10005" } }))
[2023-07-03T14:16:00.590Z INFO  safe] Identify info from PeerId("12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs"): observed address "/ip4/127.0.0.1/tcp/10005"
[2023-07-03T14:16:00.609Z TRACE safe] handle_event: Behaviour(Identify(Received { peer_id: PeerId("12D3KooWSCAKKxQprrvkkFzhkFn9fYYDxutLrjUBLU3LvRhn2mRV"), info: Info { public_key: PublicKey { publickey: Ed25519(PublicKey(compressed): f3505e14e3bad0bbe7264840f7b4baf163e2c3e0809de840a72fc3faa9c9c34) }, protocol_version: "/safe/0.1.0", agent_version: "rust-libp2p/0.43.0", listen_addrs: ["/ip4/127.0.0.1/tcp/10006"], protocols: [StreamProtocol { inner: Right("/ipfs/id/1.0.0") }, StreamProtocol { inner: Right("/ipfs/id/push/1.0.0") }, StreamProtocol { inner: Right("/libp2p/autonat/1.0.0") }], observed_addr: "/ip4/127.0.0.1/tcp/10005" } }))
[2023-07-03T14:16:00.609Z INFO  safe] Identify info from PeerId("12D3KooWSCAKKxQprrvkkFzhkFn9fYYDxutLrjUBLU3LvRhn2mRV"): observed address "/ip4/127.0.0.1/tcp/10005"
[2023-07-03T14:16:00.624Z TRACE safe] handle_event: Behaviour(Identify(Received { peer_id: PeerId("12D3KooWGqThYDJHcvzRxaoCTZp7rmrC3F1DJTHsM4unvq8PgfCa"), info: Info { public_key: PublicKey { publickey: Ed25519(PublicKey(compressed): 6849a4a549165774975fd3e4df431b8924626209b9b6510536f42ffac9f224f) }, protocol_version: "/safe/0.1.0", agent_version: "rust-libp2p/0.43.0", listen_addrs: ["/ip4/127.0.0.1/tcp/10008"], protocols: [StreamProtocol { inner: Right("/ipfs/id/1.0.0") }, StreamProtocol { inner: Right("/ipfs/id/push/1.0.0") }, StreamProtocol { inner: Right("/libp2p/autonat/1.0.0") }], observed_addr: "/ip4/127.0.0.1/tcp/10005" } }))
[2023-07-03T14:16:00.624Z INFO  safe] Identify info from PeerId("12D3KooWGqThYDJHcvzRxaoCTZp7rmrC3F1DJTHsM4unvq8PgfCa"): observed address "/ip4/127.0.0.1/tcp/10005"
[2023-07-03T14:16:00.647Z TRACE safe] handle_event: ConnectionEstablished { peer_id: PeerId("12D3KooWFtjj7AyXwLwqMBKvMTN6hSvjLea49AamTjg132vCqE3R"), connection_id: ConnectionId(11), endpoint: Listener { local_addr: "/ip4/127.0.0.1/tcp/10005", send_back_addr: "/ip4/127.0.0.1/tcp/38386" }, num_established: 1, concurrent_dial_errors: None, established_in: 81.889837ms }
[2023-07-03T14:16:00.647Z TRACE safe] handle_event: Behaviour(Identify(Sent { peer_id: PeerId("12D3KooWFtjj7AyXwLwqMBKvMTN6hSvjLea49AamTjg132vCqE3R") }))
[2023-07-03T14:16:00.655Z TRACE safe] handle_event: Behaviour(Identify(Received { peer_id: PeerId("12D3KooWP8is1eMWg5SdnDBPy5ePd5dkMXSNLcX2qFEXnEAtRxbB"), info: Info { public_key: PublicKey { publickey: Ed25519(PublicKey(compressed): c5dbc89fc5bcf2f76a277df1ad530753c432b6f1bc7ec6cb7d85521eb02cca) }, protocol_version: "/safe/0.1.0", agent_version: "rust-libp2p/0.43.0", listen_addrs: ["/ip4/127.0.0.1/tcp/10010"], protocols: [StreamProtocol { inner: Right("/ipfs/id/push/1.0.0") }, StreamProtocol { inner: Right("/ipfs/id/1.0.0") }, StreamProtocol { inner: Right("/libp2p/autonat/1.0.0") }], observed_addr: "/ip4/127.0.0.1/tcp/10005" } }))
[2023-07-03T14:16:00.655Z INFO  safe] Identify info from PeerId("12D3KooWP8is1eMWg5SdnDBPy5ePd5dkMXSNLcX2qFEXnEAtRxbB"): observed address "/ip4/127.0.0.1/tcp/10005"
[2023-07-03T14:16:00.729Z TRACE safe] handle_event: Behaviour(Identify(Received { peer_id: PeerId("12D3KooWFtjj7AyXwLwqMBKvMTN6hSvjLea49AamTjg132vCqE3R"), info: Info { public_key: PublicKey { publickey: Ed25519(PublicKey(compressed): 5a44d2ae8956b52139ad14c6e347ca3a51a42cc23ac9f70249883f31bc5f0) }, protocol_version: "/safe/0.1.0", agent_version: "rust-libp2p/0.43.0", listen_addrs: ["/ip4/127.0.0.1/tcp/10007"], protocols: [StreamProtocol { inner: Right("/libp2p/autonat/1.0.0") }, StreamProtocol { inner: Right("/ipfs/id/push/1.0.0") }, StreamProtocol { inner: Right("/ipfs/id/1.0.0") }], observed_addr: "/ip4/127.0.0.1/tcp/10005" } }))
[2023-07-03T14:16:00.729Z INFO  safe] Identify info from PeerId("12D3KooWFtjj7AyXwLwqMBKvMTN6hSvjLea49AamTjg132vCqE3R"): observed address "/ip4/127.0.0.1/tcp/10005"
[2023-07-03T14:16:03.433Z TRACE safe] handle_event: Behaviour(AutoNat(OutboundProbe(Request { probe_id: ProbeId(0), peer: PeerId("12D3KooWSCAKKxQprrvkkFzhkFn9fYYDxutLrjUBLU3LvRhn2mRV") })))
[2023-07-03T14:16:03.516Z TRACE safe] handle_event: IncomingConnection { connection_id: ConnectionId(12), local_addr: "/ip4/127.0.0.1/tcp/10005", send_back_addr: "/ip4/127.0.0.1/tcp/38390" }
[2023-07-03T14:16:03.549Z TRACE safe] handle_event: Behaviour(AutoNat(InboundProbe(Request { probe_id: ProbeId(1), peer: PeerId("12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs"), addresses: ["/ip4/127.0.0.1/tcp/36748/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs", "/ip4/127.0.0.1/tcp/39838/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs", "/ip4/127.0.0.1/tcp/49388/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs", "/ip4/127.0.0.1/tcp/10009/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs", "/ip4/127.0.0.1/tcp/38362/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs"] })))
[2023-07-03T14:16:03.549Z INFO  safe] AutoNAT InboundProbeEvent: Request { probe_id: ProbeId(1), peer: PeerId("12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs"), addresses: ["/ip4/127.0.0.1/tcp/36748/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs", "/ip4/127.0.0.1/tcp/39838/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs", "/ip4/127.0.0.1/tcp/49388/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs", "/ip4/127.0.0.1/tcp/10009/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs", "/ip4/127.0.0.1/tcp/38362/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs"] }
[2023-07-03T14:16:03.549Z TRACE safe] handle_event: Dialing { peer_id: Some(PeerId("12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs")), connection_id: ConnectionId(13) }
[2023-07-03T14:16:03.550Z TRACE safe] handle_event: ConnectionEstablished { peer_id: PeerId("12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs"), connection_id: ConnectionId(13), endpoint: Dialer { address: "/ip4/127.0.0.1/tcp/10009/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs", role_override: Dialer }, num_established: 2, concurrent_dial_errors: Some([("/ip4/127.0.0.1/tcp/36748/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs", Other(Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })), ("/ip4/127.0.0.1/tcp/39838/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs", Other(Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) })), ("/ip4/127.0.0.1/tcp/49388/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs", Other(Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) }))]), established_in: 1.403488ms }
[2023-07-03T14:16:03.550Z TRACE safe] handle_event: Behaviour(AutoNat(InboundProbe(Response { probe_id: ProbeId(1), peer: PeerId("12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs"), address: "/ip4/127.0.0.1/tcp/10009/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs" })))
[2023-07-03T14:16:03.558Z TRACE safe] handle_event: Behaviour(AutoNat(OutboundProbe(Response { probe_id: ProbeId(0), peer: PeerId("12D3KooWSCAKKxQprrvkkFzhkFn9fYYDxutLrjUBLU3LvRhn2mRV"), address: "/ip4/127.0.0.1/tcp/10005/p2p/12D3KooWQgzBoCMUQrC69bk4n29xvzhskc3nSHrbiuNVa5dSM9Dm" })))
[2023-07-03T14:16:03.558Z TRACE safe] handle_event: Behaviour(AutoNat(StatusChanged { old: Unknown, new: Public("/ip4/127.0.0.1/tcp/10005/p2p/12D3KooWQgzBoCMUQrC69bk4n29xvzhskc3nSHrbiuNVa5dSM9Dm") }))
[2023-07-03T14:16:03.558Z TRACE safe] handle_event: ConnectionEstablished { peer_id: PeerId("12D3KooWSCAKKxQprrvkkFzhkFn9fYYDxutLrjUBLU3LvRhn2mRV"), connection_id: ConnectionId(12), endpoint: Listener { local_addr: "/ip4/127.0.0.1/tcp/10005", send_back_addr: "/ip4/127.0.0.1/tcp/38390" }, num_established: 2, concurrent_dial_errors: None, established_in: 41.706254ms }
[2023-07-03T14:16:03.558Z TRACE safe] handle_event: Behaviour(Identify(Sent { peer_id: PeerId("12D3KooWSCAKKxQprrvkkFzhkFn9fYYDxutLrjUBLU3LvRhn2mRV") }))
[2023-07-03T14:16:03.632Z TRACE safe] handle_event: Behaviour(Identify(Received { peer_id: PeerId("12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs"), info: Info { public_key: PublicKey { publickey: Ed25519(PublicKey(compressed): cd822ac5b72799937c1acaf9bda545b709e4f9210a1c3de81b5e77ca236e) }, protocol_version: "/safe/0.1.0", agent_version: "rust-libp2p/0.43.0", listen_addrs: ["/ip4/127.0.0.1/tcp/10009", "/ip4/127.0.0.1/tcp/10009/p2p/12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs"], protocols: [StreamProtocol { inner: Right("/ipfs/id/1.0.0") }, StreamProtocol { inner: Right("/ipfs/id/push/1.0.0") }, StreamProtocol { inner: Right("/libp2p/autonat/1.0.0") }], observed_addr: "/ip4/127.0.0.1/tcp/36028" } }))
[2023-07-03T14:16:03.632Z INFO  safe] Identify info from PeerId("12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs"): observed address "/ip4/127.0.0.1/tcp/36028"
[2023-07-03T14:16:03.632Z TRACE safe] handle_event: Behaviour(Identify(Sent { peer_id: PeerId("12D3KooWPeasFmfkXLPniHDUAgHLTcMehvqYXKP3PELQtYJdNVTs") }))
[2023-07-03T14:16:03.640Z TRACE safe] handle_event: Behaviour(Identify(Received { peer_id: PeerId("12D3KooWSCAKKxQprrvkkFzhkFn9fYYDxutLrjUBLU3LvRhn2mRV"), info: Info { public_key: PublicKey { publickey: Ed25519(PublicKey(compressed): f3505e14e3bad0bbe7264840f7b4baf163e2c3e0809de840a72fc3faa9c9c34) }, protocol_version: "/safe/0.1.0", agent_version: "rust-libp2p/0.43.0", listen_addrs: ["/ip4/127.0.0.1/tcp/10006"], protocols: [StreamProtocol { inner: Right("/libp2p/autonat/1.0.0") }, StreamProtocol { inner: Right("/ipfs/id/push/1.0.0") }, StreamProtocol { inner: Right("/ipfs/id/1.0.0") }], observed_addr: "/ip4/127.0.0.1/tcp/10005/p2p/12D3KooWQgzBoCMUQrC69bk4n29xvzhskc3nSHrbiuNVa5dSM9Dm" } }))
[2023-07-03T14:16:03.640Z INFO  safe] Identify info from PeerId("12D3KooWSCAKKxQprrvkkFzhkFn9fYYDxutLrjUBLU3LvRhn2mRV"): observed address "/ip4/127.0.0.1/tcp/10005/p2p/12D3KooWQgzBoCMUQrC69bk4n29xvzhskc3nSHrbiuNVa5dSM9Dm"

Partial output from our production test:

Debug Output

[2023-06-29T05:52:43.590082Z DEBUG <unknown module>] Inbound dial request from Peer 12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum with dial-back addresses ["/ip4/134.209.23.129/tcp/58026/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/42904/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/56788/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/58264/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/59236/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/43442/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/59252/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/51268/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/33030/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/58272/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/43512/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/35412/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/51552/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/58034/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/56754/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/40604/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum"].    
[2023-06-29T05:52:43.590172Z DEBUG sn_networking::event] AutoNAT inbound probe: Request { probe_id: ProbeId(20), peer: PeerId("12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum"), addresses: ["/ip4/134.209.23.129/tcp/58026/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/42904/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/56788/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/58264/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/59236/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/43442/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/59252/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/51268/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/33030/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/58272/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/43512/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/35412/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/51552/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/58034/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/56754/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", "/ip4/134.209.23.129/tcp/40604/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum"] }
[2023-06-29T05:52:43.590254Z DEBUG <unknown module>] discarding 5 addresses from `NetworkBehaviour` because `DialOpts::extend_addresses_through_behaviour is `false` for connection ConnectionId(492)    
[2023-06-29T05:52:43.590266Z DEBUG <unknown module>] dialing 134.209.23.129:58026    
[2023-06-29T05:52:43.590301Z DEBUG <unknown module>] dialing 134.209.23.129:42904    
[2023-06-29T05:52:43.590312Z DEBUG <unknown module>] dialing 134.209.23.129:56788    
[2023-06-29T05:52:43.590319Z DEBUG <unknown module>] dialing 134.209.23.129:58264    
[2023-06-29T05:52:43.590325Z DEBUG <unknown module>] dialing 134.209.23.129:59236    
[2023-06-29T05:52:43.590333Z DEBUG <unknown module>] dialing 134.209.23.129:43442    
[2023-06-29T05:52:43.590341Z DEBUG <unknown module>] dialing 134.209.23.129:59252    
[2023-06-29T05:52:43.590347Z DEBUG <unknown module>] dialing 134.209.23.129:51268    
[2023-06-29T05:52:43.590356Z DEBUG <unknown module>] dialing 134.209.23.129:33030    
[2023-06-29T05:52:43.590363Z DEBUG <unknown module>] dialing 134.209.23.129:58272    
[2023-06-29T05:52:43.590369Z DEBUG <unknown module>] dialing 134.209.23.129:43512    
[2023-06-29T05:52:43.590378Z DEBUG <unknown module>] dialing 134.209.23.129:35412    
[2023-06-29T05:52:43.590384Z DEBUG <unknown module>] dialing 134.209.23.129:51552    
[2023-06-29T05:52:43.590390Z DEBUG <unknown module>] dialing 134.209.23.129:58034    
[2023-06-29T05:52:43.590396Z DEBUG <unknown module>] dialing 134.209.23.129:56754    
[2023-06-29T05:52:43.590403Z DEBUG <unknown module>] dialing 134.209.23.129:40604    
[2023-06-29T05:52:43.590418Z TRACE sn_networking::event] Dialing Some(PeerId("12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum")) on ConnectionId(492)
[2023-06-29T05:52:43.598493Z DEBUG <unknown module>] Dial-back to peer 12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum failed with error Transport([("/ip4/134.209.23.129/tcp/58026/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/42904/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/56788/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/58264/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/59236/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/43442/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/59252/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/51268/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/33030/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/58272/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/43512/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/35412/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/51552/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/58034/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/56754/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/40604/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } }))]).    
[2023-06-29T05:52:43.598637Z DEBUG <unknown module>] Connection attempt to PeerId("12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum") failed with Transport([("/ip4/134.209.23.129/tcp/58026/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/42904/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/56788/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/58264/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/59236/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/43442/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/59252/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/51268/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/33030/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/58272/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/43512/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/35412/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/51552/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/58034/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/56754/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } })), ("/ip4/134.209.23.129/tcp/40604/p2p/12D3KooWNsQpwN8rTLHyNkuWoKa4YyNxVFUyn8RQ1gaBDpmMfoum", Other(Custom { kind: Other, error: Custom { kind: Other, error: Left(Left(Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })) } }))]).    

Possible Solution

Either these addresses not be added at all as candidates, or some mechanism that puts the address without the ephemeral port first, so that results in a successful AutoNAT report?

Version

  • libp2p version (version number, commit, or branch): master

Would you like to work on fixing this bug?

Maybe.

@b-zee
Copy link
Contributor Author

b-zee commented Jul 3, 2023

Note, we observed this behavior when we updated from 0.51 to 0.52. I have tried looking for changes regarding identify, but I could not see why we are seeing this behavior with the update. We'll therefore have to postpone the updating for now until we can fix this. I will keep looking at this probably.

@b-zee
Copy link
Contributor Author

b-zee commented Jul 5, 2023

I found a possible culprit. In the following code, the candidate is passed on before address translation.

rust-libp2p/swarm/src/lib.rs

Lines 1065 to 1094 in 28c4786

self.behaviour
.on_swarm_event(FromSwarm::NewExternalAddrCandidate(
NewExternalAddrCandidate { addr: &addr },
));
// Generate more candidates based on address translation.
// For TCP without port-reuse, the observed address contains an ephemeral port which needs to be replaced by the port of a listen address.
let translated_addresses = {
let mut addrs: Vec<_> = self
.listened_addrs
.values()
.flatten()
.filter_map(|server| self.transport.address_translation(server, &addr))
.collect();
// remove duplicates
addrs.sort_unstable();
addrs.dedup();
addrs
};
for addr in translated_addresses {
self.behaviour
.on_swarm_event(FromSwarm::NewExternalAddrCandidate(
NewExternalAddrCandidate { addr: &addr },
));
}
}
ToSwarm::ExternalAddrConfirmed(addr) => {
self.add_external_address(addr);

In the previous version of this code (before #3954 @thomaseizinger), add_external_address was called only after address translation. (See here.)

I think a possible solution is not sending the event before address translation. (I.e. the first line of code above.)

b-zee added a commit to b-zee/rust-libp2p that referenced this issue Jul 5, 2023
The `NewExternalAddrCandidate` event is yielded both before and after address translation. This will
cause, in the case of TCP, ephemeral ports to be added as candidate.

In turn, that will cause protocols like AutoNAT to fail as these candidates are not actually
reachable/external.

Fixes libp2p#4153
b-zee added a commit to b-zee/rust-libp2p that referenced this issue Jul 6, 2023
The `NewExternalAddrCandidate` event is yielded both before and after address translation. This will
cause, in the case of TCP, ephemeral ports to be added as candidate.

In turn, that will cause protocols like AutoNAT to fail as these candidates are not actually
reachable/external.

We will now only yield the original candidate if translation did not apply.

Fixes libp2p#4153
b-zee added a commit to b-zee/rust-libp2p that referenced this issue Jul 6, 2023
The `NewExternalAddrCandidate` event is yielded both before and after address translation. This will
cause, in the case of TCP, ephemeral ports to be added as candidate.

In turn, that will cause protocols like AutoNAT to fail as these candidates are not actually
reachable/external.

We will now only yield the original candidate if translation did not apply.

Fixes libp2p#4153
@mergify mergify bot closed this as completed in #4158 Jul 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant