DataStorm/reliability hang (Ubuntu release CI)

[bernardnormier]

From:
https://github.com/zeroc-ice/ice/actions/runs/13063024722/job/36450166492?pr=3458

ok
process /home/runner/work/ice/ice/cpp/test/DataStorm/reliability/build/x86_64-linux-gnu/shared/writer --Ice.Default.Host=127.0.0.1 --Test.BasePort=14200 --Ice.Warn.Connections=1 --Ice.Default.Protocol=tcp --Ice.IPv6=0 --Ice.PrintStackTraces=1 --DataStorm.Node.Multicast.Enabled=0 --DataStorm.Node.Server.Enabled=0 --DataStorm.Node.ConnectTo="tcp -p 14211" --Ice.Connection.Server.IdleTimeout=0 --Ice.Connection.Client.IdleTimeout=0 --DataStorm.Node.Name=writer-app --DataStorm.Trace.Topic=1 --DataStorm.Trace.Session=3 --DataStorm.Trace.Data=2 --Ice.Trace.Protocol=1 --Ice.LogFile=/home/runner/work/ice/ice/cpp/test/DataStorm/reliability/client-013125-0005.log pid=80273 is hanging - 01/31/25 00:06:54
process /home/runner/work/ice/ice/cpp/test/DataStorm/reliability/build/x86_64-linux-gnu/shared/writer --Ice.Default.Host=127.0.0.1 --Test.BasePort=14200 --Ice.Warn.Connections=1 --Ice.Default.Protocol=tcp --Ice.IPv6=0 --Ice.PrintStackTraces=1 --DataStorm.Node.Multicast.Enabled=0 --DataStorm.Node.Server.Enabled=0 --DataStorm.Node.ConnectTo="tcp -p 14211" --Ice.Connection.Server.IdleTimeout=0 --Ice.Connection.Client.IdleTimeout=0 --DataStorm.Node.Name=writer-app --DataStorm.Trace.Topic=1 --DataStorm.Trace.Session=3 --DataStorm.Trace.Data=2 --Ice.Trace.Protocol=1 --Ice.LogFile=/home/runner/work/ice/ice/cpp/test/DataStorm/reliability/client-013125-0005.log pid=80273 is hanging - 01/31/25 00:07:24
process /home/runner/work/ice/ice/cpp/test/DataStorm/reliability/build/x86_64-linux-gnu/shared/writer --Ice.Default.Host=127.0.0.1 --Test.BasePort=14200 --Ice.Warn.Connections=1 --Ice.Default.Protocol=tcp --Ice.IPv6=0 --Ice.PrintStackTraces=1 --DataStorm.Node.Multicast.Enabled=0 --DataStorm.Node.Server.Enabled=0 --DataStorm.Node.ConnectTo="tcp -p 14211" --Ice.Connection.Server.IdleTimeout=0 --Ice.Connection.Client.IdleTimeout=0 --DataStorm.Node.Name=writer-app --DataStorm.Trace.Topic=1 --DataStorm.Trace.Session=3 --DataStorm.Trace.Data=2 --Ice.Trace.Protocol=1 --Ice.LogFile=/home/runner/work/ice/ice/cpp/test/DataStorm/reliability/client-013125-0005.log pid=80273 is hanging - 01/31/25 00:07:54
process /home/runner/work/ice/ice/cpp/test/DataStorm/reliability/build/x86_64-linux-gnu/shared/writer --Ice.Default.Host=127.0.0.1 --Test.BasePort=14200 --Ice.Warn.Connections=1 --Ice.Default.Protocol=tcp --Ice.IPv6=0 --Ice.PrintStackTraces=1 --DataStorm.Node.Multicast.Enabled=0 --DataStorm.Node.Server.Enabled=0 --DataStorm.Node.ConnectTo="tcp -p 14211" --Ice.Connection.Server.IdleTimeout=0 --Ice.Connection.Client.IdleTimeout=0 --DataStorm.Node.Name=writer-app --DataStorm.Trace.Topic=1 --DataStorm.Trace.Session=3 --DataStorm.Trace.Data=2 --Ice.Trace.Protocol=1 --Ice.LogFile=/home/runner/work/ice/ice/cpp/test/DataStorm/reliability/client-013125-0005.log pid=80273 is hanging - 01/31/25 00:08:24
process /home/runner/work/ice/ice/cpp/test/DataStorm/reliability/build/x86_64-linux-gnu/shared/writer --Ice.Default.Host=127.0.0.1 --Test.BasePort=14200 --Ice.Warn.Connections=1 --Ice.Default.Protocol=tcp --Ice.IPv6=0 --Ice.PrintStackTraces=1 --DataStorm.Node.Multicast.Enabled=0 --DataStorm.Node.Server.Enabled=0 --DataStorm.Node.ConnectTo="tcp -p 14211" --Ice.Connection.Server.IdleTimeout=0 --Ice.Connection.Client.IdleTimeout=0 --DataStorm.Node.Name=writer-app --DataStorm.Trace.Topic=1 --DataStorm.Trace.Session=3 --DataStorm.Trace.Data=2 --Ice.Trace.Protocol=1 --Ice.LogFile=/home/runner/work/ice/ice/cpp/test/DataStorm/reliability/client-013125-0005.log pid=80273 is hanging - 01/31/25 00:08:54
process /home/runner/work/ice/ice/cpp/test/DataStorm/reliability/build/x86_64-linux-gnu/shared/writer --Ice.Default.Host=127.0.0.1 --Test.BasePort=14200 --Ice.Warn.Connections=1 --Ice.Default.Protocol=tcp --Ice.IPv6=0 --Ice.PrintStackTraces=1 --DataStorm.Node.Multicast.Enabled=0 --DataStorm.Node.Server.Enabled=0 --DataStorm.Node.ConnectTo="tcp -p 14211" --Ice.Connection.Server.IdleTimeout=0 --Ice.Connection.Client.IdleTimeout=0 --DataStorm.Node.Name=writer-app --DataStorm.Trace.Topic=1 --DataStorm.Trace.Session=3 --DataStorm.Trace.Data=2 --Ice.Trace.Protocol=1 --Ice.LogFile=/home/runner/work/ice/ice/cpp/test/DataStorm/reliability/client-013125-0005.log pid=80273 is hanging - 01/31/25 00:09:24
process /home/runner/work/ice/ice/cpp/test/DataStorm/reliability/build/x86_64-linux-gnu/shared/writer --Ice.Default.Host=127.0.0.1 --Test.BasePort=14200 --Ice.Warn.Connections=1 --Ice.Default.Protocol=tcp --Ice.IPv6=0 --Ice.PrintStackTraces=1 --DataStorm.Node.Multicast.Enabled=0 --DataStorm.Node.Server.Enabled=0 --DataStorm.Node.ConnectTo="tcp -p 14211" --Ice.Connection.Server.IdleTimeout=0 --Ice.Connection.Client.IdleTimeout=0 --DataStorm.Node.Name=writer-app --DataStorm.Trace.Topic=1 --DataStorm.Trace.Session=3 --DataStorm.Trace.Data=2 --Ice.Trace.Protocol=1 --Ice.LogFile=/home/runner/work/ice/ice/cpp/test/DataStorm/reliability/client-013125-0005.log pid=80273 is hanging - 01/31/25 00:09:54
process /home/runner/work/ice/ice/cpp/test/DataStorm/reliability/build/x86_64-linux-gnu/shared/writer --Ice.Default.Host=127.0.0.1 --Test.BasePort=14200 --Ice.Warn.Connections=1 --Ice.Default.Protocol=tcp --Ice.IPv6=0 --Ice.PrintStackTraces=1 --DataStorm.Node.Multicast.Enabled=0 --DataStorm.Node.Server.Enabled=0 --DataStorm.Node.ConnectTo="tcp -p 14211" --Ice.Connection.Server.IdleTimeout=0 --Ice.Connection.Client.IdleTimeout=0 --DataStorm.Node.Name=writer-app --DataStorm.Trace.Topic=1 --DataStorm.Trace.Session=3 --DataStorm.Trace.Data=2 --Ice.Trace.Protocol=1 --Ice.LogFile=/home/runner/work/ice/ice/cpp/test/DataStorm/reliability/client-013125-0005.log pid=80273 is hanging - 01/31/25 00:10:24
process /home/runner/work/ice/ice/cpp/test/DataStorm/reliability/build/x86_64-linux-gnu/shared/writer --Ice.Default.Host=127.0.0.1 --Test.BasePort=14200 --Ice.Warn.Connections=1 --Ice.Default.Protocol=tcp --Ice.IPv6=0 --Ice.PrintStackTraces=1 --DataStorm.Node.Multicast.Enabled=0 --DataStorm.Node.Server.Enabled=0 --DataStorm.Node.ConnectTo="tcp -p 14211" --Ice.Connection.Server.IdleTimeout=0 --Ice.Connection.Client.IdleTimeout=0 --DataStorm.Node.Name=writer-app --DataStorm.Trace.Topic=1 --DataStorm.Trace.Session=3 --DataStorm.Trace.Data=2 --Ice.Trace.Protocol=1 --Ice.LogFile=/home/runner/work/ice/ice/cpp/test/DataStorm/reliability/client-013125-0005.log pid=80273 is hanging - 01/31/25 00:10:54
process /home/runner/work/ice/ice/cpp/test/DataStorm/reliability/build/x86_64-linux-gnu/shared/writer --Ice.Default.Host=127.0.0.1 --Test.BasePort=14200 --Ice.Warn.Connections=1 --Ice.Default.Protocol=tcp --Ice.IPv6=0 --Ice.PrintStackTraces=1 --DataStorm.Node.Multicast.Enabled=0 --DataStorm.Node.Server.Enabled=0 --DataStorm.Node.ConnectTo="tcp -p 14211" --Ice.Connection.Server.IdleTimeout=0 --Ice.Connection.Client.IdleTimeout=0 --DataStorm.Node.Name=writer-app --DataStorm.Trace.Topic=1 --DataStorm.Trace.Session=3 --DataStorm.Trace.Data=2 --Ice.Trace.Protocol=1 --Ice.LogFile=/home/runner/work/ice/ice/cpp/test/DataStorm/reliability/client-013125-0005.log pid=80273 is hanging - 01/31/25 00:11:24
...

[pepone]

From the client log attached to the CI job. The last messages in the log are:

-- 01/31/25 00:05:54.597 /home/runner/work/ice/ice/cpp/test/DataStorm/reliability/build/x86_64-linux-gnu/shared/writer: Session: s/2: session 'pf/1-reader-app' disconnected:
   local address = 127.0.0.1:43304
   remote address = 127.0.0.1:14212
   
-- 01/31/25 00:05:54.597 /home/runner/work/ice/ice/cpp/test/DataStorm/reliability/build/x86_64-linux-gnu/shared/writer: Session: s/2: unsubscribed topic '1' from topic '1:int'
-- 01/31/25 00:05:54.597 /home/runner/work/ice/ice/cpp/test/DataStorm/reliability/build/x86_64-linux-gnu/shared/writer: Session: s/2: trying to reconnect session with 'reader-app -t -e 1.1'

No clear why the reconnect attempt isn't made, wonder if there is a deadlock preventing it.

[pepone]

Another failure in: https://github.com/zeroc-ice/ice/actions/runs/13132682205/job/36641051882?pr=3487