No reconnection attempts after connection timeout

[haraldpdl]

AMQProxy 0.8.8

If the connection cuts out, no reconnection attempts are performed. Systemd shows the service as "active (running)" with the following being logged in the journal:

Error reading from upstream: Error reading socket: Connection timed out (IO::Error)
from usr/share/crystal/src/io/evented.cr:61:9 in 'unbuffered_read'
from usr/share/crystal/src/io/buffered.cr:261:5 in 'fill_buffer'
from usr/share/crystal/src/io/buffered.cr:83:9 in 'read'
from usr/share/crystal/src/io.cr:540:20 in 'read_fully?'
from usr/share/crystal/src/io.cr:523:5 in 'read_fully'
from tmp/amqproxy/lib/amq-protocol/src/amq/protocol/frames.cr:26:9 in 'read_loop'
from tmp/amqproxy/src/amqproxy/upstream.cr:34:7 in '->'
from usr/share/crystal/src/fiber.cr:146:11 in 'run'
from usr/share/crystal/src/fiber.cr:98:34 in '->'
from ???

It would be great if reconnection attempts can be performed or if the service could abort so Systemd can retry to start the service again.

Thanks in advance!

[haraldpdl]

Extra information:

OS: Debian 11

Package installed Systemd Service File: /lib/systemd/system/amqproxy.service

[Unit]
Description=AMQProxy server for connection and channel pooling
Documentation=https://github.com/cloudamqp/amqproxy
Requires=network.target
After=network.target

[Service]
ExecStart=/usr/bin/amqproxy --config /etc/amqproxy.ini
Restart=on-failure
DynamicUser=yes
LimitNOFILE=infinity

[Install]
WantedBy=multi-user.target

Changing Restart to the following:

[Service]
Restart=always

does not unfortunately fix the issue.

[TristanPouliquen]

Hi,

We are also facing this issue with the newest v0.8.12.

This makes our PHP processes wait indefinitely on a get call to a connection that does not exist anymore => our message processing stops working, and our processes continue running indefinitely.

OS : Ubuntu 20.04
AMQProxy: 0.8.12
PHP: 8.0

[dentarg]

amqproxy will connect to the broker if you connect to it again, are you saying that is not the case? I think you need to check your clients

[TristanPouliquen]

If the connection is closed by the server, the socket stays open so our PHP processes do not see that the connection is not open anymore, and will indefinitely wait on a response from the AMQP server : php-amqp/php-amqp#35

[dentarg]

Yeah I have been able to reproduce this – I hope to have a PR addressing it out this week