Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How do I automatically retry when there is a connection error? #13

Open
Container-Zero opened this issue Oct 31, 2023 · 22 comments
Open
Labels
feature request Feature request for future version question Further information is requested

Comments

@Container-Zero
Copy link

Describe your issue
How to observe the link status after creating the first new WebSocket\Client. Automatically try to reconnect after a delay of a few seconds when there is a link error, and stop the connection when a certain threshold is exceeded? (Similar to curl's CURLOPT_RETRY)

@sirn-se
Copy link
Owner

sirn-se commented Oct 31, 2023

The client will automatically (re)connect when used to send or receive messages, or when connect() method is explicitly called. So it would be fairly easy to implement a retry strategy when setting up the client interaction.

I will consider an internal retry strategy for future version.

@sirn-se sirn-se added the question Further information is requested label Oct 31, 2023
@Container-Zero
Copy link
Author

The client will automatically (re)connect when used to send or receive messages, or when connect() method is explicitly called. So it would be fairly easy to implement a retry strategy when setting up the client interaction.

I will consider an internal retry strategy for future version.

Oops, is there a good case for this?

It seems to be all I can do right now:

$count = 0;
do{
    try {
        $client = new WebSocket\Client($url,$header);
        $message = $client->receive();
        $count = 10;
    } catch (Exception $e) {
        $count++;
    }
}
while($count<10);

This doesn't seem very convenient.

@sirn-se
Copy link
Owner

sirn-se commented Oct 31, 2023

An exception thrown by the constructor indicates setup error. This can never be restored by retry attempt, so keep the constructor outside the try/catch block.

On the other hand, exceptions thrown by the send/receive methods might be fully recoverable and should probably not count towards a max limit. The WebSocket\Exception\ClientException indicates failed connection, and could be eligible for retry attempt.

@Container-Zero
Copy link
Author

internal retry strategy

I get it, but that doesn't seem convenient either, and I'm very much looking forward to the internal retry strategy.

@sirn-se sirn-se added the feature request Feature request for future version label Nov 1, 2023
@Container-Zero
Copy link
Author

Container-Zero commented Nov 1, 2023

An exception thrown by the constructor indicates setup error. This can never be restored by retry attempt, so keep the constructor outside the try/catch block.

On the other hand, exceptions thrown by the send/receive methods might be fully recoverable and should probably not count towards a max limit. The WebSocket\Exception\ClientException indicates failed connection, and could be eligible for retry attempt.

For now I'm going to
$client->receive();
Replace it with the use of:

function receive_auto_retry($client){
    $count = 0;
    do{
        try {
            $message = $client->receive();
            return $message;
        } catch (Exception $e) {
            if (!$client->isConnected()) {
                $client->connect();
            }
            $count++;
            $error = $e;
        }
    }
    while($count<10);
    return $error;
}

Is this a better way to use it?

@sirn-se
Copy link
Owner

sirn-se commented Nov 1, 2023

Given above snippet, I'm not sure what issue you're trying to solve. So instead I can give you an explanation what's going on and what errors might occur.

1

$client = new Client($url);

Initializes the client but does not connect to server

  • If provided URI is invalid, a BadUriException will be thrown (not resolvable)

2

$client->connect()

Attempts to connect to server and perform handshake

  • If it can't establish a connection with server, a ClientException will be thrown (possibly resolvable)
  • If handshake fails, a HandshakeException is thrown (possibly resolvable)

3

$client->receive();

Attempts to read message sent by server, will call connect() if not already connected and possibly throw exception accordingly

  • If message is invalid, a BadOpcodeException will be thrown (resolvable)
  • If read attempt times out, a ConnectionTimeoutException will be thrown (resolvable)
  • If connections has closed, a ConnectionClosedException will be thrown (resolvable)
  • If any other connection failure occur, a ConnectionFailureException will be thrown (possibly resolvable)

What to retry

  • The not resolvable will never succeed a retry attempt
  • The resolvable can be retried, but using a counter would do more damage than good
  • The possibly resolvable could be retried, but might never succeed so a counter could be beneficial

@Container-Zero
Copy link
Author

Container-Zero commented Nov 2, 2023

I keep forgetting to add that my production environment is php 7.4, so I can only use version 1.7.0.
Thanks for the explanation, but I would like to follow up:

  1. why might counters present drawbacks? What exactly are the drawbacks?
  2. does not using $client->receive(); before using $client->connect() make the probability of reporting an error increase? If it does, why does this happen?
  3. My wss server is capable of doing all interactions in one session, if I use the 'persistent'=>true configuration item, can I enable persistent connections to avoid doing a handshake connection to the server every time $client->receive(); and instead use a previously existing link as a way of enhancing the stability of the link?
  4. Since I'm on version 1.7.0, does this mean I won't get future internal retry updates? I'll just have to fulfill those requirements myself?

@sirn-se
Copy link
Owner

sirn-se commented Nov 2, 2023

Reading your 3) I think there is a misunderstanding how websockets work. Unlike HTTP, which always perform a connect/send/receive/disconnect series of operation (that could easily be retried on failure), a websocket connection will be kept open until the client or server explicitly close it. Once connected, you may perform any number of send and receive operations independently.

  1. Some failures are non-disruptive. For instance, the receive() operation may timeout. This mean a message could not be read, but the connection would still be available for additional send/receive operations. Breaking your application after 10 timeout errors is unnecessary.
  2. receive() will internally call connect() if not already connected.
  3. The persistent setting means that your OS will attempt to keep the connection even if your application closes. But as mentioned above, even without it the connection will remain open until explicitly closed. Don't use this option unless you have good reason to do so.
  4. The 1.7 will only have maintenance patches from now on.

@Container-Zero
Copy link
Author

Container-Zero commented Nov 2, 2023

Reading your 3) I think there is a misunderstanding how websockets work. Unlike HTTP, which always perform a connect/send/receive/disconnect series of operation (that could easily be retried on failure), a websocket connection will be kept open until the client or server explicitly close it. Once connected, you may perform any number of send and receive operations independently.

  1. Some failures are non-disruptive. For instance, the receive() operation may timeout. This mean a message could not be read, but the connection would still be available for additional send/receive operations. Breaking your application after 10 timeout errors is unnecessary.
  2. receive() will internally call connect() if not already connected.
  3. The persistent setting means that your OS will attempt to keep the connection even if your application closes. But as mentioned above, even without it the connection will remain open until explicitly closed. Don't use this option unless you have good reason to do so.
  4. The 1.7 will only have maintenance patches from now on.

I understand it in general, thanks for the answer.

@UksusoFF
Copy link
Contributor

In my case:
Create CLI command with Laravel.
It's keep long live connection till not failed.
Run command with supervisor.
Configure for autorestart when it fail.

Read more: https://beyondco.de/docs/laravel-websockets/basic-usage/starting#keeping-the-socket-server-running-with-supervisord

@indigoram89
Copy link

Thank you for great package! We are waiting for automatic reconnection functionality.

@indigoram89
Copy link

indigoram89 commented Mar 11, 2024

I did it like this way:

Screenshot 2024-03-11 at 13 47 55

And reconnection:

Screenshot 2024-03-11 at 13 48 44

@sirn-se
Copy link
Owner

sirn-se commented Mar 11, 2024

Hi @indigoram89

This library will automatically connect/reconnect when sending, receiving or starting the listener.

However, subscribing to a websocket server typically means that the client send some initial messages for identification and/or configuration. So we can't provide a generic restart of a listening session - a client application need to handle the communication logic for the current service itself.

So if above code works for your application, you should stick with it. Some notes though;

  • Instead of catching Handshake and ConnectionFailure, you could catch ConnectionLevelInterface. Both mentioned, and other connection-level errors, implement that interface.
  • You probably want to put the client->start part in a while(true) or it will only attempt to reconnect once

@indigoram89
Copy link

Thank you so much for you reply!

About reconnection - I imagined a method like:

$client->onReconnected(function (Client $client) {
    // some logic here...
});
  1. Thank you for ConnectionLevelInterface, I will use it.
  2. My client->start logic is placed in startListening method which calls in reconnect method, so It is ok as I guess.

@sirn-se
Copy link
Owner

sirn-se commented Mar 11, 2024

When building subscriber applications, I typically place those initial calls in the onConnect() listener.
This means they will always be called when my application connects or reconnects.

$client
    ->onConnect(function ($client, $connection) {
        $client->text($setup);
    })
    ->onText(function ($client, $connection, $message) {
        // Act on incoming message
    })
   ->onError(function ($client, $connection, $exception) {
        // Evaluate error
        $client->start(); // Restart listening session if stopped, which will reconnect if disconnected
    })
    ->start();

@indigoram89
Copy link

Yes, but I told about automatic reconnection functionality in this package.

@indigoram89
Copy link

indigoram89 commented May 28, 2024

@sirn-se Hello! Could you tell me how to stop process rightly? I do the following in Laravel:

$this->trap([SIGINT, SIGTERM, SIGQUIT], function (int $signal) use ($client) {
    $client->stop();
    $client->close();
    $client->disconnect();
});

$client->start();

And I get Phrity\Net\StreamException (Failed to select streams for reading) after I send SIGTERM to this process.

@sirn-se
Copy link
Owner

sirn-se commented May 29, 2024

@indigoram89

That depends on what you're attempting to do

  • stop - Will stop listening to server, but keep connection open. Messages can still be sent, and listening can be resumed at any point.
  • close - Will tell server that you're want to close connection, when server repsond client will disconnect. This is the orderly way to close a websocket.
  • disconnect - Will immediately disconnect without informing the server.

So you should only call one of these methods at any given point.

@indigoram89
Copy link

@sirn-se Thank you for your answer! It is useful information, but I try to understand why I get error (Failed to select streams for reading) after I call any of these methods?

@sirn-se
Copy link
Owner

sirn-se commented May 31, 2024

This error occurs when listening to a stream (as per start() method) but the stream is closed or non-readable.

So it appears you have a race condition in your code. Can not tell why based on your code snippet, I suggest you use the log function to find out exactly what's going on when you run your code.

@e-sau
Copy link

e-sau commented Jun 18, 2024

@sirn-se Hello!

How can I prevent send message from server for current or some connection?
Current Server::send method broadcast messages to all connections, but I need to exclude some of them.

@sirn-se
Copy link
Owner

sirn-se commented Jun 18, 2024

@e-sau You can call the send() method on current Connection instead of the Server instance. Exactly how to do so depends on context.

If you're inside one of the message listener methods you will get current Connection as second argument.

If you need to send to a specific collection of Connections outside the listeners, you need to collect them in your code and send the same Message on each Connection in list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Feature request for future version question Further information is requested
Projects
None yet
Development

No branches or pull requests

5 participants