Socket limit reached - server doesn't recover

[dwlodarz]

I have implemented a router and a client in .net just to see how it works in general. Then I have deployed it to S2 AppService in Azure. The problem that I experience is that once the TCP connection limit is reached (~4k) and I close all the clients the router becomes unusable. I cannot connect any new clients. It seems that the connections are not being disposed.

This is the exception I'm getting:
System.AggregateException: One or more errors occurred. (Unable to connect to the remote server) ---> System.Net.WebSockets.WebSocketException: Unable to connect to the remote server ---> System.Net.Http.HttpRequestException: Nie mozna przeprowadzic operacji na gniezdzie, poniewaz w systemie brakuje miejsca na bufor lub poniewaz zostala zapelniona kolejka ---> System.Net.Sockets.SocketException: Nie mozna przeprowadzic operacji na gniezdzie, poniewaz w systemie brakuje miejsca na bufor lub poniewaz zostala zapelniona kolejka
at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)

which means pretty much this:
" An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full"

Could you pls advise?

[darkl]

If you can write a local simple repro (i.e. does not involve Azure), then I might be able to investigate this.
@bigbearzhu do you guys run on Azure? Did you try these kind of things?

Elad

[dwlodarz]

When it comes to router it's nothing more but
Startup.cs->
app.Map("", builder => { builder.UseWebSockets(new WebSocketOptions()); wampHost.RegisterTransport(new DssConnectorWebSocketTransport(builder, log), new JTokenJsonBinding(), new JTokenMsgpackBinding()); }); wampHost.Open();

Program.cs->
WebHost.CreateDefaultBuilder(args) .UseUrls("http://localhost:5000", "http://localhost:80", "https://localhost:443") .UseStartup<Startup>() .UseSetting("detailedErrors", "true") .CaptureStartupErrors(true) .Build().Run();
and
dependencies:

[darkl]

What is DssConnectorWebSocketTransport? Please add a full repro. Preferably, upload it to https://gist.github.com.

Elad

[dwlodarz]

Oh sorry. The class you mentioned is just AspNetCoreWebSocketTransport with some logging added around opening the connection. No additional logic or changes.
I cant give you the full src code unfortuantely. But everuthing relevant is here

[darkl]

Can't reproduce. Here's an attempt. Client eventually crashes with System.Net.Sockets.SocketException (10055): An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full. Afterwards, restarting client (without touching the server), new clients are able to connect.

Elad

[darkl]

In turns out that this is an issue with IIS/IIS Express, see here. See also #279.

[narenkv]

@darkl , is this resolved as we are still facing the same issue.

[darkl]

You'll have to wait several minutes for the server to detect the disconnected clients, see this comment.