"Connection lost" between nginx and HyperFastCGI

[derFunk]

We encountered this issue already twice:

• Request sent to nginx
• Nginx should forward this request to HFC.
• The request is not reaching Mono (no log output on the mono side), and nginx writes it's infamous HTTP ERROR 499 entry into it's log file: "POST /api/0.0.1/list.json? HTTP/1.1" 499 0 "-" "BestHTTP". This 499 entry is nginx-specific and only kept in the logs, not sent out to the client. The clients request times out.
• The HFC log file doesn't have any output on that. We're calling it this way: usr/bin/mono /usr/lib/hyperfastcgi/4.0/HyperFastCgi.exe /config=/hyperfastcgi.conf /applications=/:/usr/aspnet/ /verbose /loglevels=All /logfile=/var/log/nginx/fastcgi.log.

We're using the most current revision of HFC and Mono 4.2.1.

This is our HFC config:

<configuration>
        <server type="HyperFastCgi.ApplicationServers.SimpleApplicationServer">
                <host-factory>HyperFastCgi.HostFactories.SystemWebHostFactory</host-factory>
                <threads min-worker="40" max-worker="0" min-io="4" max-io="0" />
        </server>

        <listener type="HyperFastCgi.Listeners.NativeListener">
                <apphost-transport type="HyperFastCgi.Transports.NativeTransport">
                        <multithreading>ThreadPool</multithreading>
                </apphost-transport>
            <protocol>InterNetwork</protocol>
            <address>127.0.0.1</address>
            <port>9000</port>
        </listener>

    <apphost type="HyperFastCgi.AppHosts.AspNet.AspNetApplicationHost">
                <log level="Debug" write-to-console="true" />
                <add-trailing-slash>false</add-trailing-slash>
    </apphost>

</configuration>

That's all we got from the HFC log:

[2016-01-20 16:29:34Z] Debug   HyperFastCgi
[2016-01-20 16:29:35Z] Debug   Threadpool minw=40,minio=4,maxw=100,maxio=100
[2016-01-20 16:29:35Z] Debug   Root directory: /
[2016-01-20 16:29:36Z] Debug   Listening on port: 9000
[2016-01-20 16:29:36Z] Debug   Listening on address: 127.0.0.1
[2016-01-21 10:24:51Z] Debug   HyperFastCgi
[2016-01-21 10:24:51Z] Debug   Threadpool minw=40,minio=4,maxw=100,maxio=100
[2016-01-21 10:24:51Z] Debug   Root directory: /
[2016-01-21 10:24:52Z] Debug   Listening on port: 9000
[2016-01-21 10:24:52Z] Debug   Listening on address: 127.0.0.1

After restarting HFC every request got served correctly and fast again.
On a sidenote - I had to kill -9 HFC, because a "normal kill" wouldn't work.

The question is - What can we do to investigate this issue further, other than debugging the code in an UI? (because we don't know what the root cause of this behaviour is to reproduce it). Are there any system information or logs which can help telling why HFC obviously stops responding?

[xplicit]

It looks like the client closes connection before start to receive data. I'll look on this scenario at weekend, can this produce unexpected behaviour of HFC.

[xplicit]

Should be fixed via aeadc0e