Consul connect proxy reset connection instead of timeout

[VenkateswaranJ]

I have grafana Loki running as a nomad service which gets connected to clients via consul proxy.

Main Runtime service

service {
    name = "runtime"
    connect {
        sidecar_service {
            proxy {
                upstreams {
                    destination_name = "loki"
                    local_bind_port  = 18092
                }
            }
        }
    }
}

Loki Service

service {
    name = "loki"
    port = 3100
    check {
        name     = "Loki instance health check"
        type     = "http"
        path     = "/ready"
        interval = "10s"
        timeout  = "2s"
        expose   = true
        check_restart {
            limit = 3
            grace = "2m"
        }
    }
    connect {
        sidecar_service {}
        sidecar_task {
            resources {
              cpu    = 80
              memory = 80
            }
        }
    }
}

Everything works correctly, but when I stop the Loki service, I receive a 'connection reset' error from the Consul proxy, even after reducing the client connection timeout to 1 second (default connection timeout for the Consul proxy is 5 seconds). In this case, I expect an HTTP connection timeout instead of an immediate connection reset by the proxy.

# wget --timeout=1 http://127.0.0.1:18092/ready
Connecting to 127.0.0.1:18092 (127.0.0.1:18092)
wget: error getting response: Connection reset by peer 

As you can see above, the Consul proxy reset the connection even though the client ("wget") has 1-second timeout.

Could someone please help with this? Am I missing some configuration in the nomad service spec?