You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Consul 1.6.2 introduced a change to the way JSON is decoded with PR #6680. As of that release, and current to 1.8.3, Consul fails to properly decode a catalog registration request which contains the field "Connect": null (Service.Connect), and abruptly closes the TCP connection instead of responding with an error.
This field is sent by consul-k8s's catalog sync in version 0.7.0 and earlier.
Reproduction Steps
Steps to reproduce this issue:
Create a JSON file (e.g., payload.json) with the following contents.
Attempt to register the service with /v1/catalog/register using curl.
$ curl --verbose \
--insecure \
--http1.1 \
--request PUT \
--header "Content-Type: application/json" \
--data @payload.json \
$CONSUL_HTTP_ADDR/v1/catalog/register
* Trying 10.0.0.201...
* TCP_NODELAY set
* Connected to 10.0.0.201 (10.0.0.201) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-ECDSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: CN=server.dc2.consul
* start date: Jun 18 22:28:05 2020 GMT
* expire date: Jun 18 22:28:05 2022 GMT
* issuer: C=US; ST=CA; L=San Francisco; street=101 Second Street; postalCode=94105; O=HashiCorp Inc.; CN=Consul Agent CA 276763587613377718708777645654418674711
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> PUT /v1/catalog/register HTTP/1.1
> Host: 10.0.0.201
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 711
>
* upload completely sent off: 711 out of 711 bytes
* TLSv1.2 (IN), TLS alert, close notify (256):
* Empty reply from server
* Connection #0 to host 10.0.0.201 left intact
curl: (52) Empty reply from server
* Closing connection 0
The server abruptly closes the connection without returning an HTTP response. There is no error logged when running Consul with a --log-level of DEBUG or TRACE. However, this error can be seen when running Consul under a debugger (stacktrace from Consul 1.6.2).
Failed to continue - runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation]
Unable to propogate EXC_BAD_ACCESS signal to target process and panic (see https://github.com/go-delve/delve/issues/852)
Last known immediate stacktrace (goroutine id 511):
/Users/blake/Documents/p/hashicorp/consul/agent/structs/structs.go:901
github.com/hashicorp/consul/agent/structs.(*ServiceConnect).UnmarshalJSON
/usr/local/Cellar/go/1.14.6/libexec/src/encoding/json/decode.go:860
encoding/json.(*decodeState).literalStore
/usr/local/Cellar/go/1.14.6/libexec/src/encoding/json/decode.go:384
encoding/json.(*decodeState).value
/usr/local/Cellar/go/1.14.6/libexec/src/encoding/json/decode.go:765
encoding/json.(*decodeState).object
/usr/local/Cellar/go/1.14.6/libexec/src/encoding/json/decode.go:370
encoding/json.(*decodeState).value
/usr/local/Cellar/go/1.14.6/libexec/src/encoding/json/decode.go:765
encoding/json.(*decodeState).object
/usr/local/Cellar/go/1.14.6/libexec/src/encoding/json/decode.go:370
encoding/json.(*decodeState).value
/usr/local/Cellar/go/1.14.6/libexec/src/encoding/json/decode.go:180
encoding/json.(*decodeState).unmarshal
/usr/local/Cellar/go/1.14.6/libexec/src/encoding/json/stream.go:73
encoding/json.(*Decoder).Decode
/Users/blake/Documents/p/hashicorp/consul/agent/http.go:580
github.com/hashicorp/consul/agent.decodeBody
/Users/blake/Documents/p/hashicorp/consul/agent/catalog_endpoint.go:18
github.com/hashicorp/consul/agent.(*HTTPServer).CatalogRegister
/Users/blake/Documents/p/hashicorp/consul/agent/http.go:298
github.com/hashicorp/consul/agent.(*HTTPServer).handler.func3
/Users/blake/Documents/p/hashicorp/consul/agent/http.go:490
github.com/hashicorp/consul/agent.(*HTTPServer).wrap.func1
#8537 will fix the panic, but that request would still fail with a 400 because ProxyDestination was removed in 65be587 and a704ebe made unknown fields fail validation.
If we backport this fix to 1.6.x, that request should work.
Overview of the Issue
Consul 1.6.2 introduced a change to the way JSON is decoded with PR #6680. As of that release, and current to 1.8.3, Consul fails to properly decode a catalog registration request which contains the field
"Connect": null
(Service.Connect), and abruptly closes the TCP connection instead of responding with an error.This field is sent by consul-k8s's catalog sync in version 0.7.0 and earlier.
Reproduction Steps
Steps to reproduce this issue:
Create a JSON file (e.g., payload.json) with the following contents.
Attempt to register the service with
/v1/catalog/register
usingcurl
.The server abruptly closes the connection without returning an HTTP response. There is no error logged when running Consul with a
--log-level
of DEBUG or TRACE. However, this error can be seen when running Consul under a debugger (stacktrace from Consul 1.6.2).Consul info the Server
Server info
The expectation is that Consul should return a proper HTTP error response to the requestor instead of closing the connection.
The text was updated successfully, but these errors were encountered: