rpc: client disable compression

[unclezoro]

By default, the http go client support Gzip-compressed Http response.
The cli and light binary does not disable this support via the DisableCompression
property of the HTTPTransport object. As soon as it send a Http Request to a user-controlled
server, it become possible to cause a Denial of Service via a GZIP bomb.

Steps To Reproduce:
Code:

<?php
header("Content-Encoding: gzip");
echo file_get_contents("hello.gz")
?>

file： hello.gz
the file is 9MB in size but will expand to 9GB as soon as it decompressed.

start lightd or cli with --node to this server, the process will crash.

• Updated all relevant documentation in docs
• Updated all code comments where relevant
• Wrote tests
• Updated CHANGELOG_PENDING.md

[melekes]

Thank you for discovering this 👍 It's a bit hard to believe Go's http.Transport is using an "unsafe" option by default.

// Reference https://github.com/golang/go/blob/master/src/net/http/transport.go
//
// DisableCompression, if true, prevents the Transport from
// requesting compression with an "Accept-Encoding: gzip"
// request header when the Request contains no existing
// Accept-Encoding value. If the Transport requests gzip on
// its own and gets a gzipped response, it's transparently
// decoded in the Response.Body. However, if the user
// explicitly requested gzip it is not automatically
// uncompressed.
DisableCompression bool

If true and you add a header Accept-Encoding: gzip manually in the request, then you have to handle Gzip response body by yourself.
If false and server responds with gzip file, client will attempt to decode by itself.

Note while our RPC client is vulnerable, Tendermint itself is not (because our server does not respond with gzip files / any files).

[melekes]

Also, could you please format the code with gofmt -s -w?

[unclezoro]

Also, could you please format the code with gofmt -s -w?

Thks for your suggestion, updated. please review again. @melekes

[melekes]

🍰

[melekes]

Can you also add

BUGFIXES:
- [rpc/lib/http_client] DisableCompression set to true to prevent DoS attacks using gzip bombs

to the CHANGELOG_PENDING.md?

[unclezoro]

Can you also add

BUGFIXES:
- [rpc/lib/http_client] DisableCompression set to true to prevent DoS attacks using gzip bombs

to the CHANGELOG_PENDING.md?

sure

[codecov-io]

Codecov Report

Merging #3430 into develop will decrease coverage by 0.01%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop    #3430      +/-   ##
===========================================
- Coverage    64.31%   64.29%   -0.02%     
===========================================
  Files          214      214              
  Lines        17470    17494      +24     
===========================================
+ Hits         11235    11248      +13     
- Misses        5303     5309       +6     
- Partials       932      937       +5

Impacted Files	Coverage Δ
libs/cli/helper.go	90.9% <0%> (-4.55%)	⬇️
consensus/reactor.go	71.19% <0%> (-0.6%)	⬇️
p2p/conn/connection.go	80.58% <0%> (-0.27%)	⬇️
consensus/wal.go	66.45% <0%> (+2.16%)	⬆️
libs/events/events.go	98.05% <0%> (+4.85%)	⬆️
privval/signer_service_endpoint.go	89.09% <0%> (+5.45%)	⬆️