Elastic cloud Client prevents decompression of gzip compressed responses

[laurenzonneveld]

Summary of problem or feature request

ClientBuilder::build() adds an Accept-Encoding:gzip header when elasticCloudId is provided but does not handle decompression of gzip compressed responses.

We're using Symfony\Component\HttpClient\HttplugClient as asyncHttpClient which adds gzip by default but does not handle decoding when the Accept-Encoding header is explicitly set.

As per Symfony HTTPClient maintainer:

you explicitly send the request with the "accept-encoding" header. When you do this, you disable transparent compression and you opt-in for explicit compression, which means it's now your job to effectively decompress.
Don't send the header if that's not what you want.

symfony/symfony#34238 (comment)

Thus the Elastic Client should either handle decompressen (and check whether the required zlib PHP extension is loaded) or the Elastic Client should leave responsibility for decompression to the Psr\Http\Client\ClientInterface implementation and not set the header at all, we prefer the latter.

Code snippet of problem

if (!empty($this->cloudId)) {
    $transport->setHeader('Accept-Encoding', 'gzip');
}

And in Symfony HTTPClient:

$this->inflate = !isset($options['normalized_headers']['accept-encoding']); // This prevents decompression

System details

• PHP version: 8.1.8
• ES-PHP client version: 8.3.2
• Elasticsearch version: 7.15.2

[ezimuel]

@laurenzonneveld I noticed that you are using Elasticsearch server version 7.15.2 but your PHP client is using elasticsearch-php v8.3.2. You should use elasticsearch-php v7.x, for instance the latest 7.17.0. Can you try with this version?

In version 8 we rewrote the client from stratch and we used elastic-transport-php for the HTTP transport layer. This project uses can use any PSR-18 libraries for sending the HTTP request (Guzzle by default).

Let me know if using elasticsearch-php v7 solve the issue. In the meantime, I'll check using Symfony HTTPClient and Elastic Cloud with v8.

Thanks!

[laurenzonneveld]

@ezimuel Thank you for the fast response. We have no problem using the 7.15.2 version of Elastic PHP. However, we would like to upgrade Elastic and client version 8.x since we want to make use of asynchronous requests.

For now we have patched the $transport->setHeader call out of the library which solves the issues.

We'd love to hear your findings regarding the 8.x client with Symfony HTTPClient.

[ezimuel]

@laurenzonneveld I just completed the PR #1243 to fix this issue and to add the support for Symfony HTTP Client: HttplugClient and Psr18Client. Can you test it and let me know? Thanks!

[laurenzonneveld]

@ezimuel Thanks for the update, I'll be on vacation for the next 2 weeks. We'll test as soon as I get back.

[laurenzonneveld]

Just tested, works as expected. Thank you.