Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

highWaterMark option is ignored in HTTPs request stream #32781

Closed
thunderol opened this issue Apr 11, 2020 · 5 comments
Closed

highWaterMark option is ignored in HTTPs request stream #32781

thunderol opened this issue Apr 11, 2020 · 5 comments

Comments

@thunderol
Copy link

Hi,

https request stream ignores the custom highWaterMark size option. In http request it works perfectly.

Version: All of them, even nightly
Platform: Windows 10 64-bit
Platform: https

What steps will reproduce the bug?

Execute the following code snippet using node

var https = require('https');

var url = "https://www.facebook.com";      
var options = { rejectUnauthorized: false, 
                highWaterMark: 128000  };
        
var request = https.request(url, options, function(response)   {
     // it returns 16384 with https but 128000 in http
     console.log("HTTPS readableHighWaterMark: " + response.readableHighWaterMark);  
}).end();

var http = require('http');

var url = "http://www.facebook.com";
var options = {  highWaterMark: 128000  };

var request = http.request(url, options, function(response)   {
     // it returns 16384 with https but 128000 in http
     console.log("HTTP readableHighWaterMark: " + response.readableHighWaterMark);  
}).end();

How often does it reproduce? Is there a required condition?

No. This potential bug can always be reproduced.

What is the expected behavior?

https stream readableHighWaterMark should be 128000 instead 16384.

What do you see instead?

Console Output:

HTTP readableHighWaterMark: 128000
HTTPS readableHighWaterMark: 16384
@rickyes
Copy link
Contributor

rickyes commented Apr 11, 2020

I'm try working on this.

BethGriggs pushed a commit that referenced this issue Apr 27, 2020
PR-URL: #32786
Fixes: #32781
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Andrey Pechkurov <apechkurov@gmail.com>
BridgeAR pushed a commit that referenced this issue Apr 28, 2020
PR-URL: #32786
Fixes: #32781
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Andrey Pechkurov <apechkurov@gmail.com>
BridgeAR pushed a commit that referenced this issue Apr 28, 2020
PR-URL: #32786
Fixes: #32781
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Andrey Pechkurov <apechkurov@gmail.com>
BethGriggs pushed a commit that referenced this issue Apr 28, 2020
PR-URL: #32786
Fixes: #32781
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Andrey Pechkurov <apechkurov@gmail.com>
@thunderol
Copy link
Author

thunderol commented May 5, 2020

There is still something strange...

Testing with Node.js v15.0.0-nightly20200505c17dcb3253 it returns fine now the highWaterMark in https module but it is still downloading with chunks of 16Kb

HTTPS Test
--------------------------------------------------
Response.readableHighWaterMark: 131072
Outfile WriteStream WritableHighWaterMark: 131072

Progress: 0%    16.00 kB / 21.85 MB
Progress: 0%    32.00 kB / 21.85 MB
Progress: 0%    48.00 kB / 21.85 MB
Progress: 0%    64.00 kB / 21.85 MB
Progress: 0%    80.00 kB / 21.85 MB
Progress: 0%    96.00 kB / 21.85 MB
Progress: 0%    112.00 kB / 21.85 MB
...

Testing the same file to download, to the same server in a LAN with http instead https we are receiving in "on data" callback bigger chunks

HTTP Test
--------------------------------------------------
Response.readableHighWaterMark: 131072
Outfile WriteStream WritableHighWaterMark: 131072

Progress: 0%    10.69 kB / 21.85 MB
Progress: 0%    74.69 kB / 21.85 MB
Progress: 0%    138.69 kB / 21.85 MB
Progress: 0%    202.69 kB / 21.85 MB
Progress: 1%    266.69 kB / 21.85 MB
Progress: 1%    330.69 kB / 21.85 MB
Progress: 1%    394.69 kB / 21.85 MB
Progress: 2%    458.69 kB / 21.85 MB
Progress: 2%    522.69 kB / 21.85 MB
Progress: 2%    586.69 kB / 21.85 MB
...

Should I fill a new ticket?

@rickyes
Copy link
Contributor

rickyes commented May 6, 2020

I'm trying to replicate this problem.

@puzpuzpuz
Copy link
Member

@thunderol could you create a separate GH issue? It'll be easier to track the fix this way.

@thunderol
Copy link
Author

thunderol commented May 6, 2020

Created new issue #33262

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants