https.get, TLS, *node*, *async*, and perhaps others and EPROTO type 40 error
#1098
Comments
|
Alert 40 is a generic "handshake failure" status message that the remote server sends. Actually, I can't reproduce the EPROTO error you describe. I can connect just fine with different versions of Node.js, the error is that the server certificate is not valid for confluence.bredex.de but for afdr.bredex.de (and there is no Subject Alternative Name record.) I'm going to guess you (or request) send some TLS options that the remote server doesn't like. |
|
Appreciate your testing.
Really? This is what I get, varied in all browsers, for example:
Until yesterday I didn't even know one could specify We send nothing additional on the request just a simple If the error was "trapped" then I would just tell the Author to use a different server but it trips node and restarts our server. Here's our log details for the period where the Author was trying it:Error: write EPROTO 140064048379712:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:../deps/openssl/openssl/ssl/s3_pkt.c:1500:SSL alert number 40 140064048379712:error:1409E0E5:SSL routines:ssl3_write_bytes:ssl handshake failure:../deps/openssl/openssl/ssl/s3_pkt.c:659: 2018-02-08 16:05:45.003 +00:00: Starting application... 2018-02-08 16:08:18.822 +00:00: Starting application... 2018-02-08 16:08:19.754 +00:00: Starting application... 2018-02-08 16:08:35.883 +00:00: Starting application... 2018-02-08 16:08:36.645 +00:00: Starting application... 2018-02-08 16:08:45.936 +00:00: Starting application... 2018-02-08 21:28:10.172 +00:00: Starting application... 2018-02-08 21:28:34.267 +00:00: Starting application... 2018-02-08 21:28:34.989 +00:00: Starting application... 2018-02-08 21:30:23.862 +00:00: Starting application... 2018-02-08 21:32:03.960 +00:00: Starting application... 2018-02-08 21:34:43.961 +00:00: Starting application... 2018-02-08 21:34:45.321 +00:00: Starting application... 2018-02-08 21:35:39.256 +00:00: Starting application... 2018-02-08 21:35:40.180 +00:00: Starting application... 2018-02-08 21:36:12.036 +00:00: Starting application... 2018-02-08 21:36:12.972 +00:00: Starting application... 2018-02-08 21:36:46.831 +00:00: Starting application... 2018-02-08 21:36:47.815 +00:00: Starting application... 2018-02-08 22:37:59.484 +00:00: Starting application... 2018-02-09 03:07:20.878 +00:00: Starting application... 2018-02-09 03:07:21.272 +00:00: Starting application... 2018-02-09 03:09:14.436 +00:00: Starting application... ... this drop down from stderr log write shows that it tripped our server many times (see date/time with Starting application...) and thus prevented others from being able to use the site while we were starting up. This is why I suspended the routine. So I'm not entirely sure why node of various flavors (versions and distros) is doing this. |
|
I tried it a few more times and I know now what the problem is: the server doesn't deal with SNI (Server Name Indication). It's using Let's Encrypt and they had an issue with SNI recently. The https module sets it automatically (it's the Its certificate is invalid and if you disable certificate authentication, it tells me this and only this - no status line, no HTTP headers; a badly malformed response: If the server is not under your control, then I suggest you either email them (because yes, I think it's a bug) or just move on. |
I concur (as of yesterday) however not being able to trap the error in node is also a bug. Which means that node can't handle the logic behind what we are trying to do since the server is clearly "borked". I just tried this stackoverflow answer to see if it would at least get past this node fatal error and no luck. I'm leery about trying the answer before that which sounds incredibly insecure.
That's why the routine is suspended but if we can't find a solution to trap this error it means that node isn't viable for web development... so this advice I'll ignore for the time being because I don't accept that conclusion. :) |
Works for me, the 'error' event fires as expected. If it doesn't for you, can you open a bug report with a minimal test case?
Both answers are. Disabling authentication misses the point of HTTPS. Node is right to squeal about the certificate being invalid. |
While I appreciate your insight I'd like to see your minimal test case because ours doesn't get any smaller even if I just remove the I have opened many bugs over the decades but I'm looking for help here. :) |
|
It's a one-liner:
Don't use that tone with me, will you? I'm helping you out in my spare time, you don't get to tell me what I can and cannot do. |
Not using a tone.... your paradigm of perception seems to be skewed.
Aren't we all doing this?
Hmmm... so telling me to open a bug issue isn't? Aside from your temper, which I wish you would refrain from doing that, the WFM isn't accepted without what you tried. What it boils down to is you claim it does and two servers (production and development) say it doesn't. I backed up my claims with this help issue and I would love to see what you did. That's where help comes in. I can say node has no bugs with no justification and that would be prevarication of a large magnitude. :) |
|
I just tested it using native 'use strict';
var fs = require('fs');
var https = require('https');
setTimeout(main);
function main(){
https.get({
host: 'confluence.bredex.de',
port: 443,
path: '/',
method: 'GET',
agent: false,
}, res => {
var fsStream = fs.createWriteStream('index.htm');
res.pipe(fsStream);
}).on('error', err => {
console.log(err);
});
}And the output I got is As you can see, it is not a fatal error, just ordinary error catched with |
|
I'll test that momentarily but in the meantime do you think having it wrapped in async package could be causing the |
|
As I just told our establishing owner these are the code points where we do exactly what has been mentioned: The simple request at https://github.com/OpenUserJS/OpenUserJS.org/pull/1324/files#diff-b755ddbc6b2edf964c5586d742e3afdaL1459 |
|
I think we've established that the issue is not with |
|
Your code traps it. As does a modified version of @bnoordhuis node -e 'https.get("https://confluence.bredex.de/images/icons/profilepics/default.png").on("error", function (err) { console.log("TRAPPED", err); })'So I'm still at a loss as to why it's not trapped even though it's in there at https://github.com/OpenUserJS/OpenUserJS.org/pull/1324/files#diff-b755ddbc6b2edf964c5586d742e3afdaL1488 |
|
@bnoordhuis |
|
You also need to give me time to test things out please. |
Martii
changed the title
https.get and EPROTO type 40 errorhttps.get, TLS, *node*, *async*, and perhaps others and EPROTO type 40 error
|
@Martii I think the problem with your code is that you're attaching error listener to the response instead to the request. This should work: fn.get('https://confluence.bredex.de/images/icons/profilepics/default.png', function (aRes) {
var chunks = [];
aRes.on('data', function (aChunk) {
var buf = null;
chunks.push(aChunk);
buf = Buffer.concat(chunks);
if (buf.length > 3048) {
aRes.destroy();
}
}).on('end', function () {
buffer = Buffer.concat(chunks);
try {
dimensions = sizeOf(buffer);
} catch (aE) {
aInnerCallback(new statusError({
message: '`@icon` ' + aE.message,
code: aE.code
}));
return;
}
if (!acceptedImage(dimensions)) {
aInnerCallback(new statusError({
message: '`@icon` unsupported file type or dimensions are too large.',
code: 400
}), null);
} else {
aInnerCallback(null);
}
});
}).on('error', function (aErr) {
aInnerCallback(aErr);
});Can you see the difference? |
|
Eek... alright will give that a try and report back. Thank you for your patience. |
|
@6-8-axnw1bom81v5xa3nh48c |
|
Even with that correction (inclusive of trapping of any potential response/data/end stream errors) that you were very helpful in figuring out: fn = /^http:/.test(icon) ? http : https;
fn.get(URL.parse(icon), function (aRes) {
var chunks = [];
aRes.on('data', function (aChunk) {
var buf = null;
chunks.push(aChunk);
buf = Buffer.concat(chunks);
if (buf.length > 3048) { // NOTE: KiB
aRes.destroy();
}
}).on('end', function () {
buffer = Buffer.concat(chunks);
try {
dimensions = sizeOf(buffer);
} catch (aE) {
aInnerCallback(new statusError({
message: '`@icon` ' + aE.message,
code: aE.code
}));
return;
}
if (!acceptedImage(dimensions)) {
aInnerCallback(new statusError({
message: '`@icon` unsupported file type or dimensions are too large.',
code: 400
}), null);
} else {
aInnerCallback(null);
}
}).on('error', function (aErr) { // Trap on response... also tested removed
aInnerCallback(aErr);
});
}).on('error', function (aErr) { // Trap on request
aInnerCallback(aErr);
});I get a different error now that trips the development server: events.js:137
throw er; // Unhandled 'error' event
^
RangeError [ERR_HTTP_INVALID_STATUS_CODE]: Invalid status code: EPROTO
at ServerResponse.writeHead (_http_server.js:197:11)
at ServerResponse.writeHead (/home/user/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/on-headers/index.js:55:19)
at ServerResponse.writeHead (/home/user/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/on-headers/index.js:55:19)
at ServerResponse.writeHead (/home/user/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/on-headers/index.js:55:19)
at ServerResponse.writeHead (/home/user/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/on-headers/index.js:55:19)
at ServerResponse._implicitHeader (_http_server.js:188:8)
at ServerResponse.res.write (/home/user/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/express-minify/index.js:88:14)
at MuStream.ondata (internal/streams/legacy.js:16:26)
at MuStream.emit (events.js:160:13)
at next (/home/user/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/mu2/lib/mu/renderer.js:49:16)
at _render (/home/user/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/mu2/lib/mu/renderer.js:96:3)
at Object.render (/home/user/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/mu2/lib/mu/renderer.js:15:10)
at Immediate.<anonymous> (/home/user/repo/git/OpenUserJS.org/martii/OpenUserJS.org/node_modules/mu2/lib/mu.js:196:16)
at runCallback (timers.js:756:18)
at tryOnImmediate (timers.js:717:5)
at processImmediate [as _immediateCallback] (timers.js:697:5) |
Given that this issue is about getting help for an issue and not reporting bugs to node.js, closing it because it's not a bug for node.js while the original poster is still seeking help and their issue is unresolved does not seem appropriate. @bnoordhuis ... while I appreciate you taking the time to respond to this issue, I'd have to agree with @Martii that your conversational approach on this has not been very helpful. |
|
So it looks like I've finally found the cause after a really long period. We have a generic "parking" like page for http statusCodes that looks like this: aRes.status(aOptions.statusCode).render('pages/statusCodePage', aOptions);When I examine The solution I think is to test for string'ness and pick a generic numeric say TESTED with the url in the report and a bogus domain name and sure enough a string test solves it.... this code is buried way deep in the MVC pattern is where the error was. I want to rethank @6-8-axnw1bom81v5xa3nh48c and all of those who gave up their time to respond. What is still confusing is that the stack trace had no idea what line number and what jsm this error in our code and which dependency was. The actual error is way over here in a library module that we utilize that is probably part of the express package (module) since it's how we generate a graceful failure landing page. So to finalize the issue here with a question... why didn't the stack traces include this code point or even one of our dependencies? ❓ EDIT: I'll deal with that question later. Thanks everybody. |
install, code, runtime,meta, other?):https.getand produces similar output ):Having some difficulty with an external server and
https.get. The url is https://confluence.bredex.de/images/icons/profilepics/default.png and we get:when we just try to get it with a similar, non-ES6+, version found at https://nodejs.org/api/https.html#https_https_get_options_callback
It does work with other urls say from https://s3.amazonaws.com/uso_ss/icon/13701/large.png but that server data seems to be retrievable with
https.get.Refs:
I am hoping that someone will be kind enough to point me to the correct solution as I'm burnt out after searching/testing for ~7 hours.
I am a little confused as to why
.on('error'...doesn't catch this. I don't profess to be an expert in TLS but I am at a loss at the moment.Thanks for a moment of someones time.
The text was updated successfully, but these errors were encountered: