HTTP/2 createServer() options to pass custom Request and Response classes #15560

hekike · 2017-09-22T18:10:14Z

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

http2

Motivation

Provide a simple and performant way to extend the original Http2ServerRequest and Http2ServerResponse classes in web frameworks like express or restify without messing with the original Response and Request handlers in Node core or overwriting the prototypes at every request.

Current solutions for inheritance:

express uses Object.setPrototypeOf to overwrite req and res prototypes at every request
restify extends the original Node Core IncomingMessage and ServerResponse Objects

Object.setPrototypeOf:

Warning: Changing the [[Prototype]] of an object is, by the nature of how modern JavaScript engines optimize property accesses, a very slow operation, in every browser and JavaScript engine. The effects on performance of altering inheritance are subtle and far-flung, and are not limited to simply the time spent in obj.proto = ... statement, but may extend to any code that has access to any object whose [[Prototype]] has been altered. If you care about performance you should avoid setting the [[Prototype]] of an object. Instead, create a new object with the desired [[Prototype]] using Object.create(). - MDN

How it works

class MyServerRequest extends http2.Http2ServerRequest {
  getUserAgent() {
    return this.headers['user-agent'] || 'unknown';
  }
}

class MyServerResponse extends http2.Http2ServerResponse {
  status(code, contentType) {
    this.writeHead(code, { 'Content-Type': contentType });
  }
}

const server = http2.createServer({
  ServerRequest: MyServerRequest,
  ServerResponse: MyServerResponse
};

Benchmark results

Continuously increasing concurrent and streams.
See new benchmark in this PR.

./node benchmark/scatter.js --runs 30 --set benchmarker=h2load benchmark/http2/inheritance > scatter.csv

Aggregated by requests:

cat scatter.csv | Rscript benchmark/scatter.R --xaxis requests --category version --plot scatter-plot.png --log

X axis: total number of requests
Y axis: rate of operations

aggregating variable: streams

 requests        version     rate confidence.interval
     5000       original 6157.151            46.30018
     5000         params 6118.722            52.94544
     5000 setPrototypeOf 6127.015            46.04977
    10000       original 7373.277            54.71278
    10000         params 7318.096            55.83396
    10000 setPrototypeOf 7363.882            46.79015
    20000       original 8003.976            53.02454
    20000         params 8237.659            48.35231
    20000 setPrototypeOf 7999.081            50.75252
    50000       original 7870.669            63.91681
    50000         params 8347.653            54.88024
    50000 setPrototypeOf 7838.831            62.77288

Aggregated by streams:

cat scatter.csv | Rscript benchmark/scatter.R --xaxis streams --category version --plot scatter-plot.png --log

X axis: concurrency level
Y axis: rate of operations

aggregating variable: requests

 streams        version     rate confidence.interval
     100       original 7321.768            152.6445
     100         params 7506.819            164.8038
     100 setPrototypeOf 7317.199            143.0783
     500       original 7364.365            149.7678
     500         params 7566.271            174.6796
     500 setPrototypeOf 7385.408            143.0759
    1000       original 7372.903            140.9144
    1000         params 7518.531            172.2680
    1000 setPrototypeOf 7335.217            147.1282
    2500       original 7331.950            145.9298
    2500         params 7502.725            172.1327
    2500 setPrototypeOf 7320.819            157.1094
    5000       original 7385.907            145.3467
    5000         params 7492.245            174.1755
    5000 setPrototypeOf 7302.101            149.7285
   10000       original 7330.715            155.2385
   10000         params 7446.603            190.6992
   10000 setPrototypeOf 7332.468            145.8346

From the result, aggregated by requests number you can see that this new extension type (passing custom classes as options) is significantly faster for higher concurrency and request number.

Related PR

HTTP/1 createServer() options to pass custom Request and Response classes #15752

lpinca · 2017-09-24T06:22:00Z

doc/api/http2.md

@@ -1474,6 +1474,12 @@ added: v8.4.0
 used to determine the padding. See [Using options.selectPadding][].
 * `settings` {[Settings Object][]} The initial settings to send to the
 remote peer upon connection.
+ * `ServerRequest` {Http2ServerRequest} Specifies the ServerRequest class to


Nit: I would use http2.Http2ServerRequest for consistency.

@hekike sorry I wasn't clear. I didn't mean to change the option name but it's type from {Http2ServerRequest} to {http2.Http2ServerRequest} because this is what it is used in the rest of the file.

lpinca · 2017-09-24T06:22:26Z

doc/api/http2.md

+ * `ServerRequest` {Http2ServerRequest} Specifies the ServerRequest class to
+ used. Useful for extending the original `Http2ServerRequest`.
+ Defaults to: `Http2ServerRequest`
+ * `ServerResponse` {Http2ServerResponse} Specifies the ServerRequest class to


Same here, http2.Http2ServerResponse.

lpinca · 2017-09-24T06:30:53Z

lib/internal/http2/core.js

@@ -2337,7 +2339,10 @@ class Http2SecureServer extends TLSServer {
 super(options, connectionListener);
 this[kOptions] = options;
 this.timeout = kDefaultSocketTimeout;
- this.on('newListener', setupCompat);
+ this.on('newListener', (ev) =>
+ setupCompat.call(this, this[kOptions].ServerRequest,


I think it doesn't change anything but did you try using Function.prototype.bind()? It may be a little faster and easier to read.

It doesn't even need to be bound here. setupCompat has access to this[kOptions].

Edit: In fact, to correct myself, onServerStream has access to this[kOptions] so all the .calls can be eliminated.

lpinca · 2017-09-24T06:36:28Z

benchmark/http2/inheritance.js

+ // Original prototype
+ server = http2.createServer((req, res) => {
+ MyServerResponse.prototype.status.call(res, 200, 'text/plain');
+ res.end(`User Agent: ${MyServerRequest.prototype.getUserAgent.call(req)}`


Nit: can you please remove the line feed? The next line seems to fit here.

lpinca · 2017-09-24T06:37:57Z

lib/internal/http2/core.js

@@ -2360,7 +2365,10 @@ class Http2Server extends NETServer {
 super(connectionListener);
 this[kOptions] = initializeOptions(options);
 this.timeout = kDefaultSocketTimeout;
- this.on('newListener', setupCompat);
+ this.on('newListener', (ev) =>
+ setupCompat.call(this, this[kOptions].ServerRequest,


Also doesn't need bind or call.

apapirovski · 2017-09-24T12:27:29Z

lib/internal/http2/core.js

- this.on('stream', onServerStream);
+ this.on('stream', (stream, headers, flags, rawHeaders) => {
+ onServerStream.call(this, ServerRequest, ServerResponse,
+ stream, headers, flags, rawHeaders);


Same as above, you don't need to pass through ServerRequest and ServerResponse. onServerStream has access to this[kOptions] (with small changes to move kOptions to util & import it both in core & compat)

apapirovski · 2017-09-24T12:32:18Z

benchmark/http2/inheritance.js

+ // Original prototype
+ server = http2.createServer((req, res) => {
+ MyServerResponse.prototype.status.call(res, 200, 'text/plain');
+ res.end(`User Agent: ${MyServerRequest.prototype.getUserAgent.call(req)}`


The 'original' is slower because you're having to access a prototype of an object from outside of it's scope and then you're using call on it. This should just be:

this.writeHead(200, { 'Content-Type': contentType }); res.end(`User Agent: ${this.headers['user-agent'] || 'unknown'}`);

To expand on this, I think you would actually not even want to run the original test & the prototype test against your modified version of the code. I think those should be tested against master while the PARAMS test case should run against your build.

apapirovski · 2017-09-24T12:36:26Z

benchmark/http2/inheritance.js

+ // Express style prototype extension for each request
+ server = http2.createServer((req, res) => {
+ Object.setPrototypeOf(req, MyServerRequest.prototype);
+ Object.setPrototypeOf(res, MyServerResponse.prototype);


This closure is harder to optimize than the one for the PARAMS option because it has to access MyServerRequest & MyServerResponse. You would probably want to declare these request handlers outside of main.

Edit: I guess that's not really possible though because of how the h2 benchmarks are created. That makes the comparison a bit tricky, I think.

I tried to do something like Express https://github.com/expressjs/express/blob/a4bd4373b2c3b2521ee4c499cb8e90e98f78bfa5/lib/middleware/init.js#L35 to demonstrate the benefits of the new approach. I'm not sure we should merge the benchmark as well, it's more like a demonstration of this PR.

apapirovski · 2017-09-24T12:39:36Z

Thanks for doing this @hekike! I left some small suggestions for changes but I don't know what opinion @mcollina & @jasnell will have re: whether this is something that gets added or not.

apapirovski · 2017-09-24T13:38:08Z

lib/internal/http2/core.js

- stream, headers, flags, rawHeaders);
+ onServerStream.bind(this)(this[kOptions].Http2ServerRequest,
+ this[kOptions].Http2ServerResponse,
+ stream, headers, flags, rawHeaders);


You can just do this.on('stream', onServerStream.bind(this, this[kOptions].Http2ServerRequest, this[kOptions].Http2ServerResponse)); The rest will work as expected.

jasnell · 2017-09-24T14:08:34Z

Definitely good with this approach in general but will do a review on Monday.

mcollina · 2017-09-25T08:04:28Z

@dougwilson is this something that you could make some use of in Express. Do you need it also for require('http')?

Express would be the main consumer of this feature, and if do not need it I'm 👎 as it increases the API surface area.

hekike · 2017-09-25T08:14:01Z

@mcollina restify could benefit from it as well. The current approach extends the Node Core's IncomingMessage and ServerResponse prototypes. See https://github.com/restify/node-restify/blob/master/lib/request.js#L19 cc @yunong

mcollina · 2017-09-25T09:08:14Z

Maybe it would be more important to have this PR for http1 asap, and then bring http2 up to speed. If framework authors have a use for it, I'm 👍 .

mcollina · 2017-09-25T09:09:12Z

doc/api/http2.md

+ Defaults to: `Http2ServerRequest`
+ * `Http2ServerRequest` {Http2ServerResponse} Specifies the ServerRequest class to
+ used. Useful for extending the original `Http2ServerResponse`.
+ Defaults to: `Http2ServerResponse`


We would need to add the documentation of the constructor parameters. Those are not documented atm, but if we want to support this use case we should do it.

apapirovski · 2017-09-25T12:02:21Z

If this is going to be added to http2, it will need to be allowHTTP1 aware (so accept IncomingMessage & ServerResponse) which means even larger API surface.

apapirovski · 2017-09-25T12:11:42Z

ping @Fishrock123 I saw you in expressjs/express#2812 and you also did https://github.com/pillarjs/extend-proto so figured you might have some thoughts.

hekike · 2017-09-25T12:20:56Z

@apapirovski thanks for checking it. I fixed your suggestions, here is the latest benchmark output after the changes:

By requests:

cat scatter.csv | Rscript benchmark/scatter.R --xaxis requests --category version --plot scatter-plot.png --log

aggregating variable: streams

 requests        version     rate confidence.interval
     5000       original 6017.385           103.08612
     5000         params 6005.038            94.99691
     5000 setPrototypeOf 5899.892           116.80510
    10000       original 7066.274           130.80891
    10000         params 7133.153           118.89924
    10000 setPrototypeOf 7028.677           133.74257
    20000       original 7684.556           144.31313
    20000         params 8041.344           119.24608
    20000 setPrototypeOf 7791.633           111.70869
    50000       original 7754.759           116.79295
    50000         params 8187.066           123.17560
    50000 setPrototypeOf 7827.398           102.14100

X axis: total number of requests
Y axis: rate of operations

By streams:

cat scatter.csv | Rscript benchmark/scatter.R --xaxis streams --category version --plot scatter-plot.png --log

aggregating variable: requests

 streams        version     rate confidence.interval
     100       original 7030.053            183.8673
     100         params 7270.266            232.4791
     100 setPrototypeOf 6951.486            232.5603
     500       original 7154.344            214.5401
     500         params 7466.283            210.6520
     500 setPrototypeOf 7216.214            203.2741
    1000       original 7274.672            187.8303
    1000         params 7414.865            200.5134
    1000 setPrototypeOf 7285.713            183.2550
    2500       original 7206.308            189.6652
    2500         params 7301.195            216.3267
    2500 setPrototypeOf 7090.802            213.5673
    5000       original 7002.380            216.8112
    5000         params 7312.006            192.2240
    5000 setPrototypeOf 7113.685            183.5417
   10000       original 7116.705            193.1916
   10000         params 7285.285            214.3297
   10000 setPrototypeOf 7163.500            184.8569

X axis: concurrency level
Y axis: rate of operations

evanlucas · 2017-09-25T13:11:10Z

I would be +1 on doing this for http as well as we could use it for our internal http framework too

mcollina · 2017-09-25T13:14:42Z

Can we move the discussion on adding this for require('http')? I think we might get the best value there rather than in require('http2'). Then http2 will follow http.

hekike · 2017-09-25T13:18:09Z

@mcollina good idea. The only reason why I started with http2 because I was working on restify's HTTP/2 support.

dougwilson · 2017-09-25T13:20:04Z

@mcollina at least with Express current design, it would not be able to use this. The main things I see are

This can only be set when constructing thr HTTP server instance, which is outside of Express since it sits at a higher level.
The prototype Express sets is app-specific. This means that if I have sub apps, the prototype will change as the request is entering / leaving the sub apps.

mcollina · 2017-09-25T14:33:53Z

@hekike would this means that this functionality is needed only by Restify? I'm trying to understand who needs this functionality.

yunong · 2017-09-25T18:29:29Z

@mcollina This would be great to have for both HTTP/1 and HTTP/2. We are currently running into multiple problems in restify without this. In fact, we’ve been collaborating closely with @hekike on this issue and are in full support of this PR. 🎉

yunong · 2017-09-25T18:31:51Z

@mcollina This is something that is generally useful for most frameworks since it doesn’t require directly modifying the prototype of the request and response objects. Sure this will require the frameworks to pick up this new paradigm, but it is much better than directly modifying the prototype, which is what both restify and express does today.

If this lands restify itself will be moving to using this approach, for both HTTP/1 and HTTP/2. This will also allow us to provide HTTP/2 support in restify quickly, as we have a PR lined up and ready to go that will use this approach.

apapirovski · 2017-09-25T18:35:51Z

@yunong how do you deal with allowHTTP1: true on createSecureServer in that situation? To me that seems to be the missing piece here.

hekike · 2017-09-25T18:53:58Z

@apapirovski I'm not familiar with the HTTP/1 implementation in Node, but if I understand it correctly it's not a technical difficulty "just" we need to ship them together. Right?

apapirovski · 2017-09-25T18:59:09Z

Well, there needs to be a solution for http1 but also there needs to be a solution for passing through http1 req & res prototypes through http2 to http1 when a user creates a server with allowHTTP1: true. See https://nodejs.org/dist/latest-v8.x/docs/api/http2.html#http2_http2_createsecureserver_options_onrequesthandler and

node/lib/internal/http2/core.js

Lines 2258 to 2269 in 4f4d55d

 if (socket.alpnProtocol === false || socket.alpnProtocol === 'http/1.1') { 

 if (options.allowHTTP1 === true) { 

 // Fallback to HTTP/1.1 

 return httpConnectionListener.call(this, socket); 

 } else if (this.emit('unknownProtocol', socket)) { 

 // Let event handler deal with the socket 

 return; 

 } else { 

 // Reject non-HTTP/2 client 

 return socket.destroy(); 

 } 

 }

mcollina · 2017-09-25T22:36:59Z

@hekike can you open up a PR for http? Then we will adjust this accordingly to support both http1 and http2.

hekike · 2017-09-26T07:33:22Z

@mcollina sure, it will take a couple of days.

apapirovski · 2018-02-09T17:44:53Z

doc/api/http2.md

@@ -1714,6 +1714,14 @@ changes:
 * `Http1ServerResponse` {http.ServerResponse} Specifies the ServerResponse
 class to used for HTTP/1 fallback. Useful for extending the original
 `http.ServerResponse`. **Default:** `http.ServerResponse`
+ * `Http2ServerRequest` {http2.Http2ServerRequest} Specifies the
+ Http2ServerRequest class to used.


One final comment as I was going through this: to use. or to be used. Same below.

…teServer

hekike · 2018-02-09T17:50:47Z

@apapirovski thanks, fixed!

Add optional Http2ServerRequest and Http2ServerResponse options to createServer and createSecureServer. Allows custom req & res classes that extend the default ones to be used without overriding the prototype. PR-URL: #15560 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

apapirovski · 2018-02-09T18:07:52Z

Thanks for bearing with us @hekike. This took way too long.

(FYI, I made some adjustments to the commit message to make it fit within our guidelines.)

Landed in e5f101f

Add optional Http2ServerRequest and Http2ServerResponse options to createServer and createSecureServer. Allows custom req & res classes that extend the default ones to be used without overriding the prototype. PR-URL: #15560 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

gibfahn · 2018-04-13T03:07:14Z

Will need a backport if it should go back to v8.x (if you'd like to do a backport please change the label).

Add optional Http2ServerRequest and Http2ServerResponse options to createServer and createSecureServer. Allows custom req & res classes that extend the default ones to be used without overriding the prototype. PR-URL: nodejs#15560 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

Add optional Http2ServerRequest and Http2ServerResponse options to createServer and createSecureServer. Allows custom req & res classes that extend the default ones to be used without overriding the prototype. Backport-PR-URL: #20456 PR-URL: #15560 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

Add optional Http2ServerRequest and Http2ServerResponse options to createServer and createSecureServer. Allows custom req & res classes that extend the default ones to be used without overriding the prototype. PR-URL: nodejs#15560 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

Add optional Http2ServerRequest and Http2ServerResponse options to createServer and createSecureServer. Allows custom req & res classes that extend the default ones to be used without overriding the prototype. Backport-PR-URL: #20456 PR-URL: #15560 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Anatoli Papirovski <apapirovski@mac.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

nodejs-github-bot added the http2 Issues or PRs related to the http2 subsystem. label Sep 22, 2017

lpinca reviewed Sep 24, 2017

View reviewed changes

apapirovski reviewed Sep 24, 2017

View reviewed changes

mcollina reviewed Sep 25, 2017

View reviewed changes

hekike force-pushed the feat/http2-request-response-extend branch from d4a9c75 to be96525 Compare September 25, 2017 12:16

jasnell approved these changes Sep 25, 2017

View reviewed changes

jasnell requested a review from mcollina September 26, 2017 20:00

apapirovski reviewed Feb 9, 2018

View reviewed changes

http2: add Http2ServerRequest and Http2ServerResponse options to crea…

bdca0b7

…teServer

hekike force-pushed the feat/http2-request-response-extend branch from 32ccd60 to bdca0b7 Compare February 9, 2018 17:50

apapirovski closed this Feb 9, 2018

hekike deleted the feat/http2-request-response-extend branch February 9, 2018 18:36

MylesBorins mentioned this pull request Feb 21, 2018

v9.6.0 proposal #18902

Merged

gibfahn added dont-land-on-v6.x labels Apr 13, 2018

MylesBorins mentioned this pull request May 2, 2018

v8.11.2 proposal #20478

Merged

MylesBorins removed the dont-land-on-v8.x label May 15, 2018

MylesBorins mentioned this pull request Jul 9, 2018

v8.12.0 proposal #21593

Merged

chyzwar mentioned this pull request Feb 13, 2019

[node] Improve http2 definitions DefinitelyTyped/DefinitelyTyped#33019

Closed

9 tasks

HTTP/2 createServer() options to pass custom Request and Response classes #15560

HTTP/2 createServer() options to pass custom Request and Response classes #15560

Conversation

hekike commented Sep 22, 2017 • edited Loading

Checklist

Affected core subsystem(s)

Motivation

How it works

Benchmark results

Related PR

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

apapirovski Sep 24, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

apapirovski Sep 24, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

apapirovski Sep 24, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

apapirovski commented Sep 24, 2017

apapirovski Sep 24, 2017 • edited Loading

Choose a reason for hiding this comment

jasnell commented Sep 24, 2017

mcollina commented Sep 25, 2017

hekike commented Sep 25, 2017

mcollina commented Sep 25, 2017

Choose a reason for hiding this comment

apapirovski commented Sep 25, 2017

apapirovski commented Sep 25, 2017

hekike commented Sep 25, 2017

evanlucas commented Sep 25, 2017

mcollina commented Sep 25, 2017

hekike commented Sep 25, 2017

dougwilson commented Sep 25, 2017

mcollina commented Sep 25, 2017

yunong commented Sep 25, 2017

yunong commented Sep 25, 2017

apapirovski commented Sep 25, 2017

hekike commented Sep 25, 2017

apapirovski commented Sep 25, 2017 • edited Loading

mcollina commented Sep 25, 2017

hekike commented Sep 26, 2017

Choose a reason for hiding this comment

hekike commented Feb 9, 2018

apapirovski commented Feb 9, 2018

gibfahn commented Apr 13, 2018

hekike commented Sep 22, 2017 •

edited

Loading

apapirovski Sep 24, 2017 •

edited

Loading

apapirovski Sep 24, 2017 •

edited

Loading

apapirovski Sep 24, 2017 •

edited

Loading

apapirovski Sep 24, 2017 •

edited

Loading

apapirovski commented Sep 25, 2017 •

edited

Loading