Server side hot reload #809

drewpc · 2017-01-07T23:25:06Z

This patch enables server-side hot module reload on the existing master branch. Note: this does not include any Webpack changes nor does it include any changes to ExpressJS unless required for HMR to work. It simply reloads the existing changes when Webpack successfully recompiles the client code. This is a refactor of the code from #791 to only include server side HMR changes. Discussed in #773.

UPDATE: this branch was rebased and changes from #808 were removed. It is now a standalone patch.

…ilation of webpack client code.

…equire.cache entries!

gigabo · 2017-01-09T23:33:25Z

packages/react-server-cli/src/commands/start.js

+					server.use((req, res, next) => {
+						reactServer.middleware(req, res, next, require(serverRoutes));
+					});
+				};


Looks like there's an extra level of indentation here?

Surprised eslint didn't catch that. 😕

Fixed. I don't think that eslint is working properly. asini run lint blows up and there's no script in the root level package.json to run lint. I'll look into it.

(at least in my setup it's blowing up)

Hmmm... we just run asini run test in CI. So each package is responsible for making sure that linting is part of its test script. It looks like the CLI's package.json should be running tests as part of the gulp test phase of its test script.

Wonder where that's breaking... 🤔

so...lint doesn't exist anywhere else. There's a weird mix of gulp tasks and package.json run scripts though. And it really bothers me that they're not named the same. In some packages, it's gulp build in others it's gulp, and in other cases, you have to do npm run build.

Oh, nice catch on the missing dep. Yeah, some modules (like react-server-cli) use the eslint gulp plugin instead of the executable. It's all kind of a weird mishmash right now. 😖

I'll clean it up and submit a PR. I already have the previous issue fixed.

This is the offender: https://github.com/redfin/react-server/blame/master/packages/react-server-cli/gulpfile.babel.js#L7 Need to use globs...

gigabo · 2017-01-09T23:37:48Z

packages/react-server-cli/src/compileClient.js

@@ -324,8 +336,7 @@ module.exports = {
 	const routesContent = routesOutput.join("");
 	// make a unique file name so that when it is required, there are no collisions
 	// in the module loader between different invocations.
-	const routesMD5 = crypto.createHash('md5').update(routesContent).digest("hex");
-	const routesFilePath = `${workingDirAbsolute}/routes_${isClient ? "client" : "server_" + routesMD5}.js`;
+	const routesFilePath = `${workingDirAbsolute}/routes_${isClient ? "client" : "server"}.js`;


Nice to get rid of this.

gigabo · 2017-01-09T23:41:28Z

packages/react-server-integration-tests/src/specRuntime/testHelper.js

+			Object.keys(require.cache)
+				.filter((key) => /(__clientTemp|test-temp)/.test(key))
+				.forEach((key) => delete require.cache[key]);
+


Oh, interesting. Glad you caught this... could have been a real source of frustration. 😬

gigabo · 2017-01-10T00:14:31Z

This is a cool solution. Thanks for working it out @drewpc!

LGTM 👍

jhnns · 2017-01-10T09:05:04Z

Thanks for splitting up the pull-request @drewpc. This makes reviewing it a lot easier 👍

The solution is very simple and works as long as you stay within the page lifecycle. However, this solution will probably leak memory over time in more advanced use-cases, since cleaning the require.cache is not a guarantee that all references have been removed. Especially if you do side-effecty things like opening connections (and keeping them open), this will become a problem because the connection won't be closed in this case.

To me, a cleaner solution is to have a separate application server process that receives a KILL signal when a new bundle has been generated. This will make sure that the server cleans up everything before it is restarted. A downside of this approach is that restarting a process takes longer than cleaning the require.cache. In my project setup, the server restart takes about 1s.

drewpc · 2017-01-10T16:51:17Z

@jhnns Great point! I hadn't looked deeply into this until you mentioned it. I did some basic testing on memory leaks with regard to deleting require.cache entries (modifying this code) and came to the following conclusion: my server side patch HMR solution doesn't do anything good or bad to resolve those memory leaks. In other words, if memory leaks exist in the code being required delete require.cache[<module>] doesn't "fix" that. Your solution of restarting a separate process would fix the symptom, but not the problem: memory leaks still exist.

The topic of memory leaks would be a great documentation item: patterns to avoid, ways to detect memory leaks, etc. The page cycle of react-server would be the most vulnerable, especially since it is called for each HTTP request. I'll have to look into this, but I see it as a different problem than server side HMR.

roblg · 2017-01-10T18:52:15Z

Added breaking-change to this, due the moving of the express middleware and express-state setup out of renderMiddleware. (Which is a change that has been overdue. Thanks!)

jhnns · 2017-01-11T08:56:24Z

my server side patch HMR solution doesn't do anything good or bad to resolve those memory leaks. In other words, if memory leaks exist in the code being required delete require.cache[] doesn't "fix" that.

Of course, you're right. These memory leaks exists in the code because you don't write tear-down code if you don't need it. But usually you don't just delete require.cache, so it's not an actual memory leak in production because the module is not replaced in production. It just becomes a (potential) memory leak when the cache is deleted.

But besides the memory leak I rather worry about the unknown application state after modules have been swapped out without actually tearing down everything. Your application may behave strangely in development after a while (because of stale connections, etc.). I'd rather prefer to restart the server cleanly than having to deal with an unknown application state. That's why create-react-app is also using HMR just for CSS, because CSS can be replaced without introducing side-effects.

I've explained some details of HMR in this SO answer.

# Conflicts: # packages/react-server-cli/src/commands/start.js

…to patch-server-side-hot-reload

…er serverSideHotModuleReload function from PR redfin#809 so that things are more in sync.

drewpc added 3 commits January 9, 2017 13:57

Enabled hot module reload for the server only after a successful comp…

0fefacd

…ilation of webpack client code.

Minor fix to use the for loop logic instead of .length - 1.

4405568

Gah! I forgot to modify the test runner so that it properly deletes r…

2d0b894

…equire.cache entries!

drewpc force-pushed the patch-server-side-hot-reload branch from 659816d to 2d0b894 Compare January 9, 2017 21:08

drewpc mentioned this pull request Jan 9, 2017

Consolidate HTML + Javascript Servers #807

Closed

Merge branch 'master' into patch-server-side-hot-reload

59cc6c8

gigabo reviewed Jan 10, 2017

View reviewed changes

gigabo approved these changes Jan 10, 2017

View reviewed changes

gigabo added the enhancement New functionality. label Jan 10, 2017

Fixed indentation issues.

0cc2cc5

roblg added the breaking change label Jan 10, 2017

roblg mentioned this pull request Jan 10, 2017

Make ESLint Work #818

Merged

roblg approved these changes Jan 10, 2017

View reviewed changes

karlhorky mentioned this pull request Jan 11, 2017

Server Side HMR/Webpack & Other Improvements #791

Closed

11 tasks

drewpc added 4 commits January 11, 2017 09:11

Merge branch 'master' into patch-server-side-hot-reload

69a7973

Merge branch 'master' into patch-server-side-hot-reload

23c3ba3

# Conflicts: # packages/react-server-cli/src/commands/start.js

Merge remote-tracking branch 'origin/patch-server-side-hot-reload' in…

c9c9826

…to patch-server-side-hot-reload

Merge branch 'patch-fix-eslint-again' into patch-server-side-hot-reload

7f77687

drewpc added a commit to SoprisApps/react-server that referenced this pull request Jan 11, 2017

Merged latest changes from master. Updated the branch with the pretti…

a6a83aa

…er serverSideHotModuleReload function from PR redfin#809 so that things are more in sync.

Merge branch 'master' into patch-server-side-hot-reload

e711af0

doug-wade mentioned this pull request Jan 11, 2017

Use tree shaking #237

Closed

drewpc merged commit c824631 into redfin:master Jan 12, 2017

drewpc deleted the patch-server-side-hot-reload branch January 12, 2017 02:26

drewpc modified the milestone: 0.6.0 Jan 27, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Server side hot reload #809

Server side hot reload #809

drewpc commented Jan 7, 2017 •

edited

Loading

gigabo Jan 9, 2017

drewpc Jan 10, 2017

gigabo Jan 10, 2017

drewpc Jan 10, 2017

gigabo Jan 10, 2017

drewpc Jan 10, 2017 •

edited

Loading

gigabo Jan 10, 2017

drewpc Jan 10, 2017

drewpc Jan 10, 2017

drewpc Jan 10, 2017

gigabo Jan 9, 2017

gigabo Jan 9, 2017

gigabo commented Jan 10, 2017

jhnns commented Jan 10, 2017 •

edited

Loading

drewpc commented Jan 10, 2017 •

edited

Loading

roblg commented Jan 10, 2017

jhnns commented Jan 11, 2017

Server side hot reload #809

Server side hot reload #809

Conversation

drewpc commented Jan 7, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

drewpc Jan 10, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gigabo commented Jan 10, 2017

jhnns commented Jan 10, 2017 • edited Loading

drewpc commented Jan 10, 2017 • edited Loading

roblg commented Jan 10, 2017

jhnns commented Jan 11, 2017

drewpc commented Jan 7, 2017 •

edited

Loading

drewpc Jan 10, 2017 •

edited

Loading

jhnns commented Jan 10, 2017 •

edited

Loading

drewpc commented Jan 10, 2017 •

edited

Loading