Server-side rendering

[renchap]

Using ExecJS, we should be able (optionally) to run React on the server, render the content, and then re-hydrate the components in the browser.

The steps for this would be something like:

• find how to determine the path of the pack file containing the component. This is not trivial and I think we will need to load all the packs, maybe with an option for the user to specify the correct one and avoid loading all of them
• get the packfile content. Location may vary between development (in public/packs/) and production (need to use the Webpacker manifest)
• create a new ServerSideRendering based on ExecJS, with a render(component, props) option
• change render_component to call the server-side renderer if configured to do so
• change the webpacker-react JS module to now create a new component, but hydrate an existing one
• add various improvements, as console replay in the browser, error handling, ...

[daninfpj]

I'm gonna start working on this soon. I think to load the pack files we should stick to some convention (probably configurable) like react-rails' components.js does, so we know where can find the components.

[renchap]

You can have multiple components registered in one packfile. The logic is to have separate packs for different parts of your website, but you may have multiple root components in it, especially if not using an SPA.

[daninfpj]

Exactly, that file (components.js) loads all the components you'll be using for sever-side rendering and their dependencies. I think that makes sense and simplifies the logic of trying to locate the right packfile. And in practice you don't necessarily have to include it in your views, you can include there the separate packs you mention.

[renchap]

Seems good. Maybe use a more expressive default name, like app/javascripts/components/serverRendering.js?
Also this should be a configuration option, and allow an array of files to be loaded.

[sevos]

I'd try to challenge the whole server-side rendering idea later :)

[risinglf]

Hi guys, any news about server side rendering?

[sevos]

I hate have mixed feelings with the idea of server side rendering:

1. it kills the purpose of SPAs - didn't we want to have single page apps because we wanted to take the load off the server and move to the client/browser to allow scalability^TM? And now what? Again moving back rendering to server. It seems like a roundtrip for me. Let's just use Turbolinks and call the SPA experiment a failure ;)

2. it surely adds some edge-cases (not everything can be handled outside of the browser, some libs?).

[risinglf]

@sevos for our company server side rendering is not a choice: it's mandatory. We don't have a full SPA but many simple widgets that for SEO and UX reasons need to be already present when the browser/bot loads the page.

It can of course add some edge cases due to many different libraries available, but IMHO is up you to choose one that does not need the browser env or to implement a different approach when the server side rendering is running...

What do you think?

[justin808]

@risinglf @sevos @daninfpj Why would prefer to put server rendering in this library compared to using React on Rails?

[daninfpj]

@justin808: The whole point of this library (not just server rendering) is to take advantage of Rails 5.1 (currently on rc2) first-class support for Webpack and React. This allows for a cleaner and simpler integration in my opinion.

[justin808]

@daninfpj wrote:

@justin808: The whole point of this library (not just server rendering) is to take advantage of Rails 5.1 (currently on rc2) first-class support for Webpack and React. This allows for a cleaner and simpler integration in my opinion.

We're almost done with that.

See shakacode/react_on_rails#822 and shakacode/react_on_rails#811.

and see:

https://github.com/shakacode/webpacker_lite

[justin808]

React on Rails 8.0.0 shipped with support for webpacker_lite. I think this has the server rendering support you desire.

[risinglf]

Hi guys, how is this feature progress?

[caselas]

Is there anything we can do to help?

[renchap]

I have not got time to really work on server-rendering. If you would like to have a stab at it, feel free! I outlined my ideas in this issue and I am available to discuss it further.
React 16 changes server-side rendering and hydrating, and I think it would be great to support it.

I am not opposed to a minimal and non-modular approach at first, supporting only mini_racer (via ExecJS?) to keep things simple.

[wingrunr21]

I've got a minimal approach working already if that is of interest. I'm using webpacker-react for the client side stuff but the server side is independent. It's pretty heavily influenced by react-rails as I thought their implementation was pretty clean.

I took the single point of entry approach with one "server" pack that imports all the various components. Right now the issue I'm working through is due to the way webpacker 3 enables the style-loader when HMR is enabled, you can't server side render and use HMR at the same time.

[renchap]

Nice! This seems like a good start.
There is a related issue in the Webpacker repo: rails/webpacker#842

What is the issue with style-loader and server-rendering?

[wingrunr21]

Ya, I've read through that issue. I actually don't need to turn off inline mode like is specified there. react-rails removes the client require that inline mode inserts. Simply adding a var self = self || this fixes the other error.

I'm messing with a server-only webpack config to try and figure out a clean way of having HMR and server side rendering. I'm not a huge fan of that thread's suggestion of maintaining a per-file list. IMO there should be another point of entry for server-side packs that the second config handles.

Another issue at play here is whether or not the client will then mount the same component on top of the server side render. This may not be wanted if you are basically statically rendering React components out, but if you are trying to bootstrap a SPA you probably want to do that. One thing at a time though 😄

What is the issue with style-loader and server-rendering?

style-loader basically wants a DOM to write <style></style> out to. It is actually recommended to use extract-text-plugin for server side, but we don't have control over webpacker enabling that right now. If you turn off HMR, webpacker disables the style-loader so everything works fine.

[renchap]

Well, server rendering should not worry about styles at all and not output anything related (except for css modules).
Hydrating a server-rendered component is a mandatory feature, I dont see a usecase for server-side only React (just use Ruby!).

React 16 changed quite a lot of things related to SsR’ I foudn this article useful: https://medium.com/@aickin/whats-new-with-server-side-rendering-in-react-16-9b0d78585d67
I dont know if it may help you.

Can you publish your work in progress somewhere so I can have a look and we can discuss over real code? This will be easier 🙂

[wingrunr21]

Agree to disagree on that one. We use styled-components pretty extensively. As they ship all styles via JS and subsequently write a style tag to the DOM on component tree render, you have to have a way of pulling the styles out of the render tree on the server. Luckily, they added first party support for server side rendering in v2, so that works pretty nicely.

Also, not everyone's whole stack is Ruby. We've got SPA, express, and Rails apps in production. We have a React-based component library that is shared between them. Components like our footer take an initial set of props and are rendered out. Having that statically rendered is a valid use case for us.

The issue isn't even about the appropriateness of CSS on the server. When the style-loader is enabled webpack emits additional code that causes execJS to choke. However, you need the style-loader for HMR. So, you either get HMR or server side rendering.

Sure, I can put stuff up in a gist

[renchap]

Ok I see how it works. Let me correct what I said by "server-rendering should ignore style-loader and other CSS-related webpack loaders :) For production your stylesheets will be compiled in CSS files by extract-text-plugin, and in dev your styles are inserted into a <style> tag by style-loader.

I guess we will need hooks to allow styled-components (and other similar projects) to work with webpacker-react server-rendering, as it it a library-specific feature.

The first and simplest goal of server-side rendering is to be able to call renderToString() on your component on the server and output the result into your Rails view, and then to call ReactDOM.hydrate when mounting the component client-side.

[wingrunr21]

Here's a gist of what we are using right now. Like I said, this is a pretty simple implementation. I'm still wondering if a server-only webpack config is a better approach. It would make messing around with CommonsChunk and such easier and would enable hmr to work alongside.

https://gist.github.com/wingrunr21/b2e2a1aca3083eb877a6deae9dedbd89

[tomasc]

@renchap @wingrunr21 I am about to tackle this in my app. Is there any progress on this or should I pickup from the gist @wingrunr21 posted?

[wingrunr21]

@tomasc I won't be submitting a PR to this repo for server side render support. As we continued to iterate on our solution, it became obvious that a more standalone solution was the best fit. We are prepping to open source a generic SSR solution for webpacker (which will still have full support for webpacker-react).

[tomasc]

@wingrunr21 that sounds good. If interested, I can help test your project as I am about to start dealing with SSR and solution with bare-bones webpacker would work best in my case.

[renchap]

@tomasc I havent got time to work on this yet. Feel free to tackle it if you want!
The preferred way is to open a PR as soon as you have a WIP, so we can discuss about it while it progresses.

@wingrunr21 I am curious about how you want to tackle it at the Webpacker level. Can you outline how it would work?

[BookOfGreg]

@wingrunr21 Yep that sounds interesting. Just commenting so I watch this thread (spying from react-rails 😉 )

[wingrunr21]

@BookOfGreg honestly, your SSR support formed a really solid basis for the work. It definitely gave us a good starting point on a solution that was known to work. Your project is also a reason we wanted to target a more generic webpacker solution. It's fairly difficult to parse out how to implement SSR in an express application without using something like create-react-app or next.js. We didn't think another implementation of SSR (with react-rails and react_on_rails both having implementations) would benefit from being specific to a given project.

@renchap Sure. We wanted this to work as closely as possible to how "real" SSR is done in a node environment. As I outlined before, our onus around this is because our React codebases are used across various environments. In addition, the vast majority of SSR testing by upstream users is done against a node environment. Emulating how those setups work seemed to be the optimal approach.

1. We are using a separate server config. We separated out the SSR entry point and the rest of the packs so that client side JS can still be run through things like CommonsChunkPlugin. The server config is derived from the webpacker config as much as possible We will probably push a few PRs against webpacker to aid with extracting that config (for instance, their ExtractTextPlugin configuration is not exported at all when the dev server is running and HMR is enabled)
2. We implemented adapters to support rendering the JS to the client that is inline with how client side libraries will then mount the components. You can see that in action on www.guildeducation.com. The navigation and footer are SSR components that are remounted via webpacker-react's auto mounting
3. We are currently working through more complex requirements such as using a StaticRouter from react-router or dealing with redux stores. The good part is that sticking fairly close to how node does things means these use cases can be supported in much the same way

[tomasc]

@wingrunr21 is there any updated on this please? I am eager to test or help out.

[wingrunr21]

@tomasc sorry, our November/December ended up being crazy with other work. I'm working on it this weekend and hope to have some good stuff come next week.

[tomasc]

@wingrunr21 thanks, that would be fantastic – and right on time ;-)

[kiwanska]

@wingrunr21 🤞 that you still working on this, do you have ETA in mind?

[wingrunr21]

Code is up: https://github.com/GuildEducationInc/webpacker_ssr, https://github.com/GuildEducationInc/webpacker_ssr-execjs, https://github.com/GuildEducationInc/webpacker_ssr-react

Need to write docs still (hoping today or tomorrow).

We just deployed the changes into production yesterday.

[tomasc]

@wingrunr21 I am trying to test, but have hard time setting it up.
Can you please share a few steps required to get this running with React? Thanks!

[tomasc]

Here my findings:

# config/initializers/webpacker_ssr.rb

require 'webpacker_ssr'
require 'webpacker_ssr/execjs'
require 'webpacker_ssr/react'

WebpackerSSR.configure do |config|
  config.server_bundle = 'server_bundle.js'
  config.server_manifest = 'manifest.json'
  config.renderer = :execjs
end

When assets are precompiled (bin/webpack) I get Cannot read property 'createElement' of undefined – seems React is not available, even though I have it imported on top of the server_bundle.js:

import React from 'react';
import ReactDOMServer from 'react-dom/server';

When using bin/webpack-dev-server, I am slo getting error of calling .protocol of undefined. I guess that comes from the JS that webpack dev server injects into the bundle.

Also when using the CommonsChunkPlugin to isolate all vendor libs I get following error: ExecJS::ProgramError: ReferenceError: webpackJsonp is not defined. I tried to make new WebpackerSSR::React::Plugin that would include the manifest.json file, but with no luck:

module WebpackerSSR
  class VendorBundle < ServerBundle
    def self.read(file_name)
      manifest = ServerManifest.new
      asset_path = manifest.lookup(file_name).to_s
      Webpacker.dev_server.running? ? load_from_dev_server(asset_path) : load_from_file(asset_path)
    end
  end
end

module WebpackerSSR
  module React
    module Plugins
      class Vendor < Plugin
        self.priority = -1

        def set_up_js_variables(_input)
          VendorBundle.read('manifest.js')
        end
      end
    end
  end
end

WebpackerSSR::React.register_plugin(:vendor, WebpackerSSR::React::Plugins::Vendor)
WebpackerSSR::React.config.default_plugins = %i[vendor react]

Help would be appreciated.

[wingrunr21]

Hi all,

Really sorry. We are in the middle of multiple large client launches right now and my time is being monopolized in support of those.

In the mean time:

Gemfile:

gem 'webpacker', '~> 3.2.0'
gem 'webpacker-react', '~> 0.3.1'
gem 'webpacker_ssr', '~> 1.0.0.alpha.1', github: 'GuildEducationInc/webpacker_ssr'
gem 'webpacker_ssr-execjs', '~> 1.0.0.alpha.1', github: 'GuildEducationInc/webpacker_ssr-execjs'
gem 'webpacker_ssr-react', '~> 1.0.0.alpha.1', github: 'GuildEducationInc/webpacker_ssr-react'

config/initializers/webpacker_ssr.rb

WebpackerSSR.configure do |config|
  config.renderer = :execjs
  config.server_bundle = 'server_side_render'
  config.server_manifest = 'server-manifest.json'

  config.react.default_plugins = [:react]
end

config/webpack/server.js

const path = require('path')
const ExtractTextPlugin = require('extract-text-webpack-plugin')
const ManifestPlugin = require('webpack-manifest-plugin')
const { Environment } = require('@rails/webpacker')

// Webpacker screws around with the exports so this needs to be redefined
const postcssConfigPath = path.resolve(process.cwd(), '.postcssrc.yml')
const isProduction = process.env.NODE_ENV === 'production'
const extractOptions = {
  fallback: 'style-loader',
  use: [
    { loader: 'css-loader', options: { minimize: isProduction } },
    { loader: 'postcss-loader', options: { sourceMap: true, config: { path: postcssConfigPath } } },
    'resolve-url-loader',
    { loader: 'sass-loader', options: { sourceMap: true } }
  ]
}

const extractCSSLoader = {
  test: /\.(scss|sass|css)$/i,
  use: ExtractTextPlugin.extract(extractOptions)
}

class ServerEnvironment extends Environment {
  constructor() {
    super()

    // Fix so HMR can be used at the same time
    this.loaders.set('style', extractCSSLoader)

    // Override server manifest
    const manifestPlugin = this.plugins.get('Manifest')
    manifestPlugin.opts.fileName = 'server-manifest.json'
    this.plugins.set('Manifest', manifestPlugin)
  }

  toWebpackConfig() {
    const result = super.toWebpackConfig()
    const serverSideEntry = result.entry['server_side_render']
    result.devtool = undefined
    result.output.libraryTarget = 'this'
    result.entry = () => {
      return {'server_side_render': serverSideEntry }
    }
    return result
  }
}

const environment = new ServerEnvironment()

module.exports = environment

config/webpack/development.js

const environment = require('./environment')
const serverEnvironment = require('./server')

const config = environment.toWebpackConfig()
delete config.entry['server_side_render']

const serverConfig = serverEnvironment.toWebpackConfig()

module.exports = [config, serverConfig]

config/webpack/production.js

const environment = require('./environment')
const serverEnvironment = require('./server')

const config = environment.toWebpackConfig()
config.devtool = 'hidden-source-map'

const serverConfig = serverEnvironment.toWebpackConfig()

module.exports = [config, serverConfig]

We are running against webpacker 3.2.0 (apparently there are some issues with 3.2.2).

[tomasc]

Thanks, @wingrunr21, that's very helpful, I will give it a try!

[tomasc]

PS got it to work with no issues on webpacker 3.2.2 (although I do not use styled components).

[tomasc]

@wingrunr21 would be helpful if the three projects could be released as gems already – even if as alpha.

[wingrunr21]

kk. I want to get some tests in place first. Crossing my fingers that I can use some weekend time on these projects.

[tomasc]

Thanks. Once we get the base set up I can help with refining the edges.

[wingrunr21]

Hey all,

So sorry. I have not forgotten about these projects. I'm blocking time this Friday to write documentation + make some additional improvements to the gems. Startup life...

[BookOfGreg]

@wingrunr21 I know what you mean, barely have any time to work on react-rails anymore, moved company to a startup and there's no sponsored time anymore. Honestly good luck with this effort.

[tomasc]

@wingrunr21 thanks – let me know if I can help anyhow. I have been running the webpacker-ssr group of gems in production for a while now and it works very fine so far.

[mobyjames]

To anyone having issues, it took me a while to realize i needed to have a pack named server_side_render.js that looks something like this...

import React from 'react'
import ReactDOMServer from 'react-dom/server'
import { ServerStyleSheet } from 'styled-components'
import HelloReact from 'components/hello_react'

execJsGlobal.React = React
execJsGlobal.ReactDOMServer = ReactDOMServer
execJsGlobal.ServerStyleSheet = ServerStyleSheet
execJsGlobal.HelloReact = HelloReact

You must also use react_server_component instead of the usual react_component helper.

Fantastic work!