Support asynchronous server rendering (waiting for data before rendering)

[fdecampredon]

It would seriously ease the process of building something isomorphic if componentWillMount could return a promise and that react would delay rendering until that promise is resolved. I have seen attempt of doing something like that in react-router and rrouter, however giving this responsibility to each component instead of a router module would make more sense for me.

[sophiebits]

The main reason (I believe) that this doesn't exist already is that on the client side, you basically always want to show some sort of loading indicator instead of deferring rendering. (It would also make the code significantly more complex, but we can probably deal with that.)

[fdecampredon]

There is 2 cases that I find hard to solve without that:

• On the server side, if you want to fetch data before rendering you can't delegate data retrieval to components since you don't have the information of which component will be rendered
• On the client side the first time you mount your application after receiving pre-rendered html, even if you have some sort of cache from data retrieved on the server, you might want to use async method to retrieve those data, and that would prevent react from reusing the html.

react-async solve those problems with fibers, and cache. That do the trick but in my point of view those are just hackish solutions to solve a problem that can only be solved in core.

[syranide]

Color me uninformed on this subject, @fdecampredon say that componentWillMount is async and you don't return anything immediately, what is React supposed to render until there is, nothing? If so, why not just return nothing in render if there is no data yet? (Yeah I get server-side) Also, what should happen if props change before componentWillMount fires?

Personally, it seems like it's wrong to dispatch async requests during componentWillMount unless the component really is an isolated black box and also implements a loading indicator. As far as I understand it, React components should not be mistaken for more conventional OOP instances. In the best case, a React component is tool for visualizing the data in props, if it's interactive then possibly also state. It's a view, not a view and model.

To my ears, that sounds like the problem, React components shouldn't be the ones dispatching the async requests, you fetch all the data and when that data is ready, only then do you call React.renderComponent. Same solution client-side and server-side. You also get the ability to abort with an outcome of your choice if any async request fail.

Feel free to dismiss me if I misunderstood something, but it seems that you're treating React components as view and model, when (it seems) they're meant as just the view.

[fdecampredon]

Color me uninformed on this subject, @fdecampredon say that componentWillMount is async and you don't return anything immediately, what is React supposed to render until there is, nothing? If so, why not just return nothing in render if there is no data yet? (Yeah I get server-side) Also, what should happen if props change before componentWillMount fires?

I must admit that I did not think about all the cases ^^.
This feature would be useful only the first time we mount the top level component, and on the server, it's true that otherwise In most cast you would want to display a loader indicator.

Personally, it seems like it's wrong to dispatch async requests during componentWillMount unless the component really is an isolated black box and also implements a loading indicator. As far as I understand it, React components should not be mistaken for more conventional OOP instances. In the best case, a React component is tool for visualizing the data in props, if it's interactive then possibly also state. It's a view, not a view and model.

In a way or in other you'll want that a 'top-level' component would be able to retrieve data, like it's done in the Flux sample.
In this sample things are pretty simple because retrieving the list of todo is a synchronous operation, if it was not, and in case of pre-rendering on the server, we would render a first time with no data (and loose the pre-rendered markup from server).

In the case of a simple application with one set of data displayed by one view hierarchy there is still not so much problem, you can preload data and still keep the synchronous property of your store.
Now in case of an application composed of multiple modules that you reuse across your application I would like to be able to treat those modules as separate applications that are able to 'subscribe' on different stores (that would be responsible for fetching data).

Perhaps That I get the things the wrong way but some discussion/sample around the web make me think there is something missing somewhere:

• In some sample it seems to me that @petehunt tried to achieve something similar.
• react-nested-router promote some similar mechanism in willTransitionTo, and some discussions make me feel that nobody has come with a proper solution.
• RRouter also provides some kind of mechanism to prefetch data as the component is being rendered/mounted.
• and finally react-async like I said earlier.

[mjackson]

@fdecampredon To be clear, the purpose of willTransitionTo in react-nested-router is not for loading data, specifically because returning a promise from that method will indeed block rendering new UI, which you don't want to do unless you absolutely have to.

[syranide]

@fdecampredon Everyone is still trying to figure things out, so it wouldn't surprise me if no one has a definitive answer. But I'm guessing the devs at Facebook must've run into this themselves a few times.

[tobice]

Any update on this? I've just started exploring React and I immediately run into this. Many people recommend React as a go to solution for building isomorphic apps, but as long as this is not solved I think it simply can't get the job done.

To my ears, that sounds like the problem, React components shouldn't be the ones dispatching the async requests, you fetch all the data and when that data is ready, only then do you call React.renderComponent. Same solution client-side and server-side. You also get the ability to abort with an outcome of your choice if any async request fail.

Feel free to dismiss me if I misunderstood something, but it seems that you're treating React components as view and model, when (it seems) they're meant as just the view.

If this is true then React is nothing more than a slightly different templating solution / view layer. And that would be a shame because there is such a potential. I really understand @fdecampredon when he mentions those complex applications composed of multiple modules. React would be perfect for this.

I don't think that this approach would mean treating a component as view and model. If you look at the Flux architecture, they think of React components as of controller-views that not only display data (from stores) but also invoke actions based on user interactions. And the actions then update the stores (= model). To me it sounds just like an obvious MVC architecture.

The problem is with the initial action that populates the store. It's fairly simple on the client side where the action can be invoked from the componentDidMount() method as recommended. On the server side on the other hand, we really need some special place and some kind mechanism that would delay the rendering until the action is finished and the stores are populated.

At this moment, it seems to me that the only way is React-async/Fibers + Flux. The nice thing about Flux is that we don't need some artificial cache to transport component states from the server to the client (like it's done in the original react-async example), we can simply initialize the stores and then send them to the client with the html markup (see this example).

But this solution is indeed hackish.

[mridgway]

I don't think it necessarily needs to be componentWillMount that is async; I'm not even sure it needs to be a lifecycle event. The real issue is that on the server side there is no way to analyze the component tree until after everything has been rendered to a string.

My ideal solution to solve this would be to allow "rendering" that would just build the component tree, then we would be able to traverse the tree to find components that need additional data, allow us to asynchronously load more data and "re-render" that components subtree, and then once we are ready for flushing the markup, allow us to convert that tree to a string.

This replicates what we're able to do in the browser: have a virtual DOM that we can re-render as we want. The difference is that in the browser DOM updates can be implicit. On the server we need to be explicit about when we render to a string so that we can perform updates to the virtual DOM based on async data.

[koistya]

My ideal solution to solve this would be to allow "rendering" that would just build the component tree, then we would be able to traverse the tree to find components that need additional data, allow us to asynchronously load more data and "re-render" that components subtree, and then once we are ready for flushing the markup, allow us to convert that tree to a string. — @mridgway

Yep, that would be good. Currently, rendering component twice on a server-side can be used as a workaround.

[mridgway]

I want to reference react-nexus as an example of what I'd want to be supported within React. It's essentially a rewrite of how mountComponent works except that it builds out the component tree without actually mounting it to the DOM or writing out a string. This allows you to traverse the component tree and fire off asynchronous methods while the tree is being built up. The issue with this implementation as it is, is that it throws away that first tree and then calls React.renderToString anyway, whereas it would be nice to take that pre-render tree and render/mount it.

I'm willing to work on this within core, but would need some pointers. I think one of the requirements is making ReactContext not globally referenced so that async wouldn't cause issues. Are there other globals that might run into issues with this as well?

[gaearon]

@mridgway If I'm not mistaken global ReactContext might be a compat thing and will go away in 0.14. But I might be wrong.

[mridgway]

@gaearon Yeah, that's the sense I got from the withContext deprecation.

[SpainTrain]

👍 Using react-router for this ATM. @mridgway's approach sounds very reasonable.

[anatomic]

A slightly different take on this issue is how to handle asynchronous loading of code chunks produced by a bundler (such as webpack).

As discussed in this ticket - remix-run/react-router#1402 - the issue with async rendering support is the initial render appears to be null as the relevant chunks haven't yet loaded in the first render pass, resulting in a <noscript> at the root of the DOM and a checksum fail. Everything falls into place quickly afterwards, but it does result in a flicker in the UI when working locally and worse in the field, especially if the chunks to be downloaded are of a reasonable size (say > 30KB)

The ability to split up big apps into multiple chunks is important to us and having solved the data fetching issue (we used react-router and nested routes which can be traversed prior to rendering on the server to fetch data dependencies) this is the last piece of the puzzle blocking us from moving fully to a React solution for our front-end.

[syranide]

@anatomic That's not React's responsibility, it's your job to chunk appropriately and defer rendering until all necessary chunks has been loaded. To put it differently, if one of your components has a dependency on some external library, it's obviously your problem to satisify before trying to use it, React couldn't do it even if it tried, so the same applies across the board.

Feel free to implement alternative strategies that may suit you better, say <WaitFor for={MyAsyncLoadedCompSignal} until={...} then={() => <MyAsyncLoadedComp ... />} />. But these are inherently opinionated and not something React should or even needs to offer, so it's best left to the community.

[koistya]

It's better to keep async stuff outside the scope of React components, here is an example:

import React from 'react';
import Layout from './components/Layout';
import NotFoundPage from './components/NotFoundPage';
import ErrorPage from './components/ErrorPage';

const routes = {

  '/': () => new Promise(resolve => {
    require(['./components/HomePage'], HomePage => { // Webpack's script loader
      resolve(<Layout><HomePage /></Layout>);
    });
  }),

  '/about': () => new Promise(resolve => {
    require(['./components/AboutPage'], AboutPage => { // Webpack's script loader
      resolve(<Layout><AboutPage /></Layout>);
    });
  })

};

const container = document.getElementById('app');

async function render() {
  try {
    const path = window.location.hash.substr(1) || '/';
    const route = routes[path];
    const component = route ? await route() : <NotFoundPage />;
    React.render(component, container);
  } catch (err) {
    React.render(<ErrorPage error={err} />, container);
  }
}

window.addEventListener('hashchange', () => render());
render();

See Webpack Code Splitting, React.js Routing from Scratch and react-routing (coming soon)

[anatomic]

@syranide I'll keep working away at it, but I don't think it's as binary as you've put it above. We are using react-router so that may introduce some issues to the mix as the router is a component rather than sitting outside the React environment.

If we did the <WaitFor ... /> approach, surely we'll still get a different DOM on the first render which will still cause the flicker/disappearance of content?

[syranide]

If we did the <WaitFor ... /> approach, surely we'll still get a different DOM on the first render which will still cause the flicker/disappearance of content?

If you don't want flicker (some do) it's simply a matter of waiting for all the chunks you depend on to have loaded before rendering, webpack provides this out-of-the-box with require.resolve.

PS. Yes, react-router and whatever else you're using surely complicates the solution, but it's still not React's problem to solve.

[anatomic]

If you don't want flicker (some do) it's simply a matter of waiting for all the chunks you depend on to have loaded before rendering, webpack provides this out-of-the-box with require.resolve.

I'll look into that, my understanding of require.resolve was that it was a synchronous lookup of a module's id and didn't involve a trip to the server? We're using require.ensure to manage our chunk loading.

Looking at our code again I think we can circumnavigate the problem by making react-router think it's rendering on the server but then following the standard client-side approach:

const history = new BrowserHistory();

if (typeof history.setup === "function") {
    history.setup();
}

Router.run(routes, history.location, (err, initialState, transition) => {
    React.render(<Provider redux={redux}>{() => <Router key="ta-app" history={history} children={routes} />}</Provider>, document.getElementById('ta-app'));
});

[syranide]

I'll look into that, my understanding of require.resolve was that it was a synchronous lookup of a module's id and didn't involve a trip to the server? We're using require.ensure to manage our chunk loading.

Sorry, yes I meant require.ensure. The callback is executed only when all the dependencies are satisfied, so it's a matter of just putting render/setState inside of it.

[anatomic]

Ok, that's kind of how we were doing it, thanks for your replies. This feels like something that needs to be addressed in react-router so I'll carry on the discussion there - sorry if this has been the wrong place to have this conversation!

You are right that the require.ensure is the way to go, I guess our ultimate issue was that it should be linked to the route currently being visited so is directly tied to the router. With react-router being component based that ties it to the render tree. Without my hack above we're left fighting a way to view the tree before everything is loaded async (or duplicating the routing logic to allow for relevant chunks to be loaded at the top level).

[syranide]

You are right that the require.ensure is the way to go, I guess our ultimate issue was that it should be linked to the route currently being visited so is directly tied to the router. With react-router being component based that ties it to the render tree. Without my hack above we're left fighting a way to view the tree before everything is loaded async (or duplicating the routing logic to allow for relevant chunks to be loaded at the top level).

I'm not intimately familiar with react-router, but I still imagine it simply being a case of setRoute(...) => require.ensure([], function() { require(...); setRoute(...); }) which really isn't practical so you introduce another level of indirection. A map of routes and the async require.ensure loader for each. Write a helper function asyncSetRoute(...) { loadRoute(route, function() { setRoute(...); }}, now you call asyncSetRoute instead and it will defer updating the router until everything is ready.

Pseudo-code and kind of generic, but that seems like the overall approach to me. Maybe react-router should provide this, maybe not... maybe it's ideally provided as an external helper, I'm not sure.

[NickStefan]

So after hours of research, I'm just now confirming that server side rendering is _impossible_ unless you feed everything from the top down (?).

Possible short term solutions:
A. Pre-fill a store and make server side loading synchronous

B. Feed everything from the top as one data input after async fetching your one data object.

re: 'A'. This wont work unless you already know what your render structure will look like. I need to render it in order to know my various collections/models/api dependencies. Also, how do I have the fetching be async on the client, but sync on the server, without having two different APIs?

re: 'B'. Basically the same problem as above. Are people having to make little dependency JSONs to go with each of their routes?

Are there any other short term solutions while we wait for a React supported solution? Any user land forks or plugins? https://www.npmjs.com/package/react-async ?

[gaearon]

@NickStefan

I don't understand the problem. :-(

1. Use Flux or Flux-like container (Alt, Redux, Flummox, etc) where stores aren't singletons.
2. Define static methods like fetchData on your route handlers that return promises.
3. On server, match route to components, grab fetchData from them and wait for them to complete before rendering. This will prefill your Flux or Redux store instance. Note that store instance is not a singleton—it's bound to a particular request, so requests stay isolated.
4. When the promises are ready, render synchronously with the store instance you just prefilled.

Here's a good tutorial for Redux describing this approach: https://medium.com/@bananaoomarang/handcrafting-an-isomorphic-redux-application-with-love-40ada4468af4

[NickStefan]

@gaearon I apologize if I confused you. Thank you for response. From your list, it sounds like I am correct in assuming the solution to server data dependency is to only ever define data needs at your root component (static methods / the article you linked to). If your data dependencies are defined at the root, it's much easier to pre-fill stores etc.

This is good flux practice, but isn't it potentially limiting? If I add a small component at the bottom of your view tree that needs some very different data, I then need to go edit the data dependencies at the root.

What I'm asking for is a way for deeply nested components to define asynchronous data needs.

If I have to add their needs to the root component, aren't I coupling the root to that one sub components needs?

[koistya]

@NickStefan with react-routing, for example, the async data fetching looks like this:

import Router from 'react-routing/lib/Router';
import http from './core/http';

const router = new Router(on => {
  on('/products', async () => <ProductList />);
  on('/products/:id', async (state) => {
    const data = await http.get(`/api/products/${state.params.id`);
    return data && <Product {...data} />;
  });
});

await router.dispatch('/products/123', component => React.render(component, document.body));

[max-mykhailenko]

I think that it's routing problem, react should only render page, not manage and request render data

[wmertens]

Using @graphql from Apollo makes data loading super simple, and the React-Router v4 component-based API makes for simple composable routing. Both are orthogonal to application state in Redux.

In order to do single-pass streaming rendering which waits for data fetches, it should be possible for render() to return a promise (on the client you just do as now, only on the server you return a promise).

Then @graphql can simply return a promise for the render() after data being loaded; all the rest is just plain React.

[gaearon]

A few months ago I gave a talk at JSConf Iceland that describes the upcoming async rendering features in React (see the second part): https://reactjs.org/blog/2018/03/01/sneak-peek-beyond-react-16.html. This is about client-side data fetching.

Now @acdlite gave a talk about how the same concepts can apply to enable asynchronously waiting for data on the server in React components, and progressively flushing down the markup as it’s ready: https://www.youtube.com/watch?v=z-6JC0_cOns

Hope you’ll enjoy watching these talks! I think that in a year or so we might be able to close out this issue and have an official strategy for this.

[steve-taylor]

It's possible to use ReactDOMServer.renderToStaticMarkup to walk the tree, place a bookmark where it reaches asynchronous dependencies (typically loading data from an API), and continue walking the tree as those dependencies are resolved, until the whole tree is resolved, and then do a final ReactDOMServer.renderToString to actually render to HTML, which will be synchronous because all the dependencies are now resolved.

I've taken this approach in react-baconjs: render-to-html.js. It was inspired by a comment @gaearon made in another issue recommending such an approach.

[davnicwil]

@steve-taylor yeah, this works. It's also the workaround I use in react-frontload which is a more general purpose solution to this which will work in any React app.

As you say essentially it's just faking asynchronous rendering by running synchronous rendering logic twice and waiting for all the data-loading promises you hit on the first render to resolve before you run the second.

It works well enough despite obviously being a bit wasteful (output of first render is more than just promises, it's also the actual HTML which is just thrown away). Will be amazing when true async server rendering makes it into React.

[steve-taylor]

@davnicwil nice work! I have thought about generalising the specific async SSR solution. Does react-frontload handle unlimited async depth? Asking because you said “twice”.

[davnicwil]

@steve-taylor cheers! Yep, if by depth you mean depth of component in the tree, it's unlimited. It lets you declare a data loading function on the Component itself (with a Higher Order Component) and then when that's encountered on the first render it's run and the promise is collected.

What won't work is when there are further child components that also load data asynchronously that would then be rendered dynamically depending on the result, if that makes sense. It just means the app has to be structured so that there's not this kind of nesting, which I've found in practice is not really a massive limitation. In fact in many ways it's better UX-wise because of course nested async logic means serial requests, and longer wait times, whereas flattening means parallel requests.

That said the nesting problem (indeed maybe this whole async server rendering problem) may be solved with the Suspense stuff coming into React in the near future. 🤞

[gaearon]

FYI we’ve started work on this.

https://reactjs.org/blog/2018/11/27/react-16-roadmap.html#suspense-for-server-rendering

[steve-taylor]

Awesome @gaearon!

@davnicwil react-baconjs supports unlimited depth.

[ivan7237d]

@gaearon I wonder if this would make possible support for observables in create-subscription, so that I can convert an observable firing JSX into a react component?

I'd love to have this feature for two reasons. First, in my app the state is stored in a web worker. So while retrieving bits of that state which are required by the component is an async operation, it takes like 5ms and it doesn't make sense for React to render anything while waiting for the data. Second, right now there's no universal way to convert an observable to a component: if your observable emits the first value synchronously, then what you do is subscribe to get that value, then unsub, then subscribe again to listen for changes (that's the Replay subject example in create-observable docs). And if it emits the first value asynchronously, then you initially render null and listen for changes.

[davnicwil]

@steve-taylor react-frontload now also supports unlimited depth of component nesting, with the 1.0.7 release.

Hope this library is useful for some people landing on this thread - if you're looking for an async data loading solution that works on client / server render, with very minimal integration effort, you should check it out.

[RyanRoll]

We had encountered this problem before as our app was built with React Hooks, and then we created a package react-use-api to solve it, which is a custom hook that fetches API data and supports SSR. I hope the package can help people in need.

However, we are still looking forward to waiting for an official solution, Just like react-frontload says, there's no built-in way to wait around for async data loading to happen once render begins currently.

[gaearon]

We've made more progress on this.

https://reactjs.org/blog/2020/12/21/data-fetching-with-react-server-components.html

[wmertens]

Summary of my understanding of Server Components as they relate to this topic:

You'd have an SSR setup that renders a root with Server Components. A request would be rendered and result in a stream of elements, depending on the Suspense mechanism in the Server Components.

The SSR can convert this to HTML and stream it to the client, pausing the stream every time it encounters a placeholder. The end result is SSR only requiring a single pass and providing bytes to the client as soon as they are available 🎉.

As you can see, Server Components are crucial to make this work. This means that if you want your app to work like before, as an SPA without further Server Components calls, you need to return Client Components that get their initial state from the SSR and perform data fetches on state changes.

[gaearon]

Here’s the other part that does the actual streaming SSR (work in progress, not done yet). In this world Server Components is where you do data fetching, and then the new HTML renderer turns that stream into an HTML stream that can progressively render as soon as we have some content — even before all data has loaded.

#20970

[gaearon]

Today, we're announcing React Labs — a new video series with technical deep dives with the React team members. Our first video is a Server Components Architecture Q&A deep dive. We hope you enjoy it! (It's very related to the topic in this thread.)

[gaearon]

Yesterday, we published The Plan for React 18.

tldr:

• We’ve started work on the React 18 release, which will be our next major version.
• We’ve created a Working Group to prepare the community for gradual adoption of new features in React 18.
• We’ve published a React 18 Alpha so that library authors can try it and provide feedback.

React 18 will include a lot of foundational work for Suspense. This includes a brand new streaming server renderer which uses the <Suspense> boundaries to stream HTML and to hydrate the page in chunks, dramatically improving responsiveness.

[gaearon]

React 18 is out with the new streaming renderer.

In principle it already supports asynchronous data fetching (producing rendering results as a stream). However, there are still questions about how exactly to transfer data for hydration to the client. So we shouldn't close this yet. The architecture is there but it's not quite usable yet. (However, if you're a framework author, you might be able to start tinkering with it.)

[insaaFusion]

Use useLayoutEffect hook