Thoughts on view recycling

[lelandrichardson]

Collection View Recycling in React Native

Motivation

View Recycling is a really important optimization strategy for both the iOS and Android platforms.
Highly related to the "ListView Problem" in React Native, view recycling on both platforms is the
most common way to get high-performing screens displaying lots of data.

The platform solutions for this come in the form of:

iOS

• UITableView
• UICollectionView

Android

• RecyclerView

Most of the solutions for lists in react native employ performance optimizations on the JS / React
side of the bridge, and have been somewhat successful with this. The latest FlatList implementation
has shown to be performant enough for a lot of applications.

There is still a limit though. This is still not recycling any views, and by allocating / deallocating
views rapidly on the native side, we can run into scroll performance and memory pressure issues.

I believe if we are to finally "put this issue to bed", the solution will need to involve some
ingenuity on both the Native and JS side of the bridge.

Current solutions and reading

A lot of work has been put into the "ListView Problem" on React Native already. For context, check
out:

• Good discussion on the design decisions behind ListView
• RFC for new FlatList
• Brent Vatne building li.st at ReactEurope 2016
• react-native-tableview by Pavel Aksonov
• FlatList commit in RN Core
• BindingListView from Wix
• Recycling Rows For High Performance React Native List Views
• React Native ListView Performance Revisited — Recycling Without the Bridge
• react-native-sglistview by Shaheen Ghiassy

Goals

• minimize allocation / deallocation of native views during scrolling
• don't allocate views that aren't visible
• minimize react component allocation / deallocation
• minimize data serialized across the bridge
• don't rely on JS thread for any scroll interaction
• possible to scale out to extremely large datasets
• support heterogenous lists

"Layout" Components

"Layout" native components is a term I'm creating for purposes of this RFC. A
Layout component is a native component whose direct children are expected to be
ViewPool components. The Layout component will draw its actual children (the child
views in the actual view hierarchy) from their children pools. Layouts themselves
determine how the views are used and laid out. For simplicity, I'm showing a
VerticalLayout component here, which would be your common "vertical list" layout.
Other layouts could be horizontally laid out components, grids, masonry style grids,
etc. The Layout components are in charge of emitting an onUpdate event when the
need for new views not available in its child view pools. When this event is fired,
it is expected that the owning component will then rerender (in the React sense) the
child ViewPools so that they satisfy the new needs.

const VerticalLayout = requireNativeComponent('VerticalLayout', {
  onUpdate: Function,
  totalItems: number,
  offset: number,
  limit: number,
  types: [ComponentKey],
  children: [ReactElement<ViewPool>],
});

ViewPool Components

This is a general purpose native component called a ViewPool. A ViewPool is a native
component that doesn't actually attach any views to the hierarchy. It merely intercepts
insertReactSubView and holds those views in memory for someone (a Layout) to consume.
Each ViewPool has roughly "homogeneous" views that it holds on to... at least as
can be guaranteed by the guarantee that all of the views were rendered at the top level
by the same composite react component. This component is for the most part just a way to
have a pool of react-rendered views without modifying the react native architecture too
much.

const ViewPool = requireNativeComponent('ViewPool', {
  type: string,
  children: any, // React elements of the type that `type` signifies here
});

Basic Types

// a string identifier for the react component type of the pool
type ComponentKey = string;

// Essentially an array of indexes annotated with a ComponentKey type
type Pool = {
  key: ComponentKey,
  indexes: Array<number>, // a list of indexes to use to retreive the items by index
}

Public Component Implementation

With the ViewPool and VerticalLayout building blocks, we are able to assemble a react component
with the public API that we want. Below is a rough implementation for one, though not complete.

class RecyclingVerticalListView<T> extends React.Component {
  props = {
    // the actual data source. We could provide an alternative version of this component
    // that didn't need this prop, but instead just had an `itemForIndex` method or
    // something, but I'm not going to worry about that for now as this is simpler to
    // visualize.
    items: [T],

    // the initial number of items to render
    initialItemCount: number,

    // provided a ComponentKey, return the correct react component.
    componentForKey: (key: ComponentKey) => Function,

    // provided an item, return the ComponentKey we want to use
    keyForItem: (item: T) => ComponentKey,

    // provided an item, return the props to be passed into that given item's component
    itemToProps: (item: T) => any,
  };
  state = {
    pooledChildren: [Pool],
  };
  onUpdate(
    pooledChildren: [Pool],
    limit: number,
    offset: number,
  ) {
    this.setState({ pooledChildren, limit, offset });
  }
  constructor(props) {
    super(props);
    this.state = {
      pooledChildren: null,
      offset: 0,
      limit: props.initialItemCount,
    };
  }
  render() {
    const {
      items,
      initialItemCount,
      componentForKey,
      keyForItem,
      itemToProps,
    } = this.props;
    let {
      pooledChildren,
      offset,
      limit,
    } = this.state;

    const types = items.slice(offset, limit).map(keyForItem);

    if (pooledChildren === null) {
      // first render, so we have to create the initial state ourselves.
      pooledChildren = poolItems(keyForItem, items.slice(0, initialItemCount));
    }

    return (
      <VerticalLayout
        totalItems={items.length}
        types={types}
        offset={offset}
        limit={limit}
        onUpdate={this.onPooledChildrenUpdated}
      >
        {pooledChildren.map(({ key, indexes }) => {
          const ItemComponent = componentForKey(key);
          return (
            <ViewPool
              key={key}
              type={key}
            >
              {indexes.map((index, i) => (
                <ItemComponent
                  key={i}
                  {...itemToProps(items[index])}
                />
              ))}
            </ViewPool>
          );
        })}
      </VerticalLayout>
    );
  }
}

In this case, if we have O(n) items, but O(k) of them are visible on the screen at any given
time, our render method is always O(k), and the data that we serialize across the bridge at
any given time is always O(k). Moreover, we are not serializing entire data models over the
bridge, but rather just really light-weight arrays of indexes and keys.

Additionally, consider the scenario where all of the ItemComponents implement a
shouldComponentUpdate function. In the case where an item of a given type goes offscreen, and
a new item of the same type comes onscreen, there will be NO react components mounted or unmounted,
and instead just a single componentWillReceiveProps that gets fired. This is ensured by the
fact that the keys we use for the ItemComponents are not actually the "stable ids" of the data
source, but rather

      React View Hierarchy        ───────────▶     Native View Hierarchy

┌─────────────────────────────────┐           ┌─────────────────────────────┐
│<VerticalLayout />               │           │0                            │
│┌───────────────────────────────┐│           │                             │
││<ViewPool type="A" />          ││           │            <A />            │
││┌─────────────────────────────┐││           │                             │
│││0                            │││          ┌│─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─│┐ ◁
│││                             │││           ├─────────────────────────────┤  │
│││            <A />            │││          ││1                            ││ │
│││                             │││           │                             │  │
│││                             │││          ││            <A />            ││ │
││└─────────────────────────────┘││           │                             │  │
││┌─────────────────────────────┐││          ││                             ││ │
│││1                            │││           ├─────────────────────────────┤  │
│││                             │││          ││2                            ││ │
│││            <A />            │││           │            <B />            │  │
│││                             │││          ││                             ││ │
│││                             │││           ├─────────────────────────────┤  │
││└─────────────────────────────┘││          ││3                            ││ │ Viewable
││┌─────────────────────────────┐││           │                             │  │  Screen
│││3                            │││          ││            <A />            ││ │
│││                             │││           │                             │  │
│││            <A />            │││          ││                             ││ │
│││                             │││           ├─────────────────────────────┤  │
│││                             │││          ││4                            ││ │
││└─────────────────────────────┘││           │            <B />            │  │
││┌─────────────────────────────┐││          ││                             ││ │
│││7                            │││           ├─────────────────────────────┤  │
│││                             │││          ││5                            ││ │
│││            <A />            │││           │            <B />            │  │
│││                             │││          └│─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─│┘ ◁
│││                             │││           ├─────────────────────────────┤
││└─────────────────────────────┘││           │6                            │  │
│└───────────────────────────────┘│           │            <B />            │  │
│┌───────────────────────────────┐│           │                             │  │
││<ViewPool type="B" />          ││           ├─────────────────────────────┤  │
││┌─────────────────────────────┐││           │7                            │  │
│││2                            │││           │                             │  │ Render Ahead
│││            <B />            │││           │            <A />            │  │   Distance
│││                             │││           │                             │  │
││└─────────────────────────────┘││           │                             │  │
││┌─────────────────────────────┐││           ├─────────────────────────────┤  │
│││4                            │││           │8                            │  │
│││            <B />            │││           │            <B />            │  │
│││                             │││           │                             │  ▼
││└─────────────────────────────┘││           └─────────────────────────────┘
││┌─────────────────────────────┐││                          │
│││5                            │││                          │
│││            <B />            │││                          │
│││                             │││                          ▼
││└─────────────────────────────┘││
││┌─────────────────────────────┐││
│││6                            │││
│││            <B />            │││
│││                             │││
││└─────────────────────────────┘││
││┌─────────────────────────────┐││
│││8                            │││
│││            <B />            │││
│││                             │││
││└─────────────────────────────┘││
│└───────────────────────────────┘│
└─────────────────────────────────┘

Native Implementation

I'm still trying to figure this part of things out. I'll add some notes on this shortly.

The gist of it is:

ViewPool Component

• Use insertReactSubview and removeReactSubview to intercept view insertions / deletions and hold
  onto them in memory.

Layout Component

On iOS:

• will probably implement UICollectionViewLayout
• will probably implement UICollectionViewDelegate
• will probably implement UICollectionViewDataSource
• will probably implement UICollection​View​Data​Source​Prefetching
• will dequeue cells from the child viewpools
• will use the prefetching behavior to message back to JS thread which new views it needs

On Android:

• will probably implement RecyclerView.LayoutManager
• will probably implement RecyclerView.Adapter

Known problems with this approach

This implementation is not without issue, though I think the technique can be tweaked to handle
a lot of these use cases, and configuration options provided that minimize the problems.

Item components do not maintain state

Since the React components themselves are reused here, if they are stateful, that state will
persist across different "items" in terms of the data source since they are being reused for
new items. It's worth noting that the recycling on the React side and the Native side are somewhat
decoupled, and one could easily allow for this as a configurable prop of the RecyclingVerticalListView
component. The only thing you would need to change would be to make the keys of the ItemComponents
use a "stable id" from the data source. This could just be a getItemKey prop or something. If
you don't actually need the components to be stateful (which might often be the case), being able to
reuse the component instances is a valuable performance optimization.

Need to keep up with scrolling

The native side can't ask for new views in its pool synchronously, so it will have to be smart
about how many views it allocates in the pool, and will probably need a healthy
"render ahead distance". This is really going to be a problem with any RN listview implementation
that does not want to block scrolling, and doesn't want to render the whole list at once. The
options here are to 1) block the main thread, 2) show empty cells while we wait for the JS, 3)
block the scrolling.

There is an additional nuanced problem here in that we are reusing views, and so there might be
some cases where we are scrolling too fast for the JS to apply the new props for the views that
are being recycled, and so you see a different cell's rendered view in a new cell's position. This
can be handled (we can know at runtime if the view has been repopulated with the right data yet
or not), but all this does is put us back in the situation above where we have to choose what to
do in this situation.

More things to think about

Decoration Views

Both UICollectionView and RecyclerView have concepts of "decoration views". Perhaps we should
try to figure out a clean way to abstract that out to react so we can further leverage these
concepts.

Drag and drop / reordering

Both platforms provide some support for dragging / dropping reordering and it seems like this would
be pretty easily integrated.

Unique Layout components

I haven't fully thought about this, but I think some pretty unique Layout components could be made.
The "Masonry" grid is one such idea, another is an "Excel" like spreadsheet grid. There must be many
more. Leveraging this type of generic view recycling could be pretty interesting.

Minor Details

Some methods above are moved down here for brevity, but rough implementations are here if
more understanding is desired:

function poolItems<T>(getKey: ((item: T) => ComponentKey), items: Array<T>): Array<Pool> {
  const result = [];
  const pools = {};
  for (var i = 0; i < items.length; i++) {
    const item = items[i];
    const key = getKey(item);
    if (pools[key] !== undefined) {
      const pool = [];
      pools[key] = pool;
      result.push(pool);
    }
    pools[key].push(i);
  }
  return result;
}

[talkol]

Please add a couple of relevant blog posts to the "Current Solutions and Reading" section:

• Recycling with UITableView and binding from JavaScript

• Recycling with UITableView and binding from native

[talkol]

On The Question of Binding

One of the main architecture questions we need to discuss is the question of binding.

First, let's define what "binding" is. Assuming a solution supports view recycling of some sort, binding takes place when a recycled view is connected to its new row data.

For example, if we have a Contact List that shows the name of every contact in a Text component. When a cell is recycled, the binding is the action of changing the text in the component to the value relevant for the row we want to display next inside this cell.

Why is binding so important?

Binding IMO is the main performance bottleneck for scroll throughput. Since the scroll action itself is completely native and we can assume that we have enough views ready to be recycled - the only thing holding back our scroll throughput is whether we can bind fast enough.

Before we jump into the implementation of binding, we should discuss general approaches. One big question is this - do we bind from JavaScript or from native?

Steps relevant for the binding process

We might skip some of these steps for performance reasons, but in general, we need to think about the following when binding:

• Running JavaScript render for the new row
• Updating the native view / shadow view with the new props (the actual native update)
• Relayout with the new content (since the new props might change sizing)

Binding from JavaScript - motivation

The main benefit for binding from JavaScript is that it's the most convenient approach for the developer (user of the List library) and the most flexible from a React/JSX point of view. The developer will have some sort of render function that can be used to customize each row according to its data source (as props). This render function will re-run every time we need to bind a new row.

The main issue with binding from JavaScript is performance. The bind action runs on the JavaScript thread and has to pass in both directions over the bridge. This means we are limited in the rate of renders. This process is also asynchronous, meaning the native scroll view might need a new view bound immediately, but one could not be provided.

The main places we can expect performance issues is when you scroll really fast. Another place is where we can't anticipate in advance which rows will be needed, for example when pressing on the status bar (scroll to top).

Most of the performance issues can be mitigated by intelligently pre-rendering rows. So when the native needs them, they're already ready.

Binding from native - motivation

The biggest motivation is performance. If we can successfully bind from native, the scroll performance will be on the same level as the pure native. This is because during scroll, there will be no passes over the bridge and no work to be done by the JavaScript thread which might be queued up.

The biggest problem with binding from native is that it's more difficult to implement and the library will be more restrictive to the developer using it. This is due to the binding having to be fully declarative. Unlike with binding from JavaScript, the developer will not be able to place imperative JavaScript in the render row function.

How can we implement binding from JavaScript

There are 2 main approaches: using React Root views and using regular React children.

In the React Root views approach, when we recycle we need to potentially allocate a new view (if one is not available for recycling). This can be done by creating a new React Root from native and defining the internal component to have the class we want. When we need to bind, we will simply change the props for this React Root. There is API for this (the property appProperties). A working example using this approach is available here.

The benefit of this approach is that it's much simpler and straightforward. The code is much easier to understand. There might be a little overhead of having so many React Roots. If you're concerned with this, the next approach might be better. I do recommend to do a performance comparison because the added complexity might not be justified.

In the React children approach, when we recycle we need to potentially allocate a new view (if one is not available for recycling). This can be done by pre-rendering a pool of React children in JavaScript and saving them in a native pool by overriding insertReactSubview. If we need a new view, we will take it from the pool. When we need to bind, we can simply send a native Event to JavaScript with the rowId we want to bind and the key of the React view we want to place it in. When the JavaScript code will handle this event, it will change state and place the new indexes in the props of the relevant React child. This will cause it to re-render by React and eventually change in native. A working example using this approach is available here with a blog post explaining it here.

One of the biggest issues with these two approaches is canceling pending renders. Without Fiber, whenever we queue a view for binding in JavaScript, this binding cannot be canceled. If the user scrolled fast enough and this view needs to be re-bound to a different row, this new binding will enter the queue. The result will be that the view will bind twice to one row and then another immediately resulting in a flicker. It might be possible to fix this issue with Fiber. Another option is to keep the render-ahead window large enough so this rarely happens.

How can we implement binding from native

This is much more complicated. Currently researching this. My current progress is detailed in this blog post. The biggest missing piece is handling re-layouts.

Conclusion - so how do we bind?

We'll probably need to provide both approaches. Both are required for different scenarios.

[lelandrichardson]

@talkol thanks for your thoughts!

So thinking about this a bit more with your feedback...

1. Event batching
   With your linked tableview-based experiment, we are sending a separate onChange event for every single queued/dequeued cell, which i think could overwhelm the bridge, and on fast scrolls is maybe causing a lot of immediate re-renders and wasting CPU time. I wonder if you implemented a batched event that was in lockstep with the main thread that you could send fewer events over the bridge. Related to this is your idea of "canceling" renders. I think we could potentially add some primitive operations on the bridge (if it's not already possible, which it might be) that would allow us to queue an event and ensure that the data being sent over (ie, which rows need to be rendered right now) is as fresh as it can be at the moment of the queue being flushed.

2. UICollectionView / RecyclerView rather than UITableView
   Going along with the above, all of the experiments that you've done thus far are with the UITableView. I see you have a note here about having a cell "buffer" for the UITableView so that it is dequeuing cells that are offscreen before they come on screen. I think UICollectionView and RecyclerView might be handling these types of things at a

3. fiber scheduling
   I think for re-rendering rows during scroll we can utilize the "Animation" priority in fiber, which might help a bit. After all, this is a scheduling problem at its core. cc @sebmarkbage do you have any thoughts on this?

4. Blocking main thread
   Going along with the above, i would be interested in experimenting with implementing some waiting on the main thread for the bridge queue to be flushed. I think if we could tap into fiber's priorities while scrolling, we could reasonably prevent long-running operations from happening (from for example a network request coming in while we are scrolling). If we block the main thread collectionview / recyclerview should properly maintain scroll momentum and such.

5. Native Binding
   I think this is an interesting path to develop, but I think the fact that it requires serializing the whole dataset to native to be a bit of a bummer. I think this is practical for some use cases, but often won't be, and initial render times are pretty important. I think the trick here will be ergonomics. The API will need to feel as natural as possible. Handling this binding with heterogeneous lists seems like the external API might be pretty complex.

[talkol]

Event Batching

Interesting idea. If I'm not mistaken the entire bridge does batching by default on the message level. So sending multiple events might already be batched together as a single bridge frame. Not 100% if it's batched in both directions though. Could be interesting to try.. maybe we should always render N views at once and send one event for all of them. Worth a try. I wish we had a good way to measure end to end performance so we can see how much it improves.

Regarding canceling renders, do you know if we can use Fiber to do that? This is also a scheduling problem at the core.

UICollectionView vs UITableView

I was always under the impression that UITableView is the more performant of the two because it's simpler and more predictable (easier for Apple to optimize). Apple keeps updating it with new API all the time, so they keep working on it.. for example prefetching in iOS 10. What API do you see on UICollectionView that can give us a performance boost?

Fiber

We should definitely try it out. Which version of RN will have React 16? is it 0.44?

Native Binding

I don't think sending the data source is a big deal. We can break it down to chunks if it's enormous.. but I think its effect is pretty negligible. In most of the cases this data source is coming from native anyways. Consider downloading the data source from server using HTTP, all the JSON data from the HTTP request is passing on the bridge from NSUrlConnection to be parsed in JS. Sending it to the other side is just like making another 10Kb HTTP GET, it's not somewhere we really try to cut down on. The same if it was coming from a file or some persistent storage, this would also originate in native and be sent to JS to be handled by business logic. So in a way, it's already probably being sent back and forth in its entirety.

The insane lists in native, where you have huge data sources are usually backed by something like CoreData that provides persistency, caching and juggling disk vs memory. We could add a similar layer to this which will be fed with data and will be a super performant backend for the list. It's a bit overkill though :)

[talkol]

Another aspect I want to bring up for discussion is variable heights.

It's challenging to implement with UITableView and UICollectionView because they want to know the height for every row synchronously.

One option is to implement a delegate function that calculates this height before binding the rows, but this will be difficult for us because we need render and layout to know the height.

Another option is to use auto layout, that's what we use in one of the example. The difficulty there is that auto layout works synchronously. If the render and layout are asynchronous, it won't work.

This means the main option I see is updating specific cells in the list (UITableView or UICollectionView) after layout. react-native-uitableview updates the entire table view when this happens which is inefficient. Updating individual cells will work, but there are issues if supporting adding and removing of cells. If a cell is added or removed and the layout happens async around it, all the indexes will change and this could become hell to manage.

[clarle]

Pre-fetching implementation

@talkol - Most of the performance improvements on UITableView should be available on UICollectionView as well. For instance, pre-fetching for UICollectionViews on iOS 10 is available as UICollectionViewDataSourcePrefetching.

On the Android side, API 25 has RecyclerView.LayoutManager.LayoutPrefetchRegistry which operates in a similar way.

I think these two will probably be a big win on the render-ahead implementation. I'd take advantage of them even if they're not supported by older versions.

Variable heights

Agreed with you 100% on the iOS side. I can't think of a great solution there without some sort of tradeoff.

[SudoPlz]

Amazing discussion.
I wonder if any decisions have been made.

View recycling is something that's really missing from react-native, and I can't wait enough to use it.