Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-implement tree to work with giant structures #40

Merged
merged 35 commits into from
Nov 4, 2020

Conversation

Lodin
Copy link
Owner

@Lodin Lodin commented Nov 4, 2020

Fixes #11.
Fixes #30.

This is the first PR on the road to version 3. It introduces significant changes (including breaking) for the library.

Common description

The virtual tree system was completely rewritten to fit the new challenges of big tree structures. According to #11 and #30, changing the openness state in trees with about 1 million nodes freeze the UI for several seconds. With this rewrite I'm going to solve the problem. It provides a lot of tools to work with giant trees comfortably.

  • The building process is now divided with the openness state change process. It allows applying different performance tweaks for both of them.
  • During the building process the component creates an internal representation of the tree. It means that the component requires treeWalker method only for initial tree building and won't use it for openness state changes.
  • Now the component is able to use requestIdleCallback during the build to reduce UI freezing for big trees.
  • The component also supports async actions like loading branches on the toggle button click. You can combine it with requestIdleCallback as well.

Breaking changes

treeWalker

The treeWalker function has the completely different shape now.

Old treeWalker worked for both initial tree building and changing node openness state:

Old treeWalker
function* treeWalker(  refresh
) {
  const stack = [];

  stack.push({
    nestingLevel: 0,
    node: rootNode,
  });

  // Go through all the nodes adding children to the stack and removing them
  // when they are processed.
  while (stack.length !== 0) {
    const {node, nestingLevel} = stack.pop();
    const id = node.id.toString();

    // Receive the openness state of the node we are working with
    const isOpened = yield refresh
      ? {
          id,
          isLeaf: node.children.length === 0,
          isOpenByDefault: true,
          name: node.name,
          nestingLevel,
        }
      : id;

    if (node.children.length !== 0 && isOpened) {
      for (let i = node.children.length - 1; i >= 0; i--) {
        stack.push({
          nestingLevel: nestingLevel + 1,
          node: node.children[i],
        });
      }
    }
  }
}

The new treeWalker is only for the tree building. The Tree component builds and preserves the tree structure internally.

New treeWalker
// This function prepares an object for yielding. We can yield an object
// that has `data` object with `id` and `isOpenByDefault` fields.
// We can also add any other data here.
const getNodeData = (node, nestingLevel) => ({
  data: {
    id: node.id.toString(),
    isLeaf: node.children.length === 0,
    isOpenByDefault: true,
    name: node.name,
    nestingLevel,
  },
  nestingLevel,
  node,
});

function* treeWalker() {
  // Here we send root nodes to the component.
  for (let i = 0; i < rootNodes.length; i++) {
    yield getNodeData(rootNodes[i], 0);
  }

  while (true) {
    // Here we receive an object we created via getNodeData function
    // and yielded before. All we need here is to describe its children
    // in the same way we described the root nodes.
    const parentMeta = yield;

    for (let i = 0; i < parentMeta.node.children.length; i++) {
      yield getNodeData(
        parentMeta.node.children[i],
        parentMeta.nestingLevel + 1,
      );
    }
  }
}

recomputeTree

The recomputeTree method has been completely re-worked. Now it is way more powerful than its predecessor, and the main role in this change is played by subtreeCallback function that applies to all descendants of the selected node. With it, you can do a lot of things including emulation of the old useDefaultOpenness and useDefaultHeight options.

Old recomputeTree
treeInstance.recomputeTree({
  opennessState: {
    'node-1': true,
    'node-2': true,
    'node-3': false,
  },
  refreshNodes: true,
  useDefaultOpenness: false
});
New recomputeTree
treeInstance.recomputeTree({
  'node-1': true,
  'node-2': {
    open: true,
    subtreeCallback(node, ownerNode) {
      if (node !== ownerNode) {
        node.isOpen = false;
      }
    }
  },
  'node-3': false,
});

Props

To allow working with requestIdleCallback and async actions, the PR introduces the following component properties.

async: boolean

This option allows making the tree asynchronous; e.g. you will be able to load the branch data on the node opening. All it does under the hood is preserving the tree state between tree buildings on treeWalker update, so the user does not see the tree resetting to the default state when the async action is performed.

If it is combined with the placeholder option, the tree re-building won't be interrupted by showing the placeholder; it will be shown only at the first time the tree is building.

placeholder: ReactNode | null

This property receives any react node that will be displayed instead of a tree during the building process. This option should only be used if the tree building process requires too much time (which means you have a really giant amount of data, e.g. about a million nodes).

Setting this option enables the requestIdleCallback under the hood for browsers that support this feature. For other browsers the original scenario is applied; no placeholder will be shown.

Using this feature allows avoiding UI freezes; however, it may slightly increase the time spent for the building process.

If you have an asynchronous giant tree and want to use profits of requestIdleCallback but don't want placeholder to be shown on the first render (that is probably quite small because all other data will be loaded asynchronously), set placeholder to null. No placeholder will be shown on the first render but the requestIdleCallback building will be enabled and allow avoiding freezes on tree re-building when tree becomes bigger.

buildingTaskTimeout: number

This option works in tandem with the placeholder option. With it, you can set the task timeout for the requestIdleCallback. The buildingTaskTimeout will be sent directly as the requestIdleCallback's timeout option.

This allows dropping metadata after the tree is built which reduces memory consumption.
Since this package is oriented to be as small as possible it is better to avoid ES6-only code. So instead of for..of it is better to use the old for loop.
# Conflicts:
#	__tests__/FixedSizeTree.spec.tsx
#	__tests__/VariableSizeTree.spec.tsx
#	src/Tree.tsx
#	src/VariableSizeTree.tsx
#	src/utils.ts
# Conflicts:
#	__tests__/FixedSizeTree.spec.tsx
#	__tests__/VariableSizeTree.spec.tsx
#	src/Tree.tsx
@Lodin Lodin added this to the 3.0 milestone Nov 4, 2020
@Lodin Lodin self-assigned this Nov 4, 2020
@Lodin Lodin force-pushed the fix/performance-on-big-data branch from 9345263 to 186ad4d Compare November 4, 2020 16:37
@sonarqubecloud
Copy link

sonarqubecloud bot commented Nov 4, 2020

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 0 Security Hotspots to review)
Code Smell A 1 Code Smell

97.0% 97.0% Coverage
0.0% 0.0% Duplication

@Lodin Lodin merged commit 2de16ae into master Nov 4, 2020
@Lodin Lodin deleted the fix/performance-on-big-data branch November 4, 2020 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Collapse/un freezes when there are a lot of children nodes Reduce necessity of tree full tree walk
1 participant