Add new ReactPerf

[gaearon]

This is a work in progress on implementing new ReactPerf as discussed in #6015.

Per @sebmarkbage’s request, I decided to focus on removing dependencies on internal method names. Data will be explicitly passed to the perf tool from React methods, and we will attempt to not rely on the execution order.

Rather than refactor the existing code, I chose to create a new tool side by side so I can compare their output until I’m confident about correctness. I will later add PROFILE feature gates to the calls.

• Add barebones implementation of new ReactPerf
• It should count totalTime for flushes
• It should not count totalTime twice for nested flushes (fixes a minor bug in ReactPerf)
• It should not rely on the rendering and mounting stack matching parent hierarchy
• Decide how inclusive measurements work
• Add a safety mechanism to avoid accidentally forgetting endMeasure()
• It should count exclusive time for every lifecycle method
• It should include displayNames and other component information
• It should reconstruct the parent tree
• It should count counts and created for components
• It should count exclusive times for components
• It should count inclusive times for components based on parent tree
• Make it the DebugTool
• Make sure teams that replaced ReactDefaultPerf with wtf can keep doing so
• It should implement printDOM()
• It should implement printWasted()
• Treat stateless components correctly
• Expose the new ReactPerf as react-addons-perf
• Make sure wasted measurements are useful (something’s off right now)
• Do we want to rely on owner?
• TESTS
• Remove the old ReactPerf code
• Introduce the new PROFILE gate and put calls behind it
• Consider the implications of using WeakMap in __PROFILE__ builds
• Expose React.unstable_Instrumentation
• Make sure we have new get*() methods and deprecated printDOM() and getMeasurementSummaryMap() are still there
• Expose whatever React Native needs for systrace integration and enabling PROFILE
• Verify compatibility with React ART (e.g. add stuff like this)
• Ensure we throw a meaningful error when start() is called inside the lifecycle or, better, consider providing support for that. See also React Perf: "cannot set property totalTime of undefined"  #2095, Call Perf method in setState callback #3436, https://github.com/lostthetrail/react-ssr-perf
• New ReactPerf is correct, tested, has no effect in production, is hard to break accidentally when refactoring, and does not rely on implementation details

[facebook-github-bot]

@gaearon updated the pull request.

[facebook-github-bot]

@gaearon updated the pull request.

[facebook-github-bot]

@gaearon updated the pull request.

[gaearon]

I want to verify whether my understanding of the constraints in #6015 (comment) is correct. Here is an example I came up with. @sebmarkbage @spicyj Can you please confirm whether this is right?

Requirements

Say we’ve got three components. A includes B, which includes C and D.
Let’s see how the Perf tool receives the events over time.

Regular Recursive Batch

Components mount (or updated) recursively like they normally do. Numbers correspond to the current time, phrases to events the Perf tool receives. The columns show whether the time period is counted towards inclusive and exclusive scores of the corresponding components.

TIME                       A             B           C          D

0 (begin A)          
|                       exc+inc         
1                   
|                       exc+inc     
  2 (begin B)
  |                       inc         exc+inc
  3
  |                       inc         exc+inc
  4
  |                       inc         exc+inc
    5 (begin C)
    |                     inc           inc       exc+inc
    6 (end C)
  |                       inc         exc+inc
    7 (begin D)
    |                     inc           inc                  exc+inc
    8
    |                     inc           inc                  exc+inc
    9 (end D)
  |                       inc         exc+inc
  10 (end B)
|                       exc+inc    
11 (end A)

Funny Batch

We need to make sure that it is easy to introduce out-of-stack updates later. For example, we want a top-level update of C to count towards B and A inclusive counters.

As an extra safety measure, we will absurdly include an update of A inside the C update. This probably doesn’t make sense technically but we want to make sure the calculations are completely independent of the stack order no matter how bonkers it is.

TIME                      A             B           C          D

11 (begin C)
|                        inc           inc       exc+inc
12
|                        inc           inc       exc+inc
  13 (begin A)
  |                    exc+inc
  14 (end A)
|                        inc           inc       exc+inc
15
|                        inc           inc       exc+inc
16
|                        inc           inc       exc+inc
17 (end C)

Note how the C updates is still counted towards A and B inclusive time even though they are not on the stack. On the other hand, the A update is only counted towards A even though C is currently on the stack.

Formal Input

Relationships

A.owner = undefined
B.owner = A
C.owner = B
D.owner = B

Events

(0, begin, A)
(2, begin, B)
(5, begin, C)
(6, end, C)
(7, begin, D)
(9, end, D)
(10, end, B)
(11, end, A)
(11, begin, C)
(13, begin, A)
(14, end, A)
(17, end, C)

Desired Output

Exclusive

• exc(A) = (2-0) + (11-10) + (14 - 13) = 2 + 1 + 1 = 4
• exc(B) = (5-2) + (7-6) + (10-9) = 3 + 1 + 1 = 5
• exc(C) = (6-5) + (13-11) + (17-14) = 1 + 2 + 3 = 6
• exc(D) = (9-7) = 2

exc(A) + exc(B) + exc(C) + exc(D) = 4 + 5 + 6 + 2 = 17 = total

Inclusive

• inc(D) = exc(D) = 2
• inc(C) = exc(C) = 6
• inc(B) = exc(B) + inc(C) + inc(D) = 5 + 6 + 2 = 13
• inc(A) = exc(A) + inc(B) = 4 + 13 = 17

---

Do these calculations look right? Is this the desired output for these scenarios?

[facebook-github-bot]

@gaearon updated the pull request.

[sebmarkbage]

Yea, this looks right. I can't see how steps 13-14 could possibly happen but if it could, then yea, that would be the right call.

The point of decoupling this is so that in theory, someone else could've implemented this perf tool without changing the core. Since you're effectively mapping the names of methods 1:1, it doesn't create much decoupling. If we change the algorithm, you would still have to go change the ReactNewPerf implementation.

A good guideline is to put things that are likely to change together into the same file. So, if we can't decouple, we might as well put all of ReactNewPerf into ReactCompositeComponent etc. to make it easy to refactor.

I understand that this is your later step "It should not rely on the rendering and mounting stack matching parent hierarchy".

One suggestion that I have is that you could, have start and stop timer associated with component time in ReactNewPerf. E.g. startTime(internalInstance) and then call stopTime(internalInstance) right before the recursive call. Then you call startTime(internalInstance) again right after the recursive call completes. You only calculate exclusive time. Then at the end, the summary, you sum them up according to the tree to get the inclusive time.

That way we can restructure the order completely without changing ReactNewPerf.

[gaearon]

One suggestion that I have is that you could, have start and stop timer associated with component time in ReactNewPerf. E.g. startTime(internalInstance) and then call stopTime(internalInstance) right before the recursive call. Then you call startTime(internalInstance) again right after the recursive call completes. You only calculate exclusive time. Then at the end, the summary, you sum them up according to the tree to get the inclusive time.

We could pass something like startTime(internalInstance, 'render'), startTime(internalInstance, 'mount'), so that the profiler does not really know about the lifecycle other than that “work denoted by some arbitrary label is being done for this component”. This makes it easy to add other “types” of work later that the analysis code can interpret as desired (e.g. “A spent X ms on render, Y ms on mount, Z ms on layout”). This also makes the profiler API easier to understand because, rather than memorize which methods require Perf calls you just remember to track whatever you think is worth tracking with startTime and stopTime methods.

Is this what you’re getting at?

[sebmarkbage]

Even better!

[sebmarkbage]

Ideally the tree information would be exposed through the new devtools API, so perhaps it would be sufficient for the perf tools to use that rather than building another one specifically for perf?

[sebmarkbage]

This looks good so far. I'm going to mark this as needs-revision just because we don't want to land it until it is gated behind something.

[gaearon]

Ideally the tree information would be exposed through the new devtools API, so perhaps it would be sufficient for the perf tools to use that rather than building another one specifically for perf?

Can do that. But it’s not exposed yet, is it? Meaning I’d need to add these events to the devtool code myself.

Currently I don’t have enough context about what kind of information DevTools want. I can start by firing notifications on the devtool for every lifecycle hook with the internal instance. Would that be sufficient? ReactNewPerf would tune in to "mounted" and "unmounted" events and use this as a chance to track the owners.

If I do this, we’ll need to gate devtool API by something like __DEV__ || __PROFILE__. Right now it’s __DEV__-only.

[sebmarkbage]

Currently these events are exposed through the devtools (search for hook.emit):

https://github.com/facebook/react-devtools/blob/master/backend/attachRenderer.js

(Note that the terminology is a bit confusing there. The element refers to an internal instance in React.)

The "backend" part of the devtools is an event protocol designed by @jaredly to be sufficient to track the tree. A good start might be to mirror that API.

use this as a chance to track the owners.

I think you mean the parents. We use the term "owner" for the thing that created the element which is not the same as the thing mounting it - which is the parent. To track inclusive time you need the parent.

It is unfortunate that tracking the tree might add a lot of overhead but it is probably nice to be able to visualize this in terms of a full tree.

ReactDOMInstrumentation is only for DOM specific operations so you'd want to replicate it with a ReactInstrumentation file that does the same thing.

The idea is that we can check for if (ReactInstrumentation.debugTool) to quickly bailout if nobody is listening.

Currently the devtools work in production mode because it just uses monkey patching. That sucks for optimizations and package size. Maybe a __DEV__ || __PROFILE__ check would be better.

[sebmarkbage]

Note that a __PROFILE__ flag might be difficult to add to our internal FB build but if we start deploying the from the npm package internally, that wouldn't be an issue.

[iamdustan]

:) I am really enjoying this thread. I like to refer as the current devtools work as duck punch monkey patching since it is doing duck punch detection for ReactDOM 0.13, 0.14, 0.15, and RN.

[gaearon]

Currently these events are exposed through the devtools (search for hook.emit):

Thank you, this is very helpful.

I think you mean the parents. We use the term "owner" for the thing that created the element which is not the same as the thing mounting it - which is the parent. To track inclusive time you need the parent.

Thanks for correcting me. I thought owners should be used for the inclusive calculation but I trust you that parents make more sense here. I don’t really understand the problem well enough yet to see why.

ReactDOMInstrumentation is only for DOM specific operations so you'd want to replicate it with a ReactInstrumentation file that does the same thing.

👍

Currently the devtools work in production mode because it just uses monkey patching. That sucks for optimizations and package size. Maybe a DEV || PROFILE check would be better.

Since this is all very confusing I’ll finish the implementation first, and then we’ll decide how to proceed with the feature flags both for Perf and tree tracking.

[sebmarkbage]

then we’ll decide how to proceed with the feature flags both for Perf and tree tracking.

Sure. If you want, you can also just add a temporary constant flag (always false). That way you can incrementally land your changes in master so that you don't need to deal with merge conflicts.

[gaearon]

The idea is that we can check for if (ReactInstrumentation.debugTool) to quickly bailout if nobody is listening.

Currently, in ReactDOMInstrumentation, ReactDOMInstrumentation.debugTool is always defined because it’s a thing that lets other devtools register themselves. Is the plan to reorganize this in some way, or do it differently for ReactInstrumentation? Otherwise I don’t see how you could exit early by checking it.

[facebook-github-bot]

@gaearon updated the pull request.

[facebook-github-bot]

@gaearon updated the pull request.

[facebook-github-bot]

@gaearon updated the pull request.

[facebook-github-bot]

@gaearon updated the pull request.

[facebook-github-bot]

@gaearon updated the pull request.

[ghost]

@gaearon updated the pull request.

[ghost]

@gaearon updated the pull request.

[ghost]

@gaearon updated the pull request.

[sebmarkbage]

Can you explain a bit on why the flush batch is important? Does it matter if the implementation batches or just does the work whenever?

[gaearon]

I think this comes down to the way printWasted() heuristic works. It says “count a render as wasted if no DOM operations below were made during the same batch”. I was trying to replicate the existing behavior so this is the main reason.

I think we may want to reconsider this as part of #6632. I’d leave it as is for now though.

[ghost]

@gaearon updated the pull request.

[ghost]

@gaearon updated the pull request.

[facebook-github-bot]

@gaearon updated the pull request.