Custom React-specific metadata format

[bvaughn]

The recent DevTools "named hooks" feature has been heavily optimized since its initial launch. There are some additional optimizations that we could consider (see the first comment below) but I believe the single biggest remaining bottleneck is downloading and parsing of large source map files.

Using Facebook as a case study, over 65% of the time spent parsing hook names is downloading the source-map files:

A significant portion of this time is spent on the generating the source map on the server:

Perhaps this could be optimized further, but at some point- we'll still have to download and parse a potentially large file.

Can we avoid relying on the source map entirely?

We aren't using source maps in the typical way. (We aren't displaying the original source code.) So the majority of the data contained in the source-map is useless to us. We've experimented with extending the source map so that we didn't have to parse the original code, but the biggest bottleneck (based on testing) is actually generating and downloading the source-map so this only helps a little.

What if we pre-computed hooks metadata (during compilation) and wrote it to a separate file that gets bundled along like a source map. We could generate this metadata for the compiled code, rather than the source code, so we wouldn't need to map anything at runtime. I don't have exact figures on how much faster this would be, but I estimate it would reduce the source-map file size by an order of magnitude which could have a pretty large impact on the overall performance.

If this custom metadata file is successful, we could also store some additional (lightweight) information in it such as component display names.

Adding additional metadata to this file/format should be done in a backwards-compatible way if at all possible. Still, the metadata format should be versioned in the event that a backwards breaking change occurred.

What if this metadata was unavailable?

This would be entirely opt-in. DevTools would fall back to loading and parsing the complete source-map if this custom metadata was unavailable.

[bvaughn]

More thoughts on source-map parsing performance

This comment is a brain dump since I've been thinking about this area recently. First, here is a high level overview of how DevTools currently parses and extracts hook names:

• Loop over every hook and inits a Metadata object (initializeHookSourceAndMetadata).
• Loop over every hook and fetches files (with de-duping per URL) (async loadSourceFiles).
• Loop over every fetched source and bases64 decodes internal source-map or fetches external source-map file (async extractAndLoadSourceMapJSON).
• Serialize fetched source-map JSON (or source code itself if no source-map) to Worker.
• Loop over every hook and inits a Metadata object (initializeHookParsedMetadata).
• Loop over every hook and decodes source-map mappings and extended source-map metadata (parseSourceMaps).
• Loop over every hook and maps runtime code to original source, then parses to an AST (parseSourceAST).
• Loop over every hook and extracts name from hooks metadata (if present) or AST (findHookNames).
• Serialize extracted hook names back to main thread.

Each of the above steps is in a sequence, meaning that it blocks the next steps. (In most cases this detail probably doesn't matter, but there are times when a component uses hooks from multiple source files.)

First, here are some things I considered but think probably aren't worth doing:

• Check areSourceMapsAppliedToErrors earlier on and avoid fetching and parsing source-maps if true.
  • We might not want to do this because it probably doesn't apply to most (non-Jest) environments.
• parseSourceAST could avoid mapping runtime code to original source if extended source-maps with hooks metadata present.
  • We might not want to do this because it wouldn't affect anything until/unless we decide to roll out extended source-maps.

Here are a few things that might be worth considering though:

• Parallelize the above per file. (First map the hooks tree to a unique set of files. Then load each source-map file as soon as the source file has been loaded, and parse the AST as soon as the source-map has been loaded, etc.)
  • We might not want to do this because it's not common (based on my own testing) for there to be more than one source file per component.
• Stream source maps and decode them while they are streaming in.
  • We might not want to do this because streaming decoding is slower overall. Getting an early start on the decoding might be a net win for us but the time spent decoding source-maps is very small by comparison, so the overall impact would likely be low.

[bvaughn]

To be clear, this issue is not proposing a specific technology (e.g. not a Babel plug-in). We'd probably want to try this out first within Facebook (which has its own build pipeline).

If successful, we'd publish a "spec" and rely on OSS to build plug-ins for different compilers/toolchains.

[bvaughn]

This is a nice discussion thread:
https://twitter.com/brian_d_vaughn/status/1447689511153065991