Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Distribute source maps for easier debugging in Chrome's Performance tab #20186

Closed
astoilkov opened this issue Nov 6, 2020 · 28 comments · Fixed by #26446
Closed

Distribute source maps for easier debugging in Chrome's Performance tab #20186

astoilkov opened this issue Nov 6, 2020 · 28 comments · Fixed by #26446

Comments

@astoilkov
Copy link

astoilkov commented Nov 6, 2020

I want to propose the addition of a new file in the react-dom npm package called react-dom.production.js — a non-minified version of react-dom production build.

Edit: After some discussion(see below) it seems that distributing source maps makes more sense. Source maps will help you see the real function names and explore them. (The points below apply for source maps as well.)

Why?

There are a few ways to profile React's performance — none of them provide a low-level view of what's happening. I believe the best way to profile React is using Chrome's Performance tab using a non-minified production build. Here's why:

  • Familiar. People use the Performance tab for every other performance profiling so they are familiar with how to use it.
  • Powerful. The Performance tab is extremely powerful. Years of engineering have been put into developing it.
  • Better understanding. When using the React DevTools profiler I have a common problem – I see a component being rendered slowly but I don't know what is causing it. In order to understand I need a more low-level view. Here are some questions that can be answered only with the Chrome Performance tab:
    • What's the balance between the app's code and React execution times? Should I implement some frequently updated components using custom non-React implementation?
    • What time is spend on setting attribute values, setting innerHTML and adding and removing listeners?
    • What time is spent on disposing effects? Is there a specific dispose function that is taking more than usual?
    • What time is spent on mounting effects? Is there a specific effect mount that is taking more than usual?

Disadvantages

As with every solution, there are some drawbacks to using this approach:

  • Documentation needed. In order to make sense of what's happening you will need some knowledge of the core functions in React. A little guide with the names of the functions for mount/unmount, effects, and DOM manipulations will be useful. This of course can be done by the community(you are already linking to some community posts in React's documentation).
  • Requires more skill. This isn't for everybody. It's aimed at more experienced developers. This type of profiling is a lot more overwhelming than the current approach.
  • May not fit your principles. Maybe it's not part of your principles to introduce and promote such a complicated solution. Maybe you are more interested in searching for a more elegant and minimal solution.
@bvaughn
Copy link
Contributor

bvaughn commented Nov 7, 2020

I don't understand this issue. Why does a minified build prevent you from using the browser's profiling tools?

@astoilkov
Copy link
Author

astoilkov commented Nov 7, 2020

You can use the profiling tools but it is hard to reason about what's happening because of the minified function names. I will give some examples of when it is useful to have the real names:

  • if I see flushPassiveEffectsImpl() taking a lot of time I know I should look at some of the dispose methods of effects
  • if I see a lot of calls to setValueForProperty() then I can think of lowering the number of attributes I am changing
  • if I see a lot of time spent inside an anonymous function which parent is updateDOMProperties() then I can change/remove the dangerouslySetInnerHTML

I can also answer the question "what time is used by React" which may let me use a custom implementation in performance-critical sections.

I can also click on every function I don't know about and explore the code and comments there and makes sense of what is happening.

@astoilkov astoilkov changed the title Support for profiling React production with Chrome's Performance tab Distribute non-minified React production build for easier debugging in Chrome's Performance tab Nov 7, 2020
@astoilkov
Copy link
Author

astoilkov commented Nov 7, 2020

I think I understood why you were confused. From what I wrote it seems like it isn't possible to profile the production build. I changed the title to reflect that. My proposal is for improving the experience when profiling production.

@bvaughn
Copy link
Contributor

bvaughn commented Nov 7, 2020

The React Profiler API, the DevTools extension profiler, and our experimental scheduling profiler are meant to give insight into the types of things you mention. Given the complexity of the internal React implementation, the profiling tools we maintain are probably going to provide better insight than manually inspecting React the way you describe. (That kind of manual inspection is often difficult even for those of us who work in React core daily.)

I wonder if providing source maps would help address the problem you mention of occasionally needing to step through React internals?

@astoilkov astoilkov changed the title Distribute non-minified React production build for easier debugging in Chrome's Performance tab Distribute source-maps for easier debugging in Chrome's Performance tab Nov 7, 2020
@astoilkov
Copy link
Author

I should have thought about source maps. I think this is the ideal solution. It's way better than my proposal.

Is there a way that the Profiler API and the DevTools extension shows the time spend on user code and React code separately? Also is there a way to see which part of the React code is taking the longest?

I will experiment with the scheduling profiler. This is the first time I've heard about it.

@astoilkov astoilkov changed the title Distribute source-maps for easier debugging in Chrome's Performance tab Distribute source maps for easier debugging in Chrome's Performance tab Nov 7, 2020
@bvaughn
Copy link
Contributor

bvaughn commented Nov 7, 2020

Is there a way that the Profiler API and the DevTools extension shows the time spend on user code and React code separately? Also is there a way to see which part of the React code is taking the longest?

No. These tools show the time spent in user code. This is because (a) a lot of energy has already been put into making React as fast as we can (given constraints) and (b) it's more feasible for people to performance-tune their own components and not React anyway.

@astoilkov
Copy link
Author

No. These tools show the time spent in user code. This is because (a) a lot of energy has already been put into making React as fast as we can (given constraints) and (b) it's more feasible for people to performance-tune their own components and not React anyway.

Thanks, makes sense.

What do you think about the source maps idea? Are you open to discussing and implementing it?

@bvaughn
Copy link
Contributor

bvaughn commented Nov 7, 2020

Discussing it seems fine!

@astoilkov
Copy link
Author

astoilkov commented Nov 7, 2020

Adding an issue I found on the topic: Question: Building react with sourcemaps.

@0xdevalias
Copy link
Contributor

0xdevalias commented Jul 10, 2021

I wonder if providing source maps would help address the problem you mention of occasionally needing to step through React internals?

This would definitely be useful for me. I actually just went looking to see if I could find sourcemaps for React (specifically we're using the minified production profiling versions of react, react-dom, etc.

My main use case is 'occasionally digging through things while profiling', but it would also provide better context if/when errors happen deep in React's code and are reported to Bugsnag/etc. As currently the stack traces tend to be a whole lot of minified/mangled names.

I would definitely want this to be as separate .map files rather than inlined though, as a majority of the time we wouldn't want/need the browser to be loading them.

It looks like the places referenced in #14361 for enabling sourcemaps are:

Curious, is this the only place that would need to be changed for this? If not, what would be the canonical way for enabling this (even if only to build my own versions)?

@jasonwilliams
Copy link
Contributor

jasonwilliams commented Jul 15, 2021

I got this working somewhat, I updated Rollup and all of the plugins, set sourcemaps to true and SourceMaps: true in the babel config. Then worked through the plugins. Mainly by commenting them out then trying to build and adding them back in.

Some I had to comment out though as i don't know how to add mapping support to some of these internal plugins I think closure and strip-unused-imports can be replaced by the terser plugin. Sizes could be replaced by https://www.npmjs.com/package/rollup-plugin-sizes

I've had to give up for now though, if anyone else was looking into it that's what i've done

@bvaughn
Copy link
Contributor

bvaughn commented Jul 15, 2021

@jasonwilliams Could you push your work somewhere so anyone else who's interested could look at it?

@jasonwilliams
Copy link
Contributor

jasonwilliams commented Jul 15, 2021

Sure!

Comparison: https://github.com/facebook/react/compare/main...jasonwilliams:addSourcMaps?expand=1
Branch (build file): https://github.com/jasonwilliams/react/blob/addSourcMaps/scripts/rollup/build.js

Clone it, yarn install, then run node --trace-warnings .\scripts\rollup\build.js react/index,react-dom/index

I'm sure someone who's good with Rollup could get it working fully

@0xdevalias
Copy link
Contributor

I was deep diving into this the other week as well.. was writing up notes/observations as I went.. but then got sidetracked and never completed it. Will include the notes I do have in an expandable below in case it helps anyone else:

Click to expand

In case it's of use for anyone else that stumbles across this issue:

This is where I changed the rollup build script to try and enabled sourcemaps:

diff --git a/scripts/rollup/build.js b/scripts/rollup/build.js
index 41b068509..aad155b45 100644
--- a/scripts/rollup/build.js
+++ b/scripts/rollup/build.js
@@ -235,7 +235,9 @@ function getRollupOutputOptions(
     freeze: !isProduction,
     interop: false,
     name: globalName,
-    sourcemap: false,
+    // TODO: if we change sourcemap to true here will we get sourcemaps output?
+    // sourcemap: false,
+    sourcemap: true,
     esModule: false,
   };
 }

And this is how I attempted to build the code, which resulted in the same error as the OP hit:

⇒  yarn build react/index,react-dom/index --type=UMD
yarn run v1.22.10
$ node ./scripts/rollup/build.js react/index,react-dom/index --type=UMD
 BUILDING  react.development.js (umd_dev)

Sourcemap is likely to be incorrect: a plugin (at position 6) was used to transform files, but didn't generate a sourcemap for the transformation. Consult the plugin documentation for help

error Command failed with exit code 1.

The two main sources that come up when googling that are:

Neither of which seemed particularly helpful on their own.

Modifying the build script a bit more:

diff --git a/scripts/rollup/build.js b/scripts/rollup/build.js
index 41b068509..928c7610d 100644
--- a/scripts/rollup/build.js
+++ b/scripts/rollup/build.js
@@ -159,6 +159,7 @@ function getBabelConfig(
     configFile: false,
     presets: [],
     plugins: [...babelPlugins],
+    sourceMaps: true, // Ref: https://babeljs.io/docs/en/options#sourcemaps
   };
   if (isDevelopment) {
     options.plugins.push(
@@ -235,7 +236,9 @@ function getRollupOutputOptions(
     freeze: !isProduction,
     interop: false,
     name: globalName,
-    sourcemap: false,
+    // TODO: if we change sourcemap to true here will we get sourcemaps output?
+    // sourcemap: false,
+    sourcemap: true,
     esModule: false,
   };
 }
@@ -363,7 +366,8 @@ function getPlugins(
     bundleType === RN_FB_PROD ||
     bundleType === RN_FB_PROFILING;
   const shouldStayReadable = isFBWWWBundle || isRNBundle || forcePrettyOutput;
-  return [
+
+  const thePlugins = [
     // Extract error codes from invariant() messages into a file.
     shouldExtractErrors && {
       transform(source) {
@@ -371,18 +375,23 @@ function getPlugins(
         return source;
       },
     },
+
     // Shim any modules that need forking in this environment.
     useForks(forks),
+
     // Ensure we don't try to bundle any fbjs modules.
     forbidFBJSImports(),
+
     // Use Node resolution mechanism.
     resolve({
       skip: externals,
     }),
+
     // Remove license headers from individual modules
     stripBanner({
       exclude: 'node_modules/**/*',
     }),
+
     // Compile to ES2015.
     babel(
       getBabelConfig(
@@ -393,12 +402,14 @@ function getPlugins(
         !isProduction
       )
     ),
+
     // Remove 'use strict' from individual source files.
     {
       transform(source) {
         return source.replace(/['"]use strict["']/g, '');
       },
     },
+
     // Turn __DEV__ and process.env checks into constants.
     replace({
       __DEV__: isProduction ? 'false' : 'true',
@@ -410,10 +421,12 @@ function getPlugins(
       // NOTE: I did not put much thought into how to configure this.
       __VARIANT__: bundle.enableNewReconciler === true,
     }),
+
     // The CommonJS plugin *only* exists to pull "art" into "react-art".
     // I'm going to port "art" to ES modules to avoid this problem.
     // Please don't enable this for anything else!
     isUMDBundle && entry === 'react-art' && commonjs(),
+
     // Apply dead code elimination and/or minification.
     isProduction &&
       closure(
@@ -424,9 +437,11 @@ function getPlugins(
           renaming: !shouldStayReadable,
         })
       ),
+
     // HACK to work around the fact that Rollup isn't removing unused, pure-module imports.
     // Note that this plugin must be called after closure applies DCE.
     isProduction && stripUnusedImports(pureExternalModules),
+
     // Add the whitespace back if necessary.
     shouldStayReadable &&
       prettier({
@@ -435,6 +450,7 @@ function getPlugins(
         trailingComma: 'none',
         bracketSpacing: true,
       }),
+
     // License and haste headers, top-level `if` blocks.
     {
       renderChunk(source) {
@@ -447,6 +463,7 @@ function getPlugins(
         );
       },
     },
+
     // Record bundle size.
     sizes({
       getSize: (size, gzip) => {
@@ -465,7 +482,14 @@ function getPlugins(
         };
       },
     }),
-  ].filter(Boolean);
+  ]
+
+  const thePluginsFiltered = thePlugins.filter(Boolean);
+
+  console.log(thePlugins);
+  console.log(thePlugins.length);
+
+  return thePluginsFiltered;
 }
 
 function shouldSkipBundle(bundle, bundleType) {

I get the following output:

⇒  yarn build react/index,react-dom/index --type=UMD
yarn run v1.22.10
$ node ./scripts/rollup/build.js react/index,react-dom/index --type=UMD
[
  undefined,
  {
    name: 'scripts/rollup/plugins/use-forks-plugin',
    resolveId: [Function: resolveId]
  },
  { name: 'forbidFBJSImports', resolveId: [Function: resolveId] },
  { name: 'node-resolve', resolveId: [Function: resolveId$1] },
  { transform: [Function: transform] },
  {
    name: 'babel',
    resolveId: [Function: resolveId],
    load: [Function: load],
    transform: [Function: transform]
  },
  { transform: [Function: transform] },
  { name: 'replace', transform: [Function: transform] },
  false,
  false,
  false,
  undefined,
  { renderChunk: [Function: renderChunk] },
  {
    name: 'scripts/rollup/plugins/sizes-plugin',
    generateBundle: [Function: generateBundle]
  }
]
14
 BUILDING  react.development.js (umd_dev)

Sourcemap is likely to be incorrect: a plugin (at position 6) was used to transform files, but didn't generate a sourcemap for the transformation. Consult the plugin documentation for help

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Since the array of plugins is filtered to only use the 'truthy' plugins, the 'position 6' referred to in the error corresponds with:

// Remove 'use strict' from individual source files.
{
  transform(source) {
    return source.replace(/['"]use strict["']/g, '');
  },
},

If we comment that out then run it again, we get the following error:

Sourcemap is likely to be incorrect: a plugin (at position 7) was used to transform files, but didn't generate a sourcemap for the transformation. Consult the plugin documentation for help

This seems to correspond with the following:

// License and haste headers, top-level `if` blocks.
{
  renderChunk(source) {
    return Wrappers.wrapBundle(
      source,
      bundleType,
      globalName,
      filename,
      moduleType
    );
  },
},

If we comment that out then run it again, we get the following error:

Sourcemap is likely to be incorrect: a plugin (scripts/rollup/plugins/closure-plugin) was used to transform files, but didn't generate a sourcemap for the transformation. Consult the plugin documentation for help

This seems to correspond with the following:

// Apply dead code elimination and/or minification.
isProduction &&
  closure(
    Object.assign({}, closureOptions, {
      // Don't let it create global variables in the browser.
      // https://github.com/facebook/react/issues/10909
      assume_function_wrapper: !isUMDBundle,
      renaming: !shouldStayReadable,
    })
  ),

If we look at scripts/rollup/plugins/closure-plugin.js we can see that it uses require('google-closure-compiler')

Originally posted by @0xdevalias in #14361 (comment)

@0xdevalias
Copy link
Contributor

0xdevalias commented Jul 17, 2021

I think closure and strip-unused-imports can be replaced by the terser plugin.

My notes/main area of focus in #20186 (comment) was on the closure compiler. It has the ability to generate sourcemaps already, just needs to be configured to do so.

Here's the diff I have locally, I had this working the other day for this part, but haven't tested again since if I modified this diff further.. so it might not be perfectly working right now.. Hopefully points people in the right direction but!

It also seemed like if we switched from the java closure compiler to the JS one, it might have a nicer API to use. Unsure if/how much impact that would have on running speed/etc for it though.

Click to expand/see diff
diff --git a/scripts/rollup/plugins/closure-plugin.js b/scripts/rollup/plugins/closure-plugin.js
index 258f3b6be..37c7ec52a 100644
--- a/scripts/rollup/plugins/closure-plugin.js
+++ b/scripts/rollup/plugins/closure-plugin.js
@@ -5,11 +5,21 @@ const {promisify} = require('util');
 const fs = require('fs');
 const tmp = require('tmp');
 const writeFileAsync = promisify(fs.writeFile);
+const readFileAsync = promisify(fs.readFile);
 
 function compile(flags) {
   return new Promise((resolve, reject) => {
     const closureCompiler = new ClosureCompiler(flags);
+
+    console.log(flags);
+    console.log(closureCompiler);
+    console.log(Object.keys(closureCompiler));
+
     closureCompiler.run(function(exitCode, stdOut, stdErr) {
+      // console.log(stdOut);
+      // console.log(stdErr);
+      // console.log(stdErr);
+
       if (!stdErr) {
         resolve(stdOut);
       } else {
@@ -25,11 +35,25 @@ module.exports = function closure(flags = {}) {
     async renderChunk(code) {
       const inputFile = tmp.fileSync();
       const tempPath = inputFile.name;
-      flags = Object.assign({}, flags, {js: tempPath});
+      flags = Object.assign({}, flags, {js: tempPath}, { create_source_map: tempPath + ".map"});
       await writeFileAsync(tempPath, code, 'utf8');
+
+      console.log(flags);
+
       const compiledCode = await compile(flags);
+
+      const compiledSourceMap = await readFileAsync(flags.create_source_map, { encoding: "utf8"});
+
+      console.log(typeof compiledCode);
+
+
+      console.log(tempPath)
+      console.log(flags.create_source_map)
+      // console.log(Object.keys(compiledCode));
+      // console.log(compiledSourceMap);
+
       inputFile.removeCallback();
-      return {code: compiledCode};
+      return {code: compiledCode, map: compiledSourceMap};
     },
   };
 };

@bvaughn
Copy link
Contributor

bvaughn commented Jul 18, 2021

It also seemed like if we switched from the java closure compiler to the JS one, it might have a nicer API to use. Unsure if/how much impact that would have on running speed/etc for it though.

Switching to the Java one was done intentionally to address memory and speed problems with the JS compiler. (See #12800)

@0xdevalias
Copy link
Contributor

as i don't know how to add mapping support to some of these internal plugins

I haven't looked too deeply into this, so not sure if they are covering the same use cases, but for the "Remove 'use strict' from individual source files" I came across the following when googling:


I think closure and strip-unused-imports can be replaced by the terser plugin.

I just stumbled upon this benchmark repo that includes benchmarks for google-closure-compiler and terser (plus a bunch more such as esbuild), which might be useful here:

esbuild tends to win hands down as far as speed is concerned. terser generally seems to come in near the top for minified sizes, and is a lot quicker than uglify-js (which tends to slightly win in size, but for a much longer run time). google-closure-compiler often tends to have less good minified sizes, and take longer to run. But it bounces around a bit depending on the library it was tested against, so hard to draw any 'universal conclusions' about it.

Originally posted by @0xdevalias in #14361 (comment)

@jasonwilliams
Copy link
Contributor

jasonwilliams commented Jul 23, 2021

as i don't know how to add mapping support to some of these internal plugins

I haven't looked too deeply into this, so not sure if they are covering the same use cases, but for the "Remove 'use strict' from individual source files" I came across the following when googling:

I think closure and strip-unused-imports can be replaced by the terser plugin.

I just stumbled upon this benchmark repo that includes benchmarks for google-closure-compiler and terser (plus a bunch more such as esbuild), which might be useful here:

esbuild tends to win hands down as far as speed is concerned. terser generally seems to come in near the top for minified sizes, and is a lot quicker than uglify-js (which tends to slightly win in size, but for a much longer run time). google-closure-compiler often tends to have less good minified sizes, and take longer to run. But it bounces around a bit depending on the library it was tested against, so hard to draw any 'universal conclusions' about it.

Originally posted by @0xdevalias in #14361 (comment)

That’s great research thanks @0xdevalias
I’ll do a PR of what I have so far and maybe some can jump on that and help.

it’s good news that use_strict removal is built in, that’s one plugin that can go. Google closure compiler can go. It’s possible esbuild could replace both Babel and minification but that increases scope by quite a bit so I think for now I’ll keep Babel and use terser. Secondly speed isn’t my biggest concern right now and can be optimised later on

@jasonwilliams
Copy link
Contributor

PR added: #21946

I feel its close, but one of the plugins renderChunk needs looking at. And i don't understand why some production builds have shouldStayReadable set. So it gets minified then re-beautified by prettier. This seems odd to me but im sure there's logic to it somewhere. I don't think sourcemaps for these builds are needed if the output is already beautified.

@andrelmchaves
Copy link

@jasonwilliams Could you confirm this is still an issue?

@jasonwilliams
Copy link
Contributor

@jasonwilliams Could you confirm this is still an issue?

Yes it’s still an issue

@0xdevalias
Copy link
Contributor

PR added: #21946

I feel its close, but one of the plugins renderChunk needs looking at. And i don't understand why some production builds have shouldStayReadable set. So it gets minified then re-beautified by prettier. This seems odd to me but im sure there's logic to it somewhere. I don't think sourcemaps for these builds are needed if the output is already beautified.

@jasonwilliams Thanks for your work on that PR! It's taken me forever to get back to this, but I spent some time today going through it, leaving some rather in depth context/comments/etc, as well as researching solutions to what I believe should be the rest of the remaining issues.:

A rather verbose review + deep dive research into the context of why certain changes were made here, as well as potential solutions for what I believe are the remaining issues that would block this PR from landing. At least in theory, I think all of the potential issues have solutions now. Would love to see this work able to land!

There are a few minor code cleanup 'suggestions' as well, which can all be automatically applied as a single commit as follows:

Originally posted by @0xdevalias in #21946 (review)


Would be awesome if we were able to get this feature finished up and merged in!

@bvaughn
Copy link
Contributor

bvaughn commented Jan 10, 2023

Finding myself wanting this again this morning, trying to debug invariant=321.

https://twitter.com/brian_d_vaughn/status/1612806202559459330

sebmarkbage added a commit that referenced this issue Feb 20, 2023
Update Rollup and related plugins to their most recent versions +
resolve any breaking changes/deprecations/etc along the way. I made each
change piece by piece, so the commit history tells a pretty good story
of what was changed where/how/why.

fixes #24894

For the full deepdive/context, see:

- #24894

The inspiration for this came from @jasonwilliams 's PR for attempting
to add sourcemap output support to React's builds:

- #20186
  - #21946

But I figured that it would be useful to minimise the scope of changes
in that PR, and to modernise the build tooling along the way.

If any of these updates rely on a node version later than `10.x`, then
the following PR may have to land first, otherwise things might break on
AppVeyor:

- #24891
  - #24892

Co-authored-by: Sebastian Markbage <sebastian@calyptus.eu>
github-actions bot pushed a commit that referenced this issue Feb 20, 2023
Update Rollup and related plugins to their most recent versions +
resolve any breaking changes/deprecations/etc along the way. I made each
change piece by piece, so the commit history tells a pretty good story
of what was changed where/how/why.

fixes #24894

For the full deepdive/context, see:

- #24894

The inspiration for this came from @jasonwilliams 's PR for attempting
to add sourcemap output support to React's builds:

- #20186
  - #21946

But I figured that it would be useful to minimise the scope of changes
in that PR, and to modernise the build tooling along the way.

If any of these updates rely on a node version later than `10.x`, then
the following PR may have to land first, otherwise things might break on
AppVeyor:

- #24891
  - #24892

Co-authored-by: Sebastian Markbage <sebastian@calyptus.eu>

DiffTrain build for [6b6d061](6b6d061)
[View git log for this commit](https://github.com/facebook/react/commits/6b6d0617eff48860c5b4e3e79c74cbd3312cf45a)
@markerikson
Copy link
Contributor

I'm gonna do the annoying "+1" thing - this is something I'd love to see added to the React packages! (and tbh it's kind of surprising it hasn't been yet.) I've spent a fair amount of time needing to debug into the guts of React, and it would be a lot easier to do that if the prod builds actually included sourcemaps. (I know that this would increase the size of the published packages on NPM, but hopefully it would be worth it!)

@markerikson
Copy link
Contributor

Brian asked Dan about this on Twitter:

Dan's willing to look at a PR, but A) have to stick with Closure Compiler and not switch to Terser, B) there's concerns over "commitment" to keep them working vs hypothetical future build tooling changes.

I'm willing to take a stab at a new PR to get sourcemaps working with Closure Compiler, and will try to take a look at that later this week.

@markerikson
Copy link
Contributor

markerikson commented Mar 17, 2023

Good news, folks! As of a few minutes ago, I have:

  • Found and fixed a bug with the current Rollup 2.x build config (weird tree-shaking issues in react-server-dom-webpack-server artifacts)
  • Successfully upgraded to Rollup 3.x, and generated effectively-identical build artifacts as current main with Rollup 2.x
  • Managed to generate sourcemaps for production artifacts that contain the contents of "the prod bundle right before it was minified" rather than "the original-original source files", _and appear to be 100% accurate!

Here's a screenshot of a random function from within the react-dom.production.js bundle:

image

I've got a bunch of cleanup and additional verification to do, but I'm fairly positive I have this working as desired.

Overall, I think there's 3 PRs that ought to be merged:

  1. Cleanup of the existing Rollup 2.x config, fixing that tree-shaking issue and removing a seemingly-dead "remove unused imports" plugin
  2. Upgrading to Rollup 3.x. There's an existing PR at Update rollup to v3 #26078 , and my own changes were almost identical to that.
  3. Actually applying the changes to generate sourcemaps.

I'll see if I can file the cleanup PR early Monday, and hopefully we can get the Rollup 3.x PR in shortly after that.

(Alternately, I could do all these in one PR. It's not that many additional changes. It's just nice to have more granular traceability to why each change was made.)

update

After discussing with Dan, I'm going to file the 3 separate PRs myself on Monday. Those will supersede the existing stale "generate sourcemaps" PR and the existing "Rollup 3.x" PR, and I can work on getting those pushed through ASAP.

@markerikson
Copy link
Contributor

Okay, I just filed #26446 , which adds the sourcemap generation to React's build config!

That should hopefully generate some packages via CodeSandbox CI in a few minutes, and we can now test out that the sourcemaps actually get included and do something useful :)

hoxyq pushed a commit that referenced this issue Nov 7, 2023
<!--
  Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.

Before submitting a pull request, please make sure the following is
done:

1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
  2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn test --debug --watch TestName`,
open `chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
  9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
  10. If you haven't already, complete the CLA.

Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->

## Summary

This PR updates the Rollup build pipeline to generate sourcemaps for
production build artifacts like `react-dom.production.min.js`.
 
It requires the Rollup v3 changes that were just merged in #26442 .

Sourcemaps are currently _only_ generated for build artifacts that are
_truly_ "production" - no sourcemaps will be generated for development,
profiling, UMD, or `shouldStayReadable` artifacts.

The generated sourcemaps contain the bundled source contents right
before that chunk was minified by Closure, and _not_ the original source
files like `react-reconciler/src/*`. This better reflects the actual
code that is running as part of the bundle, with all the feature flags
and transformations that were applied to the source files to generate
that bundle. The sourcemaps _do_ still show comments and original
function names, thus improving debuggability for production usage.

Fixes #20186 .




<!--
Explain the **motivation** for making this change. What existing problem
does the pull request solve?
-->

This allows React users to actually debug a readable version of the
React bundle in production scenarios. It also allows other tools like
[Replay](https://replay.io) to do a better job inspecting the React
source when stepping through.

## How did you test this change?

- Generated numerous sourcemaps with various combinations of the React
bundle selections
- Viewed those sourcemaps in
https://evanw.github.io/source-map-visualization/ and confirmed via the
visualization that the generated mappings appear to be correct

I've attached a set of production files + their sourcemaps here:


[react-sourcemap-examples.zip](https://github.com/facebook/react/files/11023466/react-sourcemap-examples.zip)

You can drag JS+sourcemap file pairs into
https://evanw.github.io/source-map-visualization/ for viewing.

Examples:

- `react.production.min.js`:


![image](https://user-images.githubusercontent.com/1128784/226478247-e5cbdee0-83fd-4a19-bcf1-09961d3c7da4.png)

- `react-dom.production.min.js`:


![image](https://user-images.githubusercontent.com/1128784/226478433-b5ccbf0f-8f68-42fe-9db9-9ecb97770d46.png)

- `use-sync-external-store/with-selector.production.min.js`:


![image](https://user-images.githubusercontent.com/1128784/226478565-bc74699d-db14-4c39-9e2d-b775f8755561.png)


<!--
Demonstrate the code is solid. Example: The exact commands you ran and
their output, screenshots / videos if the pull request changes the user
interface.
How exactly did you verify that your PR solves the issue you wanted to
solve?
  If you leave this empty, your PR will very likely be closed.
-->
github-actions bot pushed a commit that referenced this issue Nov 7, 2023
<!--
  Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.

Before submitting a pull request, please make sure the following is
done:

1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
  2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn test --debug --watch TestName`,
open `chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
  9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
  10. If you haven't already, complete the CLA.

Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->

## Summary

This PR updates the Rollup build pipeline to generate sourcemaps for
production build artifacts like `react-dom.production.min.js`.

It requires the Rollup v3 changes that were just merged in #26442 .

Sourcemaps are currently _only_ generated for build artifacts that are
_truly_ "production" - no sourcemaps will be generated for development,
profiling, UMD, or `shouldStayReadable` artifacts.

The generated sourcemaps contain the bundled source contents right
before that chunk was minified by Closure, and _not_ the original source
files like `react-reconciler/src/*`. This better reflects the actual
code that is running as part of the bundle, with all the feature flags
and transformations that were applied to the source files to generate
that bundle. The sourcemaps _do_ still show comments and original
function names, thus improving debuggability for production usage.

Fixes #20186 .

<!--
Explain the **motivation** for making this change. What existing problem
does the pull request solve?
-->

This allows React users to actually debug a readable version of the
React bundle in production scenarios. It also allows other tools like
[Replay](https://replay.io) to do a better job inspecting the React
source when stepping through.

## How did you test this change?

- Generated numerous sourcemaps with various combinations of the React
bundle selections
- Viewed those sourcemaps in
https://evanw.github.io/source-map-visualization/ and confirmed via the
visualization that the generated mappings appear to be correct

I've attached a set of production files + their sourcemaps here:

[react-sourcemap-examples.zip](https://github.com/facebook/react/files/11023466/react-sourcemap-examples.zip)

You can drag JS+sourcemap file pairs into
https://evanw.github.io/source-map-visualization/ for viewing.

Examples:

- `react.production.min.js`:

![image](https://user-images.githubusercontent.com/1128784/226478247-e5cbdee0-83fd-4a19-bcf1-09961d3c7da4.png)

- `react-dom.production.min.js`:

![image](https://user-images.githubusercontent.com/1128784/226478433-b5ccbf0f-8f68-42fe-9db9-9ecb97770d46.png)

- `use-sync-external-store/with-selector.production.min.js`:

![image](https://user-images.githubusercontent.com/1128784/226478565-bc74699d-db14-4c39-9e2d-b775f8755561.png)

<!--
Demonstrate the code is solid. Example: The exact commands you ran and
their output, screenshots / videos if the pull request changes the user
interface.
How exactly did you verify that your PR solves the issue you wanted to
solve?
  If you leave this empty, your PR will very likely be closed.
-->

DiffTrain build for [2c8a139](2c8a139)
@0xdevalias
Copy link
Contributor

For those of us who were interested in sourcemap support for easier debugging/etc, just wanted to highlight 2 other issues that would improve things even more that you might like to watch/support:

jerrydev0927 added a commit to jerrydev0927/react that referenced this issue Jan 5, 2024
Update Rollup and related plugins to their most recent versions +
resolve any breaking changes/deprecations/etc along the way. I made each
change piece by piece, so the commit history tells a pretty good story
of what was changed where/how/why.

fixes facebook/react#24894

For the full deepdive/context, see:

- facebook/react#24894

The inspiration for this came from @jasonwilliams 's PR for attempting
to add sourcemap output support to React's builds:

- facebook/react#20186
  - facebook/react#21946

But I figured that it would be useful to minimise the scope of changes
in that PR, and to modernise the build tooling along the way.

If any of these updates rely on a node version later than `10.x`, then
the following PR may have to land first, otherwise things might break on
AppVeyor:

- facebook/react#24891
  - facebook/react#24892

Co-authored-by: Sebastian Markbage <sebastian@calyptus.eu>

DiffTrain build for [6b6d0617eff48860c5b4e3e79c74cbd3312cf45a](facebook/react@6b6d061)
[View git log for this commit](https://github.com/facebook/react/commits/6b6d0617eff48860c5b4e3e79c74cbd3312cf45a)
EdisonVan pushed a commit to EdisonVan/react that referenced this issue Apr 15, 2024
<!--
  Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.

Before submitting a pull request, please make sure the following is
done:

1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
  2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn test --debug --watch TestName`,
open `chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
  9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
  10. If you haven't already, complete the CLA.

Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->

## Summary

This PR updates the Rollup build pipeline to generate sourcemaps for
production build artifacts like `react-dom.production.min.js`.
 
It requires the Rollup v3 changes that were just merged in facebook#26442 .

Sourcemaps are currently _only_ generated for build artifacts that are
_truly_ "production" - no sourcemaps will be generated for development,
profiling, UMD, or `shouldStayReadable` artifacts.

The generated sourcemaps contain the bundled source contents right
before that chunk was minified by Closure, and _not_ the original source
files like `react-reconciler/src/*`. This better reflects the actual
code that is running as part of the bundle, with all the feature flags
and transformations that were applied to the source files to generate
that bundle. The sourcemaps _do_ still show comments and original
function names, thus improving debuggability for production usage.

Fixes facebook#20186 .




<!--
Explain the **motivation** for making this change. What existing problem
does the pull request solve?
-->

This allows React users to actually debug a readable version of the
React bundle in production scenarios. It also allows other tools like
[Replay](https://replay.io) to do a better job inspecting the React
source when stepping through.

## How did you test this change?

- Generated numerous sourcemaps with various combinations of the React
bundle selections
- Viewed those sourcemaps in
https://evanw.github.io/source-map-visualization/ and confirmed via the
visualization that the generated mappings appear to be correct

I've attached a set of production files + their sourcemaps here:


[react-sourcemap-examples.zip](https://github.com/facebook/react/files/11023466/react-sourcemap-examples.zip)

You can drag JS+sourcemap file pairs into
https://evanw.github.io/source-map-visualization/ for viewing.

Examples:

- `react.production.min.js`:


![image](https://user-images.githubusercontent.com/1128784/226478247-e5cbdee0-83fd-4a19-bcf1-09961d3c7da4.png)

- `react-dom.production.min.js`:


![image](https://user-images.githubusercontent.com/1128784/226478433-b5ccbf0f-8f68-42fe-9db9-9ecb97770d46.png)

- `use-sync-external-store/with-selector.production.min.js`:


![image](https://user-images.githubusercontent.com/1128784/226478565-bc74699d-db14-4c39-9e2d-b775f8755561.png)


<!--
Demonstrate the code is solid. Example: The exact commands you ran and
their output, screenshots / videos if the pull request changes the user
interface.
How exactly did you verify that your PR solves the issue you wanted to
solve?
  If you leave this empty, your PR will very likely be closed.
-->
bigfootjon pushed a commit that referenced this issue Apr 18, 2024
<!--
  Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.

Before submitting a pull request, please make sure the following is
done:

1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
  2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn test --debug --watch TestName`,
open `chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
  9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
  10. If you haven't already, complete the CLA.

Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->

## Summary

This PR updates the Rollup build pipeline to generate sourcemaps for
production build artifacts like `react-dom.production.min.js`.

It requires the Rollup v3 changes that were just merged in #26442 .

Sourcemaps are currently _only_ generated for build artifacts that are
_truly_ "production" - no sourcemaps will be generated for development,
profiling, UMD, or `shouldStayReadable` artifacts.

The generated sourcemaps contain the bundled source contents right
before that chunk was minified by Closure, and _not_ the original source
files like `react-reconciler/src/*`. This better reflects the actual
code that is running as part of the bundle, with all the feature flags
and transformations that were applied to the source files to generate
that bundle. The sourcemaps _do_ still show comments and original
function names, thus improving debuggability for production usage.

Fixes #20186 .

<!--
Explain the **motivation** for making this change. What existing problem
does the pull request solve?
-->

This allows React users to actually debug a readable version of the
React bundle in production scenarios. It also allows other tools like
[Replay](https://replay.io) to do a better job inspecting the React
source when stepping through.

## How did you test this change?

- Generated numerous sourcemaps with various combinations of the React
bundle selections
- Viewed those sourcemaps in
https://evanw.github.io/source-map-visualization/ and confirmed via the
visualization that the generated mappings appear to be correct

I've attached a set of production files + their sourcemaps here:

[react-sourcemap-examples.zip](https://github.com/facebook/react/files/11023466/react-sourcemap-examples.zip)

You can drag JS+sourcemap file pairs into
https://evanw.github.io/source-map-visualization/ for viewing.

Examples:

- `react.production.min.js`:

![image](https://user-images.githubusercontent.com/1128784/226478247-e5cbdee0-83fd-4a19-bcf1-09961d3c7da4.png)

- `react-dom.production.min.js`:

![image](https://user-images.githubusercontent.com/1128784/226478433-b5ccbf0f-8f68-42fe-9db9-9ecb97770d46.png)

- `use-sync-external-store/with-selector.production.min.js`:

![image](https://user-images.githubusercontent.com/1128784/226478565-bc74699d-db14-4c39-9e2d-b775f8755561.png)

<!--
Demonstrate the code is solid. Example: The exact commands you ran and
their output, screenshots / videos if the pull request changes the user
interface.
How exactly did you verify that your PR solves the issue you wanted to
solve?
  If you leave this empty, your PR will very likely be closed.
-->

DiffTrain build for commit 2c8a139.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants