Bundle Analysis Configuration Feedback

[codecovdesign]

Thanks for dropping by! 👋

We'd love your feedback about the bundle analysis configuration and tool generally..

• How was your experience setting up bundle analysis in your repo?
• Did you experience any challenges?
• What did you think about the documentation?
• What would improve the set up process?
• How can bundle analysis help you further?
• Any general thoughts you'd like to share!

We greatly appreciate your time and thoughts - looking forward to hearing from you ❤

Codecov team

This issue is intended to share and collect feedback about the tool. If you have support needs or questions, please see our support page.

[houserx-jmcc]

• How was your experience setting up bundle analysis in your repo?
  • Very easy, got it on the first attempt.
• What did you think about the documentation?
  • Also very good! No questions.
• How can bundle analysis help you further?
  • It would be nice if it could be incorporated as a GitHub check so that CI could auto-block if a PR increases the bundle size by a set % or bytes
• Any general thoughts you'd like to share!
  • I deleted the bundle comment on one of my pull requests, and subsequent builds did not make the comment re-appear. It would be nice if it was able to re-create the comment in the event it got deleted and a new commit is pushed to the branch that modifies the bundle analysis.

[houserx-jmcc]

It would also be nice to set a threshold change to not trigger a comment. We use Vite for our builds and it seems there is often a 10-35kB variation between builds even without any substantive changes to the code or dependencies. Being able to only have the comment appear when the change matters would help to avoid people from learning to ignore it when it shows up every time.

[codecovdesign]

@houserx-jmcc thank you for the feedback 🙏

nice to set a threshold change to not trigger a comment

I've added the update proposal to this WIP issue: codecov/engineering-team#1350

[karl-sjogren]

We've been running this for a few weeks now and it was really easy to setup and has worked fine.

However, something that would be really useful is some way to filter or split bundles when reporting. In its simplest form an option to specify which file extensions to include/exclude would suffice.

We are using Vite for our project and import our styles into the entry JS file, however when building for deployment we split out the CSS stuff into a separate file. This removes the styles themselves from the bundle analysis, but all the other stuff such as images and fonts that were included are still there.

I'm only interested in the javascript parts (or possibly having a separate bundle for the styles/assets). Right now our bundle reports as 490kb which would be way to large for our scripts, but seeing as it is fonts in several formats and a few SVGs the size is quite OK. It makes it really hard to differentiates on changes though.

Something like this would be great to be able to do.

export default defineConfig({
  plugins: [
    codecovVitePlugin({
      enableBundleAnalysis: process.env.CODECOV_TOKEN !== undefined,
      bundleName: 'script-bundle',
      uploadToken: process.env.CODECOV_TOKEN,
      extensions: [
        '.js'
      ]
    }),
    codecovVitePlugin({
      enableBundleAnalysis: process.env.CODECOV_TOKEN !== undefined,
      bundleName: 'style-bundle',
      uploadToken: process.env.CODECOV_TOKEN,
      extensions: [
        '.css',
        '.woff',
        '.woff2',
        '.svg',
        '.png',
      ]
    })
  ]
});

Also, if you look at the screenshot above the gzip size is completely wrong for some reason.

[codecovdesign]

@karl-sjogren thanks for the great feedback! 🙏

some way to filter or split bundles when reporting

From my understanding of your feedback, based on that I created this issue here: codecov/engineering-team#1562. Would you say this accurately captures the issue? If TLDR, the goal is for you to specify the files you'd like to group to have analysis of size / changes. This way the reporting is more focused on what's important to you (generic filters for JS, etc don't do the job) - Is this understanding right?

[karl-sjogren]

Yes that looks good to me! Thanks!

[estevaolucas]

Hi. I've been using this plugin but Codecov doesn't show any data yet, only the "Configure bundle analysis" info.

Wondering what can be wrong in my setup (with Nx). Is this plugin generating some manifest file that is than uploaded to Codecov? How can I debug it. Thanks

[nicholas-codecov]

Hi. I've been using this plugin but Codecov doesn't show any data yet, only the "Configure bundle analysis" info.

Wondering what can be wrong in my setup (with Nx). Is this plugin generating some manifest file that is than uploaded to Codecov? How can I debug it. Thanks

@estevaolucas are you able to share your repo/config?

If it's working you should be seeing logs in your build informing you of what the plugin is doing, there is also a dryRun option that you can use to have the stats information outputted to a json file in your out dir. You can see more in out plugin config docs

[estevaolucas]

@nicholas-codecov thanks for you reply! I didn't notice there's a debug flag. That helped me to figure the issue on my setup: report was never uploading analysis due to missing token in enableBundleAnalysis to enable it. Thanks for you help

[nicholas-codecov]

@estevaolucas Glad you got everything all sorted! If you run into any issues let us know.

[grantwforsythe]

Great work on this feature!

Just a minor comment on the documentation and a question about future implementation methods...

The documentation references Angular in the webpack plugin but I don't think this is accurate for two reasons:

• The Angular build pipeline was completely overhauled in Angular v17 migrating from webpack to esbuild as the default bundler. (The webpack builder is still accessible, it just won't be the default going option going forward).
• Unless a custom builder is used, build configurations can not be changed as it is against Angular's CLI design principles.

I am iffy about switching to a custom builder to use this feature. Are there plans to support source maps or stats.json to analyze bundles? (I'm imagining that these files can be published the same way coverage reports are published in a CI pipeline).

[nicholas-codecov]

Hey @grantwforsythe thanks for reaching out!

We have on our roadmap to look into supporting esbuild directly, and we may also look into extending that to support specific cases that may arise in Angular. We will also be adding support for Angular when using Vite sooner than esbuild support.

We haven't looked into using source maps or stats.json files as of yet, but thank you for the suggestion and we will take a peak down those paths as well!

[grantwforsythe]

Looking forward to it! Thanks for the clarification, @nicholas-codecov

[kayluhb]

I'm trying to test this out as a POC and I'm getting this error

[codecov] `get-pre-signed-url` fetch attempt 0 failed
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 1
[codecov] `get-pre-signed-url` fetch attempt 1 failed
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 2
[codecov] `get-pre-signed-url` fetch attempt 2 failed
[codecov] `get-pre-signed-url` failed after 2 attempts
[codecov] Failed to get pre-signed URL, bad response

I'm using vite and my config looks like this

import {defineConfig} from 'vite';
import {hydrogen} from '@shopify/hydrogen/vite';
import {oxygen} from '@shopify/mini-oxygen/vite';
import {vitePlugin as remix} from '@remix-run/dev';
import tsconfigPaths from 'vite-tsconfig-paths';
import {codecovVitePlugin} from '@codecov/vite-plugin';

export default defineConfig({
  plugins: [
    hydrogen(),
    oxygen(),
    remix({
      presets: [hydrogen.preset()],
      ignoredRouteFiles: ['**/.*', '**/*.test.*', '**/styleguide.tsx'],
      future: {
        v3_fetcherPersist: false,
        v3_relativeSplatPath: false,
        v3_throwAbortReason: false,
      },
    }),
    tsconfigPaths(),
    // Put the Codecov vite plugin after all other plugins
    codecovVitePlugin({
      debug: true,
      enableBundleAnalysis: process.env.CODECOV_TOKEN !== undefined,
      bundleName: 'dist',
      uploadToken: process.env.CODECOV_TOKEN,
    }),
  ],
});

I've confirmed that my process.env.CODECOV_TOKEN is defined

What am I missing?

[nicholas-codecov]

Hey @kayluhb!

Hrrm first i just to note that we don't fully support Remix yet, as Vite does some funky things when it comes to client and server bundles that we'll be tackling shortly. Can you provide some info such as the repo you're trying to connect so we can filter our logs a bit more easily?

[kayluhb]

Hey @kayluhb!

Hrrm first i just to note that we don't fully support Remix yet, as Vite does some funky things when it comes to client and server bundles that we'll be tackling shortly. Can you provide some info such as the repo you're trying to connect so we can filter our logs a bit more easily?

Ah, bummer... Sure thing it's the tecovas-com/tecovas-hydrogen-storefront repo that I was playing around with it on.

Thanks!

[LuqiPan]

I am iffy about switching to a custom builder to use this feature. Are there plans to support source maps or stats.json to analyze bundles? (I'm imagining that these files can be published the same way coverage reports are published in a CI pipeline).

I wanted to echo the same idea for esbuild, except esbuild generates a meta.json file instead, see https://esbuild.github.io/api/#metafile for documentation.

To elaborate, right now to enable bundle analysis for bundler X, a user would need to

1. install codecov plugin for X via package manager
2. configure the bundler plugin by adding ~5-10 lines of configuration to bundler X's configuration file.

With @grantwforsythe's suggestion, a user would need to

1. configure bundler X to emit stats data, which in esbuild's case would be enabled by simply adding metafile: true to the configuration file and I'd imagine very similar for other bundlers
2. use and configure a github action that understands and uploads different formats of metadata/stats files from different bundlers to app.codedov.io, similar to how codecov/codecov-action works currently

[KyeRussell]

Hi. I just took a quick, ultimately unsuccessful run at this. Nothing I couldn't solve with a bit of a closer look. I was just trying to sneak this into an existing piece of work on my project, without giving it too much thought.

I'm using Vite, running in Docker (via Docker Compose). Building with GitHub Actions.

1. Because of the usual layers of indirection brought about by Docker, I was first tripped up by my CODECOV_TOKEN not actually making its way to Vite (and thus the Codecov plugin). Obviously the fact that this was anything but trivial is not the plugin's 'fault', but it would've been nice to be able to ask the plugin for verbose output, so I could more clearly see what was going wrong. I took about 2 minutes to look at the documentation to see if that was possible or not. If it is, and it's documented, I apologise! But it may be a sign that the docs need more work.
2. Second issue, and one that ultimately pushed me outside my timebox and had me shelve this for now, was that the plugin couldn't work out which 'provider' (CI provider, right?) it was running within. Again, I spent a couple of minutes poking through the docs, trying to work out how the plugin determined this, and/or if it could be overridden. I didn't find anything. I vague recall GHA setting an environment variable to hint to the workflow environment that it's running inside GHA. GHA's first-party documentation is a bit of a monstrosity, and I didn't want to deal with it, so I put it down for the time being.

I'll take another look at getting it running later. I'm not looking to waste anyone's time with 'solving' these ostensibly quite trivial issues for me. I'm just voicing this because it seems like a valuable first-run experience recount.

[Lee-Si-Yoon]

Hi I'm facing difficulty using @codecov/vite-plugin with react + vite + nx monorepo what could I have missed?

"vite": "5.2.7",
"@codecov/vite-plugin": "^1.1.0",

// vite.config.mts
import { nxViteTsPaths } from '@nx/vite/plugins/nx-tsconfig-paths.plugin';
import react from '@vitejs/plugin-react-swc';
import { codecovVitePlugin } from '@codecov/vite-plugin';

export default defineConfig({
  // ...other configs
  plugins: [
    react(),
    nxViteTsPaths(),
    codecovVitePlugin({
      enableBundleAnalysis: process.env.CODECOV_TOKEN !== undefined,
      bundleName: 'dist',
      uploadToken: process.env.CODECOV_TOKEN,
    }),
  ],
});

[codecov] `get-pre-signed-url` fetch attempt 1 failed
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 2
[codecov] `get-pre-signed-url` fetch attempt 2 failed
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 3
[codecov] `get-pre-signed-url` fetch attempt 3 failed
[codecov] `get-pre-signed-url` failed after 2 attempts
[codecov] Failed to get pre-signed URL, bad response

[nicholas-codecov]

@Lee-Si-Yoon Are you running this locally or in CI? If you're attempting to run this locally, you'll need to update your Vite config to load in the env vars for you, you can read more about this in the Vite env docs, and the loadEnv docs.

There's some issues with loadEnv however when running in a CI environment, that is why we provide the process.env setup by default. You can also enable debug on the config for a bit more details (we're working on improving this).

[Lee-Si-Yoon]

Hi @nicholas-codecov, thank you for prompt response

I am running this locally, if I run build like below

// vite.config.mts
import { nxViteTsPaths } from '@nx/vite/plugins/nx-tsconfig-paths.plugin';
import react from '@vitejs/plugin-react-swc';
import { codecovVitePlugin } from '@codecov/vite-plugin';

export default defineConfig(() => {
  console.log('CODECOV: ', process.env.CODECOV_TOKEN);

  return {
    // ...other configs
    plugins: [
      react(),
      nxViteTsPaths(),
      codecovVitePlugin({
        debug: true,
        enableBundleAnalysis: process.env.CODECOV_TOKEN !== undefined,
        bundleName: 'dist',
        uploadToken: process.env.CODECOV_TOKEN,
      }),
    ],
  };
});

the log I get is

> vite build

CODECOV: e******* # <- env is loaded properly
vite v5.2.7 building for production...
...
[codecov] Detecting CI provider
[codecov] Detected CI provider: Local
[codecov] Using commit: null
computing gzip size (0)...
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 1
...
[codecov] `get-pre-signed-url` fetch attempt 1 failed
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 2
[codecov] `get-pre-signed-url` fetch attempt 2 failed
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 3
[codecov] `get-pre-signed-url` fetch attempt 3 failed
[codecov] `get-pre-signed-url` failed after 2 attempts
[codecov] Failed to get pre-signed URL, bad response

I assume [codecov] Using commit: null this is culprit. What could I do further to fix this?

[nicholas-codecov]

@Lee-Si-Yoon, yep that's going to be the problem. Hrrm, so when we detect that you're running locally we attempt to run Git commands to collect information from your local Git repository. Do you have any issue executing these commands in the same directory as your app?

Collects branch name:

git rev-parse --abbrev-ref HEAD

Collects commit sha:

git rev-parse HEAD

Collects repo slug:

git config --get remote.origin.url

For reference: https://github.com/codecov/codecov-javascript-bundler-plugins/blob/main/packages/bundler-plugin-core/src/utils/providers/Local.ts

[Lee-Si-Yoon]

@nicholas-codecov, I tested out in github CI with debug mode enabled and by using grabbing the SHA from the GHA event context. But I still have problem uploading bundle.

rendering chunks...
[codecov] Detecting CI provider
[codecov] Detected CI provider: GitHub Actions
computing gzip size...
[codecov] Using commit: 3a5c***
[codecov] Using compareSha: f72***
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 1

...

[codecov] Failed to validate pre-signed URL
✓ built in 40.26s

So I also tried out CLI way

[codecov] Detecting CI provider
[codecov] Detected CI provider: Local
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 1
[codecov] `get-pre-signed-url` fetch attempt 1 failed
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 2
[codecov] `get-pre-signed-url` fetch attempt 2 failed
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 3
[codecov] `get-pre-signed-url` fetch attempt 3 failed
[codecov] `get-pre-signed-url` failed after 2 attempts
[codecov] Failed to get pre-signed URL, bad response: "400 - Bad Request"
[codecov] An error occurred: Error: Failed to get pre-signed URL

But does not work. Coverage is uploaded well so I do not think it is related to wrong access token or commit order.
Please guide me what I can do further

[nicholas-codecov]

@Lee-Si-Yoon hrrm, let me check in with my co-worker to see if we can dive a bit deeper into the backend side of things.

[neelrocketbots]

@nicholas-codecov, I tested out in github CI with debug mode enabled and by using grabbing the SHA from the GHA event context. But I still have problem uploading bundle.

rendering chunks...
[codecov] Detecting CI provider
[codecov] Detected CI provider: GitHub Actions
computing gzip size...
[codecov] Using commit: 3a5c***
[codecov] Using compareSha: f72***
[codecov] Attempting to fetch get-pre-signed-url, attempt: 1

...

[codecov] Failed to validate pre-signed URL
✓ built in 40.26s
So I also tried out CLI way

[codecov] Detecting CI provider
[codecov] Detected CI provider: Local
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 1
[codecov] `get-pre-signed-url` fetch attempt 1 failed
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 2
[codecov] `get-pre-signed-url` fetch attempt 2 failed
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 3
[codecov] `get-pre-signed-url` fetch attempt 3 failed
[codecov] `get-pre-signed-url` failed after 2 attempts
[codecov] Failed to get pre-signed URL, bad response: "400 - Bad Request"
[codecov] An error occurred: Error: Failed to get pre-signed URL

But does not work. Coverage is uploaded well so I do not think it is related to wrong access token or commit order. Please guide me what I can do further

Any updates on this? Seems like I'm facing same issue. I'm using a private repository.

[nicholas-codecov]

@neelrocketbots and @Lee-Si-Yoon we've rolled out a fix for a small bug we believe may be causing this, this morning. Let us know if you continue to see this.

[neelrocketbots]

@neelrocketbots and @Lee-Si-Yoon we've rolled out a fix for a small bug we believe may be causing this, this morning. Let us know if you continue to see this.

Hey, @nicholas-codecov Still facing same issue

[sweden-snow]

Hi,
I'm facing the same issue as @neelrocketbots locally.
However, when my build runs via github actions it works

[nicholas-codecov]

Hey @neelrocketbots, @sweden-snow, sorry about this we need to update our docs a bit more on this, but if you're running locally with Vite you'll need to use their way of loading envs locally. If you're running in CI however, if my memory serves me correctly loadEnv won't actually load the env vars when they are injected into the environment.

cc @aj-codecov we need to update our docs/onboarding with these caveats.

[md384]

I tried to set this up for webpack4 but it looks like it is not supported - I get a Cannot read properties of undefined (reading 'tapPromise') and I see the peer dependency of webpack5 in the package.json.

I don't think I see the webpack version requirements in your documentation so I think the docs should be updated to specify webpack5 only.

[nicholas-codecov]

Thanks for the feedback @md384, I have updated the docs to make this a bit clearer.

[shamrt-v]

Apologies, false alarm. Turns out the issue is related to an Nx command — not the Codecov plugin.

I've spent the last while trying to integrate the Webpack plugin with CircleCI and it's not gone well. The Circle agents keep running out of memory, at it seems as though the problem is related to the mismatch between Circle's host agents' reported CPU count (32) vs the actually available CPUs (2).

The Webpack bundle analysis tool tries to use a bunch of CPUs that aren't actually available, and we get a situation like the following, which blows the machine up:

I think we need a param in the plugin to restrict/disable parallelization, or something of the sort.

[kwonoj]

Is there a way to debug bundle analysis graph? Plugin uploaded data, and each bundle chunks are listed with correct size but only graph / change over time is empty.

Using github action and it have correct fetch depth

- name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 2

Not sure if this matters, it's self hosted instance 25.1.3 (with timeseries disabled).

[nicholas-codecov]

@kwonoj I am fairly certain that timeseries is required to get the chart working on this page. @JerrySentry can you confirm that?

[brunobuddy]

Hello I would love to use bundle analysis with my npm package which is a NestJS bundle relying on TSC rather than the bundlers listed. How can i set this up

[nicholas-codecov]

Hey @brunobuddy 👋

Have you taken a peek at our bundle-analyzer package?

I think that may meet your requirements, as it does not require a bundler to operate!

[tonynchen]

Hello, we are unable to upload the bundle for analysis. We are using GitHub Actions, and I have attached the following errors with debugging turned on.

$ tsc && vite build
vite v5.2.8 building for production...
[codecov] Sending telemetry data on issues and performance to Codecov. To disable telemetry, set `options.telemetry` to `false`.
transforming...
✓ 3948 modules transformed.
rendering chunks...
[codecov] Detecting CI provider
[codecov] Detected CI provider: GitHub Actions
[codecov] Using branch: tchen/bundle-analysis
[codecov] Using build: 130****
[codecov] Using slug: ****/console
[codecov] Using build: 130****
computing gzip size...
[codecov] Failed to get job URL: 404
[codecov] Using slug: ****/console
[codecov] Using build: 130****
[codecov] Using build URL: https://github.com/****/console/actions/runs/130****
[codecov] Using commit: e1****
[codecov] Using compareSha: f3****
[codecov] Using job: build-and-test
[codecov] Using pr: 517
[codecov] Using slug: ****/console
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 1
dist/index.html                                           1.71 kB │ gzip:     0.81 kB
.
.
.
.
dist/assets/Overview-****.js                         35.15 kB │ gzip:     7.11 kB

(!) Some chunks are larger than 500 kB after minification. Consider:
- Using dynamic import() to code-split the application
- Use build.rollupOptions.output.manualChunks to improve chunking: https://rollupjs.org/configuration-options/#output-manualchunks
- Adjust chunk size limit for this warning via build.chunkSizeWarningLimit.
dist/assets/****-****.js                 35.28 kB │ gzip:    13.02 kB
.
.
dist/assets/index-****.js                        10,565.20 kB │ gzip: 2,997.78 kB
[codecov] `get-pre-signed-url` fetch attempt 1 failed
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 2
[codecov] `get-pre-signed-url` fetch attempt 2 failed
[codecov] Attempting to fetch `get-pre-signed-url`, attempt: 3
[codecov] `get-pre-signed-url` fetch attempt 3 failed
[codecov] `get-pre-signed-url` failed after 3 attempts
[codecov] Failed to get pre-signed URL, bad response: "400 - Bad Request"
[codecov] Error getting pre-signed URL: "Error: Failed to get pre-signed URL"
✓ built in 46.65s
Finished building UI

[brunobuddy]

Hello @nicholas-codecov

Thank you for this information. I made it with the package !

However for some reason it seams that it only checks the master branch. In all PRs the bundle bot appears to say that the bundle has not changed (which is false). Once the PR is merged to master it does take the bundle change in account though.

Ex:

• This PR reduces the Bundle size. However the report still says it's 3.3 Mb, same as master.
• The bot comments to says it has not change wereas it should be showing 3.1 Mb instead of 3.3 Mb

My GitHub Action seems fine:

      - name: Build App
        run: npm run build
      - name: Bundle analysis
        run: npx @codecov/bundle-analyzer ./dist --bundle-name=manifest --upload-token=${{ secrets.CODECOV_TOKEN }}
        working-directory: ./packages/core/manifest

Is there something I did not get or is it a bug ?

[nicholas-codecov]

Hey @brunobuddy!

Yeah, testing locally with the CLI the numbers align with what you say (or even slightly better), I will check with my team member to see if anything funky happened in the backend and get back to you!