Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] can not resolve source-map on Next.js 15 #161

Open
azu opened this issue Nov 11, 2024 · 8 comments
Open

[Bug] can not resolve source-map on Next.js 15 #161

azu opened this issue Nov 11, 2024 · 8 comments

Comments

@azu
Copy link

azu commented Nov 11, 2024

Describe the bug

I've tested next-with-playwright with Next.js 15, but the coverage file refer to js file, not ts file.
https://github.com/cenfun/nextjs-with-playwright

Probably, it can not resolve source-map.

To Reproduce

  • Version:
        "monocart-coverage-reports": "^2.11.2",
        "monocart-reporter": "^2.9.10",
  • Node Version: v22.6.0
  • Have you tried the latest version (including node), and does the issue still exist? Yes
  • Steps to reproduce the actual behavior:

Coverage refer to *.js files.

[MCR] Next.js V8 Coverage Report
┌────────────────────────────────────┬──────────┬────────────┬──────────┬───────────┬─────────┬────────────────────────────────────────────────────┐
│ Name                               │    Bytes │ Statements │ Branches │ Functions │   Lines │ Uncovered Lines                                    │
├────────────────────────────────────┼──────────┼────────────┼──────────┼───────────┼─────────┼────────────────────────────────────────────────────┤
│ .next/server/app                   │          │            │          │           │         │                                                    │
│ ├ about                            │          │            │          │           │         │                                                    │
│ │ └ page.js                        │  69.14 % │    82.86 % │          │   63.16 % │ 76.47 % │ 85-90,96-100,106-110,116-120,126-130,136-140,14... │
│ └ page.js                          │  70.29 % │    87.50 % │          │   66.67 % │ 82.11 % │ 85-89,95-99,105-109,115-119,125-129,135-139,167    │
│ localhost-3000/_next/static/chunks │          │            │          │           │         │                                                    │
│ ├ app-pages-internals.js           │  44.75 % │    55.56 % │  50.00 % │   57.89 % │ 62.50 % │ 9,36-41,47-52,58-63,69-74,80-85,91-96,102-107,1... │
│ ├ app                              │          │            │          │           │         │                                                    │
│ │ ├ about                          │          │            │          │           │         │                                                    │
│ │ │ └ page.js                      │   2.90 % │    12.50 % │  50.00 % │   16.00 % │ 27.97 % │ 9,25-30,36-41,47-52,58-63,69-74,80-85,91-96,102... │
│ │ └ page.js                        │ 100.00 % │   100.00 % │  50.00 % │  100.00 % │ 99.25 % │ 9                                                  │
│ ├ main-app.js                      │          │            │          │           │         │                                                    │
│ │ ├ v=1731284496315                │  99.70 % │    99.07 % │  50.00 % │   99.09 % │ 99.17 % │ 9,449-454,658-663                                  │
│ │ └ v=1731284496316                │  99.70 % │    99.07 % │  50.00 % │   99.09 % │ 99.17 % │ 9,449-454,658-663                                  │
│ └ webpack.js                       │          │            │          │           │         │                                                    │
│   ├ v=1731284496315                │  33.63 % │    31.35 % │  20.54 % │   37.34 % │ 32.07 % │ 21,40,66,96-98,106,116,119,125-128,162-165,171-... │
│   └ v=1731284496316                │  55.71 % │    54.15 % │  34.23 % │   67.72 % │ 52.29 % │ 21,40,66,96-98,106,116,119,125-128,162-165,200-... │
├────────────────────────────────────┼──────────┼────────────┼──────────┼───────────┼─────────┼────────────────────────────────────────────────────┤
│ Summary                            │  95.15 % │    63.33 % │  27.66 % │   77.25 % │ 67.36 % │                                                    │
└────────────────────────────────────┴──────────┴────────────┴──────────┴───────────┴─────────┴────────────────────────────────────────────────────┘

Reproduce repository:

Expected behavior

Coverage for *.tsx files

Errors or Screenshots

image

Make a minimal reproduction

Additional context

I've checked https://nextjs.org/docs/messages/improper-devtool, but the result is not changed.

const nextConfig = {
    productionBrowserSourceMaps: true,
    webpack: (config, options) => {
        
        if (process.env.NODE_V8_COVERAGE) {
            if (!options.dev) {
                config.devtool = options.isServer ? false : 'source-map';
            }
        }
        
        return config;
    }
};
export default nextConfig;
@cenfun
Copy link
Owner

cenfun commented Nov 11, 2024

It seems that the Next.js v15 does not output the source-map file anymore even sets config.devtool='source-map', it always reverting devtool to default eval-source-map, we can see it in the file begin comments

/*
 * ATTENTION: An "eval-source-map" devtool has been used.
 * This devtool is neither made for production nor for readable output files.
 * It uses "eval()" calls to create a separate source file with attached SourceMaps in the browser devtools.
 * If you are trying to read the output file, select a different devtool (https://webpack.js.org/configuration/devtool/)
 * or disable the default devtool with "devtool: false".
 * If you are looking for production-ready output files, see mode: "production" (https://webpack.js.org/configuration/mode/).
 */

Currently, MCR does not support parsing sourcemap from the eval-source-map.

I'll look into how to get sourcemap from eval, not sure if it will work, but it shoud work if you can downgrad from Nextjs v15 to v14.

@nphmuller
Copy link

nphmuller commented Dec 6, 2024

Maybe it's also a good idea to support build output from turbopack (next dev --turbo). That way the devtool hack is no longer necessary, since it outputs full source maps by default.

Turbopack uses section source maps, so I guess support for section source maps would need to be added if they aren't supported yet.

https://nextjs.org/blog/turbopack-for-development-stable#development-builds-that-closely-match-production
https://tc39.es/ecma426/#index-map

Edit:
Looks like support exists already! Cool! Now I just have to get a working example, because just enabling --turbo doesn't work.

2nd Edit:

Looks like clients components just work with turbopack. Only server components and actions don't map to the source properly, and a few too many files are included.

┌────────────────────────────────────────────────────┬──────────┬────────────┬──────────┬───────────┬──────────┬─────────────────┐
│ Name                                               │    Bytes │ Statements │ Branches │ Functions │    Lines │ Uncovered Lines │
├────────────────────────────────────────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ .next/server/app                                   │          │            │          │           │          │                 │
│ ├ about                                            │          │            │          │           │          │                 │
│ │ └ page.js                                        │ 100.00 % │   100.00 % │          │           │ 100.00 % │                 │
│ └ page.js                                          │ 100.00 % │   100.00 % │          │           │ 100.00 % │                 │
│ localhost-3000/_next/static/chunks                 │          │            │          │           │          │                 │
│ ├ [turbopack]_browser_dev_hmr-client_hmr-client... │  97.71 % │   100.00 % │  50.00 % │           │  81.82 % │ 1,5             │
│ ├ [turbopack]_browser_dev_hmr-client_hmr-client... │  99.57 % │   100.00 % │  50.00 % │  100.00 % │  92.86 % │ 1               │
│ ├ _c4c472._.js                                     │  99.26 % │   100.00 % │  75.00 % │           │  93.75 % │ 1               │
│ ├ src_app_about_page_tsx_db7418._.js               │  97.14 % │   100.00 % │  50.00 % │           │  81.82 % │ 1,5             │
│ ├ src_app_layout_tsx_36ac26._.js                   │  97.08 % │   100.00 % │  50.00 % │           │  81.82 % │ 1,5             │
│ └ src_app_page_tsx_db7418._.js                     │  97.04 % │   100.00 % │  50.00 % │           │  81.82 % │ 1,5             │
│ src/app                                            │          │            │          │           │          │                 │
│ └ counter.tsx                                      │  86.88 % │    85.71 % │  66.67 % │  100.00 % │  73.91 % │ 12-17           │
├────────────────────────────────────────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ Summary                                            │  97.91 % │    95.74 % │  57.14 % │  100.00 % │  86.09 % │                 │
└────────────────────────────────────────────────────┴──────────┴────────────┴──────────┴───────────┴──────────┴─────────────────┘

This is the server-only output (commented out fixtures, only included global-teardown):

┌──────────────────┬──────────┬────────────┬──────────┬───────────┬──────────┬─────────────────┐
│ Name             │    Bytes │ Statements │ Branches │ Functions │    Lines │ Uncovered Lines │
├──────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ .next/server/app │          │            │          │           │          │                 │
│ ├ about          │          │            │          │           │          │                 │
│ │ └ page.js      │ 100.00 % │   100.00 % │          │           │ 100.00 % │                 │
│ └ page.js        │ 100.00 % │   100.00 % │          │           │ 100.00 % │                 │
├──────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ Summary          │ 100.00 % │   100.00 % │          │           │ 100.00 % │                 │
└──────────────────┴──────────┴────────────┴──────────┴───────────┴──────────┴─────────────────┘

Edit 3 (sorry for the spam):

More progress. When I include || entry.url.includes("next/server/chunks") in playwright.config.ts and global-teardown.js the server component info is there in the output. But a few other files are also there which are not relevant...

Still server-only:

┌───────────────────────────────────────┬──────────┬────────────┬──────────┬───────────┬──────────┬────────────────────────────────────────────────────┐
│ Name                                  │    Bytes │ Statements │ Branches │ Functions │    Lines │ Uncovered Lines                                    │
├───────────────────────────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼────────────────────────────────────────────────────┤
│ .next/server                          │          │            │          │           │          │                                                    │
│ ├ app                                 │          │            │          │           │          │                                                    │
│ │ ├ about                             │          │            │          │           │          │                                                    │
│ │ │ └ page.js                         │ 100.00 % │   100.00 % │          │           │ 100.00 % │                                                    │
│ │ └ page.js                           │ 100.00 % │   100.00 % │          │           │ 100.00 % │                                                    │
│ └ chunks/ssr                          │          │            │          │           │          │                                                    │
│   ├ [root-of-the-server]__adba9d._.js │  90.48 % │    95.00 % │          │   85.71 % │  90.48 % │ 51-55                                              │
│   ├ [root-of-the-server]__d3cb94._.js │  14.29 % │   100.00 % │          │  100.00 % │   6.67 % │ 5-41                                               │
│   └ [turbopack]_runtime.js            │  51.36 % │    41.16 % │  33.33 % │   27.50 % │  39.83 % │ 27-33,48-85,94,124-126,133,143-145,153-201,208-... │
│ src/app                               │          │            │          │           │          │                                                    │
│ ├ about                               │          │            │          │           │          │                                                    │
│ │ ├ actions.ts                        │  94.83 % │    88.89 % │  50.00 % │  100.00 % │  91.67 % │ 8                                                  │
│ │ └ page.tsx                          │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                                                    │
│ ├ counter.tsx                         │  74.92 % │   100.00 % │  50.00 % │  100.00 % │  60.87 % │ 12-20,24                                           │
│ ├ counter.tsx                         │          │            │          │           │          │                                                    │
│ │ ├ proxy.js                          │  38.77 % │    66.67 % │          │   50.00 % │  83.33 % │ 3                                                  │
│ │ └ proxy.js                          │  39.65 % │    66.67 % │          │    0.00 % │  83.33 % │ 3                                                  │
│ ├ layout.tsx                          │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                                                    │
│ └ page.tsx                            │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                                                    │
├───────────────────────────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼────────────────────────────────────────────────────┤
│ Summary                               │  57.32 % │    52.41 % │  34.15 % │   41.90 % │  48.15 % │                                                    │
└───────────────────────────────────────┴──────────┴────────────┴──────────┴───────────┴──────────┴────────────────────────────────────────────────────┘

@cenfun
Copy link
Owner

cenfun commented Dec 7, 2024

@nphmuller Thanks for your suggestion, it's good news.

Yes, MCR already supports section sourcemap. And the latest Next.js will use turbopack instead of webpack, we should migrate to turbopack if possible.

The above issue is not a coverage but a sourcemap issue.
However, the sourcemaps are very complex, and each tool generates a different sourcemap. For example, webpack, esbuild, swc, turbopack, rollup, etc., so many build tools generate different sourcemaps even for the same file.
Maybe the new turbopack is not stable enough to support sufficiently accurate sourcemaps, keep watching.

I am not an expert in Next.js. If you have more good suggestions or solutions to the issue are welcome, and thanks for your help.

@cenfun
Copy link
Owner

cenfun commented Dec 7, 2024

I tried updating the repo to support Nextjs turbopack (branch turbopack)
https://github.com/cenfun/nextjs-with-playwright/tree/turbopack

It works but not perfect. For example, file _c4c472._.js does not have any features for filtering.

[MCR] Next.js V8 Coverage Report
┌────────────────────────────────────┬──────────┬────────────┬──────────┬───────────┬──────────┬─────────────────┐
│ Name                               │    Bytes │ Statements │ Branches │ Functions │    Lines │ Uncovered Lines │
├────────────────────────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ localhost-3000/_next/static/chunks │          │            │          │           │          │                 │
│ └ _c4c472._.js                     │  99.26 % │   100.00 % │  75.00 % │           │  93.75 % │ 1               │
│ src/app                            │          │            │          │           │          │                 │
│ ├ about                            │          │            │          │           │          │                 │
│ │ ├ actions.ts                     │  95.11 % │    88.89 % │  50.00 % │  100.00 % │  91.67 % │ 8               │
│ │ └ page.tsx                       │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                 │
│ ├ counter.tsx                      │  85.37 % │   100.00 % │ 100.00 % │  100.00 % │  69.57 % │ 12-17,24        │
│ ├ layout.tsx                       │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                 │
│ └ page.tsx                         │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                 │
├────────────────────────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ Summary                            │  95.55 % │    97.22 % │  83.33 % │  100.00 % │  90.22 % │                 │
└────────────────────────────────────┴──────────┴────────────┴──────────┴───────────┴──────────┴─────────────────┘

(It requires the latest version monocart-coverage-reports@2.11.4)

@nphmuller
Copy link

nphmuller commented Dec 7, 2024

Very good to hear!

I’m still tinkering myself with the config to get better result. But it’s pretty complex and some solutions I found are really hacky, so hopefully we can still improve them.

I see for the counter.tsx/proxy.js problem you also found the same solution as me. My filter is a little bit different, because just filtering proxy.js would filter out actual source files with the same name (pretty unlikely but still) so I changed the filter to .tsx/proxy.js.

For the c4c472 file: I found it was the only chunk without a map file. Since I could find no way to filter this way (would be ideal for me if there was an option to exclude everything without a source map), I added this really hacky workaround in entryFilter:

    if (entry.source?.includes("sourceMappingURL") === false) {
      return false;
    }

There's also an issue with client components (counter.tsx) not showing the correct branch information. This is caused by one of the included server chunks. Still not happy about this, but I initially fixed that by including this entryFilter:

    // '[project]/src/app/counter.tsx [app-ssr] (ecmascript)
    // ssr version of client component, we should exclude these here because they conflict with the client-side bundle, which results in incorrect coverage statistics.
    if (entry.functions?.some((f) => f.functionName.includes("[app-ssr]"))) {
      return false;
    }

Again, I'm still tinkering but wanted to share what I had so far.
I'll have more time to tinker the coming week. :)

@nphmuller
Copy link

@cenfun In the output you posted the coverage of actions.ts is correct. But when I run your latest commit on the turbopack branch I get this output with incorrect actions.ts coverage. I didn't make any changes to your branch locally. Is the output still correct on your side?

┌────────────────────────────────────┬──────────┬────────────┬──────────┬───────────┬──────────┬─────────────────┐
│ Name                               │    Bytes │ Statements │ Branches │ Functions │    Lines │ Uncovered Lines │
├────────────────────────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ localhost-3000/_next/static/chunks │          │            │          │           │          │                 │
│ └ _c4c472._.js                     │  99.26 % │   100.00 % │  75.00 % │           │  93.75 % │ 1               │
│ src/app                            │          │            │          │           │          │                 │
│ ├ about                            │          │            │          │           │          │                 │
│ │ ├ actions.ts                     │ 100.00 % │    90.00 % │  50.00 % │  100.00 % │ 100.00 % │                 │
│ │ └ page.tsx                       │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                 │
│ ├ counter.tsx                      │  85.38 % │   100.00 % │ 100.00 % │  100.00 % │  69.57 % │ 12-17,24        │
│ ├ layout.tsx                       │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                 │
│ └ page.tsx                         │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                 │
├────────────────────────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ Summary                            │  96.24 % │    97.37 % │  83.33 % │  100.00 % │  91.30 % │                 │
└────────────────────────────────────┴──────────┴────────────┴──────────┴───────────┴──────────┴─────────────────┘

@cenfun
Copy link
Owner

cenfun commented Dec 9, 2024

here is my local output with turbopack branch

[MCR] Next.js V8 Coverage Report
┌────────────────────────────────────┬──────────┬────────────┬──────────┬───────────┬──────────┬─────────────────┐
│ Name                               │    Bytes │ Statements │ Branches │ Functions │    Lines │ Uncovered Lines │
├────────────────────────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ localhost-3000/_next/static/chunks │          │            │          │           │          │                 │
│ └ _c4c472._.js                     │  99.26 % │   100.00 % │  75.00 % │           │  93.75 % │ 1               │
│ src/app                            │          │            │          │           │          │                 │
│ ├ about                            │          │            │          │           │          │                 │
│ │ ├ actions.ts                     │  95.11 % │    88.89 % │  50.00 % │  100.00 % │  91.67 % │ 8               │
│ │ └ page.tsx                       │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                 │
│ ├ counter.tsx                      │  85.37 % │   100.00 % │ 100.00 % │  100.00 % │  69.57 % │ 12-17,24        │
│ ├ layout.tsx                       │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                 │
│ └ page.tsx                         │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                 │
├────────────────────────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ Summary                            │  95.55 % │    97.22 % │  83.33 % │  100.00 % │  90.22 % │                 │
└────────────────────────────────────┴──────────┴────────────┴──────────┴───────────┴──────────┴─────────────────┘

npm version 10.8.2
node version v20.18.0
OS Windows 10

Could you please try to reinstall? (remove package-lock.json and node_modules then npm install)

@nphmuller
Copy link

nphmuller commented Dec 9, 2024

I did, and I also deleted the .next folder. I also cloned the repository again, just to be on the safe side, but the coverage for actions.ts still doesn't show correctly (incorrect percentages + uncovered lines missing). It could be my machine, but I'm not sure how. Unless it's a Windows vs Mac difference. (It is a Windows vs Mac difference, see update below)

npm 10.8.2
Node v20.18.1
Mac M1 MacOS 15.1

If I have to provide more details, just let me know.

After some more tinkering, this is the config I ended up with. Might still move it to a mcr.config.js, and hopefully we can find a way to remove the hacks. But otherwise I'm happy enough with it.

global-teardown.js and fixtures.js are the same as your code, so no special filtering is done there.

This is my playwright config section:

const coverageReportOptions: CoverageReportOptions = {
  name: "Next.js V8 Coverage Report",

  all: {
    dir: "src",
    filter: "**/*{js,ts,jsx,tsx}",
  },

  entryFilter: (entry) => {
    // skip files without sourcemap 
    // this fixes _c4c472._.js showing up
    if (entry.source?.includes("sourceMappingURL") === false) {
      return false;
    }

    // client-side
    if (entry.url.includes("next/static/")) {
      return true;
    }

    // server-side
    if (
      entry.url.includes("next/server/") &&
      // without this line coverage for server actions is not detected correctly
      entry.url.startsWith("file:") &&
      // ssr version of client component, including these messes up line coverage of client components
      !entry.functions?.some((f) => f.functionName.includes("[app-ssr]"))
    ) {
      return true;
    }

    return false;
  },

  sourceFilter: (sourcePath) => {
    return (
      !sourcePath.startsWith("node_modules/") &&
      !sourcePath.startsWith("turbopack/") &&
      !sourcePath.endsWith(".tsx/proxy.js") // generated proxies for client components
    );
  },

  reports: ["v8", "console-details"],
};

It results in the following output (I added a file with no tests, to test the all config flag):

┌────────────────┬──────────┬────────────┬──────────┬───────────┬──────────┬─────────────────┐
│ Name           │    Bytes │ Statements │ Branches │ Functions │    Lines │ Uncovered Lines │
├────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ src/app        │          │            │          │           │          │                 │
│ ├ about        │          │            │          │           │          │                 │
│ │ ├ actions.ts │  94.53 % │    88.89 % │  50.00 % │  100.00 % │  91.67 % │ 8               │
│ │ └ page.tsx   │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                 │
│ ├ counter.tsx  │  88.74 % │    85.71 % │  66.67 % │  100.00 % │  73.91 % │ 12-17           │
│ ├ layout.tsx   │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                 │
│ ├ page.tsx     │ 100.00 % │   100.00 % │          │  100.00 % │ 100.00 % │                 │
│ └ unused       │          │            │          │           │          │                 │
│   └ page.tsx   │   0.00 % │     0.00 % │          │    0.00 % │   0.00 % │ 1-3             │
├────────────────┼──────────┼────────────┼──────────┼───────────┼──────────┼─────────────────┤
│ Summary        │  92.44 % │    83.78 % │  62.50 % │   90.91 % │  87.01 % │                 │
└────────────────┴──────────┴────────────┴──────────┴───────────┴──────────┴─────────────────┘

Update:
Just cloned your branch on my Windows machine and the output for actions.ts immediately works, so there might be a mac vs windows difference here. The coverage on counter.tsx is still incorrect though (but that's the same for both our outputs).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants