API routes with 3rd party dependencies fail the build

[MKrupauskas]

Describe the bug
Using third party libraries in nextjs API routes breaks the build. It might be related to netlify-lambda functions not bundling 3rd party libraries like firebase nicely. See the possibly related issue below.

To Reproduce
Steps to reproduce the behavior:

1. Create an example project
2. Create an API route /pages/api/form.js
3. Install firebase-admin and use the library inside the API route
4. Deploy code on netlify and see a failed build, even when the local build passes.

Expected behavior
The build should not break after adding and using 3rd party dependencies like firebase-admin in the API routes. These libraries should be accessible in the generated lambda functions.

Versions

• Next.js: 10

Related issues:, there seems to be a possible workaround for netlify-lambda but not if the project uses nextjs which is the main point of the issue netlify/netlify-lambda#112

Attached logs:

11:10:02 PM: Build ready to start
11:10:04 PM: build-image version: 2cee85eb7f808bf3b6e87378c5307f9411f0a332
11:10:04 PM: build-image tag: v3.8.0
11:10:04 PM: buildbot version: 487f6d01b44481b4d57253f3d6071f198697110f
11:10:04 PM: Fetching cached dependencies
11:10:04 PM: Starting to download cache of 460.3MB
11:10:07 PM: Finished downloading cache in 2.356543225s
11:10:07 PM: Starting to extract cache
11:10:15 PM: Finished extracting cache in 8.59295375s
11:10:15 PM: Finished fetching cache in 11.078571618s
11:10:15 PM: Starting to prepare the repo for build
11:10:16 PM: Preparing Git Reference refs/heads/main
11:10:20 PM: Parsing package.json dependencies
11:10:21 PM: Different publish path detected, going to use the one specified in the Netlify configuration file: 'out' versus '/' in the Netlify UI
11:10:21 PM: Different build command detected, going to use the one specified in the Netlify configuration file: 'npm run build' versus '' in the Netlify UI
11:10:21 PM: Starting build script
11:10:21 PM: Installing dependencies
11:10:21 PM: Python version set to 2.7
11:10:22 PM: Started restoring cached node version
11:10:24 PM: Finished restoring cached node version
11:10:24 PM: v12.18.0 is already installed.
11:10:25 PM: Now using node v12.18.0 (npm v6.14.4)
11:10:25 PM: Started restoring cached build plugins
11:10:25 PM: Finished restoring cached build plugins
11:10:25 PM: Attempting ruby version 2.7.1, read from environment
11:10:26 PM: Started restoring cached ruby version
11:10:26 PM: Finished restoring cached ruby version
11:10:28 PM: Using ruby version 2.7.1
11:10:28 PM: Using PHP version 5.6
11:10:28 PM: Started restoring cached yarn cache
11:10:28 PM: Finished restoring cached yarn cache
11:10:29 PM: No yarn workspaces detected
11:10:29 PM: Started restoring cached node modules
11:10:29 PM: Finished restoring cached node modules
11:10:29 PM: Installing NPM modules using Yarn version 1.22.4
11:10:29 PM: yarn install v1.22.4
11:10:29 PM: warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
11:10:29 PM: [1/4] Resolving packages...
11:10:30 PM: [2/4] Fetching packages...
11:10:47 PM: info fsevents@2.3.2: The platform "linux" is incompatible with this module.
11:10:47 PM: info "fsevents@2.3.2" is an optional dependency and failed compatibility check. Excluding it from installation.
11:10:47 PM: [3/4] Linking dependencies...
11:10:47 PM: warning "@netlify/plugin-nextjs > @sls-next/lambda-at-edge > @vercel/nft > acorn-export-ns-from@0.1.0" has incorrect peer dependency "acorn@^6.0.1".
11:10:47 PM: warning "@netlify/plugin-nextjs > @sls-next/lambda-at-edge > @aws-sdk/client-s3 > @aws-sdk/middleware-retry > react-native-get-random-values@1.7.0" has unmet peer dependency "react-native@>=0.56".
11:10:47 PM: warning " > bootstrap@5.0.1" has unmet peer dependency "@popperjs/core@^2.9.2".
11:10:47 PM: warning "firebase-admin > @firebase/database > @firebase/auth-interop-types@0.1.6" has unmet peer dependency "@firebase/app-types@0.x".
11:10:47 PM: warning " > lottie-react@2.1.0" has unmet peer dependency "prop-types@^15.5.7".
11:10:47 PM: warning " > lottie-react@2.1.0" has incorrect peer dependency "react@^16.8.0".
11:10:47 PM: warning " > lottie-react@2.1.0" has incorrect peer dependency "react-dom@^16.8.0".
11:10:54 PM: [4/4] Building fresh packages...
11:10:56 PM: success Saved lockfile.
11:10:56 PM: Done in 27.41s.
11:10:57 PM: NPM modules installed using Yarn
11:10:57 PM: Started restoring cached go cache
11:10:57 PM: Finished restoring cached go cache
11:10:57 PM: go version go1.14.4 linux/amd64
11:10:57 PM: go version go1.14.4 linux/amd64
11:10:57 PM: Installing missing commands
11:10:57 PM: Verify run directory
11:10:57 PM: ​
11:10:57 PM: ────────────────────────────────────────────────────────────────
11:10:57 PM:   Netlify Build                                                 
11:10:57 PM: ────────────────────────────────────────────────────────────────
11:10:57 PM: ​
11:10:57 PM: ❯ Version
11:10:57 PM:   @netlify/build 15.11.0
11:10:57 PM: ​
11:10:57 PM: ❯ Flags
11:10:57 PM:   deployId: 60eb501ad664aa00079bcebd
11:10:57 PM: ​
11:10:57 PM: ❯ Current directory
11:10:57 PM:   /opt/build/repo
11:10:57 PM: ​
11:10:57 PM: ❯ Config file
11:10:57 PM:   /opt/build/repo/netlify.toml
11:10:57 PM: ​
11:10:57 PM: ❯ Context
11:10:57 PM:   production
11:10:58 PM: ​
11:10:58 PM: ❯ Loading plugins
11:10:58 PM:    - @netlify/plugin-nextjs@3.3.0 from netlify.toml and package.json
11:10:58 PM: ​
11:10:58 PM: ❯ Outdated plugins
11:10:58 PM:    - @netlify/plugin-nextjs@3.3.0: latest version is 3.6.3
11:10:58 PM:      To upgrade this plugin, please update its version in "package.json"
11:10:58 PM: ​
11:10:58 PM: ────────────────────────────────────────────────────────────────
11:10:58 PM:   1. onPreBuild command from @netlify/plugin-nextjs             
11:10:58 PM: ────────────────────────────────────────────────────────────────
11:10:58 PM: ​
11:10:59 PM: info  - Using webpack 5. Reason: no custom webpack configuration in next.config.js https://nextjs.org/docs/messages/webpack5
11:11:00 PM: Next.js cache restored.
11:11:00 PM: ​
11:11:00 PM: (@netlify/plugin-nextjs onPreBuild completed in 1.5s)
11:11:00 PM: ​
11:11:00 PM: ────────────────────────────────────────────────────────────────
11:11:00 PM:   2. build.command from netlify.toml                            
11:11:00 PM: ────────────────────────────────────────────────────────────────
11:11:00 PM: ​
11:11:00 PM: $ npm run build
11:11:00 PM: > brokoli-network@0.1.0 build /opt/build/repo
11:11:00 PM: > next build
11:11:01 PM: info  - Using webpack 5. Reason: no custom webpack configuration in next.config.js https://nextjs.org/docs/messages/webpack5
11:11:01 PM: info  - Checking validity of types...
11:11:01 PM: info  - Creating an optimized production build...
11:11:35 PM: Failed to compile.
11:11:35 PM: 
11:11:35 PM: ModuleNotFoundError: Module not found: Error: Can't resolve '@firebase/app' in '/opt/build/repo/node_modules/@firebase/database/dist'
11:11:35 PM: > Build error occurred
11:11:35 PM: Error: > Build failed because of webpack errors
11:11:35 PM:     at /opt/build/repo/node_modules/next/dist/build/index.js:17:924
11:11:35 PM:     at async Span.traceAsyncFn (/opt/build/repo/node_modules/next/dist/telemetry/trace/trace.js:6:584)
11:11:35 PM: npm ERR! code ELIFECYCLE
11:11:35 PM: npm ERR! errno 1
11:11:35 PM: npm ERR! brokoli-network@0.1.0 build: `next build`
11:11:35 PM: npm ERR! Exit status 1
11:11:35 PM: npm ERR!
11:11:35 PM: npm ERR! Failed at the brokoli-network@0.1.0 build script.
11:11:35 PM: npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
11:11:35 PM: npm ERR! A complete log of this run can be found in:
11:11:35 PM: npm ERR!     /opt/buildhome/.npm/_logs/2021-07-11T20_11_35_296Z-debug.log
11:11:35 PM: ​
11:11:35 PM: ────────────────────────────────────────────────────────────────
11:11:35 PM:   "build.command" failed                                        
11:11:35 PM: ────────────────────────────────────────────────────────────────
11:11:35 PM: ​
11:11:35 PM:   Error message
11:11:35 PM:   Command failed with exit code 1: npm run build
11:11:35 PM: ​
11:11:35 PM:   Error location
11:11:35 PM:   In build.command from netlify.toml:
11:11:35 PM:   npm run build
11:11:35 PM: ​
11:11:35 PM:   Resolved config
11:11:35 PM:   build:
11:11:35 PM:     command: npm run build
11:11:35 PM:     commandOrigin: config
11:11:35 PM:     environment:
11:11:35 PM:       - FIREBASE_CLIENT_EMAIL
11:11:35 PM:       - FIREBASE_DATABASE_URL
11:11:35 PM:       - FIREBASE_PRIVATE_KEY
11:11:35 PM:       - NEXT_PUBLIC_FIREBASE_PROJECT_ID
11:11:35 PM:     publish: /opt/build/repo/out
11:11:35 PM:     publishOrigin: config
11:11:35 PM:   plugins:
11:11:35 PM:     - inputs: {}
11:11:35 PM:       origin: config
11:11:35 PM:       package: '@netlify/plugin-nextjs'
11:11:35 PM: Caching artifacts
11:11:35 PM: Started saving node modules
11:11:35 PM: Finished saving node modules
11:11:35 PM: Started saving build plugins
11:11:35 PM: Finished saving build plugins
11:11:35 PM: Started saving yarn cache
11:11:35 PM: Finished saving yarn cache
11:11:35 PM: Started saving pip cache
11:11:35 PM: Finished saving pip cache
11:11:35 PM: Started saving emacs cask dependencies
11:11:35 PM: Finished saving emacs cask dependencies
11:11:35 PM: Started saving maven dependencies
11:11:35 PM: Finished saving maven dependencies
11:11:35 PM: Started saving boot dependencies
11:11:35 PM: Finished saving boot dependencies
11:11:35 PM: Started saving rust rustup cache
11:11:35 PM: Finished saving rust rustup cache
11:11:35 PM: Started saving go dependencies
11:11:35 PM: Finished saving go dependencies
11:11:35 PM: Build failed due to a user error: Build script returned non-zero exit code: 2
11:11:36 PM: Creating deploy upload records
11:11:36 PM: Failing build: Failed to build site
11:11:36 PM: Failed during stage 'building site': Build script returned non-zero exit code: 2
11:11:36 PM: Finished processing build request in 1m31.62083348s

[lindsaylevine]

@MKrupauskas hello! can you try adding target: "experimental-serverless-trace" to your next.config.js? that was the solution for some older issues (netlify/next-on-netlify#68 and netlify/next-on-netlify#66) on this plugin's predecessor.

[MKrupauskas]

Hi! Thanks for the help, it solved the issue. Makes me wonder why this is not the default then?

[lindsaylevine]

@MKrupauskas of course! and yeah, good question. the original next-on-netlify author and i once discussed defaulting to e-s-t, and we ultimately chose not to because it increased bundle sizes for people a lot. now we have esbuild as a node_bundler option in our docs to alleviate that symptom. there's also a chance we could be switching off target: serverless in the future (to target: server), which would hopefully solve some of the root issues we see with serverless.