Vercel
Exceeding size limits (50mb) for Serverless Functions #103
Replies: 63 comments 160 replies
-
|
Here's an example of using taking screenshots inside a function, which does not exceed limits: https://github.com/vercel/virtual-event-starter-kit/blob/main/lib/screenshot.ts |
Beta Was this translation helpful? Give feedback.
-
|
I'm curious, what is the approach here in this repo, particularly with |
Beta Was this translation helpful? Give feedback.
-
|
These are my dependencies for reference (Yarn 3 + Node 14): |
Beta Was this translation helpful? Give feedback.
-
|
hey do i need process.env.AWS_REGION too in my code, what to put in this?? Kindly let me know i'm stuck with function hits the 50mb limit problem, I am also using puppeteer and chrome-aws-lambda in my API call.
|
Beta Was this translation helpful? Give feedback.
-
|
I have a similar setup, but in my case, I keep hitting the 50MB limit, what would be the best approach to make some reduction here? |
Beta Was this translation helpful? Give feedback.
-
My situation was similar to yours and despite managing to reduce the size below the 50MB limit, if there is a dependency update that increases the size a bit, chances are that the build will fail. This happened to me when I updated to the most recent Next.js version and had a PDF generator coupled in the app. So I ended up moving the PDF generator logic to a separate app. Now I only have these dependencies: The compressed size is 49.4 MB with chrome-aws-lambda 6 and puppeteer 6. It's still a small margin to work it but it's ok (for now). |
Beta Was this translation helpful? Give feedback.
-
|
For me it happens because it's uploading all my assets in /public to the serverless functions. To avoid it, I have to adding "excludeFiles" to vercel.json. This would be fine if there wasn't a related bug, where excludeFiles can't be applied to "pages/api/*", which is where the serverless functions normally are. What I had to do to fix it is moving all serverless functions to /api (outside of pages), and now excludeFiles is working. But this creates other headaches as now my api functions are not working using "yarn dev". Could you take a look into "excludeFiles" and why it doesn't work on /pages/api? |
Beta Was this translation helpful? Give feedback.
-
|
Hey @leerob , it's still not working after 4 months :( |
Beta Was this translation helpful? Give feedback.
-
|
Issue seems to be back with next 12 upgrade |
Beta Was this translation helpful? Give feedback.
-
|
Hello there, I seem that I had a related issue and I found what was causing it (at least for me). https://nextjs.org/docs/basic-features/image-optimization#local-images Well after rolling back to a version where I didn't implement this feature, it fixed the issue for me, in case this can help you. Prior to that I was just reducing the amount of pages that could appear in the dynamic route that was throwing the error; so if I had /[post].js, instead of having 30 posts I was chopping it down to 15 (less images in total) so I didn't get this error, If anyone can explain better why statically importing images with the "require" statement was causing this issue, I would love to understand it better. Kind regards |
Beta Was this translation helpful? Give feedback.
-
|
Any update here? I'm running in to this issue for the first time but it's happening with pages with |
Beta Was this translation helpful? Give feedback.
-
|
I'm running into this issue as well -- it's including my To fix this, I'm attempting to I believe this is caused by me using custom For example, the source file for the page path Here's my @leerob any advice? Thanks~ |
Beta Was this translation helpful? Give feedback.
-
|
Note: Python currently doesn't do any bundling, so this is still the correct solution mentioned: vercel/vercel#2830 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
I have the same issue using the date.py demo from the docs. |
Beta Was this translation helpful? Give feedback.
-
|
same here, very frustrating... |
Beta Was this translation helpful? Give feedback.
-
|
FWIW, I experienced this issue and clearing (I'd accidentally added a large package that pushed me over the 50mb limit, then realized and removed it, but vercel was leaving it in cache even after it was gone from |
Beta Was this translation helpful? Give feedback.
-
|
This issues is ridiculous. Thanks rf for this insight. Hope it works for me... |
Beta Was this translation helpful? Give feedback.
-
|
I try this |
Beta Was this translation helpful? Give feedback.
-
|
Hi Mr. Robinson! We have an api route that is very small and simply sends an email and should not exceed any limits. This is our api route: https://github.com/New-Millenia-Consulting/operation-medical-care/blob/main/pages/api/contact.ts However, I believe this is simply due to incorrect reporting of the error. What was actually happening was that we had a page that was statically importing many many images for Next 11's image blurs. See this commit. Once we changed all of these static imports back to standard string sources, the build passed. So I believe that in our case, the serverless function generated for the about page (in the commit linked) is the function actually exceeding the limit. There is nothing wrong with the Edit: There have been a few commits since the issue but I am able to find the exact commits where the issue occurs if you would like something reproducible. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you! This is very helpful. |
Beta Was this translation helpful? Give feedback.
-
|
This should have been fixed by vercel/next.js#26843 which is available in cc @pancho111203 |
Beta Was this translation helpful? Give feedback.
-
|
That fix works great! Thank you so much! |
Beta Was this translation helpful? Give feedback.
-
|
Yep, fixed it for me too! Thank you, Lee! |
Beta Was this translation helpful? Give feedback.
-
|
Hey @leerob, looks like the issue is back in Next.js 12. I checked, the code and the #26843 fix is still there, but the symptom is the same:
|
Beta Was this translation helpful? Give feedback.
-
|
For some reason, I have to exclude |
Beta Was this translation helpful? Give feedback.
-
|
We currently ran into this problem aswell - running here are the compiled lambda functions, wich vercel is supposed to push to aws: but instead it looks like he pushes the whole node_modules src to the lambda function: would be great if someone can advise what we can try to not include these |
Beta Was this translation helpful? Give feedback.
-
|
Hi! Are you still using this hack or did you end up finding a better one? |
Beta Was this translation helpful? Give feedback.
-
|
@lucas-janon getting |
Beta Was this translation helpful? Give feedback.
-
|
@jwang919 @lucas-janon Found a solution thanks to @darkopetrovic! |
Beta Was this translation helpful? Give feedback.
-
|
@MauriceArikoglu this doesn't seem like an option in nextConfig according to these docs? I am also having this issue with the canvas library (as dependency of pdfjs-dist) and vercel :( |
Beta Was this translation helpful? Give feedback.
-
It is not an option but still works with a warning in 13.2.4 |
Beta Was this translation helpful? Give feedback.
-
|
Got this since I haven't defined a specific version of the nuxt/vercel-builder and it had an upgrade to 0.22 which breaks. After defining a version for in the config manually, it works like a charm. Maybe that helps someone. |
Beta Was this translation helpful? Give feedback.
-
|
I just realized that by using |
Beta Was this translation helpful? Give feedback.
-
|
This is such a headache :( |
Beta Was this translation helpful? Give feedback.
-
|
Hi everyone, I'm new to deploying with Vercel and using Nuxt and I am running into this issue as well. I read somewhere that I should move everything from Here is the error: Here are my dependencies: |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for all the help Fabian, I really appreciate it. here is the package.json: and this is the vercel.json: |
Beta Was this translation helpful? Give feedback.
-
|
@jakexia72 Okay, so this seems a bit trickier than I thought. Can't find the problem in your config files. Maybe you can provide
It could be the case, that you are using some plugin or some file that is required in build step by vercel, and must be added in "serverFiles" too. For example, when you |
Beta Was this translation helpful? Give feedback.
-
|
I tried adding some more files into the severFiles section. That didn't work (but I'm also not sure which files to add). Anyways; here are the relevant documents: Nuxt.config.js Here are the build details:
|
Beta Was this translation helpful? Give feedback.
-
|
Were you able to resolve the issue in the meantime? |
Beta Was this translation helpful? Give feedback.
-
|
Unfortunately not; I have contacted Vercel support and I will update here when I have more information :) |
Beta Was this translation helpful? Give feedback.
-
|
I can't get excludes to work properly. I have a minimal repo here where my function takes <5mb and another one here where the function takes 69.61mb. I can't figure out what to exclude. Excluding Yet, clearly, the bundle contains a bunch of junk that the code does not depend on. Note: I'm not using Next. |
Beta Was this translation helpful? Give feedback.
-
|
I'm having issues with this since updating to Next12, seems like all the files from a folder that's accessed in a getStaticProps() function get taken into the bundle. The serverless function is used to read .md files to display on the specific page, based on a slug. |
Beta Was this translation helpful? Give feedback.
-
|
Having exactly the same issue here.
|
Beta Was this translation helpful? Give feedback.
-
|
+1 I'm having this issue as well |
Beta Was this translation helpful? Give feedback.
-
|
still facing the same issue, any success? |
Beta Was this translation helpful? Give feedback.
-
|
Here is the minimal reproducible repo. If I add an API route or SSR route it fails even though API/SSR route has no dependecy on the images (which have 50mb+ size and are referred only in SSG pages).
Error screenshot:
|
Beta Was this translation helpful? Give feedback.
-
|
I have a randomly appeared warning about 50mb as well on next12 and python. Why is it randomly? If I press "redeploy" button on the build page of my project, it can go well. Actually, it always goes well if I press "redeploy". from http.server import BaseHTTPRequestHandler
from datetime import datetime
class handler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header('Content-type', 'text/plain')
self.end_headers()
self.wfile.write(str(datetime.now().strftime('%Y-%m-%d %H:%M:%S')).encode())
returnWhile js-driven functions use 7-8Mb.
This makes CI/CD process unreliable and manual: i have to press 'redeploy' button every time it happens (30-40% of time, when the python function is the same) |
Beta Was this translation helpful? Give feedback.
-
|
Experiencing similar issues. Was borrowing some code from this repo (to generate og-images). The above repo seems to deploy just fine w/o any size problems. However, on our repo the serverless function balloons up to 75mb and seems to include a bunch of unrelated files. Not sure whats going on and have opened up a support ticket. Haven't received a response yet. We're on Next |
Beta Was this translation helpful? Give feedback.
-
@leerob, it's been 2 years and 0 updates on this. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, I'm also facing the same issue when trying to deploy my django app ( it's a portfolio). Though on disk the size of build files is 20 MB and the requirements size is 16 MB I still get this error. I have also tried to add I;m very new new vercel deployment. Earlier I used heroku and railway to deploy but, heroku removing the free tier I had to switch to other PaaS. I'm planning to try out vercel. If you can help me out with this it would be really great. I have also removed all the files under I want to know some ways on how to implement or config the
Please help me out thank you!! |
Beta Was this translation helpful? Give feedback.
-
|
I am also having the same problem, Next 13.4.1. I already tried every approach possible, but I can't seem to work around this. Is it possible that Next 13.4.1 is causing this? I tried using versions prior to Next 13 and it works fine. |
Beta Was this translation helpful? Give feedback.
-
|
@gpmotta21 conseguiu achar alguma solução? 🥺 |
Beta Was this translation helpful? Give feedback.
-
|
Didn't find the right answer, but managed to make it work by updating old dependencies. I saved 25mb just by updating typescript. But I'm really worried that this may become a problem in the near future |
Beta Was this translation helpful? Give feedback.
-
|
I am facing the same issue on Next 13.3.1 Large Dependencies Uncompressed size Compressed size15:01:37.048 | node_modules/.pnpm/@swc+core-linux-x64-gnu@1.3.58 49.77 MB 16.53 MB |
Beta Was this translation helpful? Give feedback.
-
|
If anyone stumbles upon this error: SWC should not be part of the serverless functions. Added the following to my next.config |
Beta Was this translation helpful? Give feedback.
-
|
I got it to work with the following experimental: {
outputFileTracingExcludes: {
'*': [
'node_modules/@swc/core-linux-x64-gnu',
'node_modules/@swc/core-linux-x64-musl',
'node_modules/@esbuild/linux-x64',
],
},
}, |
Beta Was this translation helpful? Give feedback.
-
|
The solution worked like a charm with Storybook 7.1.1! 🎉 Thanks for your help |
Beta Was this translation helpful? Give feedback.
-
|
had the same problem, suddenly my upload went from 10mb to 100mb.... Why does these dependencies are even in the node_modules of the functions anyway ? The problem does not seems to be storybook but rather something else no ? After double checking all of the dependencies are in the function, which makes no sense since 80% of it is not used by the said function. Can't we have automatic detection or something else ? that's quite crazy |
Beta Was this translation helpful? Give feedback.
-
|
@jeremytenjo can I ask you which package manager you are using? It looks like |
Beta Was this translation helpful? Give feedback.
-
|
NPM |
Beta Was this translation helpful? Give feedback.
-
|
same happend to me, I do not know why the build stage installed a lot of deps on the project which all of them are not used in project. In addition, some of them have fairly large size such as numpy. That's drives me crazy |
Beta Was this translation helpful? Give feedback.
-
|
Had this same issue when creating a static website. It seems that because The issue was that next to the markdown files, I had some static images that were getting bundled into the Functions, making them extremely big. As mentioned above, I fixed this issue by using Hope this helps somebody. Before finding this solution I tried using both |
Beta Was this translation helpful? Give feedback.
-
|
@markoleavy yarn @hanhanxue not sure if using /* instead of * would make the trick |
Beta Was this translation helpful? Give feedback.
-
|
I'm having the same issue, Next.js includes my /** @type {import('next').NextConfig} */
const nextConfig = {
/* ... */
experimental: {
outputFileTracingExcludes: {
'/*': ['public/static/work/**/*.png'],
'/**': ['public/static/work/**/*.png'],
}
},
}Size limit warning: This is on Next.js 13.4.19. |
Beta Was this translation helpful? Give feedback.
-
|
This is happening because somewhere in your code, you are including or using files from For me, I was using |
Beta Was this translation helpful? Give feedback.
-
|
Yep, you were right. I was using Thanks for the help! |
Beta Was this translation helpful? Give feedback.
-
|
Processing the images in the public folder using a pre-build script solved it for me. Thanks for this comment! |
Beta Was this translation helpful? Give feedback.
-
|
I’m struggling with that since a few days, and I’m getting nowhere. That’s the error message I get: I’m trying to remove ‘framer-motion’ (just to start) from that list, as it’s used just client side. That’s my configuration for outputFileTracingExcludes. I’m on a monorepo, next folder is at ./packages/website/. I’ve tried with and without outputFileTracingRoot, none of those works. I've also tried most of those combination individually, setting the key to various values like 'api/pdf', 'api/pdf/[slug]' 'api/pdf/*', etc. Some considerations I’ve made:
I’m using pnpm and Next 13.4.12. |
Beta Was this translation helpful? Give feedback.
-
|
I'd like to share the process I applied based on the error resolution methods provided by the talented individuals above ( I speculated that this error might be related to
In my case, the issue within the
I found that these two libraries were only dependent on From this, I confirmed the possibility of Furthermore, the
Essential libraries like "Error: The Serverless Function "_error" is 50.07mb which exceeds the maximum size limit of 50mb," I will now provide the list of Large Dependencies, along with their uncompressed and compressed sizes:
After removing Among these, I noted dependencies with a direct relationship to From this list, I included libraries that seemed unlikely to be used at runtime in the options, and I was able to successfully complete the deployment. experimental: {
outputFileTracingExcludes: {
'*': [
'node_modules/@swc/**/*',
'node_modules/@esbuild/**/*',
'node_modules/terser/**/*',
'node_modules/webpack/**/*',
],
},
} |
Beta Was this translation helpful? Give feedback.
-
|
I was facing a similar issue with the canvas module being a optional dependency of pdfjs & react-pdf. I'm using pnpm. I've managed to fix it using the following changes : "pnpm": {
"overrides": {
"pdfjs-dist>canvas": "file:../_EXCLUDED_"
}In experimental: {
outputFileTracingExcludes: {
"*": ["node_modules/.pnpm/canvas/**/*"],
},
},Important cleanup your Vercel build cache by triggering a redeploy and leaving the use build cache option unchecked. This is kind of hidden in the doc and I spent ages trying to figure out why my local |
Beta Was this translation helpful? Give feedback.
-
|
at least in nextjs 14 it seems swc is needed in some functions and will return an error at runtime complaining that @swc/helpers cant be found if 'node_modules/@swc/**/*' is included in this collection of excludes |
Beta Was this translation helpful? Give feedback.
-
|
In my case, deleting package-lock.json and running |
Beta Was this translation helpful? Give feedback.
-
|
I'm getting this error as follows: The offending file, // src/pages/sitemap.xml.jsx
import getStaticPagePaths from '@/lib/getStaticPagePaths'
import { getAllArticlePaths } from '@/lib/getAllArticles'
const Sitemap = () => {}
export default Sitemap
export const getServerSideProps = async ({ req, res }) => {
const baseUrl = `${
process.env.NODE_ENV === 'production' ? 'https://' : 'http://'
}${req.headers.host}`
const staticPagePaths = await getStaticPagePaths()
const articlePaths = await getAllArticlePaths()
const allPaths = [...staticPagePaths, ...articlePaths]
const sitemap = `<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
${allPaths
.map((path) => `<url><loc>${baseUrl}${path}</loc></url>`)
.join('')}
</urlset>
`
res.setHeader('Content-Type', 'text/xml')
res.write(sitemap)
res.end()
return {
props: {},
}
}From the Vercel logs, it's the // src/pages/articles/index,jsx
// ...
import { getAllArticles } from '@/lib/getAllArticles'
// ...
export async function getStaticProps() {
return {
props: {
articles: (await getAllArticles()).map(({ component, ...meta }) => meta),
},
}
}From this I can assume that it's the // src/lib/getAllArticles.js
import glob from 'fast-glob'
import * as path from 'path'
const importArticle = async (articleFilename, pathOnly = false) => {
const { meta, default: component } = await import(
`../pages/articles/${articleFilename}`
)
const url = `/articles/${articleFilename.replace(/(\/index)?\.mdx$/, '')}`
if (pathOnly) return url
return {
url: url,
...meta,
component,
}
}
const fetchArticleFilenames = async () =>
await glob(['*.mdx', '*/index.mdx'], {
cwd: path.join(process.cwd(), 'src/pages/articles'),
})
export const getAllArticles = async () => {
const articleFilenames = await fetchArticleFilenames()
const articles = await Promise.all(
articleFilenames.map((filename) => importArticle(filename))
)
return articles.sort((a, z) => new Date(z.date) - new Date(a.date))
}
export const getAllArticlePaths = async () => {
const articleFilenames = await fetchArticleFilenames()
const articlePaths = await Promise.all(
articleFilenames.map((filename) => importArticle(filename, true))
)
return articlePaths
}I'm primarily a backend dev doing this as a hobby project, so errors like this put me a little out of my depth. Is there something obvious I'm missing? My two thoughts are:
Can anyone provide any insight into a fix or how to dig-into the cause? I've tried adding the following to my next config, but it made no difference to the build error: const nextConfig = {
// ...
experimental: {
// ...
outputFileTracingExcludes: {
'/*': ['./src/pages/articles/**/*.jpeg'],
'/**/*': ['./src/pages/articles/**/*.jpeg'],
},
},
} |
Beta Was this translation helpful? Give feedback.
-
|
I had a very similar issue to this using MDX with remark plugins. It was including all images in my /public/ folder in the serverless functions (because I have a remark plugin that removes a parent The only solution I found that 'kind of' fixed it was using ContentLayer to create datasets and removing all the /src/lib dependencies. However -- big unfortunately -- ContentLayer being the beta product that it is, is causing other problems (randomly not working, missing articles with every deployment and so on). It's such a shame, since ContentLayer is a brilliant concept and would save a lot of stress with projects like this, but it's not production ready for large sites. |
Beta Was this translation helpful? Give feedback.
-
|
Regarding your issue, based on what I experienced with mine, your code is basically making the whole |
Beta Was this translation helpful? Give feedback.
-
Yes, you are right. I moved the images to a separate |
Beta Was this translation helpful? Give feedback.
-
|
For anyone trying to get puppeteer to work in 2023: |
Beta Was this translation helpful? Give feedback.
-
|
If anyone is using the Flask + Next.js quick start (https://vercel.com/templates/next.js/nextjs-flask-starter), I've found that creating a These is my I debugged this by building with vercel locally and I found that the |
Beta Was this translation helpful? Give feedback.
-
|
u dropped this bro 👑 |
Beta Was this translation helpful? Give feedback.
-
|
thx dude |
Beta Was this translation helpful? Give feedback.
-
|
I'm new to Vercel and trying to see if this 50 MB stuff is a deal-breaker before I spend more time on the platform. I have 2 separate backend APIs for different purposes: 1 in Node and 1 in python. For the python serverless functions, I only see a way to specify a single requirements.txt file in the root of the project. Is there a way to do this per lambda instead? Specifically, I have some routes involving auth libraries, and others involving computation with numpy etc. Including all this gets me to 60 MB. I'd like to have separate requirements files per lambda, and then I could set up multiple functions, each with a minimal set of dependancies, to help stay below the limit. Is this possible? |
Beta Was this translation helpful? Give feedback.
-
|
The 50MB is not Vercel specific, it's AWS. As long as AWS doesn't change the limit, the only way out to deploy a lambda function larger than the limit is to package the serverless function as a container image. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @alvis please could you gelaborate on "package the serverless function as a container image"? |
Beta Was this translation helpful? Give feedback.
-
|
I disagree in the sense that Vercel doesn't let me do the normal things in
python to limit the package size. Specifically, I have multiple backend API
functions and normally I could give each one a different requirements.txt
and limit the packages, but Vercel requires everything to be bundled
together and only allows a single requirements.txt. for example. Overall
it seems like the python support is super basic, so for anything like this
I'm on my own.
…On Fri, Sep 22, 2023 at 11:31 AM Alvis Tang ***@***.***> wrote:
The 50MB is not Vercel specific, it's AWS. As long as AWS doesn't change
the limit, the only way out to deploy a lambda function larger than the
limit is to package the serverless function as a container image.
—
Reply to this email directly, view it on GitHub
<#103 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAIEQPV7M6N3VX7XPXIALITX3WVNTANCNFSM5MWXFWFQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
and others
-
Hey everyone! I wanted to merge a few separate discussions into one. Occasionally, if you include large dependencies inside your functions, it can exceed the current max function limit of
50mbon Vercel. For example, a common dependency that causes this is Puppeteer, because it's shipping an entire headless Chrome browser.If you're hitting this issue, you'll often see a message like this:
We recognize that right now, it's difficult to debug why you've exceeded your limit. We are working on providing tooling to assist with debugging the contents of your functions to understand what's contributing to the size.
Note: Next.js specifically bundles all API Routes and server-rendered pages into shared Serverless Functions, separating API routes and server-rendered pages when deployed on Vercel. Additional Serverless Functions are only created if they do not fit within the 50MB limit.
Replaces:
Beta Was this translation helpful? Give feedback.
All reactions