Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix incompatibility of node versions 15.5+ and serverless-artillery causing "Empty lambda zip" errors in load tests #856

Merged
merged 2 commits into from
Jun 9, 2021

Conversation

alvaroloes
Copy link
Contributor

Description

This is a "dependency-hell" chained error:

  1. From node v15.6.0 onwards, the behavior of .end on zlib.DeflateRaw was changed
  2. This caused the library archiver to fail. Only the newest version has it fixed.
    • This library is used by the AWS CDK, so we started getting an error when deploying because the lambda's .zip were empty. We fixed this by updating the AWS CDK that used a newer version of archiver
  3. serverless-artillery uses serverless to deploy the lambda that executes de load test. serverless, in turn, uses archiver to zip the lambda's code. This means that we were getting the "empty lambda zip" error when deploying serverless-artillery for the load tests.
  4. The newest version of serverless uses the fixed version of archiver, but we can't use it because it is incompatible with the version we are using of serverless-artillery.
  5. We can't update serverless-artillery because there is not newer version :trollface:

Changes

The solution is to force the version of serverless we are using (that's compatible with serverless-artillery) to use a newer version of archiver.
The official way to do this (which is called "override dependencies") has still not reached the official version of node and NPM (official issue here)

The "un-official" way to do this is what is done in this PR:

  • We are hoisting all the dependencies of the project. This means to put all the dependencies of each package in the root node_modules folder (instead of inside each package)
  • Therefore, the good version of archiver is located at <root>/node_modules/archiver
  • The bad version, the one that serverless is using, is located in a transitive dependency in <root>/node_modules/serverless/node_modules/archiver
  • The solution is to remove the bad module.
  • This way, when the serverless module looks for archiver and it doesn't find it in its local "node_modules" folder, it will search for it in the parent directory (this is how every node module lookup is done), finding the good archiver

Checks

  • Project Builds
  • Project passes tests and checks
  • [ ] Updated documentation accordingly

Additional information

image

console.info('Serverless Artillery stack is already deployed. Skipping redeployment.')
return
}
console.info(`Serverless Artillery stack is in a wrong state: ${slsartStack?.StackStatus}. Redeploying.`)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I enhanced the check we were doing to see if we need to redeploy the serverless-artillery stack

@@ -59,6 +59,7 @@
"lint:fix": "eslint --quiet --fix --ext '.js,.ts' **/*.ts",
"compile": "ttsc -b tsconfig.json",
"clean": "rimraf ./dist tsconfig.tsbuildinfo",
"postinstall": "rimraf ../../node_modules/serverless/node_modules/archiver && echo 'Deleted buggy \"archiver\" module from \"serveless\" dependencies to use the newer hoisted one'",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the main change in this PR

Copy link
Member

@NickSeagull NickSeagull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one! Thanks :D

@boostercloud boostercloud locked and limited conversation to collaborators Jun 8, 2021
@boostercloud boostercloud unlocked this conversation Jun 8, 2021
Copy link
Contributor

@juanjoman juanjoman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing 🙌

@boostercloud boostercloud locked and limited conversation to collaborators Jun 9, 2021
@boostercloud boostercloud unlocked this conversation Jun 9, 2021
@alvaroloes alvaroloes force-pushed the fix-incompatibility-node16-serverless-artillery branch from c9b027b to 3043f07 Compare June 9, 2021 13:02
@boostercloud boostercloud locked and limited conversation to collaborators Jun 9, 2021
@boostercloud boostercloud unlocked this conversation Jun 9, 2021
@alvaroloes alvaroloes merged commit fab128f into main Jun 9, 2021
@alvaroloes alvaroloes deleted the fix-incompatibility-node16-serverless-artillery branch June 9, 2021 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants