Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip compile & packaging if --no-build is set #560

Merged

Conversation

jamesmbourne
Copy link
Contributor

@jamesmbourne jamesmbourne commented Mar 5, 2020

What did you implement:

Closes #398

Allow the --no-build flag to commands that invoke the before:package:createDeploymentArtifacts hook. This means that build output can be generated (e.g. in CI environments) and then deployed without having to re-build each time.

How did you implement it:

Use the existing build option to skip the compile and package steps for the before:package:createDeploymentArtifacts hook.

How can we verify it:

  1. Generate build output with sls webpack
  2. Run sls deploy --no-build
  3. The existing .webpack (or other as specified by --out) directory should be used an no Webpack compilation occurs

Todos:

  • Write tests
  • Write documentation
  • Fix linting errors
  • Make sure code coverage hasn't dropped
  • Provide verification config / commands / resources
  • Enable "Allow edits from maintainers" for this PR
  • Update the messages below

Is this ready for review?: YES
Is it a breaking change?: NO

@bazaglia
Copy link

@jamesmbourne Amazing PR! I tested it, but got this error when trying to bundle with sls webpack -r us-east-1:

  TypeError: Cannot read property 'stats' of undefined
      at /Users/andre/my-project/node_modules/serverless-webpack/lib/packExternalModules.js:264:27

@jamesmbourne
Copy link
Contributor Author

Hey @bazaglia, thanks for the feedback!

What is the commit hash of serverless-webpack you are on? 19031c6 should be the most recent on this branch.

Do you have a minimal reproduction of this that you'd be able to share?

@bazaglia
Copy link

@jamesmbourne I end up using a separate serverless.yml file for deployment that basically imports everything from the original serverless.yml except for the plugins, so I can delete serverless-webpack from my plugin list. In my CI then I can do multi region deployment without this plugin bundling everything again for every single region I deploy to, what takes a lot of time.

But about reproduction steps for your PR, I've just run sls webpack and the error was as I pasted above. Nothing much more than that - and I'm sorry I don't have the stack trace anymore. And yes, I've used last commit from your branch.

@tusamcalles
Copy link
Contributor

@ceilfors @franciscocpg @hassankhan @HyperBrain @nathanchapman @rdsedmundo Is there a plan to release this PR?

@rdsedmundo
Copy link
Member

See #563.

@tusamcalles tusamcalles mentioned this pull request Apr 7, 2020
@tusamcalles
Copy link
Contributor

Thanks @rdsedmundo.

Copy link
Member

@miguel-a-calles-mba miguel-a-calles-mba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worked great for me.

@bazaglia
Copy link

bazaglia commented May 6, 2020

Oi @rdsedmundo! :)

Apparently this PR is missing a review from you? I've seen many people are looking forward to using this feature. Thanks.

@rdsedmundo
Copy link
Member

I'll have a look at this and test it by tomorrow EOD, 100%.

@miguel-a-calles-mba
Copy link
Member

This PR is ready to merge. I have been using it in a production product for weeks.

@miguel-a-calles-mba miguel-a-calles-mba requested a review from a team May 6, 2020 15:05
@miguel-a-calles-mba
Copy link
Member

@rdsedmundo Please merge this PR if you are satisfied with your test results.

@miguel-a-calles-mba miguel-a-calles-mba modified the milestones: 5.3.3, 5.4.0 May 6, 2020
@miguel-a-calles-mba miguel-a-calles-mba changed the base branch from master to release/5.4.0 May 6, 2020 23:12
Copy link
Member

@rdsedmundo rdsedmundo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested and it looks good, thanks for working on that.

@rdsedmundo rdsedmundo merged commit 2e4c5f5 into serverless-heaven:release/5.4.0 May 7, 2020
@jamesmbourne jamesmbourne deleted the no-build-support branch May 15, 2020 22:08
@RishikeshDarandale
Copy link

RishikeshDarandale commented May 24, 2020

@jamesmbourne , Can you please check the package is expected to have all these?

Here is simple example:

//serverless.yml
service: multi-comple-config-example

# Add the serverless-webpack plugin
plugins:
  - serverless-webpack

custom:
  # webpack config
  webpack:
    webpackConfig: ./webpack.config.js
    includeModules:
      forceExclude:
        - aws-sdk

package:
  individually: false

provider:
  name: aws
  runtime: nodejs12.x

functions:
  web-app:
    handler: server.handler
    events:
      - http: ANY /
      - http: ANY /{proxy+}
//server.js
'use strict';

module.exports.handler = function (event, context, cb) {
  cb(null, { message: 'Server Module', event });
}
//webpack.config.js
const path = require("path");
const slsw = require("serverless-webpack");

module.exports = {
  mode: slsw.lib.webpack.isLocal ? "development" : "production",
  entry: {
    server: "./server.js",
  },
  target: "node",
  output: {
    libraryTarget: "commonjs",
    path: path.join(__dirname, ".webpack/web-app"),
    filename: "[name].js",
  },
};
// package.json
{
  "name": "multi-compile-config",
  "version": "1.0.0",
  "description": "",
  "main": "server.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node server.js"
  },
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "serverless": "^1.71.3",
    "serverless-webpack": "^5.3.2",
    "webpack": "^4.43.0"
  }
}

Note: I had checkout locally release/5.4.0 and replaced all files from lib directory to node_modules/serverless-webpack/lib.

// output with release 5.4.0

Archive:  .serverless/multi-comple-config-example.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
     642  Defl:N      618   4% 1980-01-01 00:00 a3921a38  .webpack/multi-comple-config-example.zip
    1077  Defl:N      510  53% 1980-01-01 00:00 1938c2a5  .webpack/service/server.js
      77  Defl:N       72   7% 1980-01-01 00:00 eebe0282  client.js
     157  Defl:N      129  18% 1980-01-01 00:00 dbccd4b3  node_modules/.bin/components
     328  Defl:N      219  33% 1980-01-01 00:00 0f9ad637  node_modules/.bin/parser
    1131  Defl:N      546  52% 1980-01-01 00:00 e8632c9a  node_modules/.cache/terser-webpack-plugin/content-v2/sha512/45/17/3ce3a>
    1052  Defl:N      511  51% 1980-01-01 00:00 c9606e6a  node_modules/.cache/terser-webpack-plugin/content-v2/sha512/fd/57/39ff5>
     774  Defl:N      492  36% 1980-01-01 00:00 8a2ecbd0  node_modules/.cache/terser-webpack-plugin/index-v5/22/9b/363b2a229d1909>
     774  Defl:N      494  36% 1980-01-01 00:00 9a1a03c5  node_modules/.cache/terser-webpack-plugin/index-v5/ac/e2/723097fd86f878>
  364736  Defl:N   104652  71% 1980-01-01 00:00 1dbeba55  package-lock.json
     366  Defl:N      223  39% 1980-01-01 00:00 e577b8ed  package.json
     122  Defl:N      107  12% 1980-01-01 00:00 8e00dbe4  server.js
     343  Defl:N      217  37% 1980-01-01 00:00 9e929ac6  webpack.config.js
--------          -------  ---                            -------
  371579           108790  71%                            13 files

// output with 5.3.2
Archive:  .serverless/multi-comple-config-example.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
    1077  Defl:N      510  53% 1980-01-01 00:00 1938c2a5  server.js
--------          -------  ---                            -------
    1077              510  53%                            1 file

vicary pushed a commit to vicary/serverless-webpack that referenced this pull request Jan 6, 2021
* Add copyExistingArtifacts to packageModules

* Refactor packageModules

* Set service path

* Generate artifact name from service

* Output artifacts to .webpack before copying

* Set artifact name for service packaging

* Skip webpack:compile if --no-build is set

* Add webpack:package:copyExistingArtifacts hook

* Make packageModules & packExternalModules no-op if skipCompile is set

* Refactor packageModules

* Remove artifact location setting from packageModules

* Update cleanup to check this.keepOutputDirectory

* Fix path join on Windows

Co-authored-by: Miguel A. Calles MBA <44813512+miguel-a-calles-mba@users.noreply.github.com>
vicary pushed a commit to vicary/serverless-webpack that referenced this pull request Jan 6, 2021
* Add copyExistingArtifacts to packageModules

* Refactor packageModules

* Set service path

* Generate artifact name from service

* Output artifacts to .webpack before copying

* Set artifact name for service packaging

* Skip webpack:compile if --no-build is set

* Add webpack:package:copyExistingArtifacts hook

* Make packageModules & packExternalModules no-op if skipCompile is set

* Refactor packageModules

* Remove artifact location setting from packageModules

* Update cleanup to check this.keepOutputDirectory

* Fix path join on Windows

Co-authored-by: Miguel A. Calles MBA <44813512+miguel-a-calles-mba@users.noreply.github.com>
vicary pushed a commit to vicary/serverless-webpack that referenced this pull request Jan 6, 2021
* Add copyExistingArtifacts to packageModules

* Refactor packageModules

* Set service path

* Generate artifact name from service

* Output artifacts to .webpack before copying

* Set artifact name for service packaging

* Skip webpack:compile if --no-build is set

* Add webpack:package:copyExistingArtifacts hook

* Make packageModules & packExternalModules no-op if skipCompile is set

* Refactor packageModules

* Remove artifact location setting from packageModules

* Update cleanup to check this.keepOutputDirectory

* Fix path join on Windows

Co-authored-by: Miguel A. Calles MBA <44813512+miguel-a-calles-mba@users.noreply.github.com>
vicary pushed a commit to vicary/serverless-webpack that referenced this pull request Jan 13, 2021
* Add copyExistingArtifacts to packageModules

* Refactor packageModules

* Set service path

* Generate artifact name from service

* Output artifacts to .webpack before copying

* Set artifact name for service packaging

* Skip webpack:compile if --no-build is set

* Add webpack:package:copyExistingArtifacts hook

* Make packageModules & packExternalModules no-op if skipCompile is set

* Refactor packageModules

* Remove artifact location setting from packageModules

* Update cleanup to check this.keepOutputDirectory

* Fix path join on Windows

Co-authored-by: Miguel A. Calles MBA <44813512+miguel-a-calles-mba@users.noreply.github.com>
j0k3r pushed a commit that referenced this pull request Jan 25, 2021
* Add copyExistingArtifacts to packageModules

* Refactor packageModules

* Set service path

* Generate artifact name from service

* Output artifacts to .webpack before copying

* Set artifact name for service packaging

* Skip webpack:compile if --no-build is set

* Add webpack:package:copyExistingArtifacts hook

* Make packageModules & packExternalModules no-op if skipCompile is set

* Refactor packageModules

* Remove artifact location setting from packageModules

* Update cleanup to check this.keepOutputDirectory

* Fix path join on Windows

Co-authored-by: Miguel A. Calles MBA <44813512+miguel-a-calles-mba@users.noreply.github.com>
vicary pushed a commit to vicary/serverless-webpack that referenced this pull request Mar 1, 2021
* Add copyExistingArtifacts to packageModules

* Refactor packageModules

* Set service path

* Generate artifact name from service

* Output artifacts to .webpack before copying

* Set artifact name for service packaging

* Skip webpack:compile if --no-build is set

* Add webpack:package:copyExistingArtifacts hook

* Make packageModules & packExternalModules no-op if skipCompile is set

* Refactor packageModules

* Remove artifact location setting from packageModules

* Update cleanup to check this.keepOutputDirectory

* Fix path join on Windows

Co-authored-by: Miguel A. Calles MBA <44813512+miguel-a-calles-mba@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants