-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(integ-runner): integ-runner enhancements #19865
Conversation
This PR adds a couple of enhancements to the `integ-runner` 1. Add `--profiles` option which allows you to provide a list of AWS profiles to use. This will be used in combination with the `--parallel-regions` option to run the integration tests in parallel across multiple accounts + regions. 2. Adds `--from-file` option that allows you to store a list of tests to run (or exclude) in a file, rather than passing test names as arguments. 3. Adds `--exclude` option, which when provided will excluded the tests provided as arguments 4. Adds some additional metrics. Now each test will show the total time in the logs. Also, if `--verbose` option is provided a summary will be printed which will contain metrics for the entire test. ```bash --- Integration test metrics --- Profile mb-dev + Region us-east-2 total time: 349.866 test/integ.bundling.js: 58.75 test/integ.lambda-insights-mapping.js: 61.012 test/integ.assets.lit.js: 61.121 test/integ.lambda.docker.js: 81.936 test/integ.log-retention.js: 87.047 Profile mb-dev + Region us-west-2 total time: 359.775 test/integ.layer-version.lit.js: 72.61 test/integ.autoscaling.lit.js: 82.82 test/integ.current-version.js: 84.982 test/integ.lambda.js: 119.363 Profile mb-dev + Region us-east-1 total time: 378.713 test/integ.assets.file.js: 67.915 test/integ.runtime.inlinecode.js: 68.211 ```
Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
This PR adds a couple of enhancements to the `integ-runner` 1. Add `--profiles` option which allows you to provide a list of AWS profiles to use. This will be used in combination with the `--parallel-regions` option to run the integration tests in parallel across multiple accounts + regions. 2. Adds `--from-file` option that allows you to store a list of tests to run (or exclude) in a file, rather than passing test names as arguments. 3. Adds `--exclude` option, which when provided will excluded the tests provided as arguments 4. Adds some additional metrics. Now each test will show the total time in the logs. Also, if `--verbose` option is provided a summary will be printed which will contain metrics for the entire test. ```bash --- Integration test metrics --- Profile mb-dev + Region us-east-2 total time: 349.866 test/integ.bundling.js: 58.75 test/integ.lambda-insights-mapping.js: 61.012 test/integ.assets.lit.js: 61.121 test/integ.lambda.docker.js: 81.936 test/integ.log-retention.js: 87.047 Profile mb-dev + Region us-west-2 total time: 359.775 test/integ.layer-version.lit.js: 72.61 test/integ.autoscaling.lit.js: 82.82 test/integ.current-version.js: 84.982 test/integ.lambda.js: 119.363 Profile mb-dev + Region us-east-1 total time: 378.713 test/integ.assets.file.js: 67.915 test/integ.runtime.inlinecode.js: 68.211 ``` There are also a couple of small refactors. 1. Split up `workers.test.ts` into `snapshot-worker.test.ts` and `integ-worker.test.ts` 2. Refactored the `extract_workers` functions. There used to be two functions `snapshotTestBatch` and `integTestBatch` which called `singleThreadedXXX` functions. Since each request is essentially single threaded I've moved the `singleThreadedXXX` functions into `extract_worker` and renamed them to be `integTestWorker` and `snapshotTestWorker` ---- ### All Submissions: * [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md) ### Adding new Unconventional Dependencies: * [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md/#adding-new-unconventional-dependencies) ### New Features * [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/master/INTEGRATION_TESTS.md)? * [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)? *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
This PR adds a couple of enhancements to the
integ-runner
--profiles
option which allows you to provide a list ofAWS profiles to use. This will be used in combination with the
--parallel-regions
option to run the integration tests in parallelacross multiple accounts + regions.
--from-file
option that allows you to store a list of tests torun (or exclude) in a file, rather than passing test names as
arguments.
--exclude
option, which when provided will excluded the testsprovided as arguments
in the logs. Also, if
--verbose
option is provided a summary willbe printed which will contain metrics for the entire test.
--- Integration test metrics --- Profile mb-dev + Region us-east-2 total time: 349.866 test/integ.bundling.js: 58.75 test/integ.lambda-insights-mapping.js: 61.012 test/integ.assets.lit.js: 61.121 test/integ.lambda.docker.js: 81.936 test/integ.log-retention.js: 87.047 Profile mb-dev + Region us-west-2 total time: 359.775 test/integ.layer-version.lit.js: 72.61 test/integ.autoscaling.lit.js: 82.82 test/integ.current-version.js: 84.982 test/integ.lambda.js: 119.363 Profile mb-dev + Region us-east-1 total time: 378.713 test/integ.assets.file.js: 67.915 test/integ.runtime.inlinecode.js: 68.211
There are also a couple of small refactors.
workers.test.ts
intosnapshot-worker.test.ts
andinteg-worker.test.ts
extract_workers
functions. There used to be two functionssnapshotTestBatch
andintegTestBatch
which calledsingleThreadedXXX
functions. Since each request is essentially single threaded I've moved thesingleThreadedXXX
functions intoextract_worker
and renamed them to beintegTestWorker
andsnapshotTestWorker
All Submissions:
Adding new Unconventional Dependencies:
New Features
yarn integ
to deploy the infrastructure and generate the snapshot (i.e.yarn integ
without--dry-run
)?By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license