Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Speed up CircleCI builds by increasing parallelism and resource classes #4427

Closed
wants to merge 19 commits into from
Closed

CI: Speed up CircleCI builds by increasing parallelism and resource classes #4427

Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
58b2da3
Attempt to consolidate build into test workflows
michaeldiamant Aug 18, 2022
ead4161
Try to fix the build
michaeldiamant Aug 18, 2022
118fdc2
Revert "Try to fix the build"
michaeldiamant Aug 18, 2022
28e94d7
Try to fix build steps
michaeldiamant Aug 18, 2022
f3e35e2
Restore dir creation
michaeldiamant Aug 18, 2022
2ffc7a5
Increase parallelism
michaeldiamant Aug 18, 2022
abbc24b
Merge branch 'master' into circleci_consolidate
michaeldiamant Aug 18, 2022
bd838f9
More integration parallelism
michaeldiamant Aug 18, 2022
f23630f
Checkpoint working config
michaeldiamant Aug 18, 2022
c70b961
Checkpoint matrix and filters alias usage
michaeldiamant Aug 18, 2022
6f9b989
Delete commented out steps
michaeldiamant Aug 18, 2022
6f2cfc9
Reset to as was parallelism and executors
michaeldiamant Aug 18, 2022
a2f7090
Experiment with more parallelism
michaeldiamant Aug 18, 2022
701456a
Reduce parallelism in favor of bigger executors
michaeldiamant Aug 18, 2022
cbb4486
Add explanatory topology comment
michaeldiamant Aug 19, 2022
eca634d
Consolidate build pre-requisite steps into generic_build command
michaeldiamant Aug 19, 2022
3bf9bb1
Merge branch 'circleci_consolidate' into circleci_consolidate_paralle…
michaeldiamant Aug 19, 2022
fc81e19
Add inline comment explicitly marking configuration confidence level
michaeldiamant Aug 19, 2022
a311590
Merge branch 'master' into circleci_consolidate_parallelism
michaeldiamant Aug 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# Disclaimer:
# * Unless otherwise specified, assume `resource_class` and `parallelism` values are cherry-picked values that provided a reasonable enough build-duration-to-cost tradeoff at the time of choosing.
# * There's too many variables (architecture types, CircleCI concurrency limitations, parallel pipeline runs, source code changes) to feel confident we've found a best-fit configuration.

version: 2.1

orbs:
Expand Down Expand Up @@ -473,9 +477,9 @@ jobs:
parameters:
platform:
type: string
executor: << parameters.platform >>_medium
executor: << parameters.platform >>_large
working_directory: << pipeline.parameters.build_dir >>/project
parallelism: 4
parallelism: 32
steps:
- generic_build
- generic_test:
Expand Down Expand Up @@ -508,7 +512,7 @@ jobs:
type: string
executor: << parameters.platform >>_large
working_directory: << pipeline.parameters.build_dir >>/project
parallelism: 2
parallelism: 16
environment:
E2E_TEST_FILTER: "GO"
steps:
Expand Down Expand Up @@ -540,9 +544,9 @@ jobs:
parameters:
platform:
type: string
executor: << parameters.platform >>_medium
executor: << parameters.platform >>_large
working_directory: << pipeline.parameters.build_dir >>/project
parallelism: 2
parallelism: 10
environment:
E2E_TEST_FILTER: "EXPECT"
steps:
Expand Down