Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up mac builds #116984

Merged
merged 21 commits into from
Mar 11, 2021
Merged

Speed up mac builds #116984

merged 21 commits into from
Mar 11, 2021

Conversation

TylerLeonhardt
Copy link
Member

@TylerLeonhardt TylerLeonhardt commented Feb 18, 2021

Fixes #115263

This has some minor improvements over #116739

that job took 1hr 13min: https://dev.azure.com/monacotools/Monaco/_build/results?buildId=105100&view=results
this change took 54min*: https://dev.azure.com/monacotools/Monaco/_build/results?buildId=105610&view=results

*EDIT: Shoot... I just realized that @deepak1556's change ran the tests and mine didn't... that adds probably another 10min to mine... (so 1hr 4min)

Basically the refactor is to split up signing and build jobs.

Now it works like this:

x64 build
arm build
universal build (waits on x64 and arm)
x64 publish (waits on x64 build)
arm publish (waits on arm build)
universal publish (waits on universal build)

Then we can be sure that the signing is done in parallel. For example, the arm build finishes in 10 min - that can go right to signing when it's done rather than waiting for the universal build to sign everything.

AZURE_STORAGE_ACCESS_KEY="$(ticino-storage-key)" \
AZURE_STORAGE_ACCESS_KEY_2="$(vscode-storage-key)" \
node build/azure-pipelines/common/createAsset.js \
"darwin-$(VSCODE_ARCH)" \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For x64 it is just darwin due to historical reason, changing this key will not push out this update.

x64 - darwin
arm64 - darwin-arm64
universal - darwin-universal

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added case statement

build/azure-pipelines/darwin/product-build-darwin-sign.yml Outdated Show resolved Hide resolved
@TylerLeonhardt TylerLeonhardt force-pushed the tyler/speedup_mac_builds_experiment3 branch from 656e92e to c4c515d Compare February 19, 2021 18:41
@TylerLeonhardt
Copy link
Member Author

So the most recent run of this is still going and it's already at 1hr 30min... which means there's like 30min of variance in this build due to:

  • How long it takes for dependencies to be installed
  • How long ESRP CodeSign task takes (which we run twice)

I feel like based on my testing that this one, on average, will take less time than @deepak1556's but it won't always be the case... so really I think the PR that should be accepted should be the more readable one.

But maybe @joaomoreno has other opinions.

condition: and(succeeded(), ne(variables.NODE_MODULES_RESTORED, 'true'), eq(variables['ENABLE_TERRAPIN'], 'true'))
displayName: Switch to Terrapin packages

- script: |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that signing step does not require any fancy node script except for createAsset, can you remove the node module cache steps and simplify the yarn step to just compile src/build packages.

$ cd build
$ yarn && yarn compile

build/azure-pipelines/darwin/product-build-darwin-sign.yml Outdated Show resolved Hide resolved
build/azure-pipelines/darwin/product-build-darwin-sign.yml Outdated Show resolved Hide resolved
build/azure-pipelines/darwin/product-build-darwin-sign.yml Outdated Show resolved Hide resolved
@deepak1556
Copy link
Collaborator

deepak1556 commented Feb 22, 2021

Trimming down the yarn step in the signing stage should reduce the timing a bit, lets see how much that improves for a final number comparison.

@TylerLeonhardt
Copy link
Member Author

Last 2 successful runs has taken 1hr 5min.

@deepak1556 deepak1556 added this to the March 2021 milestone Feb 23, 2021
@TylerLeonhardt TylerLeonhardt force-pushed the tyler/speedup_mac_builds_experiment3 branch from 9b14eae to 5b418bc Compare March 5, 2021 18:31
@TylerLeonhardt TylerLeonhardt marked this pull request as ready for review March 8, 2021 19:17
@TylerLeonhardt TylerLeonhardt force-pushed the tyler/speedup_mac_builds_experiment3 branch 6 times, most recently from 749fc40 to ded050f Compare March 9, 2021 23:16
@TylerLeonhardt TylerLeonhardt force-pushed the tyler/speedup_mac_builds_experiment3 branch from ded050f to b744827 Compare March 10, 2021 15:06
@TylerLeonhardt TylerLeonhardt merged commit 8fefa6a into main Mar 11, 2021
@TylerLeonhardt TylerLeonhardt deleted the tyler/speedup_mac_builds_experiment3 branch March 11, 2021 01:52
@joaomoreno
Copy link
Member

🥳

@kieferrm kieferrm mentioned this pull request Mar 15, 2021
76 tasks
@github-actions github-actions bot locked and limited conversation to collaborators Apr 25, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Streamline build time with universal builds
3 participants