Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempt to split up CI build steps into separate jobs #807

Merged
merged 11 commits into from
Feb 22, 2022

Conversation

mitchmindtree
Copy link
Contributor

Not sure if this will work just yet, but hoping this splits up the
build CI job to execute jobs in parallel, synchronising where
necessary using the <job_id>.needs.<other_job_id> feature.

Hopefully this cuts down on waiting time and makes the results a little
more readible from within the PRs.

Closes #772.

Not sure if this will work just yet, but hoping this splits up the
`build` CI job to execute jobs in parallel, synchronising where
necessary using the `<job_id>.needs.<other_job_id>` feature.

Hopefully this cuts down on waiting time and makes the results a little
more readible from within the PRs.

Closes #772.
@mitchmindtree mitchmindtree force-pushed the mitchmindtree/split-ci branch 2 times, most recently from b3d9e64 to bac46af Compare February 17, 2022 07:56
@mitchmindtree mitchmindtree self-assigned this Feb 17, 2022
@mitchmindtree mitchmindtree marked this pull request as ready for review February 17, 2022 09:03
@mitchmindtree
Copy link
Contributor Author

OK after lots of fiddling around I think this might be ready review!

Apart from splitting up the build job into its parts, this also moves the clippy check from a separate file into the main CI workflow and adds it to the list of jobs that must be successful before publishing can go ahead.

Here's what the workflow job graph looks like now:

Screenshot from 2022-02-17 20-01-57

The last commit took about 5mins altogether, but I think this is just because it was hitting the Rust cache for every job. When not hitting the cache, it seems to take 10-12 minutes (this is just from eyeballing the actions times here). Either way, I think this is a bit better than the 17-20mins we've been getting when not hitting the cache with our current long sequential build job.

Oh yeah, I also haven't re-added this as I'm unsure what we're using it for:

      - name: Set git config
        run: |
          git config --global core.bigfilethreshold 500m

Do we still need it?

@mitchmindtree mitchmindtree requested a review from sezna February 17, 2022 09:14
@adlerjohn adlerjohn requested review from Voxelot and rfuelsh February 17, 2022 23:31
@Voxelot
Copy link
Member

Voxelot commented Feb 22, 2022

It looks like some job requirements are blocking this?

Screen Shot 2022-02-21 at 9 14 20 PM

@mitchmindtree
Copy link
Contributor Author

@Voxelot I think it might only be blocked on approval:

Screenshot from 2022-02-22 16-38-42

I don't think those old CI checks need to pass, though I could be wrong!

@Voxelot
Copy link
Member

Voxelot commented Feb 22, 2022

I think it's possible to merge without CI passing, so we should probably root out why these 2 jobs are stuck? Otherwise we may never know when CI fully succeeded or failed.

@mitchmindtree
Copy link
Contributor Author

I think they're not running because they're removed as a part of this PR in favour of the new jobs (this PR only touches the github workflow and doesn't touch any code). Is that what you mean?

@Voxelot
Copy link
Member

Voxelot commented Feb 22, 2022

I guess what I'm wondering is why the removed jobs are still expected. Is this because the PR isn't merged to master yet? Or is there some kind of config setting on the repo expecting these jobs?

Copy link
Contributor

@adlerjohn adlerjohn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the repo config to not require jobs, which had the old name

@mitchmindtree mitchmindtree merged commit 45645f2 into master Feb 22, 2022
@mitchmindtree mitchmindtree deleted the mitchmindtree/split-ci branch February 22, 2022 23:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Try splitting the existing CI "build" job into multiple jobs for a potential speedup
3 participants