chore: try GitHub Action #2166

SteveLauC · 2023-10-08T01:56:13Z

What does this PR do

Our first try with GHA

This PR migrates our:

BUILD
TEST

to two custom GitHub composite actions, and uses them on x86_64-apple-darwin

Update: 33 tasks (we have 35 in total) have been migrated, see my last comment

Checklist:

I have read CONTRIBUTING.md
I have written necessary tests and rustdoc comments
A change log has been added if this PR modifies nix's API

.github/workflows/ci.yml

SteveLauC · 2023-10-08T02:47:32Z

Well, no idea why GHA is not running with this PR, [skip ci] in the PR title should not disable it(but only the Cirrus CI) as I didn't find anything related to PR title in their documentation

Jan561 · 2023-10-08T09:33:25Z

I've found something that sounds interesting:

https://github.com/cirruslabs/cirrus-cli/blob/master/INSTALL.md#github-actions

If I understand this correctly, the Cirrus CLI allows tasks configured for Cirrus to be run in other CIs, including GHA

SteveLauC · 2023-10-08T09:46:38Z

I've found something that sounds interesting:

https://github.com/cirruslabs/cirrus-cli/blob/master/INSTALL.md#github-actions

If I understand this correctly, the Cirrus CLI allows tasks configured for Cirrus to be run in other CIs, including GHA

This is cool, just installed Cirrus CLI on my host, it can execute a specific task, which means if we separate the jobs that should be migrated to GHA into new tasks, then everything should just work?

Jan561 · 2023-10-08T10:52:04Z

I hope so, that would be very nice. I'll play around a bit on my fork too and keep you updated

SteveLauC · 2023-10-08T11:51:04Z

I hope so, that would be very nice. I'll play around a bit on my fork too and keep you updated

Thanks!

I just migrated 33 tasks to GitHub Action:

macOS
cross(QEMU)
Linux Amd64 Native Builds
Latest Rust Stable
Corss-compiling
Redox
Tier3
Minver
Rustfmt

leaving the following tasks on Cirrus

FreeBSD
Linux Arm64 Native Builds

You can preview it here

Both ways are acceptable if they work!

.github/workflows/ci.yml

Jan561 · 2023-10-08T18:09:10Z

I've looked into GHA + Cirrus CLI, and I believe the primary usecases for that CLI are

You need a self-hosted runner that receives certain tasks from the Cirrus Cloud (Persistent Worker) or
You really like the Cirrus-CI configuration format and want to use them for other CIs as well (while not using Cirrus)

I didn't find a way to tell Cirrus to skip certain tasks that get handled by GHA, other than setting up a persistent worker in GHA, letting it fetch and execute some tasks from the Cirrus Cloud and stopping it afterwards.

I think a separate configuration for GHA is by far the better option.

Edit: I got it working ... at least a little bit. I marked the tasks that should run on GHA as manual, and called them explicitly in the GHA workflow.

I ended up with

'Linux arm gnueabi' Task
  Started 'Linux arm gnueabi' Task
  unsupported instance type: got nil instance which means it's probably not supported by the CLI
  'Linux arm gnueabi' Task skipped in 0.0s!

No idea why this fails, and it's probably not worth the effort to get this working, so I suggest we should just use your approach

SteveLauC · 2023-10-09T02:02:40Z

Thanks for taking the time to investigate this tool!

Jan561

I've checked out your workflow files on my fork. These are the changes I needed to implement to make the CI pass.

.github/actions/build/action.yml

.github/actions/test/action.yml

.github/workflows/ci.yml

SteveLauC · 2023-10-10T13:38:16Z

@Jan561 Thanks for testing it on your fork, I can see that it is working here. And I am actually surprised by how fast GHA is, it finishes 33 tasks in 6 minutes, unbelievable...

Jan561 · 2023-10-10T14:00:36Z

You're welcome, I'm mind blown as well. And I've double-checked: no quotas, no fees, no limits, completely free (at least for me as a personal account, but I remember reading that this applies to all open-source projects)

Jan561 · 2023-10-12T14:09:16Z

Optional, but we could allow GHA to check all branches, by setting

on:
  pull_request:
    types: [opened, synchronize, reopened]
  push:
    branches:
      - '*'

Not required for the nix repository though, but this would allow forks to use the CI on other branches as well

SteveLauC · 2023-10-13T03:24:26Z

Optional, but we could allow GHA to check all branches, by setting
on:
  pull_request:
    types: [opened, synchronize, reopened]
  push:
    branches:
      - '*'
Not required for the nix repository though, but this would allow forks to use the CI on other branches as well

Yeah, this is a nice feature, our contributors can self-check their patches on the fork's CI with this

SteveLauC · 2023-10-17T03:28:53Z

Friendly ping @asomers, could I suggest giving this PR a review when you have time, so that we can handle other PRs without running out of Cirrus credits

asomers · 2023-10-21T04:08:28Z

Sorry I've been taking a while, but I think I'll be able to do it this weekend.

…

On Mon, Oct 16, 2023, 9:29 PM SteveLauC ***@***.***> wrote: Friendly ping @asomers <https://github.com/asomers>, could I suggest giving this PR a review when you have time, so that we can handle other PRs without running out of Cirrus credits — Reply to this email directly, view it on GitHub <#2166 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA7VOOAGWFHVNOAU7ABXOTX7X3QBAVCNFSM6AAAAAA5XKNJEGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRVGYYDMMBVGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

.github/workflows/ci.yml

SteveLauC · 2023-11-05T02:37:40Z

Let me remove the migrated tasks in our Cirrus CI so that we can finally merge this PR

We have a macro skip_if_cirrus!() to skip a CI test under Cirrus:

nix/test/common/mod.rs

Lines 54 to 62 in b28132b

    
           #[cfg(any(target_os = "linux", target_os = "android"))] 
        
           #[macro_export] 
        
           macro_rules! skip_if_cirrus { 
        
               ($reason:expr) => { 
        
                   if std::env::var_os("CIRRUS_CI").is_some() { 
        
                       skip!("{}", $reason); 
        
                   } 
        
               }; 
        
           }

It is only defined under Android and Linux, given that we have Linux tests on both Cirrus (Linux/arm64) and GHA, we should probably provide the following macros as well, and document them in CONTRIBUTING.md:

We detect if we are running on GHA with the GITHUB_ACTIONS env variable, ref: Check if we're in a GitHub Action / Travis CI / Circle CI etc. testing environment

#[cfg(any(target_os = "linux", target_os = "android"))]
#[macro_export]
macro_rules! skip_if_gha {
    ($reason:expr) => {
        if std::env::var_os("GITHUB_ACTIONS").is_some() {
            skip!("{}", $reason);
        }
    };
}

#[cfg(any(target_os = "linux", target_os = "android"))]
#[macro_export]
macro_rules! skip_if_ci {
    ($reason:expr) => {
        if std::env::var_os("GITHUB_ACTIONS").is_some() || std::env::var_os("CIRRUS_CI").is_some() {
            skip!("{}", $reason);
        }
    };
}

asomers · 2023-11-05T16:45:07Z

@SteveLauC do you know why GHA isn't running? I don't see anything in the repository settings that we would have to enable.

SteveLauC · 2023-11-06T00:12:08Z

@SteveLauC do you know why GHA isn't running? I don't see anything in the repository settings that we would have to enable.

I actually have the same question:

Well, no idea why GHA is not running with this PR, [skip ci] in the PR title should not disable it(but only the Cirrus CI) as I didn't find anything related to PR title in their documentation

One guess is that when I originally created this PR, I added [skip ci] to the PR title, which could probably disable the GHA as well?

Jan561 · 2023-11-06T15:07:33Z

It should work out of the box when the workflow files appear on the main branch

asomers · 2023-11-06T22:23:36Z

@SteveLauC how about we revert the .cirrus.yml changes and merge this PR. Hopefully both CI engines will run then, and we can reapply the .cirrus.yml changes in a second PR.

SteveLauC · 2023-11-06T23:31:07Z

@SteveLauC how about we revert the .cirrus.yml changes and merge this PR. Hopefully both CI engines will run then, and we can reapply the .cirrus.yml changes in a second PR.

Yeah, this makes sense, just discarded my last commit

chore: try GitHub Action

30c63d0

SteveLauC commented Oct 8, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

SteveLauC changed the title ~~chore: try GitHub Action [skip ci]~~ chore: try GitHub Action Oct 8, 2023

fix branch name

d7477dd

SteveLauC mentioned this pull request Oct 8, 2023

Out of $$$ for CI #2161

Closed

migrate 33 tasks

93813fd

SteveLauC commented Oct 8, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

SteveLauC commented Oct 8, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

Jan561 suggested changes Oct 10, 2023

View reviewed changes

.github/actions/build/action.yml Show resolved Hide resolved

.github/actions/test/action.yml Show resolved Hide resolved

.github/workflows/ci.yml Show resolved Hide resolved

add RUSTFLAGS and RUSTDOCFLAGS to env

eb765a4

Jan561 approved these changes Oct 10, 2023

View reviewed changes

SteveLauC mentioned this pull request Oct 13, 2023

chore: move cargo fmt to pre-commit script #2163

Closed

3 tasks

SteveLauC requested a review from asomers October 14, 2023 11:57

SteveLauC mentioned this pull request Oct 30, 2023

Add implementation of PTRACE_{GET,SET}REGSET #2044

Merged

asomers requested changes Nov 4, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

.github/workflows/ci.yml Outdated Show resolved Hide resolved

.github/workflows/ci.yml Outdated Show resolved Hide resolved

respond to review

e012b10

SteveLauC force-pushed the GHA branch from 334d2e4 to e012b10 Compare November 6, 2023 23:30

asomers approved these changes Nov 7, 2023

View reviewed changes

SteveLauC added this pull request to the merge queue Nov 8, 2023

Merged via the queue into nix-rust:master with commit 6868489 Nov 8, 2023
35 checks passed

SteveLauC deleted the GHA branch November 8, 2023 01:22

SteveLauC mentioned this pull request Nov 8, 2023

chore: remove migrated CI tasks from Cirrus #2182

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: try GitHub Action #2166

chore: try GitHub Action #2166

SteveLauC commented Oct 8, 2023 •

edited

Loading

SteveLauC commented Oct 8, 2023

Jan561 commented Oct 8, 2023

SteveLauC commented Oct 8, 2023

Jan561 commented Oct 8, 2023 •

edited

Loading

SteveLauC commented Oct 8, 2023 •

edited

Loading

Jan561 commented Oct 8, 2023 •

edited

Loading

SteveLauC commented Oct 9, 2023

Jan561 left a comment

SteveLauC commented Oct 10, 2023

Jan561 commented Oct 10, 2023

Jan561 commented Oct 12, 2023

SteveLauC commented Oct 13, 2023

SteveLauC commented Oct 17, 2023

asomers commented Oct 21, 2023 via email

SteveLauC commented Nov 5, 2023 •

edited

Loading

asomers commented Nov 5, 2023

SteveLauC commented Nov 6, 2023

Jan561 commented Nov 6, 2023

asomers commented Nov 6, 2023

SteveLauC commented Nov 6, 2023

chore: try GitHub Action #2166

chore: try GitHub Action #2166

Conversation

SteveLauC commented Oct 8, 2023 • edited Loading

What does this PR do

Checklist:

SteveLauC commented Oct 8, 2023

Jan561 commented Oct 8, 2023

SteveLauC commented Oct 8, 2023

Jan561 commented Oct 8, 2023 • edited Loading

SteveLauC commented Oct 8, 2023 • edited Loading

Jan561 commented Oct 8, 2023 • edited Loading

SteveLauC commented Oct 9, 2023

Jan561 left a comment

Choose a reason for hiding this comment

SteveLauC commented Oct 10, 2023

Jan561 commented Oct 10, 2023

Jan561 commented Oct 12, 2023

SteveLauC commented Oct 13, 2023

SteveLauC commented Oct 17, 2023

asomers commented Oct 21, 2023 via email

SteveLauC commented Nov 5, 2023 • edited Loading

asomers commented Nov 5, 2023

SteveLauC commented Nov 6, 2023

Jan561 commented Nov 6, 2023

asomers commented Nov 6, 2023

SteveLauC commented Nov 6, 2023

SteveLauC commented Oct 8, 2023 •

edited

Loading

Jan561 commented Oct 8, 2023 •

edited

Loading

SteveLauC commented Oct 8, 2023 •

edited

Loading

Jan561 commented Oct 8, 2023 •

edited

Loading

SteveLauC commented Nov 5, 2023 •

edited

Loading