Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port main CI from Azure Pipelines to GitHub Actions #3978

Merged
merged 29 commits into from
May 12, 2022

Conversation

kinke
Copy link
Member

@kinke kinke commented May 7, 2022

No description provided.

@kinke kinke force-pushed the gh_actions_test branch 6 times, most recently from deb7a9a to 01ea63c Compare May 7, 2022 15:42
@kinke
Copy link
Member Author

kinke commented May 7, 2022

Okay, looks like this (AFAICT, undocumented) approach (from https://stackoverflow.com/a/70516118) would work to replace Azure Pipelines by GH Actions in a pretty smooth way, keeping separate 'includable' YAML files for a bunch of grouped steps. In the UI, such 'action' (sub)steps aren't expanded into the including job, but are nested in the action's step, and manually specified names for these nested steps aren't displayed unfortunately. Edit: Oh, and the action steps are apparently not collapsible separately either, so you can only collapse the whole action or display it all. Pretty bad for some test action covering LDC unittests, lit-tests, dmd-testsuite and defaultlib tests, we'd probably have to grep for the error again...

@kinke kinke changed the title GH Actions test wrt. composable local actions GH Actions test wrt. composite local actions May 7, 2022
@kinke kinke force-pushed the gh_actions_test branch 22 times, most recently from 786ffdb to b5f2554 Compare May 8, 2022 18:45
@kinke kinke force-pushed the gh_actions_test branch from a932c80 to 6cc4194 Compare May 11, 2022 00:15
@kinke kinke force-pushed the gh_actions_test branch 2 times, most recently from af6b7a7 to 1784cb4 Compare May 11, 2022 02:50
@kinke kinke force-pushed the gh_actions_test branch 2 times, most recently from ca32003 to 5e6b401 Compare May 11, 2022 17:17
@kinke kinke force-pushed the gh_actions_test branch from 5e6b401 to 1fe9799 Compare May 11, 2022 18:05
@kinke kinke force-pushed the gh_actions_test branch from 979d0e4 to 15eb2b0 Compare May 11, 2022 21:38
@kinke
Copy link
Member Author

kinke commented May 12, 2022

Getting there. Some observations wrt. GitHub Actions vs. Azure Pipelines:

  • Nested steps in composite actions:
    • aren't inlined into the job, but nested in the action step
    • don't get their name displayed, but the first line of the command (annoying, hopefully changed someday)
    • aren't collapsible (annoying)
    • aren't displayed at all if skipped by if condition
    • can have explicit inputs, allowing to significantly reduce the usage of env vars
    • can show their extra env variables (by expanding their Run header line)
    • have their per-output-line timestamps collected, which can be displayed in the GUI (edit: oh, available for Azure too - with sub-second granularity even)
  • Integration is obviously much better - less clicks to get to a CI log, much simpler setup/administration (e.g., no need for a Microsoft account, and no need to set up a token for GitHub release artifacts upload). It's so smooth that the test runs unexpectedly created a CI release in my personal fork (the workflow runs automatically/accidentally on my fork too…). ;) - So forks (on GitHub) should get Continuous Deployment basically for free.
  • Being able to use 'foreign' GH Actions is nice and opens up new possibilities. Installing the latest LDC version as host compiler, installing ninja system-wide, but also more advanced stuff like fetching existing GitHub release artifacts (e.g., the Travis and Cirrus CI artifacts), so that the sha256sums.txt file contains those hashes too (as long as those 'foreign' jobs are already completed), and removing all existing artifacts (from the CI release) before uploading the new ones - so that I don't have to periodically clean up those CI release artifacts manually anymore.

Otherwise, it seems to be mostly the same thing, with some YAML and GUI differences on top of the same (?) agents+image infrastructure, incl. their general slowness (2 (Linux/Windows) / 3 (macOS) CPU cores only) and sporadic failures.

While going over the stuff, I've restructured things a bit to fit the new constraints; e.g., Windows is less exotic now and handled in the OS-generic actions. It's also mostly using bash now too, which also fixes previously missing || exit /b suffices for all batch file commands etc.

@kinke kinke marked this pull request as ready for review May 12, 2022 01:31
@kinke
Copy link
Member Author

kinke commented May 12, 2022

Oh, seems like GH Actions doubled the number of concurrent CI jobs to 20 for the Free plan (https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits). I was thinking about moving the supported-LLVM-versions jobs to Azure Pipelines, but that seems superfluous now...

@kinke kinke force-pushed the gh_actions_test branch from 7daf0cd to 9dc564e Compare May 12, 2022 17:47
@kinke kinke force-pushed the gh_actions_test branch from f85e3bf to 6a174e3 Compare May 12, 2022 18:08
@kinke kinke merged commit 991fc36 into ldc-developers:master May 12, 2022
@kinke kinke deleted the gh_actions_test branch May 12, 2022 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant