ci: refactor GitHub Actions CI to use reusable workflows, respect DEP_OPTS, build multiprocess depends with Clang
#6400
Conversation
kwvg
changed the title
kwvg
force-pushed
the
ci_stuff
branch
2 times, most recently
from
15cdb62 to
1d1ff41
Compare
kwvg
changed the title
|
This pull request has conflicts, please rebase. |
|
This pull request has conflicts, please rebase. |
kwvg
force-pushed
the
ci_stuff
branch
2 times, most recently
from
3642091 to
3cc8c70
Compare
There was a problem hiding this comment.
light utACK 3cc8c70
I have concerns about this PR; but I'm not sure how to communicate them, and don't have good actionable feedback.
I'm concerned the patch-set as is will make the GitHub CI / build.yml harder to understand and harder to improve; but I also cannot at this moment figure out an alternative way to accomplish what is needed to be accomplished. Some of the commits just "feel" wrong, but ultimately it's just for CI, so it's not that big of a deal, and I don't have ways imagined for how to improve the situation; so probably best path is to merge, and maybe I'll figure out something I like more later.
.github/workflows/build.yml
Outdated
| echo "dep-matrix=$(jq -r '.dep' -c .github/workflows/matrix.json)" >> $GITHUB_OUTPUT | ||
| echo "src-matrix=$(jq -r '.src' -c .github/workflows/matrix.json)" >> $GITHUB_OUTPUT |
There was a problem hiding this comment.
I kinda hate using jq in the build.yml like this; but I guess it's ok
There was a problem hiding this comment.
The alternative would be having to maintain two JSON files (they would have to be JSON if we want to share params between them without resorting to duplication), which wouldn't be the worst thing in the world but imo since src will be reading from dep for values and they are configuring the same actions run, just different jobs, consolidation would be better.
Plus we'd be using jq later on regardless to read through the matrix definition later on to find dep_opts and host from depends_name.
There was a problem hiding this comment.
these 2 JSON seems doens't have any dependency on each other, maybe better to have 2 files?
Is there any case, when you need both of them at once?
There was a problem hiding this comment.
Because they the second section (src) refers to the first section (dep) later on and they ultimately represent two halves of one workflow.
dash/.github/workflows/build.yml
Lines 126 to 136 in 3cc8c70
Examples of these are explained in the PR description.
.github/workflows/build.yml
Outdated
| - build_target: linux64_nowallet | ||
| host: x86_64-pc-linux-gnu | ||
| depends_on: linux64 | ||
| matrix: ${{ fromJson(needs.build-image.outputs.src-matrix) }} |
There was a problem hiding this comment.
I fear this overcomplicates build.yml and will make it harder to improve / debug / logic on
There was a problem hiding this comment.
Outputs have to be generated in the previous job in order to be used in the current job and creating a new job just to read the matrix seems a bit wasteful, so the build image step is being used to also read, generate and export as output, the matrices needed for the next two jobs.
We can probably try reusable workflows (source) in the future, it'll be more GitLab-like but would invite duplication and a different kind of convoluted syntax. Depends on how reliable this setup proves to be.
|
I think I found a way to do this with no jq/json and no duplication either (and bring two CI-s a bit closer to each other while at it), pls check UdjinM6@c40a595 (CI: Github + Gitlab) |
|
@UdjinM6, I like the idea of going back to honoring the One of the arrangements are to use the Another example is the SQLite build, which has a different This can be trivially fixed by ensuring all the So the proposed approach does deduplicate between GitHub and GitLab configurations but requires duplication in every matrix definition script. If this is an acceptable tradeoff, I'll cherry-pick the changes. Thoughts? |
|
Interesting... So we were using wrong depends for these build targets in CI all this time. Smth like 94e37ef (CI: https://github.com/UdjinM6/dash/actions/runs/11999294004 + https://gitlab.com/UdjinM6/dash/-/pipelines/1558281977) on top of my previous commit should fix it (NOTE: |
|
Also, I wonder why we bump clang to 18 manually via 77795a5 instead of doing it via corresponding backports. Can you clarify this pls? |
kwvg
changed the title
kwvg
changed the title
kwvg
changed the title
DEP_OPTS, build multiprocess depends with Clang
|
GitHub Actions run for cfdd2c6, https://github.com/kwvg/dash/actions/runs/13245494176.
Note: The GitLab build failures for |
WalkthroughThe changes introduce several new GitHub Actions workflow files and update existing CI configurations. A new workflow for building Docker containers automates the process of container building, tagging, and pushing to the GitHub Container Registry. Additional workflows are provided for dependency caching and source code builds, each accepting predefined inputs, performing environment setup, managing caches, and uploading artifacts. The main GitHub build workflow has been restructured to reference these reusable workflows by defining separate jobs for container building, dependency processing for multiple targets, and source code building. In parallel, the GitLab CI configuration has been updated, renaming the variable from Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🔭 Outside diff range comments (2)
.github/workflows/build.yml (1)
1-1: Merge Conflict AlertMultiple conflict errors were reported by the pipeline for this file. Please resolve these conflicts (as indicated by conflicting PR numbers) before merging to ensure a clean CI integration.
🧰 Tools
🪛 GitHub Actions: Check Potential Conflicts
[error] 1-1: Conflict detected in .github/workflows/build.yml. This file is involved in multiple conflicting pull requests.
.gitlab-ci.yml (1)
1-1: Merge Conflict Alert in GitLab CIThe pipeline failures indicate multiple merge conflicts with various PRs. Please resolve these conflicts to guarantee a stable and consistent CI configuration.
🧰 Tools
🪛 GitHub Actions: Check Potential Conflicts
[error] 1-1: Conflict detected with PR number 5971.
[error] 1-1: Conflict detected with PR number 5606.
[error] 1-1: Conflict detected with PR number 6421.
[error] 1-1: Conflict detected with PR number 6387.
[error] 1-1: Conflict detected with PR number 6380.
[error] 1-1: Conflict detected with PR number 6564.
[error] 1-1: Conflict detected with PR number 6563.
[error] 1-1: Conflict detected with PR number 6559.
🧹 Nitpick comments (11)
ci/test/00_setup_env_native_multiprocess.sh (2)
12-12: Update DEP_OPTS with explicit compiler settings.
ReplacingDEP_OPTS="DEBUG=1 MULTIPROCESS=1"withDEP_OPTS="MULTIPROCESS=1 CC=clang-18 CXX=clang++-18"clearly communicates the desired multiprocess mode and compiler versions. Ensure that switching from an older clang (e.g. clang-16 as previously discussed) to clang-18 is intentional and that downstream build scripts are compatible.
1-17: Resolve pipeline conflict concerns.
The pipeline reports a conflict in this file. Confirm that these changes are harmonized with other outstanding modifications to avoid integration issues.🧰 Tools
🪛 GitHub Actions: Check Potential Conflicts
[error] 1-1: Conflict detected in ci/test/00_setup_env_native_multiprocess.sh. This file is involved in multiple conflicting pull requests.
ci/test/00_setup_env_native_qt5.sh (2)
12-12: Clear DEP_OPTS for QT5 testing.
ChangingDEP_OPTSfrom"NO_UPNP=1 DEBUG=1"to an empty string removes previous build flags. Please verify that QT5 tests do not require any of these options and that this aligns with the updated CI strategy.
1-19: Address reported pipeline conflicts.
This file is flagged for potential conflicts with other pull requests. Ensure any overlaps are resolved to maintain CI stability.🧰 Tools
🪛 GitHub Actions: Check Potential Conflicts
[error] 1-1: Conflict detected in ci/test/00_setup_env_native_qt5.sh. This file is involved in multiple conflicting pull requests.
.github/workflows/build-src.yml (1)
43-44: Quote GITHUB_OUTPUT in echo statements.
To avoid word splitting or globbing issues, please enclose the output file variable in double quotes. For example, modify the lines as follows:-echo "HOST=${HOST}" >> $GITHUB_OUTPUT -echo "PR_BASE_SHA=${{ github.event.pull_request.base.sha || '' }}" >> $GITHUB_OUTPUT +echo "HOST=${HOST}" >> "$GITHUB_OUTPUT" +echo "PR_BASE_SHA=${{ github.event.pull_request.base.sha || '' }}" >> "$GITHUB_OUTPUT".github/workflows/build-depends.yml (1)
70-70: Quote environment variable expansions in the build step.
To prevent potential word splitting in the shell, please quote the variables in theenvcommand. For example, update the step as follows:- - name: Build depends - run: env ${{ steps.setup.outputs.DEP_OPTS }} HOST=${{ steps.setup.outputs.HOST }} make -j$(nproc) -C depends + - name: Build depends + run: env "${{ steps.setup.outputs.DEP_OPTS }}" HOST="${{ steps.setup.outputs.HOST }}" make -j$(nproc) -C depends🧰 Tools
🪛 actionlint (1.7.4)
70-70: shellcheck reported issue in this script: SC2046:warning:1:85: Quote this to prevent word splitting
(shellcheck)
.github/workflows/build.yml (3)
36-43: Depends Job for linux64_multiprocess: Multiprocess ConfigurationThe "depends-linux64_multiprocess" job sets
build-target: linux64_multiprocessand uses the container output appropriately. Consider confirming that any multiprocess-related caching or dependency nuances are addressed in thebuild-depends.ymlworkflow.
79-87: Source Build Job (linux64_fuzz): Fuzz Testing ConsiderationsThe "src-linux64_fuzz-build" job forwards parameters appropriate for a fuzz build. Since fuzz testing typically has unique resource or flag requirements, verify that the build and test scripts are well-tuned for fuzzing.
88-96: Source Build Job (linux64_multiprocess): Multiprocess Build ValidationThe "src-linux64_multiprocess-build" job uses outputs from the
depends-linux64_multiprocessjob. Please confirm that the multiprocess settings are fully supported by both the dependency and source build workflows, and consider additional checks for potential race conditions..gitlab-ci.yml (2)
46-50: Enhanced Logging in Build-Depends TemplateThe added echo statements outputting
BUILD_TARGET,HOST, andDEP_OPTSimprove debugging visibility during the dependency build phase. Just ensure that these logs do not inadvertently expose sensitive configuration details.
79-80: Base Template Logging EnhancementsThe echo commands for
HOSTandDEP_OPTSin the.base-templateimprove traceability of environment variables. Consider conditionally reducing log verbosity in production to avoid excessive output.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
.github/workflows/build-container.yml(1 hunks).github/workflows/build-depends.yml(1 hunks).github/workflows/build-src.yml(1 hunks).github/workflows/build.yml(1 hunks).gitlab-ci.yml(11 hunks)ci/test/00_setup_env_native_multiprocess.sh(1 hunks)ci/test/00_setup_env_native_nowallet.sh(1 hunks)ci/test/00_setup_env_native_qt5.sh(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- ci/test/00_setup_env_native_nowallet.sh
🧰 Additional context used
🪛 GitHub Actions: Check Potential Conflicts
ci/test/00_setup_env_native_qt5.sh
[error] 1-1: Conflict detected in ci/test/00_setup_env_native_qt5.sh. This file is involved in multiple conflicting pull requests.
ci/test/00_setup_env_native_multiprocess.sh
[error] 1-1: Conflict detected in ci/test/00_setup_env_native_multiprocess.sh. This file is involved in multiple conflicting pull requests.
.gitlab-ci.yml
[error] 1-1: Conflict detected with PR number 5971.
[error] 1-1: Conflict detected with PR number 5606.
[error] 1-1: Conflict detected with PR number 6421.
[error] 1-1: Conflict detected with PR number 6387.
[error] 1-1: Conflict detected with PR number 6380.
[error] 1-1: Conflict detected with PR number 6564.
[error] 1-1: Conflict detected with PR number 6563.
[error] 1-1: Conflict detected with PR number 6559.
.github/workflows/build.yml
[error] 1-1: Conflict detected in .github/workflows/build.yml. This file is involved in multiple conflicting pull requests.
🪛 actionlint (1.7.4)
.github/workflows/build-depends.yml
70-70: shellcheck reported issue in this script: SC2046:warning:1:85: Quote this to prevent word splitting
(shellcheck)
.github/workflows/build-src.yml
35-35: shellcheck reported issue in this script: SC2086:info:8:24: Double quote to prevent globbing and word splitting
(shellcheck)
35-35: shellcheck reported issue in this script: SC2086:info:9:71: Double quote to prevent globbing and word splitting
(shellcheck)
.github/workflows/build-container.yml
28-28: shellcheck reported issue in this script: SC2086:info:3:30: Double quote to prevent globbing and word splitting
(shellcheck)
28-28: shellcheck reported issue in this script: SC2086:info:4:29: Double quote to prevent globbing and word splitting
(shellcheck)
🔇 Additional comments (24)
ci/test/00_setup_env_native_multiprocess.sh (1)
10-10: Add HOST variable for unified architecture.
The new variableexport HOST=x86_64-pc-linux-gnusolidifies consistency across CI environments. Please verify that all related scripts use the same value.ci/test/00_setup_env_native_qt5.sh (1)
10-10: Introduce HOST variable for consistency.
Addingexport HOST=x86_64-pc-linux-gnuensures uniformity across CI scripts. This is in line with similar changes elsewhere..github/workflows/build-container.yml (2)
1-9: Container build workflow structure is solid.
The new workflow, defined withworkflow_call, correctly outputs the container path and tag. TheDOCKER_DRIVERsetting is properly configured for container builds.
26-33: Prepare variables step utilizes consistent naming.
The script correctly transforms branch and repository names to lowercase. Consider adding error handling ifGITHUB_REFdoes not follow the expected format.🧰 Tools
🪛 actionlint (1.7.4)
28-28: shellcheck reported issue in this script: SC2086:info:3:30: Double quote to prevent globbing and word splitting
(shellcheck)
28-28: shellcheck reported issue in this script: SC2086:info:4:29: Double quote to prevent globbing and word splitting
(shellcheck)
.github/workflows/build-src.yml (1)
1-45: Build source workflow exhibits clear modularity.
The overall structure—from checkout and initial setup through caching and artifact upload—is well organized. Ensure that all environment variables (e.g.,BUILD_TARGETand keys) remain in sync across workflows.🧰 Tools
🪛 actionlint (1.7.4)
35-35: shellcheck reported issue in this script: SC2086:info:8:24: Double quote to prevent globbing and word splitting
(shellcheck)
35-35: shellcheck reported issue in this script: SC2086:info:9:71: Double quote to prevent globbing and word splitting
(shellcheck)
.github/workflows/build.yml (11)
16-19: Container Job Setup: Reusable Workflow InvocationThe new "container" job calls the reusable workflow (
build-container.yml) to build and push the Docker container. Please ensure that the reusable workflow correctly exports the expected output (notably thepath) consumed by downstream jobs.
20-27: Depends Job for arm-linux: Parameter AlignmentThe "depends-arm-linux" job correctly passes
build-target: arm-linuxand references the container output via${{ needs.container.outputs.path }}. Confirm that the downstreambuild-depends.ymlworkflow is updated to expect these parameters.
28-35: Depends Job for linux64: Input ValidationThe "depends-linux64" job passes the appropriate parameters (with
build-target: linux64) and reuses the container output correctly. Verify that caching and dependency handling work as intended in the corresponding workflow.
44-51: Depends Job for linux64_nowallet: Target VerificationThe "depends-linux64_nowallet" job is configured for the no-wallet build target using
build-target: linux64_nowallet. Ensure that the downstream processes adjust for the absence of wallet-related components.
52-60: Source Build Job (arm-linux): Consistent Parameter PassingThe "src-arm-linux" job uses the reusable
build-src.ymlworkflow, passingbuild-target: arm-linux, the container output, and the dependency key. Verify that all inputs are consistently interpreted by the source build workflow.
61-69: Source Build Job (linux64): Platform ConsistencyThe "src-linux64" job correctly references
build-target: linux64and propagates bothcontainer-pathand the dependency key from the appropriate depends job. Ensure that any platform-specific quirks for linux64 are handled in the workflow.
70-78: Source Build Job (linux64_cxx20): C++20 ConfigurationThe "src-linux64_cxx20-build" job sets
build-target: linux64_cxx20. Please double-check that the build environment and flags for C++20 are correctly configured in the reusable source build workflow.
97-105: Source Build Job (linux64_nowallet): No-Wallet Build AccuracyThe "src-linux64_nowallet-build" job appears correctly set up with
build-target: linux64_nowalletalong with matching outputs from its dependency. Ensure that any wallet-specific steps are excluded from this target.
106-114: Source Build Job (linux64_sqlite): SQLite Integration CheckThe "src-linux64_sqlite-build" job assigns
build-target: linux64_sqliteand passes the required container and dependency parameters. Confirm that the SQLite-specific build options are being implemented correctly downstream.
115-123: Source Build Job (linux64_tsan): Thread Sanitizer EnablementThe "src-linux64_tsan-build" job is crafted for ThreadSanitizer builds. It correctly retrieves the dependency key from
depends-linux64_multiprocess. Please ensure that TSan flags, environment setup, and any sanitizer-specific steps are integrated properly.
124-132: Source Build Job (linux64_ubsan): Undefined Behavior Sanitizer SetupThe "src-linux64_ubsan-build" job, configured for UBSan builds with
BUILD_TARGET: linux64_ubsan, appears sound. Verify that any undefined behavior checks and related compiler flags are managed as expected..gitlab-ci.yml (8)
178-178: Variable Update for arm-linux BuildThe update to
BUILD_TARGET: arm-linuxis correctly applied. Be sure that downstream scripts referenceBUILD_TARGETinstead of the oldHOSTvariable.
185-186: Variable Update for win64 TargetThe job for x86_64-w64-mingw32 now uses
BUILD_TARGET: win64, maintaining consistency with the new naming convention. Confirm that any Windows-specific build scripts are adjusted accordingly.
189-190: Linux Build Target ConsistencyThe update to
BUILD_TARGET: linux64in the x86_64-pc-linux-gnu job aligns with the overall refactoring. Ensure that any remaining references to the old variable naming are updated.
196-197: No-Wallet Build Target UpdateThe variable change to
BUILD_TARGET: linux64_nowalletfor the corresponding job is appropriate. Verify that this change correctly disables wallet functions within the CI scripts.
203-204: Multiprocess Build Target UpdateThe assignment
BUILD_TARGET: linux64_multiprocessis applied correctly for the multiprocess builds. Ensure that all related pipelines and caching mechanisms utilize this new identifier consistently.
210-211: Darwin Build Target ConfigurationThe job for x86_64-apple-darwin now sets
BUILD_TARGET: mac, streamlining the naming convention. Please confirm that the macOS build environment is properly adjusted with this change.
215-327: Consistency Across Build JobsAll build jobs extending the
.build-template(such as arm-linux-build, win64-build, linux64-build, and linux64_cxx20-build) now use the updatedBUILD_TARGETvariable. It is important to verify that all extended templates and build scripts uniformly refer toBUILD_TARGETand that no remnants of the oldHOSTnomenclature persist.
330-398: Test Job Configuration UpdatesThe test job definitions (e.g., linux64-test, linux64_sqlite-test, linux64_tsan-test, linux64_ubsan-test, linux64_multiprocess-test, and linux64_nowallet-test) have been updated to reflect the new
BUILD_TARGETsetting. Ensure that any internal script references and environment configurations in test phases are aligned with these changes.
|
This pull request has conflicts, please rebase. |
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
|
GitHub Actions run for a976777, https://github.com/kwvg/dash/actions/runs/13250110979.
|
There was a problem hiding this comment.
Actionable comments posted: 0
🔭 Outside diff range comments (1)
.github/workflows/build.yml (1)
1-1:⚠️ Potential issueResolve Merge Conflicts in build.yml
The pipeline report indicates that there is a conflict in this file. Please resolve any merge conflicts before merging the PR.🧰 Tools
🪛 GitHub Actions: Check Potential Conflicts
[error] 1-1: Conflict detected in .github/workflows/build.yml. This file is involved in multiple conflicting pull requests.
🧹 Nitpick comments (3)
.github/workflows/build-container.yml (1)
26-33: Improve Shell Variable Quoting in the "Prepare variables" Step
In the shell script block (lines 29–32), the outputs are appended using$GITHUB_OUTPUTwithout quotes. Although the current usage may work, it is best practice to quote variables to avoid potential word splitting or globbing issues. A suggested change is:- echo "tag=${BRANCH_NAME}" >> $GITHUB_OUTPUT - echo "repo=${REPO_NAME}" >> $GITHUB_OUTPUT + echo "tag=${BRANCH_NAME}" >> "$GITHUB_OUTPUT" + echo "repo=${REPO_NAME}" >> "$GITHUB_OUTPUT"This improves the robustness of the script.
🧰 Tools
🪛 actionlint (1.7.4)
28-28: shellcheck reported issue in this script: SC2086:info:3:30: Double quote to prevent globbing and word splitting
(shellcheck)
28-28: shellcheck reported issue in this script: SC2086:info:4:29: Double quote to prevent globbing and word splitting
(shellcheck)
.github/workflows/build-src.yml (1)
33-45: Enhance Variable Quoting in the "Initial setup" Step
In the Initial setup step (lines 33–45), outputs are appended to$GITHUB_OUTPUTwithout using quotes. For consistency and reliability, please consider quoting the variable. For example:- echo "HOST=${HOST}" >> $GITHUB_OUTPUT - echo "PR_BASE_SHA=${{ github.event.pull_request.base.sha || '' }}" >> $GITHUB_OUTPUT + echo "HOST=${HOST}" >> "$GITHUB_OUTPUT" + echo "PR_BASE_SHA=${{ github.event.pull_request.base.sha || '' }}" >> "$GITHUB_OUTPUT"This change will prevent potential issues with word splitting.
🧰 Tools
🪛 actionlint (1.7.4)
35-35: shellcheck reported issue in this script: SC2086:info:8:24: Double quote to prevent globbing and word splitting
(shellcheck)
35-35: shellcheck reported issue in this script: SC2086:info:9:71: Double quote to prevent globbing and word splitting
(shellcheck)
.github/workflows/build-depends.yml (1)
69-71: Quote Environment Variables in the Build Depends Command
The build command on line 70 uses unquoted step outputs when setting environment variables. To prevent word splitting, please enclose the interpolated values in quotes. For example:- - name: Build depends - run: env ${{ steps.setup.outputs.DEP_OPTS }} HOST=${{ steps.setup.outputs.HOST }} make -j$(nproc) -C depends + - name: Build depends + run: env "${{ steps.setup.outputs.DEP_OPTS }}" HOST="${{ steps.setup.outputs.HOST }}" make -j$(nproc) -C dependsThis change enhances stability when the variable values contain spaces or special characters.
🧰 Tools
🪛 actionlint (1.7.4)
70-70: shellcheck reported issue in this script: SC2046:warning:1:85: Quote this to prevent word splitting
(shellcheck)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
.github/workflows/build-container.yml(1 hunks).github/workflows/build-depends.yml(1 hunks).github/workflows/build-src.yml(1 hunks).github/workflows/build.yml(1 hunks).gitlab-ci.yml(11 hunks)ci/test/00_setup_env_native_multiprocess.sh(1 hunks)ci/test/00_setup_env_native_nowallet.sh(1 hunks)ci/test/00_setup_env_native_qt5.sh(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
- ci/test/00_setup_env_native_qt5.sh
- ci/test/00_setup_env_native_nowallet.sh
- ci/test/00_setup_env_native_multiprocess.sh
- .gitlab-ci.yml
🧰 Additional context used
🪛 GitHub Actions: Check Potential Conflicts
.github/workflows/build.yml
[error] 1-1: Conflict detected in .github/workflows/build.yml. This file is involved in multiple conflicting pull requests.
🪛 actionlint (1.7.4)
.github/workflows/build-container.yml
28-28: shellcheck reported issue in this script: SC2086:info:3:30: Double quote to prevent globbing and word splitting
(shellcheck)
28-28: shellcheck reported issue in this script: SC2086:info:4:29: Double quote to prevent globbing and word splitting
(shellcheck)
.github/workflows/build-depends.yml
70-70: shellcheck reported issue in this script: SC2046:warning:1:85: Quote this to prevent word splitting
(shellcheck)
.github/workflows/build-src.yml
35-35: shellcheck reported issue in this script: SC2086:info:8:24: Double quote to prevent globbing and word splitting
(shellcheck)
35-35: shellcheck reported issue in this script: SC2086:info:9:71: Double quote to prevent globbing and word splitting
(shellcheck)
⏰ Context from checks skipped due to timeout of 90000ms (9)
- GitHub Check: Build (win64, x86_64-w64-mingw32, win64)
- GitHub Check: Build (linux64_ubsan, x86_64-pc-linux-gnu, linux64)
- GitHub Check: Build (linux64_tsan, x86_64-pc-linux-gnu, linux64)
- GitHub Check: Build (linux64_sqlite, x86_64-pc-linux-gnu, linux64)
- GitHub Check: Build (linux64_nowallet, x86_64-pc-linux-gnu, linux64)
- GitHub Check: Build (linux64_fuzz, x86_64-pc-linux-gnu, linux64)
- GitHub Check: Build (linux64_cxx20, x86_64-pc-linux-gnu, linux64)
- GitHub Check: Build (linux64, x86_64-pc-linux-gnu, linux64)
- GitHub Check: Build (arm-linux, arm-linux-gnueabihf, arm-linux)
🔇 Additional comments (10)
.github/workflows/build-container.yml (2)
1-8: New Workflow Declaration Looks Good
The overall structure for building and pushing the Docker container is clear and well structured.
34-55: Docker Setup and Image Build Steps are Correct
The steps for setting up Docker Buildx, logging into the container registry, and configuring the build/push action are well implemented..github/workflows/build-src.yml (2)
1-18: New "Build source" Workflow Declaration Approved
The workflow inputs and environment settings are clearly defined and meet the requirements.
47-85: Remaining Steps are Well-Structured
The checkout, cache restore, ccache management, build execution, and artifact upload steps are clearly defined and consistent with the project’s CI practices..github/workflows/build-depends.yml (3)
1-18: "Build depends" Workflow Declaration is Clear
The new workflow clearly defines required inputs and outputs for dependency caching, and the overall structure looks good.
34-45: Initial Setup Block is Well-Implemented
The Initial setup step correctly sources the required matrix script and computes the dependency hash. The use of quotes with"$GITHUB_OUTPUT"is correct here.
47-68: Caching Steps are Configured Correctly
The actions for caching dependency sources and restoring cached dependencies are implemented appropriately using the generated hash and host-specific paths..github/workflows/build.yml (3)
15-19: Container Job Setup is Correct
The job invoking the reusablebuild-container.ymlworkflow (lines 16–19) is concise and correctly configured.
20-59: Depends Jobs Configuration is Consistent
The various dependency jobs (e.g.depends-arm-linux,depends-linux64, etc.) are cleanly defined and properly depend on the container job. Ensure that the container-path input is correctly resolved across all jobs.
60-149: Source Build Jobs Are Well-Organized
The multiple source build jobs properly reuse thebuild-src.ymlworkflow and reference the appropriate dependency outputs. The inputs across these jobs (build-target, container-path, and depends-key) are consistently and clearly set.
There was a problem hiding this comment.
Not sure about c9708b7 but it's probably fine.
And it seems to be working as expected otherwise: https://github.com/UdjinM6/dash/actions/runs/13265321830 so
ACK a976777
Additional Information
Depends on ci: use
actions/cacheto manage depends cache #6406We have to move away from matrices as matrix-based job variants cannot be individually accepted as dependencies for the next job, only the whole job can. This means that the next job cannot start until the last matrix variant of the dependency succeeds. This also means that if any variant fails in the dependency job fails, every job forward also ceases.
The approach taken in this PR is a mirror of the current implementation on GitLab CI, defining a reusable unit ("templates" in GitLab parlance, "reusable workflows" in GitHub) and defining each variant, now coming with a unique job name that can be specified as dependencies for the job. This should reduce stalling from slower variants.
DEBUG=1had to be dropped as it resulted in the runners running out of free space (build). Despite this only being a problem with GitHub Actions, this change also affects GitLab CI as we now readDEP_OPTSfrom matrix scripts instead of the CI configuration file.We are reading from the matrix script to ensure that both CI instances build with the same configuration and to reduce code complexity arising from passing around details between different workflows.
Potential
DEP_OPTSconflicts (as discussed here) have been circumvented by passing the expected cache key wholesale instead of reconstructing it (commit). We do still expect thatHOSTwill be defined correctly in the matrix script (theHOSTof both depends and build should match) but as differingHOSTs are incompatible regardless, this shouldn't be a problem.outputcache-primary-key, the depends cache step was split in two so that the output from therestoreaction is can be exported from the workflow (source).As
DEP_OPTSis a new parameter that could require rebuilding depends, it needs to be incorporated into the cache key. The most straightforward way to do it is to append the hash of the file that definesDEP_OPTSto the cache key. Unfortunately, this requires us to hardcode the name of the individual file (e.g.,00_setup_env_native_qt5.shforlinux64) asmatrix.shis just a dispatcher and comes with the drawback that any change to the script could result in a cache miss due to a changed hash.This has been mitigated by hashing the build variant name and the environment variables that influence the build (source).
headtrims the output ofsha256sumto 64 characters, the hash itself isn't being trimmed but everything after it is (trailing hyphen and newline). This is also whyecho -nis used in some places, to avoid newline addition resulting in a different hash value.Currently there doesn't seem to be a way to have full control on a reusable workflow's name, it remains a fixed "[caller name] / [reusable workflow name]" and attempting to remove the name altogether will still result in a trailing slash (source).
Breaking Changes
None expected.
Checklist