piotr-yuxuan/walter-ci

<scherz>Walter Kohl is the younger son of Helmut Kohl. Like his father he likes to break down walls and reunify friends under one common fundamental law.</scherz>

Problem

It's extremely tedious to maintain workflows on more than two repositories. What if you want to make a change? What if you have had a different idea to improve CI on some later project? I think that losing all my time propagating changes by replication and copy/paste is an hindrance. That's the job of a machine, so let a machine do it.

Goal

The goal of Walter is to automate and standardise code grooming as much as possible so that an individual contributor can scale to more than two projects on GitHub.

Solution

Walter is an open-source Clojure-oriented CICD system. It intends to remove YAML boilerplate as much as possible while not impeding user freedom. It contains:

• A GitHub action "piotr-yuxuan/walter-ci@main" that brings useful commands like clojure CLI, leiningen, Babashka, some Leiningen profiles, the amazing Practicalli's deps.edn and Walter's own executable: walter.
• Walter's own executable contains code that is just difficult enough but not complex. For example, forwarding secrets or updating workflow files from Walter to other managed repositories.
• Predefined, opinionated workflows, expressed in edn. They are the same for all managed repositories. They will all behave the same way, as controlled by Walter. They will be maintained in sync so you may avoid the hell of Jenkinsfile that diverges over time.

It uses frugal means to yield vast power:

• An action that provides standard and community-maintained tools;
• No configuration for managed repositories. It's fine if Walter can't manage your edge-case repository, just write and maintain your own workflows. Walter's action will still provide a convenient base though.
• Leverage community tools as much as possible. « Not invented here » is a feature, not a bug so Walter code footprint is quite small.
• A small set of edn->yaml reader macros and predefined DRY steps to avoid repetition where possible;
• Some GIT_* environment variables;
• A runtime executable for operations that can not easily be expressed in bash.
• Favour bash as much as possible to express simple, composable steps.

Usage

• To let Walter manage your repositories, fork this repository and update managed-repositories.edn.

• To use the action in your workflow:

name: My simple workflow
jobs:
  my-job:
    runs-on: ubuntu-latest
    steps:
      # Hello 👋
      - uses: piotr-yuxuan/walter-ci@main
      - run: echo "Enjoy 👍"

Supported operations

• Performance tests, unit tests with great expressive power
• Report vulnerabilities
• Create and deploy releases
• User-defined release version
• Conditional checks and human approval before a release
• Distinguish between Walter action and walter cli tool, allow the use of different/fixed versions
• Upgrade dependencies
• (not yet) Enforce and fix reverse-domain-based project group name if deploying to Clojars (mandatory)
• Generate list of licenses
• (not yet) Add .java-version
• Run quality scan
• Lint files
• Sort namespace forms

It doesn't require any addition to code of the managed repository, except the installation step when workflow files are copied. It is a goal of this project to stay free from any project-level configuration duplicate hell. It is completely fine if some projects don't fit in it.

Toward version 1.0.0

• Rewrite README.md, make it clearer, and better structured
• Better explanation for each steps
• Externalise steps.edn and make it support self-reference and reader macros
• Proper documentation and getting started
• Add logging with custom level as a required argument with default value
• Reduce cache time, cache more at once and better
• Write tests for each command
• Enable release with pseudo version: major, minor, and patch bumps the current version, or a tag or commit sha may be provided.
• Custom walter jar version should accept latest, a tag, a branch name or a commit sha.
• Support at least one other codebase type so that the abstraction is here and may later be expanded to other codebases like Rust, KDB, stream processing jobs, libs, cli, and so on :-)
• Sign commits from Walter with a special key.
• Licence is derived from :licence value in project.clj, or fails otherwise.