Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Working CI with non-gui phases #4

Merged
merged 60 commits into from
Nov 9, 2023
Merged

Working CI with non-gui phases #4

merged 60 commits into from
Nov 9, 2023

Conversation

hmelder
Copy link
Collaborator

@hmelder hmelder commented Oct 29, 2023

This is currently a draft PR aiming to get a fully automated toolchain up and running.

Features:

  • Sandboxed building using Debian's pbuilder, and pdebuild along with debhelper utilities
    • pbuilder hooks for dependency management via local repository (See pbuilder/)
    • Transparent pbuilder management
  • Fast building using Github Actions
    • Phases clearly separated for easy debugging
    • Packages uploaded as artefacts
    • Option to skip sandboxing for workers*
  • Build utility should be aware of dependencies and build phases
  • Automatic packaging of origin tarballs required by quilt
  • Architecture, and release independent
  • Storing debian/ package descriptions with metadata separate from the source (more on that below)
    • Transparently fetching sources from git

Traditionally, debian/ configuration is shipped along side the code. A maintainer would grab the latest release and upload it to the corresponding git repository of the debian package.

Modern distributions like NixOS, VoidLinux, or ArchLinux aggregate all package descriptions inside a single mono-repo to avoid complex ownership hierarchies or teams. This lowers the entry barrier of updating a package significantly:

  • Complex dependencies can be managed from one synchronised state, instead of hundreds of independent repositories which need manual coordination (-> time intensive)
  • Patching package descriptions is easier
  • Simpler CI/CD
  • Automatic tools to notify the current maintainer of a new release

I chose to do the same for this repository. The resulting structure:

  • Debian package file in `source/debian´
  • Additional metadata (name, upstream, installation order (non-sandbox)) in source/description as JSON
  • cmd The build utility. Invoked via ./cmd/main.sh.
  • pbuilder: Configuration, and hooks for pbuilder

Definitions:

  • debootstrap: A tool for bootstrapping a debian root filesystem
  • pbuilder: A tool that uses debootstrap to build a minimal rootfs, and manages the building of debian packages within this rootfs.
  • debuild: The standard way of building packages directly on the HOST system
  • debhelper: An aggregation of ancient and painful to use build scripts (The debian/rules file is a Makefile with multiple pre-configured "pipelines")
  • To speed-up the building of packages with GitHub Actions, I chose to implement an additional option that uses a completely different way of building the packages natively. This should not be used on a normal machine!

@hmelder hmelder marked this pull request as ready for review November 5, 2023 15:03
@hmelder hmelder changed the title Add PBuilder Scripts and rename packages Working CI with non-gui phases Nov 5, 2023
@hmelder
Copy link
Collaborator Author

hmelder commented Nov 5, 2023

Ready to review!

@hmelder hmelder requested a review from gcasa November 5, 2023 15:30
@hmelder
Copy link
Collaborator Author

hmelder commented Nov 5, 2023

The workflow uploads all the packages as compressed tarballs containing the debian packages.

See: https://github.com/gnustep/tools-packaging/actions/runs/6762034966

@hmelder
Copy link
Collaborator Author

hmelder commented Nov 5, 2023

Turns out we need QEMU for ARM64 building. I have looked at https://github.com/uraimo/run-on-arch-action, but they do not support bookworm.

I would suggest postponing the cross CI for now

@hmelder hmelder requested a review from ivucica November 8, 2023 16:52
@hmelder
Copy link
Collaborator Author

hmelder commented Nov 9, 2023

Merged as discussed in email. Further work with cross-building is following

@hmelder hmelder merged commit 0c1766d into main Nov 9, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

1 participant