Unifying archive package structure

[khos2ow]

Current State

As of now the apt URLs for each release have hardcoded Regolith version in it and it's a bit unpredictable when it comes to upgrading to another release (Regolith, or underlying OS: Debian, Ubuntu). Currently the apt URL is as follow:

https://regolith-desktop.org/testing-debian-testing-amd64
                             |       |      |       |
                             |       |      |       * Architecture
                             |       |      * Distro Version/Codename
                             |       * Distro Name
                             * Regolith Stage

and apt source entry is as follow:

deb [arch=amd64] https://regolith-desktop.org/testing-debian-testing-amd64 testing main
                                              |       |      |       |     |       |
                                              |       |      |       |     |       * Component
                                              |       |      |       |     * Distro Version/Codename
                                              |       |      |       * Architecture
                                              |       |      * Distro Version/Codename
                                              |       * Distro Name
                                              * Regolith Stage

Proposed Change

This proposal suggests to cleanup and shorten the url with the following format:

http://archive.regolith-desktop.com/ubuntu/unstable noble main
                                    |      |        |     |
                                    |      |        |     * Component
                                    |      |        * Distro Version/Codename
                                    |      * Regolith Stage
                                    *Distro Name

and apt source entry as follow:

deb [arch=amd64] http://archive.regolith-desktop.com/ubuntu/unstable noble main
          |                                          |      |        |     |
          |                                          |      |        |     * Component
          |                                          |      |        * Distro Version/Codename
          |                                          |      * Regolith Stage
          |                                          * Distro Name
          * Architecture

Changes

1. move Architecture out of url to deb source file entry
2. change base domain from regolith-desktop.org to archive.regolith-desktop.com
3. move Distro Name as the top level path of the package archive
4. move Regolith Stage out as the second level path of the package archive
5. move Distro Version/Codename out to the distribution section of its own
6. create corresponding Component section for Regolith Stage
   • main for every Regolith Stage
   • fixed version number (e.g. v3.1, v3.2, etc) for stable releases (e.g. release-3_2, release-3_1, etc)
7. Regolith Stage will not ever have the Regolith versions (e.g. release-3_1, etc)

Note that item 2 is my personal preference and I think it looks cool to have this, but ultimately this is the only optional item in this proposal.

Examples

The followings are some example of what the deb apt urls will look like:

• ubuntu/amd64
  • deb [arch=amd64] http://archive.regolith-desktop.com/ubuntu/stable noble main
  • deb [arch=amd64] http://archive.regolith-desktop.com/ubuntu/stable jammy v3.1
  • deb [arch=amd64] http://archive.regolith-desktop.com/ubuntu/stable jammy v3.2
  • deb [arch=amd64] http://archive.regolith-desktop.com/ubuntu/experimental noble main
  • deb [arch=amd64] http://archive.regolith-desktop.com/ubuntu/unstable mantic main
  • deb [arch=amd64] http://archive.regolith-desktop.com/ubuntu/testing mantic main
• debian/arm64
  • deb [arch=arm64] http://archive.regolith-desktop.com/debian/stable bookworm main
  • deb [arch=arm64] http://archive.regolith-desktop.com/debian/experimental bookworm main
  • deb [arch=arm64] http://archive.regolith-desktop.com/debian/unstable testing main
  • deb [arch=arm64] http://archive.regolith-desktop.com/debian/testing testing main

Considerations

Backward Compatibility

The proposed solution can be backward compatible because the changes are only in the URLs and with redirect rules on the running webserver most probably can be achieved.

Multiple Versions

Multiple versions of the same package must be supported with this new proposal because Regolith Stage (i.e. Regolith stable releases: e.g. release-3_1, etc) is being moved out of the url. This raises two points:

1. it is not currently possible to have multiple versions of the same package to be published using rerepo
   • alternative packaging tools (such as freight or aptly) most probably will solve this though
2. is it at all needed (and supported) to have multiple versions of the same package?
   • what is the compatibility matrix of Regolith? Can 3.2 be installed on Jammy or Focal? Can 3.1 (or 3.0) be installed on Noble? etc. If the answer to these questions are no, then that ultimately means there won't be a need to have multiple packages per distro/version/package triad

[khos2ow]

@kgilmer @SoumyaRanjanPatnaik FYI.

I opened the issue at Voulage repo first to have a discussion and if we get to a point that we would likely wanted to move forward then it'd be great to create another discussion thread in main repo or discussions page for broader community visibility and engagements.

[kgilmer]

@khos2ow your proposal is fantastic! Perhaps this falls into the "multiple versions" block you have above ~ I also was unable to get reprepro to host the same package across multiple architectures. But, as a consequence of having to create a seperate debian repository for every stage/dist/codename/arch touple, we sidestep collisions in parallel builds. So, I think the deb workflow will require some changes there as well such that various builds do not compete.

freight looks cool! aptly seems a bit more than we need, upon first glance. Do you have a preference? Do you see risks?

[kgilmer]

Also we will still need a way of encoding the regolith stage somewhere, as it's common for users to consume from testing. Perhaps as a prefix to the host, or the first element of the path?

[khos2ow]

freight looks cool! aptly seems a bit more than we need, upon first glance. Do you have a preference? Do you see risks?

I also agree that Aptly might be a bit overkill for our use case and Freight seems to be exactly what we want. And deb workflow definitely needs to change, I think less on the build side, and more on the publish side.

One big difference is that, as far as I could gather, freight needs to run on the web server where the packages are hosted, as opposed to current state of building, packaging, generating deb repo, rsync everything.

[khos2ow]

Also we will still need a way of encoding the regolith stage somewhere, as it's common for users to consume from testing. Perhaps as a prefix to the host, or the first element of the path?

The way I'm thinking about this is point 5 and 6 from the original proposal:

5. move Regolith Stage out and append it to distribution section as needed
6. Regolith Stage will not ever have the Regolith versions (e.g. release-3_1, etc)

That means differentiation between stages will be right there in the deb source file, BUT, release version slug will never be visible anywhere in the deb url and/or deb source file.

• Noble/Stable: deb [arch=amd64] https://archive.regolith-desktop.com/ubuntu noble main
• Noble/Unstable: deb [arch=amd64] https://archive.regolith-desktop.com/ubuntu noble-unstable main
• Noble/Testing: deb [arch=amd64] https://archive.regolith-desktop.com/ubuntu noble-testing main

And in all of the cases above, Regolith versions will just be packages (plural) inside the dist/ and pool/. At least this versioning part is theory and depends on 1) ability to use freight and such and more importantly 2) if multiple regolith version per distro release could be possible. (e.g. can we have Regolith 3.0, 3.1, and 3.2 on Noble? Or is it just 3.2 on Noble and nothing else is supported?)

[kgilmer]

deb [arch=amd64] https://archive.regolith-desktop.com/ubuntu noble-unstable main

👍

can we have Regolith 3.0, 3.1, and 3.2 on Noble? Or is it just 3.2 on Noble and nothing else is supported?

From the project start, the only target that had been supported but is no longer is Ubuntu 18.04. @SoumyaRanjanPatnaik and I recently discussed this topic and thought it best to only support the last + current LTS release, plus any current non-LTS release for Ubuntu. (I'm counting all official Debian releases as "LTS"). So, I plan to deprecate support for Bullseye and Focal in 3.2. I haven't done it yet because I want to know if something in 3.2 fails on those older platforms in beta before the release. But this is the policy, LMK what you think:

Regolith releases target the last + current LTS release, plus a latest non-LTS release for Ubuntu if exists.