Skip to content

Commit

Permalink
doc: document DistGit cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
praiskup committed Apr 9, 2024
1 parent 83a1da8 commit d80b76a
Show file tree
Hide file tree
Showing 3 changed files with 34 additions and 0 deletions.
2 changes: 2 additions & 0 deletions doc/developer_documentation/architecture.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. _architecture:

Basic Copr build process / Architecture diagram
-----------------------------------------------

Expand Down
31 changes: 31 additions & 0 deletions doc/maintenance/dist-git.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
.. _maintaining_distgit:

DistGit VM maintenance
======================

The Copr DistGit machine serves as a "proxy" middle-man between the actual source
code location and our farm of builders. See :ref:`architecture` for more
info. There's no data-consistency promise regarding Copr's DistGit data;
our primary goal is to ensure effective Copr Builds. Consequently, we
only strive to keep the "proxy" sources for the **existing packages**
within still **existing Copr projects**.

Cleaning up DistGit
-------------------

Copr is a place where CI is done frequently, leading to the generation of
a significant amount of "source" data. Often, each CI build produces a
different tarball that is uploaded and stored on the Copr DistGit machine.
Copr projects are also often short-lived (e.g., Packit). This
necessitates Copr to conduct regular DistGit cleanup to prevent the source
data consumption from quickly exploding.

Currently, we have two house-cleaning procedures for DistGit:

1. `Cleaning up sources for outdated builds <https://github.com/fedora-copr/copr/blob/main/dist-git/run/dist-git-clear-tarballs.py>`_

2. `Cleaning up old repositories and sources for deleted Copr projects <https://github.com/fedora-copr/copr/blob/main/dist-git/run/prune-dist-git.py>`_

Both of these procedures are not enabled "by default" on every Copr
deployment. However, Fedora Copr has automated these scripts using cron.

1 change: 1 addition & 0 deletions doc/maintenance_documentation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,4 @@ This section contains information about maintenance topics. You may also be inte
email_templates
Sending notifications and removing data from outdated chroots <how_to_delete_outdated_chroots>
Monitoring the service <maintenance/monitoring>
Maintaining DistGit machine <maintenance/dist-git>

0 comments on commit d80b76a

Please sign in to comment.