Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Perform Backend's resultdir cleanup #3495

Closed
praiskup opened this issue Nov 1, 2024 · 6 comments
Closed

Perform Backend's resultdir cleanup #3495

praiskup opened this issue Nov 1, 2024 · 6 comments
Assignees
@praiskup
Labels
fedora-copr-admin Tasks that need to be done by Fedora Copr administrator

Comments

@praiskup
Copy link
Member

praiskup commented Nov 1, 2024

There's the /usr/bin/copr-backend-resultdir-cleaner utility that we probably
never executed. Except that it removes chroot_scan sub-directories, it also
log-out some directories that we could remove ("manually", after reviewing the
list of occurences).

This ticket is the "administrator" task that needs to be done (babysit the script, etc.).
@praiskup praiskup added the fedora-copr-admin Tasks that need to be done by Fedora Copr administrator label Nov 1, 2024
@github-project-automation github-project-automation bot moved this to Needs triage in CPT Kanban Nov 1, 2024
praiskup added a commit to praiskup/copr that referenced this issue Nov 1, 2024
@praiskup
backend: manpage for copr-backend-resultdir-cleaner && logger fix
675d740 
Build the manual page from an updated --help output.  The logger needs
to be initialized in `_main`, otherwise the log-file can mistakenly be
created under the `root` user before the `script_requires_user()` check
is done.

Relates: fedora-copr#3495
praiskup added a commit to praiskup/copr that referenced this issue Nov 1, 2024
@praiskup
backend: copr-backend-resultdir-cleaner to skip module dirs
71231c9 
Relates: fedora-copr#3495
praiskup added a commit to praiskup/copr that referenced this issue Nov 5, 2024
@praiskup
backend: manpage for copr-backend-resultdir-cleaner && logger fix
31c52b4 
Build the manual page from an updated --help output.  The logger needs
to be initialized in `_main`, otherwise the log-file can mistakenly be
created under the `root` user before the `script_requires_user()` check
is done.

Relates: fedora-copr#3495
praiskup added a commit to praiskup/copr that referenced this issue Nov 5, 2024
@praiskup
backend: copr-backend-resultdir-cleaner to skip module dirs
4239100 
Relates: fedora-copr#3495
praiskup added a commit that referenced this issue Nov 6, 2024
@praiskup
backend: manpage for copr-backend-resultdir-cleaner && logger fix
4cb6af2 
Build the manual page from an updated --help output.  The logger needs
to be initialized in `_main`, otherwise the log-file can mistakenly be
created under the `root` user before the `script_requires_user()` check
is done.

Relates: #3495
praiskup added a commit that referenced this issue Nov 6, 2024
@praiskup
backend: copr-backend-resultdir-cleaner to skip module dirs
fa9ac64 
Relates: #3495
@praiskup praiskup moved this from Needs triage to In 3 months in CPT Kanban Nov 6, 2024
@nikromen nikromen moved this from In 3 months to In Progress in CPT Kanban Nov 11, 2024
@praiskup
Copy link
Member Author

Before I start the cleanup, number of inodes is ~59M

/dev/mapper/copr--backend--data-copr--backend--data 664791040 59639988 605151052    9% /var/lib/copr/public_html

Script started ~ Fri Nov 22 01:15:46 PM UTC 2024

@praiskup
Copy link
Member Author

Still running, but the effect is already visible:

[copr@copr-be ~][PROD]$ date && df -i /var/lib/copr/public_html
Mon Nov 25 04:26:23 AM UTC 2024
Filesystem                                             Inodes    IUsed     IFree IUse% Mounted on
/dev/mapper/copr--backend--data-copr--backend--data 664791040 44762982 620028058    7% /var/lib/copr/public_html

@praiskup
Copy link
Member Author 

TODO DIR APPSTREAMFAIL: /var/lib/copr/public_html/results/ellana/spotify-qt/fedora-rawhide-x86_64/appdata
WARNING:__main__:TODO DIR APPSTREAMFAIL: /var/lib/copr/public_html/results/ellana/spotify-qt/fedora-rawhide-x86_64/appdata

real    4281m50.238s
user    23m47.398s
sys     56m24.556s

$ df -i /var/lib/copr/public_html
Filesystem
/dev/mapper/copr--backend--data-copr--backend--data 664791040 43585645 621205395    7% /var/lib/copr/public_html

@praiskup
Copy link
Member Author

59639988 -> 43585645 => only ~27% of inodes down, I thought it would be more close to ~50% :-( but still good.

@praiskup
Copy link
Member Author

I backed up the (two) logs:
https://copr-be.cloud.fedoraproject.org/archive/issues/copr-3495/

We still can process the TODO files like APPSTREAMFAIL, etc.

praiskup added a commit to praiskup/copr that referenced this issue Nov 28, 2024
@praiskup
resultdir-cleaner: remove old appstream failures
0b9a5d1 
These are no longer that frequent as we disabled appstream-builder by
default long time ago.

Relates: fedora-copr#3495
@praiskup praiskup mentioned this issue Nov 28, 2024
Merged
praiskup added a commit to praiskup/copr that referenced this issue Nov 28, 2024
@praiskup
backend: resultdir-cleaner: remove old appstream failures
cf2b239 
These are no longer that frequent as we disabled appstream-builder by
default long time ago.

Relates: fedora-copr#3495
praiskup added a commit to praiskup/copr that referenced this issue Nov 28, 2024
@praiskup
backend: resultdir-cleaner: remove old appstream failures
b328003 
These are no longer that frequent as we disabled appstream-builder by
default long time ago.

Relates: fedora-copr#3495
nikromen pushed a commit that referenced this issue Dec 2, 2024
@praiskup @nikromen
backend: resultdir-cleaner: remove old appstream failures
e6fdb07 
These are no longer that frequent as we disabled appstream-builder by
default long time ago.

Relates: #3495
@praiskup
Copy link
Member Author

praiskup commented Dec 6, 2024

Run, 43.6M => 43.2M. Done.

@praiskup praiskup closed this as completed Dec 6, 2024
@nikromen nikromen moved this from In Progress to Done in CPT Kanban Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@praiskup praiskup

Labels
fedora-copr-admin Tasks that need to be done by Fedora Copr administrator
Projects
CPT Kanban
Archived in project
Milestone
No milestone
Development

No branches or pull requests
1 participant
@praiskup