auto prune fails for simple operations with fallocate EINVAL

[dustymabe]

Auto pruning (#2847, #2866) will fail for simple operations (like rpm-ostree install htop && OSTREE_SYSROOT_OPTS=early-prune /usr/bin/ostree admin finalize-staged) with an fallocate error:

[root@cosa-devsh ~]# rpm-ostree install htop
Checking out tree d7dbc6b... done
Enabled rpm-md repositories: fedora-cisco-openh264 fedora-modular updates-modular updates fedora updates-archive
Importing rpm-md... done
rpm-md repo 'fedora-cisco-openh264' (cached); generated: 2023-03-14T10:56:46Z solvables: 4
rpm-md repo 'fedora-modular' (cached); generated: 2023-04-13T20:30:47Z solvables: 1082
rpm-md repo 'updates-modular' (cached); generated: 2018-02-20T19:18:14Z solvables: 0
rpm-md repo 'updates' (cached); generated: 2023-05-31T17:25:42Z solvables: 13915
rpm-md repo 'fedora' (cached); generated: 2023-04-13T20:37:10Z solvables: 69222
rpm-md repo 'updates-archive' (cached); generated: 2023-05-31T18:06:06Z solvables: 14547
Resolving dependencies... done
Checking out packages... done
Running pre scripts... done
Running post scripts... done
Running posttrans scripts... done
Writing rpmdb... done
Writing OSTree commit... done
Staging deployment... done
Freed: 24.6 MB (pkgcache branches: 0)
Added:
  htop-3.2.2-2.fc38.x86_64
Changes queued for next boot. Run "systemctl reboot" to start a reboot
[root@cosa-devsh ~]# OSTREE_SYSROOT_OPTS=early-prune /usr/bin/ostree admin finalize-staged
Copying /etc changes: 6 modified, 1 removed, 35 added
error: Checking if bootfs has space: fallocate: Invalid argument
[root@cosa-devsh ~]# rpm -q ostree
ostree-2023.3-2.fc38.x86_64

If I add some diagnostic code I can see the length passed to fallocate is 0, which triggers an EINVAL.

diff --git a/src/libostree/ostree-sysroot-deploy.c b/src/libostree/ostree-sysroot-deploy.c
index 425abe8b..1b778daa 100644
--- a/src/libostree/ostree-sysroot-deploy.c
+++ b/src/libostree/ostree-sysroot-deploy.c
@@ -2576,6 +2576,7 @@ auto_early_prune_old_deployments (OstreeSysroot *self, GPtrArray *new_deployment
       /* it wasn't in current_bootcsums; add */
       net_new_bootcsum_dirs_total_size += bootdir_size;
     }
+  g_printerr ("net_new_bootcsum_dirs_total_size is %lu\n", net_new_bootcsum_dirs_total_size);
 
   {
     gboolean bootfs_has_space = FALSE;

[root@cosa-devsh ~]# OSTREE_SYSROOT_OPTS=early-prune /usr/bin/ostree admin finalize-staged
Copying /etc changes: 6 modified, 1 removed, 35 added
net_new_bootcsum_dirs_total_size is 0
error: Checking if bootfs has space: fallocate: Invalid argument

[dustymabe]

A simple solution to this problem is to just skip the check if net_new_bootcsum_dirs_total_size is 0. I can try to open a PR for that, but I'm thinking maybe the fix is at a higher level?

[cgwalters]

A simple solution to this problem is to just skip the check if net_new_bootcsum_dirs_total_size is 0. I can try to open a PR for that, but I'm thinking maybe the fix is at a higher level?

Without digging into this at all, I would say it makes total sense to add that check. Even if there's a higher level fix, having that sanity check is low cost and helps keep things robust.

(OK I spent a minute looking at the code, and I think the only real fix is that check)

Likely the bug here is when there are no kernel changes at all, and that isn't covered in the CI test for this flag right now.

[dustymabe]

Potential fix in #2871