Proxmox VE LXC Filesystem Trim

[tteck]

Proxmox VE LXC Filesystem Trim

With traditional hard disk drives (HDDs or spinner disks), there isn't a significant performance penalty for leaving empty blocks or unused space on the disk. This is because HDDs use spinning magnetic platters and mechanical read/write heads to access data. Therefore, the physical location of data on the disk doesn't impact performance significantly, and there's no need to actively manage unused space to maintain performance.

However, solid-state drives (SSDs) operate differently. SSDs use NAND flash memory to store data, and they have a limited number of write cycles per cell. As data is written, erased, and rewritten to the SSD, performance can degrade over time due to the way NAND flash memory works. Specifically, write operations are faster on empty blocks than on blocks that have been previously written and need to be erased before new data can be written.

fstrim is a command used with SSDs to inform the drive which blocks of data are no longer considered in use by the filesystem so that the drive can internally erase them and prepare them for new data. This helps maintain consistent performance on SSDs over time and prevents degradation due to inefficient write operations.

Similarly, thin-provisioned storage technologies like LVM-Thin, Ceph, and qcow2 allocate storage space dynamically as needed. Without proper management, unused blocks within these storage systems can continue to occupy space, causing storage usage to grow unnecessarily.

Therefore, it's essential to periodically trim SSDs and manage unused space in thin-provisioned storage systems to optimize performance and prevent unnecessary consumption of storage resources.

VMs are configured to run fstrim automatically on a weekly basis, for LXC containers, it's necessary to perform fstrim manually or set up a separate automated process.

So, I created a script to run fstrim on all LXC containers (that are not selected to be skipped). This is designed to work with SSDs on ext4 filesystems only.

So, lets test it

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/misc/fstrim.sh)"

[jcastro]

Ran it on my Prox 8 for docker, npm and adguard LXC. All went ok! thanks

[joyshmitz]

Tried it on my home lab ProxMox 8, 7 containers, worked great.

Thanks for the help.

[0xk1f0]

Trimmed all of my 8 LXC-Containers on my Proxmox 8 instance without issues.

Neat script, thanks!

[towerhand]

There were no errors on my end running Proxmox 8.1.4 with 24 LXC containers. Thanks for the script.

I think this is a good candidate for an unintended cron script since you mentioned VMs are configured to run fstrim automatically weekly.

[tteck]

For a test:

30 0 * * 0 pct list | awk "/^[0-9]/ {print \$1}" | while read ct; do template=$(pct config $ct | grep -q "template:" && echo "true" || echo "false"); if [ "$template" = "false" ]; then pct fstrim $ct; fi; done >> /var/log/fstrim-cron.log 2>/dev/null

[eikaramba]

i tested the first plain cron job and got this result:
/bin/sh: 1: pct: not found

the same for the second one:
/bin/sh: 1: pct: not found

[tteck]

Where are you creating the crontab? Proxmox host?

[eikaramba]

yes exactly - very weird. i haven't found any clue why i get this error in the crontab but the command executed in shell works flawessly it seems

[Xpl0itU]

The error happens because cron doesn't have a PATH set, so you have to include the full path of the pct binary, this crontab entry should work

30 0 * * 0 /usr/sbin/pct list | awk '/^[0-9]/ {print $1}' | while read ct; do /usr/sbin/pct fstrim ${ct}; done

[onethree7]

you could add that your script can take -f option (like 'force trim all' , if this is chosen whiplash gets bypassed).
then you can offer users a dialog in the normal first run of the script via whiplash "would you like to add this script to crontab" the script could write itself into crontab with the -f option then. (or place a manual method like in your previous comment, and dont bother -f either) :D I dont think the whiplash dialog is neccessary at all here generally. Who would want to skip an LXC

I think thats only natural and most will choose this workflow and cron-setup.

Thanks for the good work.

[tteck]

🤣 It's called "whiptail"

[onethree7]

sh1t really :D Sorry i only used dialog on bsd before

whiplash sounds cooler though

[Mellowlynx]

Thanks, Ran without problems here.

[brusarp]

Very helpful! Thanks... However, I think it is better to correct some terms... In the first window of the script it says "The selected containers will be subjected to..." In the second one it is asked to "select those to exclude..." A sort of linguistic short circuit and cause of possible misunderstandings in my opinion...

[tteck]

Corrected, first window now, "The LXC containers will undergo the fstrim command. Proceed?".

[tteck]

The script now displays the data percentage used before and after trimming.

[Calimerorulez]

On some of my LXCs, percentage before and after is shown as empty. Only a percentage sign is displayed.

[Calimerorulez]

My 'problematic' LXCs do run of a SSD too, not from a HDD, fwiw.

[Calimerorulez]

The ones that don't show the percentages are running on the same Samsung SSD. Maybe that's why.

[tteck]

I bet the Samsung SSD is set as type Directory instead of LVM-Thin.

[Calimerorulez]

You are right! :)

[tteck]

If the contents of that Samsung SSD aren't critical to you, consider repurposing it as a thinpool for efficient thin provisioning. This can be easily achieved through the GUI.

[divemasterjm]

for me it worked

…

On Mon, Feb 26, 2024 at 9:24 AM tteckster ***@***.***> wrote: Let's start with a plain cron job, nothing fancy. 30 0 * * 0 pct list | awk '/^[0-9]/ {print $1}' | while read ct; do pct fstrim ${ct}; done — Reply to this email directly, view it on GitHub <#2505 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALECS4U6CMHS3LC7EX25HGTYVRBCFAVCNFSM6AAAAABDL3ZX7CVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DKOBZGI4TQ> . You are receiving this because you commented.Message ID: ***@***.***>

[divemasterjm]

this one doenst work for me. Cron ***@***.***> pct list | awk "/^[0-9]/ {print \$1}" | while read ct; do template=$(pct config $ct | grep -q "template:" && echo "true" || echo "false"); if [ "$template" = "false" ]; then pct fstrim $ct; fi; done >> /var/log/fstrim-cron.log 2>/dev/null /bin/sh: 1: pct: not found

…

On Mon, Feb 26, 2024 at 9:46 AM tteckster ***@***.***> wrote: For a test: 30 0 * * 0 pct list | awk "/^[0-9]/ {print \$1}" | while read ct; do template=$(pct config $ct | grep -q "template:" && echo "true" || echo "false"); if [ "$template" = "false" ]; then pct fstrim $ct; fi; done >> /var/log/fstrim-cron.log 2>/dev/null — Reply to this email directly, view it on GitHub <#2505 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALECS4SRTO6FOXBQBEIWHS3YVRDYFAVCNFSM6AAAAABDL3ZX7CVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DKOBZGUYTQ> . You are receiving this because you commented.Message ID: ***@***.***>

[AJolly]

So what happens if you're running ZFS?

[nirav-mj]

Sorry I'm new to this, but am I supposed to run the script in LXC container which is running Ubuntu or do I need run this on the Proxmox host? Thanks!

[tteck]

Run in the Proxmox shell

[nirav-mj]

Thank you!! so the script will auto detect my LVM-THIN SSD disk and start the trim process?

[nirav-mj]

I just ran it, and it seem to work. Is there way to see the details how much was trimmed? the screen went by quickly

[tteck]

Is there way to see the details

It doesn't store logs.

the screen went by quickly

Not so quick that you couldn't read the before/after stats.

[nirav-mj]

lol, I was using Guacamole SSH connection and it was quick, anyway I ran it again using putty and I could see it. Thank you very much for all your time and effort you deserve a coffee.

[aroundmyroom]

could the bytes saved in human readable form like 10GB / 100MB? one LXC went from 99,5% to 9.3% ... ;)

[tteck]

[bazb]

Please add a cron version to the .io page

[nayzm]

+1, would be super useful.

[NeoPlayer13]

Or a way to add the containers to skip as a shell parameter.

[AT-StephenDetomasi]

Is this script actually needed on ZFS? As long as you trim the host disks, running additional trims for LXCs should not be necessary as ZFS is a block-level filesystem. Happy to be corrected though.

[tteck]

Read the instructions.

[pylchung]

When I run the script I get a message that root file system is not ext4. My boot SSD is ZFS, but my SSD for VMs/LXC are LVM-Thin. Can I not use this script?

[tteck]

You cannot, sorry.

[pylchung]

Thanks for confirming. I managed to run trim on each LXC manually using pct fstrim <ctid>.