version 1.9.26

.github/workflows/ci-ubuntu-latest.yml

@@ -20,6 +20,7 @@ jobs:
         codespell virtnbdmap
         codespell virtnbd-nbdkit-plugin
         codespell README.md
+        codespell Changelog
     - name: Python code format test
       run: |
         sudo pip3 install black==22.3

Changelog

@@ -1,3 +1,12 @@
+Version 1.9.26
+---------
+ * virtnbdbackup: transient vm backup: validate the state of the checkpoint
+   during redefine: this can help to detect situations where the bitmap is
+   broken.
+ * virtnbdmap: Fix typo in log message.
+ * Update README regarding port usage during remote backups.
+ * Add notes regarding stability of the features, thanks @pipo
+
 Version 1.9.25
 ---------
  * Fix typo in log messages.
@@ -112,7 +121,7 @@ Version 1.9.6
 ---------
  * virtnbdrestore: add detection for disks with volume notation, reset
  type to file during restore so virtual machine config is adjusted to
- new restore pathes. (#81)
+ new restore paths. (#81)
 
 
 Version 1.9.5
@@ -152,10 +161,10 @@ Version 1.9.1
  * Fix disk detection with volume notation. (#81)
  * Backup and restore qcow image related settings: during backup
  qemu-img info is called on qcow based image devices and qcow specific options
- are stored as json dump. If information is existant, on restore the
+ are stored as json dump. If information is existent, on restore the
  original settings of the qcow image (such as cluster size or other options)
  are now applied during image creation. (#80)
- * Fix some race conditions during restore of multiple disks and seqences: NBD
+ * Fix some race conditions during restore of multiple disks and sequences: NBD
  connection handle was not closed, following qemu-nbd calls would fail due
  to locked files or already in use tcp ports.
 
@@ -285,7 +294,7 @@ Version 0.89
 ---------
 * virtnbdrestore: check if additional boot files such as nvram/kernel images
 exist and if not, restore them to the original path instead of just warning
-the user about manual steps beeing required.
+the user about manual steps being required.
 
 Version 0.88
 ---------
@@ -326,7 +335,7 @@ before assignment
 
 Version 0.82
 ---------
-* Move common arguments to seperate file.
+* Move common arguments to separate file.
 
 Version 0.81
 ---------
@@ -341,7 +350,7 @@ Version 0.80
 
 Version 0.79
 ---------
-* Add -U/--uri option: can be used to specifiy libvirt connection URI, if
+* Add -U/--uri option: can be used to specify libvirt connection URI, if
 authfile is specified, use openAuth to authenticate against libvirt daemon.
 * Add --user and --password options: can be used to authenticate against
 libvirt daemon.
@@ -367,7 +376,7 @@ Version 0.76
  specified mountpoints are freezed and thawed. (#60)
 * virtnbdrestore:
  + Option -c would only adjust virtual machine config for the first disk.
- + Option -c now correctly removes exluded disks from the adjusted virtual
+ + Option -c now correctly removes excluded disks from the adjusted virtual
  machine configuration.
 * Add --version option for all utilities.
 * Vagrant scripts: test installation/execution after creating rpm packages.
@@ -381,7 +390,7 @@ Version 0.75
 Version 0.74
 ---------
 * virtnbdbackup: add some more debug messages around freezing/thwaing
-filesytems and backup job operation.
+filesystems and backup job operation.
 
 Version 0.73
 ---------
@@ -413,11 +422,11 @@ Version 0.69
 * virtnbdbackup: exit early if removing checkpoints fails during full
 backup.
 * virtnbdrestore: do not write zeroes during restore of image, as the
-resulting zeroed regions are then reported as beeing "data": this would result
+resulting zeroed regions are then reported as being "data": this would result
 in further backups of the virtual machine to be thick provisioned, as even
 zeroed regions are saved. (#56)
 * virtnbdrestore: dont adjust vm config if option -c is missing.
-* virtnbdrestore: now removes existant backing stores if adjust setting is
+* virtnbdrestore: now removes existent backing stores if adjust setting is
 enabled: usually the case if virtual machine operated on snapshot during
 backup.
 
@@ -426,7 +435,7 @@ Version 0.68
 * Code cleanup at different places. Make the output helper wrapping.
 * Use Elementree from lxml for easier access via xpath, adjust dependencies
 * Adjust pylintrc
-* virtnbdrestore: add option -c: adjusts required pathes within restored
+* virtnbdrestore: add option -c: adjusts required paths within restored
 virtual machine config so it can be defined using virsh. (#54)
 * virtnbdrestore: add option -D: in combination with option -c can be used to
 register virtual machine on libvirt daemon after restore.
@@ -510,11 +519,11 @@ Version 0.57
 
 Version 0.56
 ---------
-* Fix for Issue (#40): redefining checkpoints failes
+* Fix for Issue (#40): redefining checkpoints fails
 
 Version 0.55
 ---------
-* Correct some wrongly catched exceptions
+* Correct some wrongly caught exceptions
 * Update README
 
 
@@ -559,10 +568,10 @@ passed by the --sequence option.
 Version 0.48
 ---------
 
-* Internal code changes: start using propper exceptions if stream format
+* Internal code changes: start using proper exceptions if stream format
 is not parseable
-* virtnbdbrestore: add --sequence option, allows to specify a comma seperated
-list of backup files whose informations should be dumped. Next step is to
+* virtnbdbrestore: add --sequence option, allows to specify a comma separated
+list of backup files whose information should be dumped. Next step is to
 implement restore of a specified sequence.
 
 
@@ -690,7 +699,7 @@ With the recent changes, virtnbdbackup now detects
 if the backup source is offline and then:
 
  * Forces the backup mode to "copy": no full or
- incremental backup supported because we cant create
+ incremental backup supported because we can't create
  an checkpoint via libvirt API. Copy type backup
  is like a full backup just without checkpoint.
  * Forces the amount of workers to 1
@@ -739,7 +748,7 @@ devices not possible within docker containers.)
 
 Version 0.32
 ---------
-Introduce new `virtnbdmap` utlity to allow simple single file and instant recovery, see:
+Introduce new `virtnbdmap` utility to allow simple single file and instant recovery, see:
 
  https://github.com/abbbi/virtnbdbackup#single-file-restore-and-instant-recovery
 
@@ -753,7 +762,7 @@ Version 0.30
 ---------
 Various bugfixes, added some small features in `virtnbdrestore`: it now
 attempts to use the original disks name for the disk files and copies the
-virtual machine config file. Fixed a few pylint warnings aswell.
+virtual machine config file. Fixed a few pylint warnings as well.
 
 Updated readme and improved testsuite.
 
@@ -844,8 +853,8 @@ Version 0.16
 Version 0.15
 ---------
 * Minor code changes, move some functions to common class
-* Use makedirs for targetfolder to support nested target pathes.
-* Updated README with more informations about backup workflow.
+* Use makedirs for targetfolder to support nested target paths.
+* Updated README with more information about backup workflow.
 
 Version 0.14
 ---------
@@ -863,7 +872,7 @@ it is assumed everything is OK.
 
 Now virtnbdbackup writes the data to a partial file first
 and renames the file as last step. During incremental
-backup the target directory is checked for possible existance
+backup the target directory is checked for possible existence
 of partial backups and backup is aborted with error.
 
 Version 0.11

README.md

@@ -96,18 +96,25 @@ this project: [qmpbackup](https://github.com/abbbi/qmpbackup)
 
 # Prerequisites
 
-Obviously you require a libvirt/qemu version that supports the incremental backup
-features. Following, you will find a short overview which distributions ship the
-required versions.
+Obviously you require a libvirt/qemu version that supports the incremental
+backup features. Since libvirt v7.6.0 and qemu-6.1 the required features are
+[enabled by default](https://libvirt.org/news.html#v7-6-0-2021-08-02) and are
+considered production ready: everything will work out of the box.
+
+Following, you will find a short overview which older libvirt
+versions may require further adjustments to the virtual machine config.
+
 
 ## Libvirt versions <= 7.6.0 (Debian Bullseye, Ubuntu 20.x)
 
 If you are using Debian Bullseye or Ubuntu 20.x, the included libvirt version
-already supports the features required, but does not enable them by default!
+already has an almost complete support for incremental backup, although it
+doesn't work properly with migration or some block jobs.
 
-To enable the incremental backup feature on these libvirt versions 
-change the virtual machine config using `virsh edit <vm>` like so: (the
-first line must be changed, too!):
+If you don't want to use migration or other blockjobs you can enable the 
+incremental backup feature on these libvirt versions change the virtual 
+machine config using `virsh edit <vm>` like so: (the first line must be 
+changed, too!):
 
  ```
   <domain type='kvm' id='1' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
@@ -121,7 +128,9 @@ first line must be changed, too!):
 
 `Note`:
 > You must restart the virtual machine after enabling the feature!
- 
+> Upstream libvirt strongly discourages enabling the feature on production
+> systems for these libvirt versions.
+
 ## RHEL/Centos Stream, Alma, Rocky Linux
 
 ### Version <= 8.5
@@ -483,6 +492,11 @@ virtnbdbackup -U qemu+ssh://root@hypervisor/system --ssh-user root -d src -o  /t
 
 See also: [Authentication](#authentication)
 
+`Note`:
+> If you want to run multiple remote backups at the same time you need to pass
+> an unique port for the NBD service used for data transfer via --nbd-port
+> option for each backup session.
+
 ### NBD with TLS
 
 By default disk data received from a remote system will be transferred via

libvirtnbdbackup/__init__.py

@@ -15,4 +15,4 @@
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 """
 
-__version__ = "1.9.25"
+__version__ = "1.9.26"

libvirtnbdbackup/virt/checkpoint.py

@@ -224,7 +224,10 @@ def redefine(domObj: libvirt.virDomain, args: Namespace) -> bool:
         try:
             domObj.checkpointCreateXML(
                 checkpointConfig.decode(),
-                libvirt.VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE,
+                [
+                    libvirt.VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE,
+                    libvirt.VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE_VALIDATE,
+                ],
             )
         except libvirt.libvirtError as e:
             log.error("Redefining checkpoint failed: [%s]: %s", checkpointName, e)

man/virtnbdbackup.1

@@ -1,5 +1,5 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.48.1.
-.TH VIRTNBDBACKUP "1" "June 2023" "virtnbdbackup 1.9.25" "User Commands"
+.TH VIRTNBDBACKUP "1" "June 2023" "virtnbdbackup 1.9.26" "User Commands"
 .SH NAME
 virtnbdbackup \- backup utility for libvirt
 .SH DESCRIPTION
@@ -48,7 +48,7 @@ Backup only disk with target dev name (\fB\-i\fR vda)
 Exclude disk(s) with target dev name (\fB\-x\fR vda,vdb)
 .TP
 \fB\-f\fR SOCKETFILE, \fB\-\-socketfile\fR SOCKETFILE
-Use specified file for NBD Server socket (default: \fI\,/var/tmp/virtnbdbackup.4066\/\fP)
+Use specified file for NBD Server socket (default: \fI\,/var/tmp/virtnbdbackup.1145544\/\fP)
 .TP
 \fB\-n\fR, \fB\-\-noprogress\fR
 Disable progress bar

man/virtnbdmap.1

@@ -1,5 +1,5 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.48.1.
-.TH VIRTNBDMAP "1" "June 2023" "virtnbdmap 1.9.25" "User Commands"
+.TH VIRTNBDMAP "1" "June 2023" "virtnbdmap 1.9.26" "User Commands"
 .SH NAME
 virtnbdmap \- map virtnbdbackup image files to nbd devices
 .SH DESCRIPTION

man/virtnbdrestore.1

@@ -1,5 +1,5 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.48.1.
-.TH VIRTNBDRESTORE "1" "June 2023" "virtnbdrestore 1.9.25" "User Commands"
+.TH VIRTNBDRESTORE "1" "June 2023" "virtnbdrestore 1.9.26" "User Commands"
 .SH NAME
 virtnbdrestore \- restore utility for libvirt
 .SH DESCRIPTION
@@ -40,7 +40,7 @@ Process only disk matching target dev name. (default: None)
 Disable progress bar
 .TP
 \fB\-f\fR SOCKETFILE, \fB\-\-socketfile\fR SOCKETFILE
-Use specified file for NBD Server socket (default: \fI\,/var/tmp/virtnbdbackup.4071\/\fP)
+Use specified file for NBD Server socket (default: \fI\,/var/tmp/virtnbdbackup.1145549\/\fP)
 .TP
 \fB\-r\fR, \fB\-\-raw\fR
 Copy raw images as is during restore. (default: False)

virtnbdmap

@@ -53,7 +53,7 @@ def checkDevice(args: Namespace, device: str) -> None:
     """Check if /dev/nbdX exists, otherwise it is likely
     nbd module isn't loaded on the system"""
     if not device.startswith("/dev/nbd"):
-        logging.error("Target device [%s] seems not to be a ndb device?", device)
+        logging.error("Target device [%s] seems not to be an NBD device?", device)
 
     if not lib.exists(args, device):
         logging.error(