Update July 2021: I have seen errors with external snapshots of volumes on versions of QEMU/KVM/libvirt from Ubuntu 20 Focal. Adding note on using internal snapshot on volume backed by qcow2.
Internal snapshots created on QEMU copy-on-write (qcow2) disks are the most commonly used snapshot when using libvirt. It is easy to see why; they are the default type exposed by ‘virsh snapshot-create-as‘, the snapshot is embedded as an overlay in the existing disk file so no need to edit the domain xml, reversion is fully supported with ‘virsh snapshot-revert‘, and it has support in the virt-manager GUI.
# stop host so memory state is not preserved virsh destroy mydomain # take snapshot internal to qcow2 file virsh snapshot-create-as mydomain mysnapshot # revert snapshot virsh snapshot-revert mydomain mysnapshot
In contrast, external snapshots do not have libvirt support for snapshot-revert so it must be done manually, the virt-manager GUI does not expose them, and a new qcow2 file is created (with backing file pointing at original disk file) which means the domain xml must be edited with this new file path.
However, in return, external snapshots have better support for no-downtime state capture, more control over snapshots of disk/memory state, and also support guest domain with disk types such as ‘volume’ by explicitly listing the disk specification.
External snapshot for simple guest domains
Creating external snapshot
If your guest domain is a simple VM where all the <disk> are of type=”file”, then creating an external snapshot of a powered-on VM’s state (excluding the RAM) looks like below. For this example, assume there is a hard drive at ‘hda’ and CD-ROM at ‘hdb’.
# name of domain, snapshot, and target disk device thedomain="mysimpledomain" snapshotname="simple-test" targetdisk="hda" # look at '<disk>' types, should be just 'file' types virsh dumpxml $thedomain | grep '<disk' -A5 # show block level devices and qcow2 paths (hda,hdb,..etc) virsh domblklist $thedomain # create snapshot in default pool location # file name is $thedomain.$snapshotname virsh snapshot-create-as $thedomain --name $snapshotname --disk-only # list snapshot virsh snapshot-list $thedomain
Reverting external snapshot
To see the mechanism underneath and prepare variables for the revert of the snapshot, execute the commands below.
# notice path to hda has now changed to snapshot file virsh domblklist $thedomain # <source> has changed to snapshot file virsh dumpxml $thedomain | grep '<disk' -A5 # pull default pool path from xml pooldir=$(virsh pool-dumpxml default | grep -Po "(?<=path\>)[^<]+") echo "default pool dir: $pooldir" # should see two files starting with $thedomain # the one named $thedomain.$snapshotname is the snapshot cd $pooldir ls -latr $thedomain* # snapshot points to backing file, which is original disk sudo qemu-img info $thedomain.$snapshotname -U --backing-chain # capture original backing file name so we can revert backingfile=$(qemu-img info $thedomain.$snapshotname -U | grep -Po 'backing file:\s\K(.*)') echo "backing file: $backingfile"
To do the revert we need to modify the domain xml back to the original qcow2 file, delete the snapshot metadata, and finally the snapshot file.
# stop VM virsh destroy $thedomain # edit hda path back to original qcow2 disk virt-xml $thedomain --edit target=$targetdisk --disk path=$backingfile --update # validate that we are now pointing back at original qcow2 disk virsh domblklist $thedomain # delete snapshot metadata virsh snapshot-delete --metadata $thedomain $snapshotname # delete snapshot qcow2 file sudo rm $pooldir/$thedomain.$snapshotname # start guest domain virsh start $thedomain
The guest domain should now be in the original state.
External snapshot for guest domains that need disk specification
If you need more control over the external snapshot created OR your guest domain is not a simple VM supported by the default syntax in the section above, then you will need explicitly set the ‘–diskspec’ flag to create the external snapshots for your disk devices.
For example, if your guest domain disk is of type “volume” instead of the standard “file”, then listing the blk devices will output something like below.
# working with guest domain that has <disk type='volume'> # the minimal snapshot-create-as syntax will not work $ virsh domblklist myadvdomain Target Source ------------------------------------------------ vda xenial1.qcow2 hdd /data/kvm/pool/xenial1-cloudinit.iso
And you will see an error like below when trying to create an external snapshot using the minimal syntax.
# error when trying to take external snapshot of domain where disk type is not 'file' $ virsh snapshot-create-as myadvdomain --name adv-test --disk-only error: unsupported configuration: cannot generate external snapshot name for disk 'vda' without source
Creating external snapshot, specify –diskspec
This means that libvirt cannot determine how the snapshot should be done, and we need to explicitly define the snapshot for each disk device with the ‘–diskspec’ flag. Here is how you would create the external snapshot using the guest domain above where there is a volume disk device ‘vda’ and CD-ROM ‘hdd’.
thedomain="myadvdomain" snapshotname="adv-test" targetdisk="vda" # pull default pool path from xml pooldir=$(virsh pool-dumpxml default | grep -Po "(?<=path\>)[^<]+") echo "default pool dir: $pooldir" # look at '<disk>' types, can have type other than file virsh dumpxml $thedomain | grep '<disk' -A5 # show block level devices virsh domblklist $thedomain # create snapshot in default pool location # if this fails, read section below # about internal snapshot for qcow2 volumes virsh snapshot-create-as $thedomain --name $snapshotname --disk-only --diskspec vda,file=$pooldir/$thedomain.$snapshotname --diskspec hdd,snapshot=no # list snapshot virsh snapshot-list $thedomain
Reverting external snapshot taken with –diskspec
The reversion for an external snapshot taken with diskspec follows the exact same instructions as the section above, “Reverting external snapshot”.
Internal snapshots for qcow2 volumes
Although the section above on external snapshots for volumes was working on Ubuntu 14 and 16, I’ve noticed that on Ubuntu 20 it no longer works for volumes backed by qcow2 files (QEMU=4.2,virsh=6.0.0). I now get the error below when trying to create the external snapshot.
# if this fails, use an internal snapshot $ virsh snapshot-create-as $thedomain --name $snapshotname --disk-only --diskspec vda,snapshot=external,file=$pooldir/$thedomain.$snapshotname --diskspec hdd,snapshot=no error: internal error: Child process (/usr/bin/qemu-img create -f qcow2 -o backing_fmt=qcow2,backing_file= /data/kvm/pool/microk8s-1-192.168.122.210.adv-test) unexpected fatal signal 6: qemu-img: /build/qemu-rbeYHu/qemu-4.2/block.c:5845: bdrv_img_create: Assertion `full_backing' failed.
However, if the ‘vda’ disk is qcow2 backed (as our is), you can create an internal snapshot as long as you use ‘–diskspec’ to avoid a snapshot on the other ‘hdd’ that is an ISO.
# specify that vda is an internal snapshot, and skip hdd $ virsh snapshot-create-as $thedomain --name $snapshotname --disk-only --diskspec vda,snapshot=internal --diskspec hdd,snapshot=no # show snapshots $ virsh snapshot-list $thedomain # remove snapshot $ virsh snapshot-delete $thedomain $snapshotname # snapshot now gone, reverted back to original disk state $ virsh snapshot-list $thedomain
REFERENCES
libvirt.org, explanation of libvirt not supporting external snapshots – which is required since snapshots cannot be taken of volumes (only disk)
kashyapc, snapshot discussion from LinuxCon Eu-2012 and slides
virt.fedoraproject, discussion of deleting external snapshots and modifying xml file path
libvirt.org, reference for snapshot-create and commands
systutorials.com, virt-xml manual page
access.redhat.com, internal versus external snapshots
mathiashueber.com, installing latest qemu/libvirt by compilation or ppa
launchpad.net jacobzimmerman – libvirt/qemu
libvirt.org, discussion of state capture of VMs
libvirt.org, live merge disk chain
libvirt.org, qemu backup internals
qemu.readthedocs.io, iSCSI LUN for QEMU
bugzilla.redhat.com, performance isues with internal snapshots
NOTES
Standard snapshot errors to watch for
Below are errors you will face if you try to perform standard snapshot operations on guest domains that do not support it, or try to manipulate external snapshot using libvirt (which it does not support).
# error when trying to take snapshot of domain where disk type is not 'file' error: internal error: Child process (/usr/bin/qemu-img snapshot -c test1) unexpected exit status 1: qemu-img: Expecting one image file name # error when trying to delete external snapshot with libvirt error: unsupported configuration: deletion of 1 external disk snapshots not supported yet
Determining version of QEMU, KVM, virsh
$ sudo apt show qemu-system-x86 # below from Ubuntu 20 Focal $ kvm --version QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.17) Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers $ qemu-system-x86_64 --version QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.17) $ virsh --version 6.0.0