KVM: creating and reverting libvirt external snapshots

Update July 2021: I have seen errors with external snapshots of volumes on versions of QEMU/KVM/libvirt from Ubuntu 20 Focal.  Adding note on using internal snapshot on volume backed by qcow2.  

Internal snapshots created on QEMU copy-on-write (qcow2) disks are the most commonly used snapshot when using libvirt.  It is easy to see why; they are the default type exposed by ‘virsh snapshot-create-as‘, the snapshot is embedded as an overlay in the existing disk file so no need to edit the domain xml, reversion is fully supported with ‘virsh snapshot-revert‘, and it has support in the virt-manager GUI.

# stop host so memory state is not preserved
virsh destroy mydomain

# take snapshot internal to qcow2 file
virsh snapshot-create-as mydomain mysnapshot

# revert snapshot
virsh snapshot-revert mydomain mysnapshot

In contrast, external snapshots do not have libvirt support for snapshot-revert so it must be done manually, the virt-manager GUI does not expose them, and a new qcow2 file is created (with backing file pointing at original disk file) which means the domain xml must be edited with this new file path.

However, in return, external snapshots have better support for no-downtime state capture, more control over snapshots of disk/memory state, and also support guest domain with disk types such as ‘volume’ by explicitly listing the disk specification.

External snapshot for simple guest domains

Creating external snapshot

If your guest domain is a simple VM where all the <disk> are of  type=”file”, then creating an external snapshot of a powered-on VM’s state (excluding the RAM) looks like below.  For this example, assume there is a hard drive at ‘hda’ and CD-ROM at ‘hdb’.

# name of domain, snapshot, and target disk device
thedomain="mysimpledomain"
snapshotname="simple-test"
targetdisk="hda"

# look at '<disk>' types, should be just 'file' types
virsh dumpxml $thedomain | grep '<disk' -A5

# show block level devices and qcow2 paths (hda,hdb,..etc)
virsh domblklist $thedomain

# create snapshot in default pool location
# file name is $thedomain.$snapshotname
virsh snapshot-create-as $thedomain --name $snapshotname --disk-only

# list snapshot
virsh snapshot-list $thedomain

Reverting external snapshot

To see the mechanism underneath and prepare variables for the revert of the snapshot, execute the commands below.

# notice path to hda has now changed to snapshot file
virsh domblklist $thedomain

# <source> has changed to snapshot file
virsh dumpxml $thedomain | grep '<disk' -A5

# pull default pool path from xml 
pooldir=$(virsh pool-dumpxml default | grep -Po "(?<=path\>)[^<]+")
echo "default pool dir: $pooldir"

# should see two files starting with $thedomain
# the one named $thedomain.$snapshotname is the snapshot
cd $pooldir
ls -latr $thedomain*

# snapshot points to backing file, which is original disk
sudo qemu-img info $thedomain.$snapshotname -U --backing-chain

# capture original backing file name so we can revert
backingfile=$(qemu-img info $thedomain.$snapshotname -U | grep -Po 'backing file:\s\K(.*)')
echo "backing file: $backingfile"

To do the revert we need to modify the domain xml back to the original qcow2 file, delete the snapshot metadata, and finally the snapshot file.

# stop VM
virsh destroy $thedomain

# edit hda path back to original qcow2 disk
virt-xml $thedomain --edit target=$targetdisk --disk path=$backingfile --update

# validate that we are now pointing back at original qcow2 disk
virsh domblklist $thedomain

# delete snapshot metadata
virsh snapshot-delete --metadata $thedomain $snapshotname

# delete snapshot qcow2 file
sudo rm $pooldir/$thedomain.$snapshotname

# start guest domain
virsh start $thedomain

The guest domain should now be in the original state.

External snapshot for guest domains that need disk specification

If you need more control over the external snapshot created OR your guest domain is not a simple VM supported by the default syntax in the section above, then you will need explicitly set the ‘–diskspec’ flag to create the external snapshots for your disk devices.

For example, if your guest domain disk is of type “volume” instead of the standard “file”, then listing the blk devices will output something like below.

# working with guest domain that has <disk type='volume'>
# the minimal snapshot-create-as syntax will not work
$ virsh domblklist myadvdomain
Target Source
------------------------------------------------
vda xenial1.qcow2
hdd /data/kvm/pool/xenial1-cloudinit.iso

And you will see an error like below when trying to create an external snapshot using the minimal syntax.

# error when trying to take external snapshot of domain where disk type is not 'file' 
$ virsh snapshot-create-as myadvdomain --name adv-test --disk-only
error: unsupported configuration: cannot generate external snapshot name for disk 'vda' without source

Creating external snapshot, specify –diskspec

This means that libvirt cannot determine how the snapshot should be done, and we need to explicitly define the snapshot for each disk device with the ‘–diskspec’ flag.  Here is how you would create the external snapshot using the guest domain above where there is a volume disk device ‘vda’ and  CD-ROM ‘hdd’.

thedomain="myadvdomain"
snapshotname="adv-test"
targetdisk="vda"

# pull default pool path from xml
pooldir=$(virsh pool-dumpxml default | grep -Po "(?<=path\>)[^<]+") echo "default pool dir: $pooldir"

# look at '<disk>' types, can have type other than file
virsh dumpxml $thedomain | grep '<disk' -A5

# show block level devices
virsh domblklist $thedomain

# create snapshot in default pool location
# if this fails, read section below
# about internal snapshot for qcow2 volumes
virsh snapshot-create-as $thedomain --name $snapshotname --disk-only --diskspec vda,file=$pooldir/$thedomain.$snapshotname --diskspec hdd,snapshot=no

# list snapshot
virsh snapshot-list $thedomain

Reverting external snapshot taken with –diskspec

The reversion for an external snapshot taken with diskspec follows the exact same instructions as the section above, “Reverting external snapshot”.

Internal snapshots for qcow2 volumes

Although the section above on external snapshots for volumes was working on Ubuntu 14 and 16, I’ve noticed that on Ubuntu 20 it no longer works for volumes backed by qcow2 files (QEMU=4.2,virsh=6.0.0).  I now get the error below when trying to create the external snapshot.

# if this fails, use an internal snapshot
$ virsh snapshot-create-as $thedomain --name $snapshotname --disk-only --diskspec vda,snapshot=external,file=$pooldir/$thedomain.$snapshotname --diskspec hdd,snapshot=no

error: internal error: Child process (/usr/bin/qemu-img create -f qcow2 -o backing_fmt=qcow2,backing_file= /data/kvm/pool/microk8s-1-192.168.122.210.adv-test) unexpected fatal signal 6: qemu-img: /build/qemu-rbeYHu/qemu-4.2/block.c:5845: bdrv_img_create: Assertion `full_backing' failed.

However, if the ‘vda’ disk is qcow2 backed (as our is), you can create an internal snapshot as long as you use ‘–diskspec’ to avoid a snapshot on the other ‘hdd’ that is an ISO.

# specify that vda is an internal snapshot, and skip hdd
$ virsh snapshot-create-as $thedomain --name $snapshotname --disk-only --diskspec vda,snapshot=internal --diskspec hdd,snapshot=no

# show snapshots
$ virsh snapshot-list $thedomain

# remove snapshot
$ virsh snapshot-delete $thedomain $snapshotname

# snapshot now gone, reverted back to original disk state
$ virsh snapshot-list $thedomain

 

REFERENCES

libvirt.org, explanation of libvirt not supporting external snapshots –  which is required since snapshots cannot be taken of volumes (only disk)

kashyapc, snapshot discussion from LinuxCon Eu-2012 and slides

virt.fedoraproject, discussion of deleting external snapshots and modifying xml file path

libvirt.org, reference for snapshot-create and commands

systutorials.com, virt-xml manual page

access.redhat.com, internal versus external snapshots

redhat.com, using ‘snapshot-delete –metadata <domain> <snapshotname>’ to delete external snapshot record

mathiashueber.com, installing latest qemu/libvirt by compilation or ppa

launchpad.net jacobzimmerman – libvirt/qemu

libvirt.org, discussion of state capture of VMs

libvirt.org, live merge disk chain

libvirt.org, live disk backup with active blockcommit for live merging external overlay snapshot back into base and switching back to base

libvirt.org, qemu backup internals

ubuntu.com, libvirt

qemu.readthedocs.io, iSCSI LUN for QEMU

libvirt.org, release notes

bugzilla.redhat.com, performance isues with internal snapshots

stackoverflow, slow guest performance using external snapshot, need to make swapping/caching less aggressive

NOTES

Standard snapshot errors to watch for

Below are errors you will face if you try to perform standard snapshot operations on guest domains that do not support it, or try to manipulate external snapshot using libvirt (which it does not support).

# error when trying to take snapshot of domain where disk type is not 'file'
error: internal error: Child process (/usr/bin/qemu-img snapshot -c test1) unexpected exit status 1: qemu-img: Expecting one image file name

# error when trying to delete external snapshot with libvirt
error: unsupported configuration: deletion of 1 external disk snapshots not supported yet

Determining version of QEMU, KVM, virsh

$ sudo apt show qemu-system-x86

# below from Ubuntu 20 Focal

$ kvm --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.17)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

$ qemu-system-x86_64 --version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.17)

$ virsh --version
6.0.0