KVM Backups - DO NOT USE

stacksofplates

I figured I'd do a quick write up of a simple way to do backups with KVM.

First you have to add the qemu-guest-agent to the VMs you want to back up.

yum install qemu-guest-agent

or

apt-get install qemu-guest-agent

Make sure the agent is running and enabled (not 100% sure if you need to run the systemctl enable or not).

systemctl start qemu-guest-agent
systemctl enable qemu-guest-agent

Then you need to add the agent channel in the VM. You can do this with either Virt-Manager or directly editing the VM xml file. Since I'm lazy, I'll just do it through Virt-Manager. Open the VM and click Add New Hardware, then select Channel and the org.qemu.guest_agent.0 and click Finish.

Now we can back up the VM. I just wrote a little script that does this (This method is assuming your .qcow2 images have the same names as the VMs). It's also backing up to an NFS server mounted at /backup and getting the images from /data/VMs.

#!/bin/bash

today=$(date +"%m-%d-%Y")

for i in VM1 VM2 VM3 VM4
do
  # Export XML
  virsh dumpxml $i > /backup/$i.xml
  # Freeze VM filesystems
  virsh domfsfreeze $i
  # Create VM snapshot
  qemu-img create -f qcow2 -b /data/VMs/$i.qcow2 /data/VMs/$i-snap-$today.qcow2
  # Thaw VM filesystems
  virsh domfsthaw $i
  # Take backup from VM snapshot
  qemu-img convert -O raw /data/VMs/$i-snap-$today.qcow2 /backup/$i-$today.img
  # Delete snapshot
  rm -f /data/VMs/$i-snap-$today.qcow2
done

Add it to a cron job and that's it. This method doesn't compress the images however. To do that you can just add a line to tar the image in the for loop.

Edit: this method is incorrect. Please use this method instead: https://mangolassi.it/topic/12537/kvm-snapshot-backup-script

Dashrender

In light of the XS HVD size limitation of 2 TB is KVM a better choice? or moving to native Xen instead of XenServer?

scottalanmiller

@Dashrender said in KVM Backups:

In light of the XS HVD size limitation of 2 TB is KVM a better choice? or moving to native Xen instead of XenServer?

XS limitations over Xen should not prompt a consideration of KVM. It's only the XS interface itself that has the limitation.

Dashrender

So should we be looking to do a fall back to Xen? I'm guessing in doing so we'll also loose XO support as well?

scottalanmiller

@Dashrender said in KVM Backups:

So should we be looking to do a fall back to Xen? I'm guessing in doing so we'll also loose XO support as well?

By default, yes. I believe that you can apply the XAPI API (is that redundant?) to the Xen install and then attach with XO. Would be a good project to do.

stacksofplates

@scottalanmiller said in KVM Backups:

@Dashrender said in KVM Backups:

So should we be looking to do a fall back to Xen? I'm guessing in doing so we'll also loose XO support as well?

By default, yes. I believe that you can apply the XAPI API (is that redundant?) to the Xen install and then attach with XO. Would be a good project to do.

Ha I always wondered if it was, but then you mention XAPI and not API and people have no idea what you're saying.

scottalanmiller

I think that XAPI is its actual name, so API is actually needed.

olivier

"Plain" Xen or KVM would need an agent installed on each host to expose an "enough advanced" API (probably something ugly like libvirt+custom scripts). Take a look a oVirt project for this.

Anyway, XO is only working on XAPI (which is the project name of this Xen API). APIs can be confusing because there is various level of APIs: low level APIs (like lib-xl or a part of lib-virt), and more "turnkey"/complete API like XAPI, which handle a lot of stuff (not only the hypervisor, but also the glue around it).

That's why XAPI project is more than just an API, but a "toolstack". See http://wiki.xen.org/wiki/Choice_of_Toolstacks

For example, Amazon got its own toolstack (not public) on top of Xen.

KOOLER

Aren't there any agent-less KVM backups available? Anything like Veeam?

scottalanmiller

@KOOLER said in KVM Backups:

Aren't there any agent-less KVM backups available? Anything like Veeam?

None that I know of. Scale does an agentless backup but you have to have a Scale cluster, not just KVM.

KOOLER

@scottalanmiller said in KVM Backups:

@KOOLER said in KVM Backups:

Aren't there any agent-less KVM backups available? Anything like Veeam?

None that I know of. Scale does an agentless backup but you have to have a Scale cluster, not just KVM.

I was under impression they license somebody's else technology, don't they?

scottalanmiller

@KOOLER said in KVM Backups:

@scottalanmiller said in KVM Backups:

I was under impression they license somebody's else technology, don't they?

For their basic backup, no it's all internal. For more advanced features they work with third parties but don't license it, it's sold separately.

KOOLER

@scottalanmiller said in KVM Backups:

@KOOLER said in KVM Backups:

@scottalanmiller said in KVM Backups:

I was under impression they license somebody's else technology, don't they?

For their basic backup, no it's all internal. For more advanced features they work with third parties but don't license it, it's sold separately.

Do you use their agentless backup? How do you find it against say Unitrends or Veeam? thanks!

scottalanmiller

It's very basic. Just full images and automated exports. Nothing extensive. But for firms needing a basic, free backup mechanism, it gets them that. In most cases you would still want something more robust, like StorageCraft, Unitrends or the like. A Veeam product for it would be awesome. But it is enough to get automated data protection in place.

KOOLER

@scottalanmiller said in KVM Backups:

It's very basic. Just full images and automated exports. Nothing extensive. But for firms needing a basic, free backup mechanism, it gets them that. In most cases you would still want something more robust, like StorageCraft, Unitrends or the like. A Veeam product for it would be awesome. But it is enough to get automated data protection in place.

See Inbox

stacksofplates

Qemu has some kind of incremental backup, but I haven't investigated much.

I just want to point out that one advantage to exporting to a raw .img file is that you can mount the image and pull files directly. If you need to import, just have qemu convert the .img back to a .qcow2 file and use the XML dump to rebuild the VM.

Dashrender

@scottalanmiller said in KVM Backups:

It's very basic. Just full images and automated exports. Nothing extensive. But for firms needing a basic, free backup mechanism, it gets them that. In most cases you would still want something more robust, like StorageCraft, Unitrends or the like. A Veeam product for it would be awesome. But it is enough to get automated data protection in place.

Is the move away from an agent based backup really worth that much?

Didn't we have this conversation a few months ago?

scottalanmiller

@Dashrender said in KVM Backups:

@scottalanmiller said in KVM Backups:

It's very basic. Just full images and automated exports. Nothing extensive. But for firms needing a basic, free backup mechanism, it gets them that. In most cases you would still want something more robust, like StorageCraft, Unitrends or the like. A Veeam product for it would be awesome. But it is enough to get automated data protection in place.

Is the move away from an agent based backup really worth that much?

Didn't we have this conversation a few months ago?

Unlike a lot of people, I'm fine with agent based. It might not be ideal, but it isn't bad. The big move here, though, is to "free and included."

Dashrender

I gotcha... So there is a free and included one in KVM, but no Xen or XS right?

scottalanmiller

@Dashrender said in KVM Backups:

I gotcha... So there is a free and included one in KVM, but no Xen or XS right?

There is a free, included one in Scale and one in XO.