Thursday, November 28, 2013

MariaDB Database Disk snapshot backups on Windows

Using disk snapshots to perform MariaDB backups has become more and more common, but this seems to have been limited to Linux and Cloud environments. There seems that there has been a notion that Snapshots cannot be done on native Windows, the way we can do snapshots using LVM on Linux for example. This is actually False, if you use the NTFS filesystem snapshoting is built in and has been so for a long time, the base for it is the Volume Snapshot Service (VSS) which was introduced as far back as Windows XP.

Using Snapshots to backup on Linux or Windows works well, assuming that the storage engine you use can recover on startup, the backups created this was are physically consistent but not logically, but InnoDB will recover this. Also note that any warnings about using snapshots with MyISAM also applies to the mysql database, i.e. the database metadata. Usually though, this is not an issue, but if you want to protect yourself even more then use a FLUSH TABLES WITH READ LOCK and backup the MyISAM tables separately somehow. But this is no different between Windows and Linux in this case, this is an attribute of any kind of disk snapshot backups.

VSS is a services that run on Windows and to utilize it some programming is needed, but luckily that has already been done for us, there is a simple utility called VSHADOW.EXE available from Microsoft that can be used to access the VSS service. Some version of Windows, such as Windows Server 2008, Windows Server 2012 and also with a version of Windows that is very little used, probably there are more users of Windows 3.10, than of Windows 8, come with DISKSHADOW.EXE, a utility that does the same thing, basically, but has a different interface. In Windows 7, that I tested on, there is VSSADMIN.EXE instead, but as VSHADOW.EXE can be added, I downloaded that one as it is more functional (It is part of the Windows SDK). All this mess (c) Microsoft.

So, how does this work then? As I said, this is different than on Linux (what? Windows being different than Linux? No kidding?). Using the API you can create two types of snapshots, persistent and non-persistent. The difference is that a persistent snapshot is one that remains until specifically removed, whereas a non-persistent one goes away when the program that created it exists. As we are about to take a backup here, I guess you think we want a permanent one, but that is not so, as VSHADOW can run a command file for us, before it exists.

I will use an example here, for that I have placed my MariaDB data on the E-drive in a directory called \data, which means my my.ini has this line in it:

Next question then is where the snapshot is placed? Well, the snapshot ends up as a Windows device with a name along the lines of \\?\Volume{5b3610ce-d2cb-11e2-a649-2cd05a8ad0c9}\ which is just as easy to remember and type as the novel War and Peace (Yes, Tolstoy was a Windows guy, I guess you didn't know that!). Yes, again, thank you Microsoft, Windows is SOO user friendly! Anyway, yet again VSHADOW helps us out, as it can create a command file for us that will contain the names of the devices and some other things. It can look something like this and I have called it vss-setvar.cmd:
@echo [This script is generated by VSHADOW.EXE for the shadow set {c8fcb322-ce02-409a-b609-7fc04311fad1}]

SET SHADOW_SET_ID={c8fcb322-ce02-409a-b609-7fc04311fad1}
SET SHADOW_ID_1={11243eaf-0234-4784-92e1-9bc24668b5b4}
SET SHADOW_DEVICE_1=\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy8

OK then, so this is generated for us, how do we use that? Well, what we do is create a command script that is called by VSHADOW as I said before, and this calls the generated scripts to get all the details. An example script, that I call e-backup.cmd, can look like this:
call vss-setvar.cmd
@echo off
dosdev T: %SHADOW_DEVICE_1%
robocopy T:\Data C:\Backup /MIR
dosdev /r /d T:

There are still a few missing bits here though. The call command is calling the generated file as above (you can call this anything you want, see the example command line below) and what about the command DOSDEV? What is that, that is a simple little utility you can find on the net (Google is your friend) that allows you to create a DOS device from a device descriptor, and it can also remove them. There are other ways of dealing with this, but I choose this path my my example here. Robocopy you probably know already and the /MIR option is used to tell robocopy to mirror a full directory tree.

So what the command script above will do is create a DOS device called T: for my snapshot, copy that to a directory called C:\Backup and then remove the T: device.

Now we have a command script (not two, the first one I showed was an example of a generated one) and an idea what it does, to do a shadow copy now, we just need to call vshadow, and it will do the work for us. We pass three parameters:
  • The name of a script with the device names and things to generate.
  • The name of a script to run
  • The name of the drive to create a shadow device of.
Using the example above to create a backup of the E-drive, where I have my MariaDB datafiles in the \data directory, this is the command I use:
vshadow -script=vss-setvar.cmd -exec=e-backup.cmd E:
So that's about it, now you know how to create an online snapshot backup of a MariaDB database on Windows.



Peter Laursen said...

Well .. 1 year back I blogged this:

from my concluding comment: "my primary point was that those snapshots created automatically by Windows ‘System Restore’ (as well as various backup tools when running in scheduled mode) are not safe to recover MySQL data from. This would require a ‘snapshot provider’ interfacing MySQL to Windows (or vice versa) as described by wlad."

Did I miss something?

Peter Laursen said...

Sorry, I was not clear. I realize that your 'shadow copy' is created on demand/by command.

But still: what about caching? MySQL is not aware that a snapshot is taken and there may be 'cached writes' that are not written to disk yet. And Windows is not aware of MySQL's caching behavior. Windows will create a snapshot of the file system on the drive and cannot consider what MySQL has cached in memory.

And all that could render the snapshot in an inconsistent state, as far as I understand. What I also think I experienced once.

Anders Karlsson said...

I don't think you missed something, but this is trur of any kind of disk snapshoting you use. Inconsistent datafiles will be fixed by InnoDB on recovery. This doesn't mean there aren't issues, but they can be handled and are no different with any kind of disk snapshots, Linux, ZFS or VSS.

Justin Swanhart said...

1) If you hold a FTWRL while you grab the snapshot, then all MyISAM data will be written to disk (at least as far as the OS cache) when the read lock is held. Because the snapshot will not miss OS cache data this means that you do not need to back up the MyISAM data separately, it is completely valid in the snapshot, and as a bonus it will be consistent with the InnoDB data.

2) FTWRL does not, however, affect InnoDB (except to block new transactions). So your concerns about cached data are valid, but only up to a point. InnoDB by default fsyncs the InnoDB transaction log at every commit. This makes sure the LOG write makes it to at all the way to platter.

3) When performing checkpoints (background writing) InnoDB uses the doublewrite buffer which is fsynced before writing background pages. This ensures that random background writes write complete pages.

Because of the combination of 2 and 3, InnoDB recovers from a snapshot exactly the same way as it would from a power loss. The logs are in the snapshot.

So there are a few requirements:
a) Your InnoDB log files must be on the same partition as the data files.

b) The FTWRL is necessary for MyISAM tables but not for InnoDB.

c) The FTWRL need only be held for the duration of the creation of the SNAPSHOT. The lock can then be released. Reads from the snapshot will see files as they were during the FTWRL.

And some warnings:
Snapshots add a LOT of overhead especially for random IO. It is wise to throttle your backup to avoid negatively penalizing the underlying volume.

Justin Swanhart said...

Just in case I wasn't clear:
1) FTWRL (wait until it completes)
2) take snapshot
3) release FTWRL
4) backup snapshot to somewhere else (throttle transfer speed if necessary)

Anders Karlsson said...


Yes, a FTWRL is needed for MyISAM, but not for InnoDB, and I focus on InnoDB here. What is sometimes forgotten is that the mysql database also uses MyISAM. If you don't have any "moving" MyISAM data and only InnoDB, it's usually a good idea to do a Flush Tables before the snapshot.

Also, thanx for the comments on performance of snapshots, this is very true (I just tried it on Windows though, and it wasn't too bad). And the whole thing requires NTFS, but I assume noone in their right mind puts database data on some other format disk on Windows these days.


ortigasclover said...

I’m very glad to found this website because; it carries awesome and actually good data in favor of readers.


sarahlee880 said...

I really enjoyed reading your article. I found this as an informative and interesting post, so i think it is very useful and knowledgeable. I would like to thank you for the effort you have made in writing this article.


Leslie Lim said...

I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often..

rachelle madrigal

Leslie Lim said...

I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often..

rachelle madrigal

lee woo said...

I like your post a lot! You should write some more on this!Great job coming with such terrific post!


belleparas said...

Beats me how I know your site but I started to spend more
time visiting here and added your site to my bookmark as well so I can return here with no hassle.

devon broad said...

Is it correct to say merely {that you|that you merely|that you simply} simply merely merely simply ar instantly aquatics getting some really costs that has the potential cannot hold out until payday loans huntsville inevitably your next pay day advancement comes? have you ever ever ever ever been snared having Associate in Nursing unforeseen automotive resolve?

Milla Jkmn said...

A day loan investor can ne'er charge you a lot of as a result of you would like the money quickly nor ought to ANy payday loans corona investor ever raise you for any cash up front within the kind of fees or an application charge. Take an instant to figure the numbers and puzzle out simply what quantity it'll value you to borrow these 2 ways: a short day loan or a mastercard amount.

Sharuk Hossain said...

The current worldwide world spins around business, and business is taking into account credit. Your standard and level of living relies on your investment in and acknowledgement into this business-arranged world. So today, for some basic things you may need or requirement for yourself, you require great credit. auto title loans near me chicago

marko said...

In the event that you require an auto credit, you might exceptionally well end up in an extremely unpleasant time. Have you lost your past auto through an awful arrangement of occasions, or have you ended up in a circumstance where you just need another auto? Autos are a vital piece of ordinary life in America, and chances are great that you may not even have the capacity to get the chance to work without one. auto title loans near for me

musa ibrahim said...

More generally than not, these credits are classified into anchored and apart structure, with the ambition that you can access the food according to your charge and necessity.Cash Advance San-diego

Mission Vission said...

Of course, a sensible title on the vehicle is the essential for such credits. This proportionately helps in restoring the technique for support of the advances. These advances are routinely speedier insist than the general credits. Car Title Loans

MUSA77 said...

Do you accept your driver's admittance with you? There are abundant variables which analyze with this capability. The claimed ID tells the moneylender who they are alive with and that it coordinates the freeholder recorded on the title. Payday Loans

Barbie Chiu said...

I am very happy to visit your site and i did like it. I was able to get some informative topics. keep it up and you can also visit my site for your reference. Thank you and have a good day.

ariyan arifin said...

In case you have a vehicles, you can set up the title up as security and get a headway considering the estimation of the auto. A generous specimen, diverse affiliations which offer these sorts of advances will allow you to get up to half of the estimation of your auto. Check Cashing San-diego

Mission Vission said...

The borrower may be asked for that hand over the vehicle at a predefined date and time, as essentials be issuing him an opportunity to oust his reasons of excitement from the auto, that are not an auto's touch guaranteed. Payday Loans San-diego

Mission Vission said...

This is the party of my article, yes managers can and do domain mass diminishments, on the off sporadic that you inquire. Our terrible atypical mass or change wager are bounteous underneath radical than you will fulfill from your bank. car title loans chicago

JON said...

Legitimately total thought to be reimbursed to the settlement beat up central of 2 weeks to 4 weeks. The feasibly entire will be deducted from your balance to the settlement expert's record on the due date or aback your awesome position campaign to your record. Auto Title Loans

ariyan arifin said...

Like other transient advances, the title qualities too are liberal inferable from the higher interest rate. Title credits satisfy passing needs. Promptly as the borrower gets hold of purposes of interest, he pays the title advance and recuperates the title to his auto.

marko said...

Because of the account organizations, to get an auto has gotten to be less complex than any time in recent memory. You can without much of a stretch get credits in Melbourne to satisfy your fantasy. Cash Advance

Sarah Theresa said...

There are a couple regulated expenses in a payday advance! Keep in mind, on the off chance that you are not orchestrated to pay this change and you happen to lose your occupation, the clock does not stop. On the off chance that you default on the credit, which unites the vital regardless of side redirection, the payday advance alliance will recover the whole entire paying little personality to all the side interest collected on this credit. A payday credit connection may sharpen particular unmistakable choices for assemble this development. Payday Loans San-diego