Consolidated backup

[Strangekid]

We have just successfully installed Virtual Centre 2.5 and ESX server 3.5 on some machines and am now faced with the problem of backup strategies. The data I have on the servers that are to be virtualised is very important and has to be kept for 20 years. Backup is therefore a big consideration.

We need to backup the data from 8 Oracle and 2 SQL Server databases which are to be held on Virtual machines (on a SAN) and also images of the machines themselves. We have been looking at the option of consolidated backup. Now it says in the docs that it can backup individual files but I get the impression that you cannot tell it as you could with Arcserve or Veritas on a physical machine to back up the database instance.

My question is what is the best solution for our backup. Our plan at the moment consists of using Consolidated backup to take images of the Virtual machines and store them in a tape library. We would then install a client on each machine to backup the database instances seperately. This also assumes we can connect to the tape library somehow from the Virtual machines, quite a newbie to this area so assuming that there would be some kind of agent that allowed us to link to a proxy and then perform the backup to tape. I hope that there is a better way to do it than this because it will involve buying a new license for backup software everytime a new VM is added. Is there some kind of centralised methodology we can follow?

I.e Can we configure the proxy server to backup the DB block on its own and a full image rather than just using it to take a whole image and having the hassle of maintaining multiple installs of the backup client on each machine.

Cheers for help and advice

Strangekid

[akutz]

If you are intent on backing up DB data that lives in a VM you will definitely need to use an agent inside of a VM to back that data up. Otherwise if you treat the VM as the backup container your DB backups will be crash consistent at best. My recommendation is this. For VMs that host I/O intensive applications, use an agent-based backup from with inside the VM. For VMs that are little more than front-ends to other data, such as web-servers, print-servers, perhaps file-servers, back the entire VM up as crash-consistency won't be an issue. There are of course in-the-middle problems like an Active Directory domain controller where there is a database to worry about, but it is not subject to the same database backup tools. For these situations it is probably okay to treat the VM as the backup container as well.

As for your other questions, VCB can backup individual files inside Windows VMs, but it will have no concept of what a database is.

[Strangekid]

I have heard about a tool called NetBackup which is available from Symantec. I was told by a colleague that if you install this on a proxy server you can then use this server to backup entire machine installs and databases on other machines. Is this correct do you know? From what you said in the last post this does not sound right. Do you know if you can put what is effectively a license server on the proxy and then distribute this out to the machines you wish to backup.

[dmorgan]

OK. I have just gone through exactly what you are going through in our office. We use ARCserve for backup, Fibre attached Tape Library, and Lot's of Oracle database servers, some Virtual, some not. This is what I ended up with. I use consolidated backup to do bare metal backups (fullvm) of the virtul database servers. If they are windows machines, I sometimes do file level backups as well, but not of the databases themselves. I looked at scripting pre and post commands to put the oracle databases into backup mode, assuming archivelog mode is enabled that is, backing them up, and then back again after the backup. This is possible, but I am not an Oracle guru, and seemed like too much work for me. Instead, I settled on using Arcserve oracle agents to back up just the databases. This in conjunction with the fullvm, baremetal backup, provides full recoverability for the server. Yes, it requires two seperate backups to tape. However, we only do the full backup of the machine once a month. The database backups operate much more frequently. Only the database really changes on these servers, so restoring the latest full, and then the latest database on top of that, has worked well for us. If you are an Oracle guru, and your databases are all in archivelog mode, you may just be able to script pre and post commands to be sent to the database servers to quiese the database and put it in backup mode, then run the VCBMounter scripts, backup up the machine at a fullVM or file level, unmount the drive(s), and then revert the database out of backup mode. That would give you a single backup, with everything on one job, however I found this to be too time consuming of a project for my meager oracle skills.

Don

If you found this or any other post helpful please consider the use of the Helpfull/Correct buttons to award points

[polysulfide]

You can do database dumps and backup those. SQL maintenance / backup can do regular dumps. Backup the dumps with the file agent. If Oracle is unix you can put the dumps on their own VMDK and back that up.

If you're handy at scripting, you could stop the database service on the host before you do a snapshot and the snapshot will be suitable for recovery.

Any solution you come up with yourself that isn't officially supported by someone should be thoroughly tested.

If it was useful, give me credit

http://communities.vmware.com/blogs/polysulfide

VI From Concept to Implementation

[dmorgan]

The problem with stopping the database service is that means the database is down. That is unacceptable for our production environments. Thus, we had to have a hot backup solution. This can be done, if the oracle database is in archivelog mode, where by all changes are written to logs, while the database is being backed up, and once that backup is complete, the changes are then applied. However, if you stop the service, the database is down. I could script that in 5 seconds, however we cannot have production databases, some as large as 1TB or larger, down for the period of time it would take to perform the backup. Thus, I went with the agent for the database, which uses RMAN i believe, and a fullvm backup for the system on a monthly basis.

If you found this or any other post helpful please consider the use of the Helpfull/Correct buttons to award points

[kudzu1]

There are three basic ways of backing up a DB in a VM

1) Client and DB agent inside the VM. This is essentially the same backup method as if the DB was hosted on a physical machine

2) For VCB style backups you could use the VMware pre and post processing scripts that are installed via VMware Tools, you could write a script to quiesce or shutdown the DB before the VCB snap is taken. The DB shutdown script would have to be written specifically for your DB and environment.

3) The third method would be to do nothing and back up the DB using VCB. This would create a crash-consistent copy of the DB and is not recommended for most situations.

[dmorgan · Accepted solution]

Exactly the conclusions I came to for our backup solutions... The crash consistent version is not really an option, especially for live, production databases. So, that leaves writing the scripts myself to quiese the database and put it in backup mode, and including those commands in the VCB pre and post scripts, and using the appropriate database agent from the tape backup software manufacturer. In my case, I am not a databae guru, and scripting these commands was more complex than just using the agents. Plus, we only really need a fullvm image for disaster recovery on a monthly basis, the server really doesnt change other than the database. The database level backup can occur on a more frequent basis, and if anything really goes wrong, that database backup could be used to restore the database to another server as well.

If you found this or any other post helpful please consider the use of the Helpfull/Correct buttons to award points

[Strangekid]

Cheers for all your help guys. Like dmorgan says the databases are production databases and can therefore not be allowed to go down. I think the solution we will go with is the installing of agents on each machine. | am a developer and I know a little stuff about Oracle, so tempted to try that but as the saying goes 'why bark when you've got a dog'. I know that it will be more expensive than scripting ourselves but funding is not an issue for our client (lucky them :smileyblush: ). I take it we will have to buy a license for each of the Virtual servers that use the agent i.e. 8 Oracle agent licenses and 2 SQL server agent licenses?

Thanks again for all your help.

[dmorgan]

Glad to help! One thing I forgot to mention is that if you do one backup job of just the database using the agent, and a fullvm using VCB, is that you will be backing up some redundant data. If, by chance, you have the database housed on a separate drive than the rest of the OS, you could choose to not back up that drive, so as to cut down on the backup time. Otherwise, you will be backing up the database with your fullVM VCB backup, as well as via the agent, although only the agent backup would be useful.

Yes, you have to buy agents for each system just the same as if they were physical machines. This is the only reason I even considered scripting the quiese commands to put our Oracle databases in backup mode as opposed to using the agents. I am an IT guy, not a developer, so I had no desire to re-invent the wheel. However, I do have a budget, and purchasing the number of agent licences I would need can add up fast. In the end, I was able to weasel a good volume discount from a reseller, so it wasn't too bad.

If you found this or any other post helpful please consider the use of the Helpfull/Correct buttons to award points

[Strangekid]

Cheers again for that. So we need individual agent licenses for each of the VM's. We need one license for Backup server. Do we need license for a client for each of the virtual machines for sitting underneath those agents? Sorry if I have misunderstood any of this but seems that the client sits on the same VM as the agent and I am curious if we need licenses for both so we would end up with:

10 x client licenses.

8 x Oracle agent licenses.

2 x SQL server agent liscenses

1 x Backup server license.

Thanks for your patience and help.

[dmorgan]

No problem, that is what these forums are for, questions and answers. I ask twice as many questions as I answer, I am sure. :smileyblush:

So far as licences go, it really depends on the tape backup software, but they all tend to operate the same way. We happen to use Arcserve. Most all of the tape backup software companies license by machine being backed up, as well as per agent on each of those machines... That is where they make their money, i think. The base product may not cost all that much, but if you want to back up databases, open files, exchange, etc... the agents are really where it's at. Some software companies do unlimited licenses for workstations, others sell workstation backup agents in packs of 5 or 10. You will have to check with whatever tape software company you are going with for the specifics, but I would recommend going with one on the list of compatible software with VCB. How they integrate with VCB will differ between software companies, Arcserve provides pre and post scripts that call vcbmounter.exe to mount the VM's before, and unmount them after backup.

We have 1 license for the Master server, this is installed on our Tape Backup server that is Fibre attached to our library, and to the SAN, for VCB. This server also has the VSS option installed.

On our Oracle servers, we have the standard arcserve client agent installed, plus open file agent, plus oracle agent. On the Exchange server, we have the Exchange agent installed.

If you found this or any other post helpful please consider the use of the Helpfull/Correct buttons to award points