DR / Real Time Replication for Hot Site

[Technologist]

Hello all,

I am new to the forums and VMWare Infrastructure 3. I have been lurking around and sponging up information like crazy. I am currently in training for VMWare and have it already installed at work. It is being configured at work and needs some tweaking.

So here is the setup I have (TWO sites with the following config):

Dell Blade servers connected via iSCSI to a Dell Power Vault 1950 / MD3000 storage array

Power Vault server is running "Windows Unified Data Storage Server" which then connects to MD3000 which houses the HD's

4.5 TB total storage

One 1TB LUN for the OS's

One 2TB "Data" NTFS Volume

We want to be able to use the built in replication in windows (Distributed File System aka DFS) to replicate VM's and / or Data to 2nd location in real time so that if primary location goes down we can bring up the secondary location.

We wanted to (try) to avoid products like esx ranger and anything else that costs extra $$ and use the built in Windows replication.

Here is where I am getting hung up: I dont think DFS can replicate a VM file. The VM file on Windows Storage Server is actually a *.VHD file and all the guest operating systems live in the file. DFS will not be able to back up a SQL database for example, cause all it will see if the VHD file. So here is my problem. I wanted windows (DFS) to be able to see data like a SQL database and replicate it real time to the secondary location for DR purposes and have it ready in a hot site scenario weher we could bring the other site online.

Possible solution or strategy would be to create a "C" drive only VM running Windows Server 2003, install Micro$ofts iSCSI initiator software and have the VM attach to my NTFS volume as a local drive letter.

Question: Would using M$'s iSCSI initiator create a "D" drive from within the VM to the NTFS partition actually create a VMDK file?

OK, I have probably just confused myself and half the people reading this post heh

Bottom line: Is there a way to use DFS within Windows to do real time replication between sites of to create an effective DR solution?

--T

[epping]

wow this is a complex way of doing things, i have never heard of using DFS, how had you ended up with VHD files and not vmdk ?? were they windows virtual machines.

replication is normally done by the SAN controllers, however i would go down the esx ranger option if you dont have the $$

sorry cant really help abou the DFS never even thought of it

[Technologist]

Using Windows Storage Server to create a "Virtual Disk" on a volume on our NAS actually creates a "*.vhd" file. I actually agree that it is lame(?) that is creates a vhd file...

OK, if the DFS thing is throwing everybody does anybody know of a good way to have a hot site with real time replication? It doesnt matter what the tool is... (altough if someone has a DFS option then shoot). I welcome all suggestions at this point.

The plan is to be able to have a mirror of our vmware and data at our second site. If one burns down, the other can kick in.

--c

[Texiwill]

Hello,

To get a realtime replication of the VMFS on which the VMDK resides, the storage you use needs to support some form of mirroring. Business Continuity products exist for most Enterprise class SANS (Business Copy/Hot Copy, Continuous Access, etc). Some can produce a snapshot in time of the entire LUN (VMFS) for backup purposes and others will maintain the mirror until one side breaks. However these are at the storage level and not the ESX level. vReplicator and its ilk are not realtime replication tools but more snapshot in time tools. If your storage does not support something like HP's Continuous Access, then you may want to look within the Guest for a realtime replication product. Each Guest has something like that available. I imagine DFS would run within the Guest OS and make copies to remote VMs in the same fashion as the Storage can do. However, the network bandwidth required for all VMs would be pretty high I would imagine.

So it boils down to, can the Storage you are using provide what you want at the LUN level, or does the Guest OS provide some form of mechanism to perform a realtime replication. ESX does not have this capability yet. With the fact that most SANs do provide this, I am not sure this will be in the mix for the future (it is an EMC company after all).

Best regards,

Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education. Available on Rough Cuts at http://safari.informit.com/9780132302074

[hugop]

Have you considered Asynchronous mirroring using products such as Datacore's AIM?

[Technologist]

This was a helpful response. Basically from what I can tell there is no "silver bullet" way to do real time mirroring of storage level and esx level data. I am intrigued by the guest OS realtime replication but then wouldnt I need to have all the same or similar guests running at the other location.

There are so many ways to go on this... It would be great to find a product that can sync to sites at the esx level so if the data was kept in a virtual file it wouldnt matter cause the entire esx vm would be sync'd, with the data in it. The vm could then just be fired up at the hot site in an emergency. Does vReplicator mirror the VM and the data in the vm as well?

Is there anybody here using vReplicator?

[Technologist]

I looked on their website and did not find a product called "AIM". Could it be called something else?

[hugop]

AIM = Asynchronous IP Mirroring, its an optional component that you can buy for SANmelody or SANsymphony or as part of the Melody DR2 package for £3999 or $8000, which would give you two SANmelody licenses plus the AIM option and thin provisioning.

Melody datasheet is here: http://www.datacore.com/downloads/SANmelody_Datasheet_eng.pdf

Customer success story is here: http://www.DataCore.com/downloads/CS_Basler_US%20FINAL.pdf

Vizioncore vReplicator datasheet is here: http://www.vizioncore.com/DataSheets/vReplicatorDatasheet_HighQ.pdf

I'm helping a customer with similar requirements to you.

[Texiwill]

Hello,

I think doing the replication at the Storage level (whether using SAN Melody) or something built into the SAN is the safest way to do this sort of thing. EMC has this for most if not all their SANs/iSCSI servers as does HP, and all the rest. So that is the way I would go. Individual agents for each of the VMs would be pretty costly I would presume, not that the SAN options are cheap either. Since EMC has their mechanism I just do not see VMware putting it into the VMFS which is ideally where it would need to go.

Best regards,

Edward L. Haletky, author of the forthcoming 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', publishing January 2008, (c) 2008 Pearson Education. Available on Rough Cuts at http://safari.informit.com/9780132302074

[mreferre]

You might want to have a look at this deck:

Massimo.

[AWo]

Take a look here: http://www.doubletake.com/products/virtualization/default.aspx

I have no experience with VMWare and DoubleTake, but I guess it is worth looking at. We use it at some very large customers (e.g. as a backup mechanism for a 24/7 baggage claim system at a large airport in the south of germany, backups a SQL database in realtime to a second system). You can do realtime replication as well as scheduled and multi-level replication. It only replicates modified blocks.

AWo

[del1798]

I don't think that DFS will do what you want it to do. DFSR can't replicate a file that is locked. For instance, if a user has an Excel spreasheet open for a full day, that file won't replicate the changes until the user closes the file. The same will go for databases; while the SQL/Oracle/etc. services are running and locking the DB files, the DB cannot be replicated until it is unlocked.

Doubletake is a great fit for what you want to accomplish.

[jasper9890]

Is this envrionment already in production? I saw your LUN sizes and think you're REALLY going to want to do some research on that. You are going to have some major scsi reservation issues when you start building up your VM's on single luns like that. I've had to find the sweet spot for my environment (amount of I/O and VM sizes), which comes to about 275gb luns, with no more than 10 to 15 VM's on each. Any more than that and i start having some major issues. Right now i'm up to 10 luns of this size and almost 100 vm's and it's going perfect, previous when i had a few large luns i would have vm's locking up and rebooting randomly because of this.

Also, you don't get the advantage you think you would out of putting OS on one lun and data on another. Two factors impact here - snapshotting and power on/off will impact both luns instead of one when they are fired off, which can cause a little performance degragation to other machines, as well as now i forget what i was going to say - something to do with overhead of splitting disks up, i would have to go back at old posts and look that up sorry. Just warning you to do the research if you have not proved this configuration out yet!

All the above is of course my humble opinion, but comes from some hard learned experience...

[korman]

I see where you are going here and I think that you are onto something. I have used Doubletake to successfully replicate databases and GSX vmdk files in the past. The product does an awesome job for a host based replication . Where it lacks a bit is during resyncs and in my case, the business requires that we run in DR as prod and then replicate back. I too was considering implementing something similar to what you are describing. This would require two Doubletake licenses one on the Source and one on Target about $1800 each. That is a heck of a cheaper then any real-time Array based replication that I am aware of?

DT would be installed on the Unified Windows Storage OS and Configured to replicate to a DR site in real-time asynchronously. One might argue that this configuration is "Array" based replication on the cheap as your Array is a Windows Server running DT software.

Some Brain storming ...

You could schedule VSS snaps shots every 15 minutes in DR so you have a bunch of running recovery points. If a vm on the DR side did not start you could restore the VMDK or VHD files from previous versions.

ESX Host(s) in the Prod Site can Access the Windows Storage via NFS or ISCSI. And still use vmotion

NFS might have advantages over ISCSI for VMFS data stores in this configuration. If you present ISCSI luns to your ESX hosts and configure them as VMFS data stores there would be a lot of layers of complexity, pretty ugly.

ISCSI from Windows Target

(VM)NTFS (ESX)VMFS (ESX)VMDK (Win Storage).VHD (Win Storage) NTFS

NFS Windows Target would look something like this as you are basically putting the OS on a file share.....

(VM)NTFS (ESX)VMFS (ESX)VMDK (WinStorage)NTFS

Use RDM for Data and OS Luns which will be served up via ISCSI. Choose to ONLY replicate the necessary VHD files in DT replication sets. Put the VM Config Directory on NFS VMFS Datastore. And choose not to replicate the Configuration folders. This way you can have an identically configured ready to go server in DR and you will not be replicating the VMs page file

You get the benefits of DT's efficient Compression and Async replication which helps WAN utilization. Version 5.0 does byte level changes.

I would build a highly redundant teamed NIC access point to the Storage Server

This is not by any means a solution for heavy IO servers and I have never tested this