replication of virtual machines setup between ESX hosts

[puzzledtux]

I have a test bed of 40 odd virtual machines hosting applications which are being accessed by QA and devel team. The VM's are setup on 2 ESX hosts ,managed by VI client and using SAN storage. The older setup has FC connected SAN and the newer hosts is connected to ISCSI SAN.

We now would want to replicate the entire setup onto a upgraded hardware (again 2 ESX hosts) without disturbing the current environment. The applications within the VM's are IP and hostname specific for functionality.

So basically I want the current setup (GroupA) to be replicated into another (GroupB) and both co-existing simultaneously for a period of time until the new setup (GroupB) is verified and then discard GroupA. There happens to be a further challenge that any updates made in VM's of GroupB be also synced with GroupA VM's.

Now evaluating VMware Lab Manager I have managed to replicate the GroupA setup onto a newer ESX host using network fencing feature. So both the setups are properly working without any conflicts. I would now like to know if there is a way to sync the data using any available tools?

[Texiwill]

Hello,

Data Synching would have to exist within the VM for now, there are no VMDK level tools that would do this except to perhaps make backups and restore backups periodically...

Personally I would opt to use Lab Manager to do what you want to do, test everything then use Lab Manager to do the final move, that way the VMs are exact copies at time of shutdown of the older systems. Alternatively if you could just SVMotion the VMs over when ready.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll

Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links

[puzzledtux]

I was searching for one such tool which actually synchronizes the entire virtual machines from GroupA to GroupB and found that vReplicator does perform this operation. However it seems that the target VM needs to be powered-down.

Also your suggestion of final SVMotion is accepted as one of the last stages of migration. But I needed both the setups running simultaneously over a period of time for proper migration verification by the team.

[lamw]

Texiwill has a good point about using something like Lab Manager to help you mange your scenario. Though if you're looking to just "replicate" the existing environment to another and you have shared storage or even NFS mounted storage and you don't want any down time, you can take a look at the following free backup script:

http://communities.vmware.com/docs/DOC-8760

Even though this script is meant to be used as backing up your Virtual Machines, it does in fact allow you to backup live running Virtual Machines to another datastore (FC/iSCSI SAN, LOCAL or NFS). If you're looking to get an exact copy so long as there are not too many changes going on in the system (say apps stop writing or writes to external storage), this would get pretty close in replicating your VMs 1 for 1. Once copied onto the new storage that the current ESX Server can see, you can then bring the replicated VMs up in the new environment so long as it's configured with the same portgroups/etc.

Hope this helps

[puzzledtux]

I am already evaluating LabManager and have setup a fenced network which has the replicated virtual machines up and running perfectly. But the challenge is to sync any updation to the virtual machines in GroupA to corresponding virtual machine in GroupB with both the VM's powered on!!

I am off to grab a bite for now..be back in a while to check out the forwarded link

[lamw]

Oh in that case, it sounds like you may have to rely on some type of guestOS agent that does the data sync. I'm not aware of any, but I'm sure there is something out there that does that.

[puzzledtux]

Anybody in here who has used esXpress who can share views with respect to virtual machine level replication between two hosts and failover support?

[Texiwill]

Hello,

In order to do the replication the VMs on the target host have to be powered off. Source host can be powered on. You would backup using esXpress and restore using esXpress to the target host. Or backup directly to the target host over SSH/SCP. But then you will have to by hand register the VMs.

Use of any backup solution to do this would require creating a backup, then restoration, then registration. You could easily write your own scripts to do the same thing as well using vcbMounter from the command line and SSH to run commands using pre-shared keys.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll

Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links

[puzzledtux]

So does it also mean that esXpress does not provide any single button failover wherein the instance of virtual machine will be powered on the target host?

[Texiwill]

Hello,

Last time I looked it did not. No 'straight' VMDK backup tool provides 100% of this capability. Veeam Backup has the tools to do replication but I am unsure if it registers and powers on the VM as well. vReplicator is the other one that I know about.

I still think you may be better off writing a script specific to your situation, which is quite a bit different than standard backup tools. I am not sure I would ever want replication to automatically power on or even register my VMs until they are necessary and for that I generally write a script to perform this function for me.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll

Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links

[lamw]

As suggested, your best bet is to write a custom script that'll handle your case specifically. You can look at the backup script I've written which will work for both ESX and ESXi, then in terms of recovering or powering on the machines on the new host, you'll have to script that process. It won't be too difficult, you just need to use "vmware-cmd -s register /full/path/to/vmx" or on ESXi, it would be "vim-cmd solo/registervm /full/path/to/vmx". Once registered, you can power on all newly registered hosts and possibly put a sleep in between so you don't have a boot storm via a script or manually power on systems as you need

[puzzledtux]

Hmm..so currently as of all the tools I have known for now does not perform replication with the target virtual machine in powered on state (or there are some which I haven't explored!).

Btw I liked vReplicator as it performs a neat replication and also allows failover in its management interface. However for some reasons it fails to replicate when the source virtual machine has prior snapshots taken.

[Texiwill]

Hello,

Hmm..so currently as of all the tools I have known for now does not perform replication with the target virtual machine in powered on state (or there are some which I haven't explored!).

If you want Source and Target VMs powered on during duplication, then you are not looking at any VMDK level tool but a tool that works from within the VM. There are several of those available for the OS you are running within the VM.

Btw I liked vReplicator as it performs a neat replication and also allows failover in its management interface. However for some reasons it fails to replicate when the source virtual machine has prior snapshots taken.

Many 'backup' based tools do the same thing. They do not like snapshots as they create their own snapshots and copy JUST the main disk, which could ignore all other snapshots in play.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

Blue Gears and SearchVMware Pro Blogs: http://www.astroarch.com/wiki/index.php/Blog_Roll

Top Virtualization Security Links: http://www.astroarch.com/wiki/index.php/Top_Virtualization_Security_Links

[petedr]

This is an overview of esXpress and replication. It uses its backups to perform the replication and it is a vmdk replication. We usually recommend for the VMs to be registered ahead of time on the target host and then esXpress can replicate the vmdks.

With the current 3.1 product it does what we call simple replication. What that means is from the source host a VM is backed up ( Delta/Full backups ). These backups are sent to a target ( ftp/smb/ssh or shared vmfs ). Then the replicated to host looks for those backups and automatically restores them.

With the upcoming 3.5 product it will do what we call advanced replication from our new DeDup backups. With advanced replication the backups will be sent to our DeDup appliance and then the replicated to host will look for those backups and do the automatic restore. However the difference now is it now inject only the changed blocks into the restored to vmdk making for a much faster restore process.

If you have any other questions about esXpress you can always private message or email me.

Pete@esXpress

[Gostev]

Hello,

Last time I looked it did not. No 'straight' VMDK backup tool provides 100% of this capability. Veeam Backup has the tools to do replication but I am unsure if it registers and powers on the VM as well. vReplicator is the other one that I know about.

Hello, thank you for mentioning our product.

Yes, Veeam Backup does that (automatic registration and power on). We do provide with a one-click failover to a replica from UI - both to the latest state, and earlier point of time (which is unique capability to the market). This could come quite handy in case of software disaster, when you notice issue/corruption a few replication cycles after it happens (with other replication solutions, failing over to latest state in this case will do you no good).

As for Veeam Backup replication engine, it provides true block-level incremental replication, and does not require source VM to run in snapshot mode. It does, however, requires target VM to be powered down. puzzledtux, if you have any questions about Veeam Backup replication, please send me a personal message and I will be glad to answer those (or better yet, you can post those on Veeam forums!).

Thank you!

[puzzledtux]

Thank you guys for all the information! I shall get back to all of you once I manage to set things up at my end.

Regs

Avinash

[puzzledtux]

@Gostev

I have been trying the Veeam Backup tool for replication purpose. Is there a way I can replicate the virtual machine from source onto a backup ESX host with only changes in ESX networking options. Like for eg. I have a virtual machine registered and powered on in source ESX host esx37.com.

I want the replica virtual machine to be powered on esx22.com (target/backup) with another network i.e VMNetwork-22. (this port group is configured on host esx22.com)

Source esx37.com:

Virtual Machine: Windows 2003 Enterprise Server

Network: VMNetwork-37

Datastore: Storage1 (SAN storage)

Target esx22.com:

Virtual Machine: Windows 2003 Enterprise Server_REPLICA

Network: VMNetwork-22

Datastore: Storage2 (SAN storage)

[Gostev]

Hello Avinash, you can do this, but you will need to change the network setting on replica manually, right before initiating the failover. This is because all changes to replica's VM settings (VMX file) will be overwritten during each replication cycle. I will investigate if changing replica network is something we could easily automate to better support your use case.

[puzzledtux]

Hmm..changing the network settings manually before failover works fine and the cluster virtual machines defined within that port group communicate properly in fenced mode as per my requirement. Its a tough task when the cluster size grows though. But now I am bigger problems as I missed earlier to gather information that the virtual machines which are managed by the VMware Lab Manager fail to take snapshots from the VC, this also prevents replication process since snapshots cannot be taken!

Also in Veaam Backup, is there a possible way I can write a small script to change the network settings in replica virtual macines. Since its a "bat" file I cannot figure out a way to do it on Windows. Btw I was impressed with Veeam backup with respect to its ability to capture and replicate the source virtual machine's existing snapshosts onto the target! I think vReplicator doesn't work with existing snapshots on source virtual machines(or I am missing something here).

Also I was looking at something from Veeam which allows to sync data from failover virtual machine to the source virtual machine once its up. I checked vReplicator does this stuff.

Phew! Quite a few road blocks already in the migration test plan so far.