Long Distance vMotion

[Dryv]

Hi Guys,

Please help me gain some knowledge here as I am really very confused. I understand (or think I understand) vMotion at the same physical site level... i.e. all hosts connect to the same VLAN for vMotion and all hosts can see the same storage LUNS.... off you go vMotion should be a doddle...

But I'm really struggling with the concept of vMotion Long Distance... i.e. to a different physical site where the same VLAN may not exist and indeed the same storage LUNS may not exist... can someone please (please) explain to me how this concept would work or how such a vMotion activity can be implemented? vMotion to me means being able to move a VM from one host to another without any downtime of the VM, thats why I am struggling to understand the concept of a Long Distance vMotion with different networks and storage at different site.

Confused Dryv

[MKguy]

"Long Distance vMotion" has the same basic requirements and works with the exact same assumptions like "ordinary" vMotions.

You need the same stretched layer 2 network (VLAN) in both sites for your VM network (and possibly for the vMotion vmkernel interfaces of the hosts since routed vMotion is not supported until ESXi 6.0) and both source and destination hosts need access to the same storage LUNs.

It can be done as long as you meed bandwidth and latency requirements for your site links, people have done it since years, but it has some drawbacks and most network engineers will roll their eyes when you say that you want to span a flat layer 2 network across geographically dispersed sites. See:

http://blog.ipspace.net/2015/02/before-talking-about-vmotion-across.html

[Dryv]

MKguy,

Thank so much for clarifying this for me... I just get the impression people use the term without really understanding the complexities of stretching network and san for this to work... so effectively are you saying we need to meet pre-reqs that apply for creating a vMSC inorder to get long distance vMotion working?

Also, if you can help me:

- what does vmotion without shared storage mean? as this implies you dont need shared storage for a vMotion activity? is this even possible? I hear this alot... the term shared nothing vmotion I hear people throwing about a lot!

- with v6 round the corner and supporting routed vMotion, is the stretched storage requirement still there?

so if I said to you :

"yes we can just vmotion VMs from one site to another"

and the underlying environment was configured as follows:

- Single vCenter for ALL sites

- Clusters per site (allowing HA/DRS/vMotion within a cluster only)

- No stretched vLANS between the sites

- Routing in place between all sites

- No SAN replication or SAN stretching between the sites

What would you say to me?

Is there any possible (or even remotely possible) way "we can just vMotion VMs from one site to another"?

Sorry for ranting....

Dryv

[MKguy · Accepted solution]

so effectively are you saying we need to meet pre-reqs that apply for creating a vMSC inorder to get long distance vMotion working?

Yes, all basic clustering requirements like for vMSC apply here as well. It's the same stretched cluster thing with a "cooler" name to imply support for distances longer than a metro area.

- what does vmotion without shared storage mean? as this implies you dont need shared storage for a vMotion activity? is this even possible? I hear this alot... the term shared nothing vmotion I hear people throwing about a lot!

vSphere 5.1 introduced shared-nothing vMotion ("enhanced vMotion"), that is basically storage vMotion and compute vMotion at the same time over the network. You don't need shared storage between source and destination host but you still need layer 2 connectivity for the vmkernel and VM networks. Obviously this will take a lot more time than a regular vMotion with shared storage because you need to pre-copy the whole VM disks. See here for more info:

http://wahlnetwork.com/2012/11/21/leveraging-vmwares-enhanced-shared-nothing-vmotion/

so if I said to you :

"yes we can just vmotion VMs from one site to another"

and the underlying environment was configured as follows:

- Single vCenter for ALL sites

- Clusters per site (allowing HA/DRS/vMotion within a cluster only)

- No stretched vLANS between the sites

- Routing in place between all sites

- No SAN replication or SAN stretching between the sites

What would you say to me?

Is there any possible (or even remotely possible) way "we can just vMotion VMs from one site to another"?

In this scenario you technically can, through the enhanced shared-nothing vMotion mentioned above. But without stretched VLANs your VM won't be reachable in the other site unless you do some fancy magic at the physical network layer or decide to manually re-IP the VM or implement network virtualization like NSX.

Still, any type of vMotion between hosts is only officially supported if the vMotion vmkernel interfaces are on the same layer 2 broadcast domain. Apparently you can get a kludgy VMware support approval like this http://cumulusnetworks.com/blog/routed-vmotion-why/.

This will change with vSphere 6 which officially supports routing vMotion traffic though.

Also note that in this scenario there is no DRS or HA between sites since your clusters are split. But the usually the whole idea behind vMSC or its longer distance brother is to have a single stretched cluster.

I suggest you read some of the posts from Ivan I linked in my post before where he makes some good rants about the whole topic from a networkers point of view.

[Dryv]

Perfect Perfect Perfect!

So just read the link on enhanced vMotion, thats awesome...the example is using datastores presented over NFS... but I'm assuming it would be the same for FC presented LUNS?

Interestingly as well in the article, it mentions all hosts are in the same cluster... which we dont have... so could this still work? Even when v6 comes out and routed vMotion is supported, I still the same issue in getting the VM to be accessible once it goes over...maybe they are building in an IP change (like SRM) when doing such a migration.

Going through Ivans stuff now... really interesting and exactly my concerns.

Appreciate your time.

Dryv

[MKguy]

So just read the link on enhanced vMotion, thats awesome...the example is using datastores presented over NFS... but I'm assuming it would be the same for FC presented LUNS?

Yes, it's independent from the underlying storage types. You can migrate from local to local, local to FC, FC to iSCSI, iSCSI to NFS or whatever.

Interestingly as well in the article, it mentions all hosts are in the same cluster... which we dont have... so could this still work?

Yes, it still works provided all normal vMotion requirements that apply within a cluster as well are met (hosts with same CPU generation/EVC level, same dvSwitch/portgroup name etc). You can test of that with your existing 5.1/5.5 infrastructure already.

I still the same issue in getting the VM to be accessible once it goes over...maybe they are building in an IP change (like SRM) when doing such a migration.

That's definitely not happening, the whole process is basically the same as usual, it's oblivious to the network on the other side (as long as same dvSwitch/portgroup name) and assumes it "just works". With v6 you can change the destination (d)vSwitch/portgroup of a vMotion as well, and even vMotion to a host managed by another vCenter, but it won't account for re-configuring the IP inside the Guest OS.

Here is some more info on the v6 vMotion enhancements:

vMotion Enhancements for vSphere 6 Announced - Wahl Network

http://www.wooditwork.com/2014/08/26/whats-new-vsphere-6-0-vmotion/

[Dryv]

MKGuy! How do I become as knowledgeable as you?

...clearly a master for a reason.... I need to work with people like you...

Thank you so much for your help.

Dryv.

[MKguy]

Hah, there are many more knowledgeable people in this community or elsewhere who would be more fun to work with as well, but I'm glad if I could help out.