PaulEsson
Contributor
Contributor

Moving VMs with RDMs between clusters

Jump to solution

Folks,

We have two, two-node clusters with all nodes running ESX3.5 U1. The clusters have different CPU types which prevent VMotion between clusters. All nodes are FC attached and share storage on an EMC CLARiiON CX3-80. Currently the VMFS datastores, which consist mostly of OS disks, are shared across the clusters but the RDMs (data drives) are presented specifically to one or other of the clusters. We are planning a significant re-organisation of our infrastructure that means effectively swapping clusters for the majority of our VMs. Is there are more efficient means of migrating than detaching the RDMs, cold migrating the VMs and adding the RDMs back in on the other side?

As you can tell my experience of VMware internals is currently fairly limited, but I'm learning - only sometimes the hard way!

Regards,

Paul Esson

Tags (3)
0 Kudos
1 Solution

Accepted Solutions
dmauder
Enthusiast
Enthusiast

Hi Paul:

Everyone here has great suggestions. However, cold migrating a VM that has RDM's will effectively convert or clone the contents of the RDMs into a Virtual Disk. The original contents of the RDM will be left intact, and the only way to avoid the conversion with a cold-migrate is to remove the RDM's from your VM configuration first (as you already mentioned). Using Storage Vmotion will only move the RDM pointer files and effectively does nothing with the RDM's themselves. You'd still have to move them on your storage array to present to the other cluster.

Check out the following KB article for a little more detail:

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1005241&sl...

-Dave

-Dave

View solution in original post

0 Kudos
9 Replies
Gerrit_Lehr
Commander
Commander

Hi Paul, welcome to the forum.

You can use storage-vmotion to move the VMs from one datastore to another. However, this will not change the ESX Server is is executed on but this can be changed afterwards. Another approche, wich will require a downtime to switch over is the vmware converter. But SVmotion should be the way to go. In 3.5 there is only a third party GUI for svmotion but using the rcli for svmotion is pretty simple, too.

Kind Regards,

Gerrit Lehr

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".

Kind regards, Gerrit Lehr If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".
0 Kudos
BryanMcC
Expert
Expert

Unfortunately due to the CPU differences you may not get away with anything but a cold migration.. You can however present the LUNs that you use for your RDMs to all of your hosts and cold migrate with the RDMs attached.. We did that exact thing here before using EVC.

Help me help you by scoring points.

Help me help you by scoring points.
0 Kudos
BryanMcC
Expert
Expert

Gerrit:

I believe he is stating that the RDMs are the only LUNs that all of the hosts cannot see. If that is the case he simply needs to present them to the other hosts and do a cold migration for the machines due to the CPU bit mask differences.

LUNs are LUNs and as long as your hosts see all LUNs including the RDMs then there should be no need for sVmotion. The cold migration should be relatively quick considering there is no machine activity in memory so you would obe looking at seconds for migration after the RDM Luns are presented to all hosts.

Help me help you by scoring points.

Help me help you by scoring points.
0 Kudos
dmauder
Enthusiast
Enthusiast

Hi Paul:

Everyone here has great suggestions. However, cold migrating a VM that has RDM's will effectively convert or clone the contents of the RDMs into a Virtual Disk. The original contents of the RDM will be left intact, and the only way to avoid the conversion with a cold-migrate is to remove the RDM's from your VM configuration first (as you already mentioned). Using Storage Vmotion will only move the RDM pointer files and effectively does nothing with the RDM's themselves. You'd still have to move them on your storage array to present to the other cluster.

Check out the following KB article for a little more detail:

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1005241&sl...

-Dave

-Dave

View solution in original post

0 Kudos
PaulEsson
Contributor
Contributor

Dave,

Thanks for the document link, its the confirmation I was looking for.

Regards,

Paul Esson

0 Kudos
BryanMcC
Expert
Expert

I want to be sure that everyone is on the same page as I did not read in your post that you were trying to reloacte storage for the VMs... Here is the info from the link above..

Cold Migration

With file relocation:

  • Any non-RDM virtual disks are physically moved to the destination.

  • The virtual machine configuration files are physically moved to the destination.

  • RDMs or Raw LUNs themselves, cannot be moved, as they are raw disks presented from the SAN. However, the pointer files can be moved. The contents of the RDM are copied into a VMDK file on the destination, effectively converting or cloning it into a virtual disk. This also applies when the virtual machine is not changing hosts. The original RDM is left intact.

  • If you wish to cold migrate a virtual machine without cloning or converting its RDMs, remove them from the configuration of the virtual machine before migrating. Re-add them to the configuration when completed.

Without file relocation:

  • The virtual machine registration changes, but the files are left untouched.

The RDM and files would not move or a VMDK would not change as long as you were only changing hosts and not datastores.

Help me help you by scoring points.

Help me help you by scoring points.
0 Kudos
PaulEsson
Contributor
Contributor

Bryan,

You are correct it is not my intention to relocate storage. I had initially thought about the potential of simply presenting the RDMs to the other cluster as you suggested but did not fully understand how the relationship would be maintained on the other cluster. Presumably I have to be able to keep the LUN IDs consistent, what if I can't do that?

Regards,

Paul Esson

0 Kudos
BryanMcC
Expert
Expert

ESX works with LUN UUID and not LUN ID so I am not sure it would cause an issue...

You could always present the RDM luns to the allclusters and go through validation during the cold migration. If it is an issue it will tell you then. If it is then simply remove it and readd it post migration.

In the future it is probably a good idea to present all LUNs including RDMs to all hosts in the same datacenter unless you are using some crazy number of LUNs like 256.

It leaves fleibility for tasks like you are trying to perform now.

Hope that helps...

Help me help you by scoring points.

Help me help you by scoring points.
PaulEsson
Contributor
Contributor

Bryan,

The exercise you described in an earlier post was that between clusters and do you recall if the LUN IDs were consistent?

Regards,

Paul Esson

0 Kudos