VMware Cloud Community
danfr1
Contributor
Contributor

Arbitrary VMotion in a large host farm

Hi,

Suppose I have 300 hosts all managed by a single vCenter. Can VMotion of guests be performed arbitrarily between any 2 hosts? How does one configure the environment to support such a scenario? Assume that no DRS/HA is used, i.e. not clusters are defined.

Thanks,

Dan

Reply
0 Kudos
13 Replies
WaffleSniffer
Enthusiast
Enthusiast

Hi Dan

Assuming you have the correct licensing, you can VMotion VM's between host without problems. VMotion only requires a vCenter server not a cluster.

The requirements to support VMotion are the same as if you had a cluster however... i.e. you need to have the host networking configured identically on all of the hosts either by standard switches or distibuted, and of course have the required VMotion and VM networking portgroups configured. And of course the VM's need to reside on shared storage that is accessible by all hosts.

Hope this helps

Adam

Reply
0 Kudos
danfr1
Contributor
Contributor

Adam,

Thanks for your reply. Just to make sure I understand... With regards to shared storage, can it be configured so that 300 hosts can share storage and thus allow VMotion between them? Is this supported by VMFS?

Dan

Reply
0 Kudos
WaffleSniffer
Enthusiast
Enthusiast

Hi Dan

In a word.. no.

I'm afraid (as far as is stated in the Config Maximums doc) you can only have 64 hosts per VMFS volume, so I can't see how you'd be able to support VMotion between all 300 hosts... but surely this is only hypothetical... you're not actually considering setting this up, right? 😃

Cheers

Adam

Reply
0 Kudos
danfr1
Contributor
Contributor

Thanks, Adam.

Is there a way to get an official response from VMware to this question?

Thanks,

Dan

Reply
0 Kudos
WaffleSniffer
Enthusiast
Enthusiast

Hi Dan

Not sure how easily it would be to get an official response direct from VMware about this (you might get lucky and have an employee post here!)... however, the Configuration Maximums document is official and seems to state that it's not possible. 😃

Best regards

Adam

Reply
0 Kudos
dburgess
VMware Employee
VMware Employee

Certainly the hosts per VMFS is your most obvious boundary, so it is definetly not supported..

You are also approaching the upper limits of the tested VC configuration maximum also which is currently 300. How many VM's would you be planning to host?

I'd be curious to know why you might want to do this? I can conceveive of a scenario with groups of ESX servers bridged by transition lun's which might allow you to pass the VM from group to group via Storage vMotion - so 8 groups of 32 hosts each connected to their own shared LUNS plus one or two hosts in that group that are also sharing a group of LUNS that 'star' the 8 groups together. So a sVmotion to the transition luns, a normal vMotion to one of the other groups, and another sVmotion to get you from the transition LUN could take you to any group in the cluster. With a final vmotion to get you to an arbitary node in the group..

RParker
Immortal
Immortal

so I can't see how you'd be able to support VMotion between all 300 hosts... but surely this is only hypothetical

you confused him, by offering a POSSIBLE scenario, now he thinks it's POSSIBLE despite the official document stating the limit.

Reply
0 Kudos
RParker
Immortal
Immortal

Is there a way to get an official response from VMware to this question?

There is a document that states for the record limits to ESX. That document states that you can NOT have more than 64 hosts sharing a datastore.

Therefore as soon as you connect the 65th host to a datastore you will get an error, and therefore it can NOT be done.

There is your "official" response.

Reply
0 Kudos
RParker
Immortal
Immortal

>Suppose I have 300 hosts all managed by a single vCenter.

Well ESX is a tool that requires planning. Throwing a bunch of hosts and attaching them all to the same datastore is happhazard. So what you need to do is plan out your environment, decided an effective method for sharing your VM's, because you are asking a "hypothetical" question (300 hosts which seem to me to be vaporware at this point) and you wouldn't use vmotion as you describe.

Your VM's are organized in an orderly fashion, and you wouldn't WANT all those hosts sharing the same datastores (for performance reasons, despite the limitations of the product) it's not necessary to set up such a scenario for vMotion.

You also cannot have more than 200 hosts managed by a single vCenter anyway, so this also tells me you need to read the manual and understand it fully before making any crazy configurations. If you plan this out logically you will eventually understand why this proposed idea would not make sense.

Reply
0 Kudos
danfr1
Contributor
Contributor

Thanks for your reply,

Dan

Reply
0 Kudos
dburgess
VMware Employee
VMware Employee

So as Rparker comments - I didn't want to confuse you here - as my first line suggest this is definetly an 'edge' case and is not supported as you describe.

My thoughts about the star shaped grouping is an idea that might be made to work if you had a sufficiently good reason (I can't think of one) but our customers are always challenging us with new ideas - like can I run XP on ESX?, that question has started a whole industry!

You would have to drop the concept of a cluster completely as it has it's own limits in any case and you cannot at the moment vmotion or svomotion across a cluster boundary. I am genuienly interested in why you might want to do this though, as always there may be another way to skin the cat..

Reply
0 Kudos
danfr1
Contributor
Contributor

Hi,

How about the following scenario:

Place each VM in a separate volume. When wishing to do a VMotion, mount the volume on the destination host, perform the VMotion and then dismount the volume from the original host. The downside is a huge number of volumes, however, in this fashion VMotions can be performed arbitrarily among a large number of hosts. mount-vmfs and umount-fms are used.

Thanks,

Dan

Reply
0 Kudos
hrobinson
VMware Employee
VMware Employee

Dan,

Our ears are burning. I'd surely like to understand the use case why you would want to do this. Yes, it is correct that today VMFS accommodates up to 64 ESX hosts. The suggested star configuration is ingenious and would allow you to scale beyond that. I guess my question is why you would want to do this?

Thanks

Henry Robinson

VMware

Reply
0 Kudos