ESX 3.5u3, Vmotion and isolation - help please.

[NeilP]

Hi all,

We are quite new to Vmware (Well, to ESX and Vcenter anyway) and would appreciate it if you could help with a couple of issues we are having.

After our ESX/Vcenter trial and successful test environment we bought the Enterprise ESX package with 2 new servers (plus one for Vcenter).

Manual Vmotion is working fine however I have a couple of questions and would appreciate it if you could point me in the right direction.

Config..

2x ESX3.5u3 servers and 1 VCenter2.5u3 server.

6 nics in each (not all used).

VSwitch0 - 2 NICs active in team..... VM port group (VM Network), Virtual machines (3 test machines so far), Service console port (Service Console). Connected to our main L3 switch stack.

VSwitch1 - 1 NIC active...... Service console port (Service console 2), VMKernel Port (iSCSI). Connected to a switch with connectivity to our NETAPP Filer via iSCSI. No router here.

Questions.

1) I would like to optimise the configuration to add a separate VMKernel port to move Vmotion to a new VSwitch2. At present, the iSCSI port has been set up for VMotion on vSwitch1.

Does this sound ok and is it worth doing for performance reasons? My question is whether the new Nic needs to be able to see the iSCSI datastore "directly" on the same subnet (No router here)?

Can someone provide an optimum design using between 4-6 NIC's please?

2) I have been doing some isolation testing, unplugging the 2 nics on Vswitch0 for the active host (with the 3 vms on it). I was expecting the host to power down the machines and move them across, but they stay running and dont move. Am I missing a few steps? I have set the HA on the cluster to "Shut down VM" in the host isolation response settings.

Many thanks in advance.

NP

[Yattong]

Hey

1) I would like to optimise the configuration to add a separate VMKernel port to move Vmotion to a new VSwitch2. At present, the iSCSI port has been set up for VMotion on vSwitch1.

Does this sound ok and is it worth doing for performance reasons? My question is whether the new Nic needs to be able to see the iSCSI datastore "directly" on the same subnet (No router here)?

You can and should place vmotion on a separate network from your iSCSI network.

Can someone provide an optimum design using between 4-6 NIC's please?

There are so many ways in doing this, as other posts will show.

One way could be

vSwitch0 - SC - 2 pnics

vSwitch1 - VM - 2 pnics

v-Switch2 - iSCSI / VMotion - 2 pnics (alternate active/standby config)

2) I have been doing some isolation testing, unplugging the 2 nics on Vswitch0 for the active host (with the 3 vms on it). I was expecting the host to power down the machines and move them across, but they stay running and dont move. Am I missing a few steps? I have set the HA on the cluster to "Shut down VM" in the host isolation response settings.

You still have a 2nd Service console port group providing heartbeats to other ESX servers, hence it knows it is not isoldated.

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

~y

[Yattong]

actually,

If you are using the software initiatior and are already providing some type of redundancy for the Service console by placing it onto 2 different networks, then you could get away with just 1 pnic on vSwitch0

vSwitch0 - SC - 1 pnic

vSwitch1 - VM - 2 pnics

vSwitch2 - iSCSI / SC / vmotion - 3 pnics

iscsi having 2 active pnics 1 standby

vmotion having 1 active pnic and 1 or 2 standby

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

~y

[depping]

you would need to unplug all service consoles before isolation occurs.

with 6 nics I would do:

vswitch 0 - 2 nics - Service Console + VMotion

Service console active on first nic and standby on second nic, and for VMotion the other way around

vswitch 1 - 2 nics - iSCSI

Just set it to load balancing on virtual port id, so this vswitch would include a vmkernel and secondary service console

vswitch 2 - 2 nics - VM's

just two nics and a whole bunch of portgroups for your vlans...

Duncan

Blogging:

If you find this information useful, please award points for "correct" or "helpful".

[NeilP]

Thanks for the reply..

Just to clarify a few points before I award - hope you dont mind.

1) If vMotion is placed on a separate network, does it need to be able to access the iSCSI Filer directly?

2) The 6 NIC design you listed has the iSCSI and Vmotion on the same network is that correct?

3) Is it a good idea to have a second service console (I have seen documents stating that it is useful to have 2)? If so, do you have any pointers to ensure a failover takes place when either of the service consoles lose connectivity?

Many thanks again,

NP

[Yattong]

1) If vMotion is placed on a separate network, does it need to be able to access the iSCSI Filer directly?

no, the vmotion network does not need access to storage

2) The 6 NIC design you listed has the iSCSI and Vmotion on the same network is that correct?

yes, if only to add redundancy,

Duncans suggestion also fits the bill

3) Is it a good idea to have a second service console (I have seen documents stating that it is useful to have 2)? If so, do you have any pointers to ensure a failover takes place when either of the service consoles lose connectivity?

Have a look in to the HA best practices pdf.

http://www.vmware.com/files/pdf/VMwareHA_twp.pdf

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

~y

[NeilP]

Thanks Yattong.

The iSCSI/VMotion side of the ESX Server network is isolated from the rest of our network and is not accessible from my or most machines.

If I removed the service console from the Virtual Machine network (on your vSwitch2), would I lose connectivity to the VM's? I know I would not be able to access the ESX servers directly from my machine.

Cheers,

NP

[satishgte]

Hi

As per your question no. 1 iscsi storage traffic and vmotion with one vSwitch is not a good planning.

vSwitch1 = you can team with 2 Phy Nic for iscsi traffic and VM network becasue teaming provide redundancy and loadbalancing, becasue it connected to storage so there is more i/o transfer always.

Create vSwitch 2 for vmotion

[Yattong]

Then you could go with Duncans solution.

Ensuring that both nics on vSwitch0 are trunked to both management and vmotion networks.

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

~y

[NeilP]

Thanks for you help, all..

I am still a bit hazy on a couple of points, but the changes I have made seem to have allowed me to separate my Vmotion and iSCSI. It works manually.

Now onto the isolation failover problems..

NP

[depping]

it's not a problem, it's working as designed.

Duncan

Blogging:

If you find this information useful, please award points for "correct" or "helpful".