networking vmotion and iSCSI question

[jzsjr]

Hello,

I have read several threads in the archives concerning how to setup vmotion and how to setup iSCSI but I was hoping someone could help me with my specific situation.

I have currently two ESX 3.5 host and am in the process of setting up my third. The first two were set up by an outside contractor. I am setting up the third one myself and reading a great deal about the process. In my readings I am questioning the initial setup of the first two host. Please look at the attached image for the details. Under vSwitch0 there is the service console and an item that says it is the iSCSI link but it is not a vmkernel port which in setting up the third esx host puzzled me greatly since I am trying to mimmick the first two configs. My conclusion is that the port group labeled VMkernel-iSCSI is not doing the iSCSI communication but that is all being handled under vSwitch2 which is a real VMkernel port; thus the vSwitch2 is handling both vMotion and iSCSI traffic.

I am aware that the vmkernel port for iSCSI should be under the vswitch0 (for chap ability) with the service console and that the vmotion should be on it's own switch using a vkernel port also (but on a different subnet).

My question is is this the right conclusion and how can I fix this without hosing my iSCSI connections and also still have the ability to vmotion? Is it as simple as unchecking the vmotion box under this vmkernel port and relabeling this one for iSCS and then setting up a new vmkernel port for vmotion using a different nic and subnet?

thanks,

Jim

[weinstein5]

You are correct that should be a vmkernel port - I am betting what is happening is the vmkernel port that is enabled for vmotion is being used to communicate to the iSCSI target - as long as the service console port can route to the iSCSI target they vmkernel port does not need to coexist on the same virtual switch -

To fix the issue I would use the power of vmotion - vmotion the vm's to one of the hosts - if you want to configure it the way you describe - on the host that now has no vm's add a vmkernel port to vSwitch0 with an IP address can get to the iSCSI target preferably on the same segment - change the up address on the vmekrnel port used for vmotion - rescan the iSCSI SAN to make sure all still works - if it does - connect to the ESX host and change the IP address on the vmkernel port used for vmoiton and vmoiton the vms back tot he first host - repeat rhe changes so they are configured similarly -

Some things to keep if you are using DRS set it to manual and now your physical networking to make sure the entwork paths are valid -

[jzsjr]

Okay,

If I leave things alone, since the console is routable I guess the best course of action is to move the teamed nic in vswitch0 so it is a teamed nic with vswitch2 (giving the iSCSI and vmotion two nics and the console one). Attached is an image of how I have setup up ESX3 which is yet to be in production so it can work with ESX1 and ESX2.

thanks,

Jim

[Texiwill]

Hello,

Having iSCSI on the same pair of vSwitch as the SC is not a bad idea with the independent vMotion vSwitch. So 2 for SC/iSCSI, 1 for vMotion, 2 for VMs is a good split of your network resources giving you the redundancy on iSCSI you will need.

Ideally you would have 2 for SC, 2 for iSCSI, 2 for vMotion, 2 for VMs.

Or. 1 for SC, 1 for vMotion (tied together on the same vSwitch), 2 for iSCSI, 2 for VMS.

You really need 6-8 pNICs to get full redundancy, performance, and security.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[jzsjr]

I agree 6-8 nics is ideal. I have 6 nics in all the machines but the issue on mine is that vMotion and iSCSI are on the same switch so that is why I gave two nics to that switch. I know this is not ideal but I guess I'll try it out and see what happens. We certainly do not vMotion all that much so I would not think that it would be much of an issue. I really wish there could be two vmkernel ports on the same subnet. That would also be an easy fix. I read where someone was able to do it using command line from the console.

thanks again,

Jim

[Texiwill]

Hello,

You can have multiple vmkernel ports on the same subnet, BUT only one will be used. Usually the one defined for vMotion.... But that depends on routing you can set up specific routes using esxcfg-route for the vmkernel ports. Not something I would do.

Remember how vMotion works, you really want that as fast as possible. Generally, people use 'vMotion/SC' tied together but you need VLANs for that. And iSCSI data on its own vSwitch and Pair of pNICs. But that is the common method. Anything that works, but if you do vMotion you could impact your storage network. I would run some tests and see if there is any impact to other systems.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[jzsjr]

Well, I guess it has been fine for us since this is the way it has been setup for two years. I think teaming two nics for the iSCSI and vMotion that will be on the same vmkernel will help out a good bit.

thanks for all the responses,

Jim