VCenter 2.5 Networking advice

[Hogwilde1]

I'm Currently running VCenter Server 2.5 with 3 hosts. I am currently noticing that the VMKernal (ISCSI) even though the switch has 2 network adapters it seems to be only actually using one adapter. I have configured the storage paths to Round Robin to test it. I am just wondering if anyone would have any other suggestions for evening out the network card usage.

all of my host currently have the following network setup

Service Console on production network with 1 adapter

VM network with 3 adapters on the production network

Service Console (2nd) and VMKernel with 2 adapters on a separate isolated physical switch for ISCSI

Any helpful info would be greatly appreciated

Message was edited by: Hogwilde1

I should probably add that my ISCSI target has 3 network adapters attached to the physical ISCSI network switch and that VCenter sees 3 paths to each datastore

[yann_bizeul · Accepted solution]

Hi,

Do you provide 2 distinct IP network for different iSCSI paths ?

I'm not sure about this, but I had to configure xen server with NetApp iSCSI storage, and it looked like the best practices were to put two ethernet cards of the host on two distinct IP networks (and eventually two distinct switching hardware).

The NetApp controllers were configured the same way, each port of each controller on the two different networks (and one port of each controller connected to different switch).

The idea is to be able to ping two remote iSCSI controllers through two distinct hardware path.

Now, I guess that if you have only one controller on your storage device, you won't we able to use different networks, but in that case, I don't see how the TCP/IP stack will be able to balance traffic on two different cards that are on the same network. AFAIK, the TCP IP stack chooses one arbitrary adapter for a designated network and will stick on it.

Let me know what you find, I'm still trying to figure that out, but this config worked fine with my customer

Message was edited by: yann.bizeul

I should add that you posted on the wrong forum. This question is not specific to vCenter, but more related to vSphere ESX storage connectivity

[Hogwilde1]

yann,

In your configuration you must need 2 VMKernals mapped to 2 different IP address'?

I would assume that since my Hosts see 3 separate paths to each defined ISCSI storage drive

that it sees 3 different IP address' (IE the 3 network cards on my ISCSI target). I see the 3 different

HBA's (IE VMHBA32:0:0, VMHBA32:2:0, and VMHBA32:3:0) is there a way to derive the IP address'

related to each?

Is there a way to copy or move this thread to the one you suggest?

I am kind of new to the form.

[yann_bizeul]

I think that if you see 3 different paths, your network config is ok.

Though it Is not sufficient to get traffic to got on all paths : your storage server must be able to accept traffic on all port, on dual controller storage system, this is usually not the case.

If so, you must set the multipath policy to round-robin in your datastore property, in the "manage path" subwindow.

I don't know if you can move a topic

[AWo]

As far as I know round robin uses one path at a time but changes the path after a specific amount of I/O's.

AWo
VCP / VMware vEXPERT 2009

\[:o]===\[o:]

=Would you like to have this posting as a ringtone on your cell phone?=

=Send "Posting" to 911 for only $999999,99!=

[yann_bizeul]

Makes sense...

Does ESX provide a concurrent IO mechanism on iSCSI or FC links ?

[AWo]

Does ESX provide a concurrent IO mechanism on iSCSI or FC links ?

RR is the only one for load balancing.

Have you connected each NIC with its own VMkernel port and then associated both VMkernel port with the initiator? That is needed for load balancing with the software iSCSI.

AWo
VCP / VMware vEXPERT 2009

\[:o]===\[o:]

=Would you like to have this posting as a ringtone on your cell phone?=

=Send "Posting" to 911 for only $999999,99!=

[yann_bizeul]

I was not in the same situation, but that's what I'd do, yes.

Now, if I understand how RR works, some I/O will be sent to a path, then after a while, it will be balanced to another path. Which does not provide twice the bandwidth am I right ?

But I understand too that this policy is set "per-lun", so an answer may be that you should create two datastore so that while one is commiting I/O on a path, the other will use the other path. Does that makes sense ?

[AWo]

Yes, that's my understanding, too. When one path is used at a given point of time and the path used is changed over time, only more LUN's will lead to load balancing.

AWo
VCP / VMware vEXPERT 2009

\[:o]===\[o:]

=Would you like to have this posting as a ringtone on your cell phone?=

=Send "Posting" to 911 for only $999999,99!=

[Hogwilde1]

I inserted and attached a diagram hopefully it comes through :-). The extra VMKernal must use extra CPU and memory resources on the host but it is much? In the scenario that I diagramed I would end up with 6 paths to each LUN 3 on each IP subnet.

[yann_bizeul]

We can't see anything in your emf file You should save it with a better resolution and post it using JPEG.

I think the overhead of adding VMkernel ports is not significative

I don't see much in your schema, but if each LUN is attached to 3 NICs, you would end up with 3x2 path for each, that's true (Two NICs on host, 3 NICs on target). Now I don't see what are the 4 objects between the switch and the LUNs. If those are NIcs, and each can make I/O on LUNs, then you ill have 8 paths.

FYI, the max paths for a iSCSI LUN is 8

[Hogwilde1]

I tried saving it as a JPEG but the text didn't come through.

I have 3 LUNS on one target with 4 nics if I setup 2 subnets and put 2 nics on each subnet (=4 paths?)

then on the Hosts setup the 2 VMKernals one nic on each subnet (=2 paths?) your right the total would be 8

Would I then use Round Robin or Preferred Path?

Should I check into MPIO on the target that is using the Microsoft ISCSI software?

Thanks

[yann_bizeul]

Ok, iSCSI MPIO is not thats simple.

Take a deep breath :

http://virtualgeek.typepad.com/virtual_geek/2009/09/a-multivendor-post-on-using-iscsi-with-vmware-vs...

[Hogwilde1]

Thanks for the link to the article.. That was very helpful and gives me one more reason to upgrade to VSphere 4.