Nic Binding for iSCSI Help!

[ranjitcool]

Hey Guys,

I asked about this before http://communities.vmware.com/message/1876168 but couldn't get far.

I will ask again - lets start with WHY do we need to bind nics for iscsi traffic?

I have two separate vmk vswitches each with 1 nic for iscsi traffic - which are separate than the management vmk and service console vswitch which is on a completely diff vlan and subnet.

High latency was reported, vms became unresponsive, host dropped from vcenter -  the fix was to bind the nics to vmk1 and vmk2, restart management agents and network on the host, then mark a path as disabled (flip the iscsi path) to fix the issue without a host reboot.

So is this a bug? What I think happened or am being told is vmware is pumping iscsi traffic over the management vmk? Why in the heck would it do that unless its a known bug?

Any comments are greatly appreciated.

R

[chriswahl]

IP storage traffic shouldn't traverse the management vmknic unless there are no vmknics on the IP storage subnet. As long as you have created a vmknic on the IP storage subnet, it will pick that one for sending traffic. Otherwise, it uses the default gateway on the management vmknic.

It sounds like you're using different subnets, so just make sure the vmknic for IP storage has an IP on that same subnet as your storage array. I'm not sure why you are having to bind the vmknic. I typically use binding when the management and IP storage subnet are the same (sad but true for some) or when you want to achieve MPIO.

You can watch the traffic traverse the IP storage vmknic using ESXTOP. If you still see iSCSI storage traffic go out the management vmknic, there may be a network issue between your IP storage vmknic and the target array (such as routing issue, jumbo frames, something like that).

[ranjitcool]

Hello,

Thank you for your reply.

I see nic binding scenario you mentioned and that makes sense.

However, how do i differentiate traffic in esxtop for nics - i know the N shows packtes sent and recieved but how to tell what those packets are or were?

I dont recall seeing any packets (high numbers) going through vmnic1 - which is a trunked port for SC and vmk (vmotion).

I do see traffic like 3000tx for vmk2 and a few packets like 50tx for vmk2.

I will look into if I am able to ping from the sc to the vmk ips.

Let me try that - anything else - what about if vmk nics were saturated - what happens then ?

R

[chriswahl]

You could copy a large file over iSCSI and watch ESXTOP, the active vmknic should show a lot of throughput and packets being transferred.

Saturation doesn't matter in this scenario from an uplink selection standpoint. The hypervisor will always route traffic over the first available vmknic on the correct subnet, or over the bound iSCSI vmknic. It doesn't load balance.

[ranjitcool]

Yeah, I got to try that.

Remember vmk1 has only 1 vmnic and vmk2 (separate switch) has a separate nic vmnic.

R

[chriswahl]

In that case it will only use one vmknic on one vswitch, unless you are using 2 different subnets. Once it finds a vmknic on the right subnet, it stops looking any further.

[ranjitcool]

Thanks,

So I guess, the question remains, why vms became unresponsive and why the hyp got kicked out of vcenter.

service console ip range - 10.252.xxx.xxx

vmk1 - 192.168.100.xxx

vmk2 - 192.168.100.yyy

ISCSI Target - 192.168.100.aaa  and .bbb

Do you think may be vmk default route is 10.252.xxx.y  - it may have tried to send traffic to the gateway?

I am trying to see if I am able to ping the target ips from the vm without using vmkping.

R