vmkping to san storage packet loss

[cafe99]

Hi,

    Today, I have one of the ESXi4. Hosts (USB booted, not no physical storage on local) have a problem.

    Datastorage Indentification (inactive).

   ssh to the host and try to do a vmking to the SAN storage.

   When I do a vmkping -D

   I got this :

   But when I vmkping to the ISCSI to the SAN Storage (DELL MD3200i),  I got  packet loss.

I got packet loss and I can not connect to DELL SAN Storage (MD3200i).  so I got no datastorge on this host.

And I have another ESXi4 host with local harddisk I do not have this problem.

Hope someone can give me some help.  Thanks!

Choong Leng

[john23]

- is Managment network and iscsi network is same?

-what it the output of esxcfg-route on the both the hosts??

[cafe99]

Hi John,

   Thanks for the reply.

- is Managment network and iscsi network is same?

  No..  The Management netwwork is on 192.168.1.0/24

          The iSCSI is on 10.10.10.0/24

-what it the output of esxcfg-route on the both the hosts??

  The output of esxcfg-route is the same :

   from host: 192.168.1.66  (with local storage)

   from host: 192.168.1.68 (USB bootup, no local storage)

   Pls help.

[john23]

From this I find configuration side no issue .

One more stuff i also observed.. not all the packets are dropped (2 packet drop) .

What error messages you are seeing in logs while rescanning iscsi?

Did you try from any other network spare port?

[a_p_ · Accepted solution]

Did you already double check the physical connections (cables) from the host to the switch and the switch port settings!?

Are you using Jumbo frames? If yes, how did you configure the MTU size for the physical switch?

André

[cafe99]

Hi John,

>> What error messages you are seeing in logs while rescanning iscsi?

  Where can I locate the logs files ?

  Thanks!

[cafe99]

Hi,

    I did not use jumbo Frame.

    This is a new installation, It was runnin fine for a day, just today this issue coming up.

    I will check the cable and the switch as well.

   Regards,

Choong Leng

[a_p_]

In new installations issues like this often are caused by the network configuration. How did you setup the iSCSI network (using EtherChannel)? Also make absolutely sure spanning-tree is setup correctly (usually spanning-tree portfast).

André

[cafe99]

Ok, I had found the reason why the packet was loss. It was due to one VM that have a very huge latency to the SAN Storage. I kill the VM and re-start the host.

But, the other host still not able to connect to the SAN storage.

I can vmkping to the SAN Storage.

~ # vmkping 10.10.10.241

PING 10.10.10.241 (10.10.10.241): 56 data bytes

64 bytes from 10.10.10.241: icmp_seq=0 ttl=64 time=0.710 ms

64 bytes from 10.10.10.241: icmp_seq=1 ttl=64 time=0.250 ms

64 bytes from 10.10.10.241: icmp_seq=2 ttl=64 time=5.434 ms

--- 10.10.10.241 ping statistics ---

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max = 0.250/2.131/5.434 ms

~ # vmkping 10.10.10.242

PING 10.10.10.242 (10.10.10.242): 56 data bytes

64 bytes from 10.10.10.242: icmp_seq=0 ttl=64 time=0.722 ms

64 bytes from 10.10.10.242: icmp_seq=0 ttl=64 time=0.864 ms (DUP!)

64 bytes from 10.10.10.242: icmp_seq=0 ttl=64 time=0.908 ms (DUP!)

64 bytes from 10.10.10.242: icmp_seq=0 ttl=64 time=0.996 ms (DUP!)

64 bytes from 10.10.10.242: icmp_seq=1 ttl=64 time=0.259 ms

64 bytes from 10.10.10.242: icmp_seq=2 ttl=64 time=0.240 ms

--- 10.10.10.242 ping statistics ---

3 packets transmitted, 3 packets received, +3 duplicates, 0% packet loss

round-trip min/avg/max = 0.240/1.330/0.996 ms

~ # vmkping 10.10.10.243

PING 10.10.10.243 (10.10.10.243): 56 data bytes

64 bytes from 10.10.10.243: icmp_seq=0 ttl=64 time=0.737 ms

64 bytes from 10.10.10.243: icmp_seq=1 ttl=64 time=0.235 ms

64 bytes from 10.10.10.243: icmp_seq=2 ttl=64 time=0.241 ms

--- 10.10.10.243 ping statistics ---

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max = 0.235/0.404/0.737 ms

~ # vmkping 10.10.10.244

PING 10.10.10.244 (10.10.10.244): 56 data bytes

64 bytes from 10.10.10.244: icmp_seq=0 ttl=64 time=0.696 ms

64 bytes from 10.10.10.244: icmp_seq=1 ttl=64 time=7.554 ms

64 bytes from 10.10.10.244: icmp_seq=2 ttl=64 time=0.242 ms

--- 10.10.10.244 ping statistics ---

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max = 0.242/2.831/7.554 ms

~ # vmkping 10.10.10.242

PING 10.10.10.242 (10.10.10.242): 56 data bytes

64 bytes from 10.10.10.242: icmp_seq=0 ttl=64 time=0.277 ms

64 bytes from 10.10.10.242: icmp_seq=1 ttl=64 time=0.266 ms

64 bytes from 10.10.10.242: icmp_seq=2 ttl=64 time=0.243 ms

--- 10.10.10.242 ping statistics ---

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max = 0.243/0.262/0.277 ms

~ # vmkping 10.10.10.243

PING 10.10.10.243 (10.10.10.243): 56 data bytes

64 bytes from 10.10.10.243: icmp_seq=0 ttl=64 time=0.286 ms

64 bytes from 10.10.10.243: icmp_seq=1 ttl=64 time=14.707 ms

64 bytes from 10.10.10.243: icmp_seq=2 ttl=64 time=0.260 ms

--- 10.10.10.243 ping statistics ---

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max = 0.260/5.084/14.707 ms

~ #

~ # esxcli network neighbor list

Neighbor      Mac Address        vmknic  Expiry(sec)  State 

10.10.10.61   00:50:56:74:21:0f  vmk1    1191               

10.10.10.241  78:2b:cb:37:c1:39  vmk1    205                

10.10.10.245  78:2b:cb:34:4e:cb  vmk1    100                

10.10.10.248  78:2b:cb:34:4e:ce  vmk1    95                 

192.168.1.1   00:0c:42:c0:50:fe  vmk0    979                

192.168.1.71  00:0c:29:88:d0:f5  vmk0    1157               

~ #

I still can't connect to the SAN, so I remove the VMKernel network, and re-create the VMkernel Network as before.

Still Can't connect.

So, I use the dell MD3200i Storage Manager to remove the host port identifier for the host and hopefully that I can re-conect by adding it back but I see no unclaim host port identifier.

I even manually enter the host port identifer, but no luck.  I try to scan the vmhba33, still can't connect to the SAN.

Help..

[a_p_]

Well, this doesn't answer my questions!

Please provide detailed information about your network setup (virtual and physical)!

André