VMware Cloud Community
mogmios
Contributor
Contributor

Infiniband not working..

vmnic0    0000:01:00.00 nx_nic      Down 0Mbps     Half   00:25:b3:00:a2:98 1500   NetXen HP NC522SFP Dual Port 10GbE Server Adapter
vmnic1    0000:01:00.01 nx_nic      Down 0Mbps     Half   00:25:b3:00:a2:9c 1500   NetXen HP NC522SFP Dual Port 10GbE Server Adapter
vmnic2    0000:08:00.00 e1000e      Up   1000Mbps  Full   00:1b:21:78:7e:01 1500   Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
vmnic3    0000:08:00.01 e1000e      Up   1000Mbps  Full   00:1b:21:78:7e:00 9000   Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
vmnic3.p1 0000:03:00.00 ib_ipoib    Down 0Mbps     Half   00:02:c9:21:11:ad 1500   Mellanox Technologies MT25208 InfiniHost III Ex
vmnic3.p2 0000:03:00.00 ib_ipoib    Down 0Mbps     Half   00:02:c9:21:11:ae 1500   Mellanox Technologies MT25208 InfiniHost III Ex
vmnic4    0000:09:00.00 e1000e      Up   1000Mbps  Full   00:1b:21:78:7e:03 1500   Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
vmnic6    0000:09:00.01 e1000e      Up   1000Mbps  Full   00:1b:21:78:7e:02 1500   Intel Corporation 82571EB Gigabit Ethernet Controller (Copper)
vmnic7    0000:0b:00.00 igb         Down 0Mbps     Half   00:30:48:cc:f5:aa 1500   Intel Corporation 82575EB Gigabit Network Connection
vmnic8    0000:0b:00.01 igb         Down 0Mbps     Half   00:30:48:cc:f5:ab 1500   Intel Corporation 82575EB Gigabit Network Connection


I've added two Infiniband cards to each of my ESXi 4.1 servers but have been unable to get even a link light to come on although the cards show in ESXi. For the Mellanox cards I had to install some beta drivers I got from the company and that seemed to work without a problem. For the NetXen cards they seemed to be found by ESXi out of the box.


If I connect my iSCSI device to my switch by the same cables the link lights come on and everything seems to work but if I plug into the ESXi machines there is nothing. The only way I've yet got a link light out of them is by plugging the same cable into the two ports on the same card.


Is there something else I need to do?

Also, is there any way to cause the pnics to be listed in a more logical order and to stop moving around when I apply system patches? Something similar to Linux where you can assign a specific device MAC to an interface?

0 Kudos
1 Reply
mogmios
Contributor
Contributor

FYI for anyone interested. You absolutely must have a subnet manager running on the network before Infiniband will show a physical connection. We decided Infiniband was a pain and switched to plain 10G ethernet which has been working great.

0 Kudos