I've been going through some network failover testing and have discovered that some VMs are connecting to our NetApp SAN via iSCSI but over the LAN connection.
Some config info.
We have a few HP DL380 G4 and G5 Servers.
Each has 6 Gigabit Network cards, (3x2port)
We've setup the following network configuration:
Switch Name Num Ports Used Ports Configured Ports Uplinks
vSwitch0 32 3 32 vmnic0
PortGroup Name Internal ID VLAN ID Used Ports Uplinks
Service Console portgroup0 0 1 vmnic0
Switch Name Num Ports Used Ports Configured Ports Uplinks
vSwitch1 64 6 64 vmnic3,vmnic1
PortGroup Name Internal ID VLAN ID Used Ports Uplinks
vmkernel_nfs portgroup7 0 1 vmnic1,vmnic3
ext_vswitch1 portgroup2 0 2 vmnic1,vmnic3
Switch Name Num Ports Used Ports Configured Ports Uplinks
vSwitch2 64 5 64 vmnic4,vmnic2
PortGroup Name Internal ID VLAN ID Used Ports Uplinks
service_console_iscsiportgroup5 0 1 vmnic2,vmnic4
vmkernel_iscsi portgroup4 0 1 vmnic2,vmnic4
Switch Name Num Ports Used Ports Configured Ports Uplinks
vSwitch3 64 3 64 vmnic5
PortGroup Name Internal ID VLAN ID Used Ports Uplinks
vmkernel_vmotion portgroup6 0 1 vmnic5
So that basically means we have:
vSwitch0 using vmnic0 which is the LAN Service Console which talks to Virtual Center and cold migrations happen
vSwitch1 using vmnic1 and vmnic3 which is the teamed LAN network for the VMs. There is also an NFS connection running through here to connect to a nearstore hosting the Templates and ISO's
vSwitch2 using vmnic2and vmnic4 is the teamed SAN connection. We run a vmkernel and service console connection for iscsi across here
vSwitch3 using vmnic5 is a dedicated VMotion NIC
The LAN and SAN networks are isolated and cannot route between each other
We can connect to the SAN all happily. Our ESX box has the IP address of the SAN card on the filer and does its discovery all well.
After that things seem to be a little strange. It seems as though the ESX box sometimes talks to the Filer for iSCSI over the LAN.
I can see connections on the Filer from the LAN IP address. The filer is also connected to the LAN for NTFS data.
Now it looks as though there could be connections to the SAN but across the VMotion NIC as well.
So my question is, how can I force iSCSI to go only over vSwitch2 as having multiple vmkernel connections means at the moment iscsi can go anywhere?
I know I can get the filer to only accept iSCSI over its SAN connection but this still leaves the possibility that iSCSI will go over the VMotion network from the ESX host which is not teamed and should be dedicated to VMotion.
I'm thinking the discovery that is correctly going over the SAN network is picking up the filer name and then browsing for the filer name which can possibly take two paths, one over the LAN which would use DNS or over the SAN which would be by browsing. Now maybe I could amend the hosts file to only be able to talk to the filer over the SAN IP address but I don't want to go through this and have somethnig else to configure but it would be much better to beable to force iSCSI traffic over vSwitch2
Any ideas?
http://WoodITWork.com
