We have an ESX 4 environment with 3 hosts (esx1, esx2, and exs3) all with the same exact network settings (except their IP's, obviously). Along with those 3 hosts we have a NetApp FAS2040 set up as an NFS datastore for all 3 hosts. All of these are connected to 2 switches. Both switches are identical and have the same exact configurations (for redundancy etc.). We also have 3 VLAS set up, one for our local LAN (192.168.200.0/22), one for NFS between the 3 hosts and the NetApp (192.168.100.0/24), and one for VMotion (192.168.50.0/24). The NFS and VMotion VLANs are confined to just those 2 switches. This has been working great ever since it was first set up and still is working great. However, we are adding a new physical storage unit (a FreeNAS box), and this is where our issues comes in.
The FreeNAS box has 2 nics on it. We set up one nic on the local LAN and the other nic on the NFS VLAN. I was able to add the FreeNAS as a datastore to esx2 successfully. We were even able to migrate VM's to it and run the VM's from it. However, esx1 and esx3 refuse to connect to it, throwing the "Cannot connect to NFS server" error. Digging deeper, we tried pinging and vmkping'ing the hosts and the FreeNAS. We were able to vmkping the FreeNAS from the NFS vmk on esx2 and we were able to ping the NFS vmk on esx2 from FreeNAS, but esx1 and esx3 cannot ping FreeNAS and the other way around as well. All 3 esx hosts are able to ping eachother. We have tried everything we can think of and are at a complete loss as to where to go from here.
What would cause this? Any thoughts or suggestions on how to fix it?
