clarification on static routes

[IB_IT]

Hi all,

I have recently aquired a new vmware env.  For mgmt and vmotion they are on two separate subnets, two separate vswitches.  kb 2001426 tells how to set up static routes, however I am getting errors when attempting to add.  I am running esxi 5.0 EDIT: 5.0 Update 2

~ # esxcfg-route -a 10.80.2.0 255.255.255.0 10.80.2.254

Adding static route 10.80.2.0 to VMkernel

Error: Duplicate route to network 10.80.2.0/24 found.  Please delete the old route first.

So I  check the routing, and I only see one entry for the network in question:

VMkernel Routes:

Network          Netmask          Gateway          Interface

10.80.1.0        255.255.255.0    Local Subnet     vmk0

10.80.2.0        255.255.255.0    Local Subnet     vmk2

169.254.1.0      255.255.255.0    Local Subnet     vmk3

default          0.0.0.0          10.80.1.254      vmk0

I attempt to delete this route, however I get the error

Error: Unable to delete route to 10.80.2.0/24 this route is automatically created based on the  IP address and netmask of one of the VMkernel TCP/IP  interfaces

So I guess I am looking for clarification...what exactly does "Local Subnet" mean when looking at the vmkernel routes?  The default gateway for the server is a different subnet, so if the 10.80.2.x route is going thru "Local Subnet", does that mean it's finding it's own subnets gateway?  I'm not sure how it is smart enough to do that.  I've read other posts where ppl with similar issues are told to check esxtop.  I don't see how that would help as all that would tell me is which vmnic is used.  If the routing is screwed up then I don't know how esxtop would tell me.  Anyway, any help would be great.  

[NealeC]

Hello IB_IT

The reason there is already a route to 10.80.2.0 is becuase vmk2 has an ip configured on that subnet.  If that interface is on that subnet there is no need to use a route to get to it.  If ESX needs to send any traffic to 10.80.2.0 it will just use VMK2  no need for a default gateway.

Can you display the config using the following commands

     esxcli network vswitch standard list      # list current vswitch configuration
  esxcli network vswitch dvs vmware list    # list Distributed Switch configuration
  esxcli network ip interface list          # list vmkernel interfaces and their configuration
  esxcli network nic list                   # display listing of physical adapters and their link state

If you can post the output of those it will be possible to explain a little more, but essentially you don't need a route as your host has a NIC with an IP on that subnet.  If that NIC isn't plugged in to that subnet, then it's IP is configured wrong 🙂

[vfk]

Just for clarification, what issues are you experiencing?  As above,  any traffic to 10.80.2.0 it will just use VMK2.

[IB_IT]

No issues at all in terms of performance as far as I can see.  Curiosity at this point and just more of a warm and fuzzy for me to know that it's working correctly before I can check it off and move on. 

[IB_IT]

NealeC wrote:

The reason there is already a route to 10.80.2.0 is becuase vmk2 has an ip configured on that subnet.  If that interface is on that subnet there is no need to use a route to get to it.  If ESX needs to send any traffic to 10.80.2.0 it will just use VMK2  no need for a default gateway.

Perfect thanks...I wasn't sure about this but it sounds like a static route is not needed.  My fear was that since all the hosts can talk to each other on the mgmt interface vmk, and that's the interface that is the default gateway, then that's what it will try to route across.  I ran into this in the Windows world where the only fix was to set persistent, static routes before the traffic started behaving as expected.  It sounds like vmware's rules are different inside ESXi.  I don't suppose there is any way to verify this?  Maybe I could get my network admin to monitor the traffic on 10.80.2 since vmotion is the only thing on there.  See if he sees the traffic when I vmotion.