Beacon Probing - Question about ARP

[geico]

Hi All,

We are experiencing lot of HSRP flapping in our customers Cisco switches(Catalyst 6509) connected to the ESX servers. We believe this may be due to aggressive ARP timers set in the ESX servers.ARP traffic are going in to the switches around 200ms intervals and cause the switches drop some of the traffic and high CPU spikes. This is only happening in the switch vlans connecting ESX servers.I tried searching the vmware site to find about the nature of the beacon probing used in vNICsetups thinking this maybe the culprit of ARP generation.Unfortunately i was unsuccessful in finding out any thing describing the type of beacons sent out, pings, ARP, etc..

I appreciate any comments or experienced shared here..

[weinstein5]

I do not think it is beacon probing - best way to check is switch failure detection to to link state only and if theproblem goes away you know what the issue but I do not think that is it - also do you have multiple physical nics connected to your your virtual switches? if you do what is the vswitch load balancing set to?

[geico]

Unfortunately we cant change the failure detection to Link status Only just yet as there are multiple servers involved. The load balancing is set to Route based on Source MAC.

I found that the beaconing is L2 broadcast frames with NIC BIA as the source MAC and the destination MAC is broadcast (see below). But did not see any counters or timers involved for sending out beacons. I am not expert in Vmware but to me looks like the ESX servers are generating huge number of ARP requests even before current ARP times out.

Thanks

http://www.vmware.com/files/pdf/vmi_cisco_network_environment.pdf

The beacon frames are Layer 2 frames, with Ethertype 0x05ff with source MAC address equal to the

burnt-in-address of the NIC card (not the VMware MAC address) and a broadcast destination address.

Frames are sent on every VLAN that the vSwitch is on.

[Erik_Zandboer]

Hi,

Setting the vswitches to "link state only" should not be a problem, even with multiple ESX servers. Why did you use "route based on source MAC" ? This is from the old ESX 2.5 days, not used very much any more (if used at all). Most used is port-ID based balancing, or (if you have etherchannels in place) using the session based balancing method based on source and target MAC. An ESX host should not gegerate that much ARPs, certainly not enough to trip HSRP on a 6509....

BTW, I have seen some very strange behaviour if your firmware revisions within the 6500 enclosure are incompatible. You should check the various firmwares inside for compatibility issues amongst each other. Seen CPU bursts to 100% during alternation of ACLs and stuff (scary!!) if your firmware versions are out of sync... Even complete hangups of the enclosure. I would certainly make sure you run most current release versions of firmware within the Cisco(s).