esxi 5 server flapping ips

[gaganvmware]

Could you help me to understand

Network team  reported flapping in the router logs were 0050.56b2.6200 > 10.101.192.57 > xyzserver1

  4 :44:36 PM: johny: 0050.56b2.6201 > 10.10.19.58 > xyzserver2

15 PM: johny: 0050.56b2.000f > 10.10.19.40 > xyzserver3

I checked logs and could not find any discrepancies that server changed their ips at the day and time network team mentioned . what could be the reason. all esxi servers connected to vlan 15 on the physical switch and esxi service console did not configure any vlan tag .its set None. 

Where i can go and find the reason or its network router issue ?

[rickardnobel]

From what I understand these IPs belongs to virtual machines? Did the network team report any more information, as in if they actually saw any issues coming from this?

Did it happen just once or have been repeated over time?

[gaganvmware]

it happened only once on 5/9 and never happened before and after that. No , flapping did not create any issue as i did not find anything in logs .it just came in the router log and they sent us to explain . is there anything i can find them in log as i checked every where.

[rickardnobel]

I think there is little to less information to say anything. It is not even clear if there are "flapping" in the IP-to-MAC tables or the MAC-to-port tables.

The VMs are of course moving sometimes with vMotion and possible this at some point confused some network device and caused the logs. If it just was on a single occation I would guess that you should not worry too much.

[gaganvmware]

" The VMs are of course moving sometimes with vMotion and possible this at some point confused some network device and caused the logs. If it just was on a single occation I would guess that you should not worry too much.""

Is there any other reason can be for host ips "flapping" other than you explained above ?

VMware KB: Physical Link Down with Loss of Redundancy

Note: These alarm can also be triggered if there is a limit placed on the the Broadcast traffic on the physical upstream switches. ESX/ESXi hosts rely on Broadcast Traffic and if there are limits set, it may cause flapping on the vmnic during vMotion

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=101281...

i verified everything with network team and could not find anything ?

[rickardnobel]

Do you know if you are using "Beacon probing" or "Link State" on your vSwitches as network failure detection?

Ordinary vMotion should not generate much broadcasts * and should not in normal situations not be seen as flapping by the physical switches.

* 10 broadcasts per vMotion is sent, with fakes broadcasts from the vmkernel pretending to be the VM: http://rickardnobel.se/vswitch-notify-switches-setting

[gaganvmware]

I set link state only with route based on the originating virtual port id ., yes ,yes.

where i can find this issue in vmware logs as i found almost everywhere. network team gave up . they said we gave you what we saw on device.

[rickardnobel]

gaganvmware wrote:

I set link state only with route based on the originating virtual port id ., yes ,yes.

The Network Failure Detection setting is something else than the NIC teaming policy, i.e. either "Link State" or "Beacon Probing". Could you just check that you have the default of Link State?

[gaganvmware]

its link state . i need to dig dipper to find the reason for network flapping issue.can i find this in any log ?

more inofrom network team

Host 0050.56b2.0010 in vlan 192 is flapping between port Gi4/35 and port Gi4/37

[rickardnobel]

gaganvmware wrote:

Host 0050.56b2.0010 in vlan 192 is flapping between port Gi4/35 and port Gi4/37

Here is something quite specific at least. Do you know what ESXi ports that connects to those switch ports above? You would need to locate that to tell which vSwitch and vmnics are involved into this.

[gaganvmware]

given above mac address currently belongs to window vm attached to the g 4/15 has been migrated from g/14 ( corrected ports now.. wrong above ) .

""" MAC  0050.56b2.0010 was learnt on gi 4/14 and now it is being learnt on gi 4/15, the reason why the message says the MAC flapped."""

this mac belongs to window vm that has to be migrated between esx servers in cluster and this mac vm was migrated 7 times on 5/13 .

is this a big deal ? is this normal .. right ?

drs migrates these vms in cluster among esx server for load balance .. network team saying its issue .. as The Switch does not expect the a MAC address to switch ports.

??

[rickardnobel · Accepted solution]

gaganvmware wrote:

this mac belongs to window vm that has to be migrated between esx servers in cluster and this mac vm was migrated 7 times on 5/13 .

is this a big deal ? is this normal .. right ?

If you know that the VM with the actual MAC address was in fact moved by DRS / vMotion several times that day and you know the two switch ports mentioned in the log is connected to the two ESXi hosts then everything is perfectly normal.

It might seem a bit unusual for the physical switch that a MAC address changes location several times during the day, but it is nothing more "strange" than a user walking around a large office and connecting to the physical network at different locations, sends some packets, and then disconnect and later connect at some other physical position in the network. The switches should have no problem with that fact.

The only important is that the MAC address must never be learned from different locations at the same time. That would mean you have a network layer two loop, which is very dangerous.

In your situation I would say you can safely ignore the switch error log.