BaluRP
Contributor
Contributor

Intermittent network issues particularly with linux vm's

Dear members,


I am experiencing a weird network connectivity issue in our environment.
Really appreciate if you could go through below details and provide your valuable suggestions.


Scenario:

1. Linux vms will lose network connectivity after vmotion to another host

2. Observed both on machines with single ip as well as multiple ips.

3. On machines with multiple ips, the connecivity will be fine on primary ip ( eg: ifcfg-eth0)

   but other virtual interfaces will face issues (eg: ifcfg-eth0:0, ifcfg-eth0:1, ifcfg-eth0:2 etc)

4. During the issue, ping from virtual interfaces to default gateway will not work

5. Sometimes existing ssh sessions would remain fine but new sessions will not connect

6. Observed only on linux vms ( redhat/centos 6 and 7 )

7. When this happens to a particular vm, no connectivity issues observed on peer vms or on host

8. VM adaptor is vmxnet3

Workarounds:

1. Restart network service ( will break existing connections)

2. Add a new virtual network interface from vsphere console

I cannot isolate a particular area of issue yet

1. Occurs irrespective of hosts

2. Other vms on the box will work fine

3. All vms are on same dv port group

4. All vm's on same subnet

Configuration:

Dvswitch using 14 uplinks from  7 esxi hosts ( active 7, standby 7)

VM port group settings:

Load balancing: Route based on the originating virtual port ID

Network failover detection: Link status only

Notify switch: yes

Failback: yes

Vcenter : 5.5 build 3660016

Esxi    : 5.5, 3568722

vmtools : 9355, 10240 version is available but not deployed on all machines.

---------------------------------------------------------------------------------------------------------------------------------------------------------

I cannot understand why this issue is resolved, if i add additional network adaptor , later i remove it.

which make me think something is there on virtual adaptors and network stack on linux vms.

Does anyone know any similar issues with linux vms.

what I can do to test, investigate, or troubleshoot the issue?


Thank you very much.

0 Kudos
4 Replies
Alaponuko
Contributor
Contributor

maybe is not related, but you can check MAC conflict if are manual or try to rename the virtual switch in the different ESXi using the same names in all,

I mean if you have 1 ESXi with two virtual switch call Sw1 and Sw2 and the second ESXi use the same names for the virtual switch.

I hope this help you

0 Kudos
Bala_16
Contributor
Contributor

Thank you for your reply.

The mac address is assigned from vswitch itself, i have double checked for any duplicates though.

The vms are on dvswitch and all standard switches are with same name.

0 Kudos
vHaridas
Expert
Expert

can you remove original Virtual NICs from VM and Add new NIC, configure IP settings.

Then try to vMotion and see what happens.

Please consider awarding points for "Correct" or "Helpful" replies. Thanks....!!! https://vprhlabs.blogspot.in/
0 Kudos
BaluRP
Contributor
Contributor

Thank you Haridas.

I did that once and didn't make any changes. this particularly happen with linux vms with virtual interfaces inside OS.like eth0:1, eth0:2

primary interface will be responding in and out from subnet but sub interfaces not respond to outside subnet.

Last week when i issued ifup eth0:1, it started working.

Sometimes, it will work when i reconnect the vnic to vswitch.

I think this could be related with networking but i am not a network person.

Trying to isolate the the issue to vnics/vswitches.

0 Kudos