VMware Cloud Community
anderwd
Contributor
Contributor

Network Drops - extream newbie

I have recently been hired with a company running VMWare ESX, and we are experieincing a scenario where the network just shuts off for a second. Looking at the Graphs, I see that Average Ram Usage Spikes up and the Granted Dips down below the Usages (seems very strange how can the Granted dip?) at the same time this occurs, everything on the network graph drops to nothing, then comes back on.

I am trying to determine if the server is over tasked and under specked and really have no ideas where to start. All the system status are shwoing me green lights, so it does not appear to be undersized?

Reply
0 Kudos
11 Replies
PaulSvirin
Expert
Expert

I think getting familiar with esxtop tool will be useful to proceed in solving this problem.

Here's the PDF:

---

iSCSI SAN software

http://www.starwindsoftware.com

--- iSCSI SAN software http://www.starwindsoftware.com
anderwd
Contributor
Contributor

Outstanding - I will check it out and see if I make head or tails of it. I would also like to apologize for my vague question and limited specs - I was at the end of my rope for the day and not thinking straight! I will let you know how it goes and get this discussion closed properly.

Reply
0 Kudos
anderwd
Contributor
Contributor

I'll be ding donged it just happened again. I was reading the PDF, watching esxtop, and I had the Real-Time graphs from the VIC running behind, but honestly cannot say that I notcide anything on esxtop, the CPU load looks higer than my other two (bouncing around 38.00 - 40.00) the other two virtual servers 1.5 - 3.0 and 4 - 5 ( this last one is citrix server and I can tell when some logs on, but that is about it). Over all the CPU average is .07, .07, .08. Looking at the vNic dispaly (and I dont think this version matches that pdf - things are a little different or not there on my version)

7:58:30am up 216 days 19:59, 68 worlds; CPU load average: 0.07, 0.07, 0.08

PORT ID UPLINK USED BY DTYP DNAME PKTTX/s MbTX/s PKTRX/s MbRX/s %D

16777217 Y vmnic0 H vSwitch0 15.42 0.01 18.91 0.01

16777218 N 0:NCP H vSwitch0 0.00 0.00 0.00 0.00

16777219 N 0:CDP H vSwitch0 0.00 0.00 0.00 0.00

16777220 N 0:vswif0 H vSwitch0 1.11 0.01 2.38 0.00

16777223 Y vmnic1 H vSwitch0 2015.75 2.36 2019.09 2.91

16777262 N 2219:SERVER1 H vSwitch0 0.64 0.00 2.07 0.00

16777264 N 2224:SERVER2 H vSwitch0 14.31 0.01 17.33 0.01

16777266 N 2213:SERVER3 H vSwitch0 2016.07 2.36 2018.77 2.91

SERVER3 is my problem child - and I see that it shows the most traffic - but if it is showing me something telling about the issue, I am not capable of knowing what that might be.

I have determined that is only occurring on SERVER3 - everything will be going fine, then I get a big V on my real time graph as everything Usage, Transmit rate, and receive rate all drop to nothing - the entire episode takes about 1 minute - or at least just enough time for everything to hose.(Attached a jpeg of the graph)

Reply
0 Kudos
anderwd
Contributor
Contributor

Would it also be possible that I am looking at that totally irrelevant intermittent drop and assuming that it is the cause of my other issues incorrectly.

Reply
0 Kudos
petedr
Virtuoso
Virtuoso

Were there any snapshot processing occurring when you had the intermittent network drops, mainly deletes or reverts.

www.phdvirtual.com, makers of esXpress

www.thevirtualheadline.com www.liquidwarelabs.com
Reply
0 Kudos
anderwd
Contributor
Contributor

No, we are using VRanger Pro to do snapshots at night when no one is working. The server is just running normal daily operations. Got a big one today, the network was down for a good solid minute/minute and a half - but no one was actively doing anything. I was watching PSTools Process Monitor at the same time on the server as I was watching the graphs on VIC - VIC showed the big V drop on the Network - but PS Tools and my RDP connection to the server kept right on going without skipping a beat. So needless to say - the deeper I dig the more confused I get!

Reply
0 Kudos
petedr
Virtuoso
Virtuoso

Thanks for the clarification. Snapshots processing has at times caused intermittent network drops mainly on deletes of larger snapshots which is why I mentioned.

Any other thoughts I'll follow up.

www.phdvirtual.com, makers of esXpress

www.thevirtualheadline.com www.liquidwarelabs.com
Reply
0 Kudos
anderwd
Contributor
Contributor

I think we have determined that it is not the network at all, that it is the single CPU assigned to this VM. How hard would it be to add another Virtual CPU to this virtual machine?

Reply
0 Kudos
petedr
Virtuoso
Virtuoso

You should be able to increase the number of CPUs (virtual process) from the vmware client via Edit Settings for this VM (Hardware/CPUs).

I do think you will need to power off the VM first to make that change.

www.phdvirtual.com, makers of esXpress

www.thevirtualheadline.com www.liquidwarelabs.com
anderwd
Contributor
Contributor

Thanks for all your help and information. I think we will give he a whirl!

Reply
0 Kudos
petedr
Virtuoso
Virtuoso

Not a problem, thanks for the helpful.

www.phdvirtual.com, makers of esXpress

www.thevirtualheadline.com www.liquidwarelabs.com
Reply
0 Kudos