VMware Horizon Community
Wasisnt
Contributor
Contributor

How to increase VM performance when the ESXi host has plenty of resources

We are trying to increase the performance of a Windows 7 64 bit View VM where the end user runs huge reports etc in Excel. Right now it has 4GB of RAM and 2 vCPUs where the second one was just added.

The host it runs on has plenty of RAM and CPU resources available so I was wondering if there was anything different we can do since the VM doesn't seem to be running low on memory or processor resources.

Isn't there a way to give that VM more dedicated access to the host CPUs so it doesn't have to use the scheduler? And would that do any good in this situation?

I also read that adding a second processor can hurt performance but cant seem to narrow down what situations this applies to.

Reply
0 Kudos
18 Replies
LarryBlanco2
Expert
Expert

Lot's of variables here.  One performance metric over looked is VM to core ratio. I have some older DL 580's and have plenty of horsepower to go around but am running into high CPU ready numbers due to the fact that they just can't handle all the VM's wanting time.  Plenty of cycles, just not enough cores to meet scheduling demands. I rarely go over 50% on CPU but have high ready times.  This translates to poorer performance and the VM's have to wait.

The only thing I can think of right now is setting the CPU resoruce Allocation to High or Custom and adding your own share value.  This will likely have a negative affect on other vm's as their slice of the pie will get smaller as 1 persons grows.

Now if that one person is the CEO or CFO forget the rest and do as your told.  Smiley Happy

Larry

Reply
0 Kudos
Wasisnt
Contributor
Contributor

The hardware its running on is an IBM x3560 X5 with Xeon E7-4870 processors (40 CPUs x 2.4Ghz) and 72GB of RAM.

How do you check CPU ready data?

Reply
0 Kudos
Linjo
Leadership
Leadership

What about storage performance?

// Linjo

Best regards, Linjo Please follow me on twitter: @viewgeek If you find this information useful, please award points for "correct" or "helpful".
Reply
0 Kudos
Wasisnt
Contributor
Contributor

I found CPU ready under the performance tab and then CPU. Its at 0 and has been that way for a long time and hasn't moved at all. ESXTOP shows %RDY at 0.03 to 0.06.

Disk usage is really low too plus the performance issue only occurs when crunching the numbers in the spreadsheet.

Reply
0 Kudos
Wasisnt
Contributor
Contributor

The %WAIT number is around 470 and I found this info from a VMware KB.

Wait, %WAIT:

  • This  value represents the percentage of time the virtual machine was waiting  for some VMkernel activity to complete (such as I/O) before it can  continue.

  • If the virtual machine is unresponsive and the %WAIT value is proportionally higher than %RUN, %RDY, and %CSTP, then it could indicate that the world is waiting for a VMkernel operation to complete.

  • You may observe that the %SYS is proportionally higher than %RUN. %SYS represents the percentage of time spent by system services on behalf of the virtual machine.

  • A high %WAIT value can be a result of a poorly performing storage device where the  virtual machine is residing. If you are experiencing storage latency and  timeouts, it may trigger these types of symptoms across multiple  virtual machines residing in the same LUN, volume, or array depending on  the scale of the storage performance issue.

  • A high %WAIT value can  also be triggered by latency to any device in the virtual machine  configuration. This can include but is not limited to serial  pass-through devices, parallel pass-through parallel , and USB devices.  If the device suddenly stops functioning or responding, it could result  in these symptoms. A common cause for a high %WAIT value is  ISO files that have been left mounted in the virtual machine  accidentally that have been deleted or moved to an alternate location.  For more information, see Deleting a datastore from the Datastore inventory results in the error: device or resource busy (101....

  • If  there does not appear to be any backing storage or networking  infrastructure issue, it may be pertinent to crash the virtual machine  to collect additional diagnostic information.

Reply
0 Kudos
Wasisnt
Contributor
Contributor

And does giving the VM more shares make a difference when there is no competition for resources with any of the VMs?

Reply
0 Kudos
LarryBlanco2
Expert
Expert

Giving the VM more CPU shares when there is no contention for CPU resources will not make any difference.   So we've determined that the CPU is not he issue. 

What about memory.   U have an extrodinary amount of CPU resoruces but just 72GB of ram.  Usually when I see a large server like this I'll see 128-192GB of ram accompaning it.   I know u stated that the server has plenty of RAM.  Jsut verify that memory is not over commited and that the VM is not balloning mmory or swapping to disk.

The graphs would show this information. Something similar to this:

http://communities.vmware.com/servlet/JiveServlet/download/1383571-29371/ballooned.jpg

If there is no balooning or swapping of memory then we need to investigate Storage I/O as being the bottleneck or Network I/O.

Are u using iSCSI, NFS, or FC?

Larry B.

Reply
0 Kudos
Wasisnt
Contributor
Contributor

The balooning level is 0 across the chart.

We are using FC storage.

Reply
0 Kudos
mittim12
Immortal
Immortal

What kind of latency do you see on the storage side?  

Reply
0 Kudos
LarryBlanco2
Expert
Expert

I agree, what kind of latency do u see on the storage I/O side.  U can use  ESXTOP or the performance graphs.   On ESXTOP we can drill down to the VM disk by pressing "v" and that will give u the VM disk info.  Look at LAT/rd & LAT/wr.. Latency Read and Latency Write.

Copy a files a few GIG's in size to and from the VM and tell us your results.

Larry B.

Reply
0 Kudos
Wasisnt
Contributor
Contributor

I dont see the same LAT/rd & LAT/wr columns you are referring to but will post a screenshot of ESXTOP. The numbers range from about 4 to 33 in the CMDS/s and WRITE/s and dont seem to go above 33 when copying  4GB to his VM. The VM in question is JMCGREEVY-VM.

Reply
0 Kudos
LarryBlanco2
Expert
Expert

It may be the version of ESXi u are running.

I beleive the CMD/s should be somewhat higher.   Pretty much what CMD/s is the  Reads/s + Writes/s, which translates to IOPS/second

U'll need to look at the performance graphs in the vSphere client for the VM, we'll start at the vm' and select the "switch to" to "Virtual Disk"

once viewing the virtual disk select READ RATE, READ LATENCY, WRITE RATE, WRITE LATENCY.

Do the 4GB file transfer to and from the VM.  Post the graph with the results.

Larry B.

Reply
0 Kudos
LarryBlanco2
Expert
Expert

Also from the esxtop.   with 33 iops per second  you are writing at a rate of 0.17MB/s  which is pretty slow.   U should be writing at a rate of atleast 20MB/s and for a FC link, it should be alot higher than that.

So it seems that disk may be the bottleneck here.   let's get the graphs and see what we find.

Larry B.

Reply
0 Kudos
Wasisnt
Contributor
Contributor

Here is the graph. I copied large files from 4:22 to 4:27 and it doesnt look any different from any of the other times.

Reply
0 Kudos
LarryBlanco2
Expert
Expert

Can u add the READ RATE and WRITE RATE as well.  it'll tell us how fast the data is being written to and read from the disk subsystem

Thanks,

Larry B.

Reply
0 Kudos
LarryBlanco2
Expert
Expert

Were u able to find the issue with your disks?

Larry B.

Reply
0 Kudos
Wasisnt
Contributor
Contributor

We have been playing around and it appears that the performance hit only happens when running a certain procedure in Excel. I also tried it on my laptop that has a SSD drive, 6 GB of RAM and an i7 processor and it did the same thing on my laptop.

I was told to test with 6 processors on the VM even though I knew it wouldnt make a difference which it didnt and now may have to test with processor affinty just to satisfy their curisoty. It doesnt look like its storage related.

Reply
0 Kudos
LarryBlanco2
Expert
Expert

Ahh, ok.  glad u were able to narrow it down and that it is not and issue with your vdi environment.

Larry B.

Reply
0 Kudos