5 Replies Latest reply on Dec 21, 2010 8:10 PM by Scissor

    Sluggish VMs

    DavidMcKnight Novice

      I have what my users are describing as sluggish VMs.

       

      One case in point:

       

      [VM]

      Windows XP 32bit fully patched

      Default Hardware from VMware template

      Dual Processor (2 x 2.4 GHz Xeon) / 2 Gig RAM

      30 Gig C: Drive

      no reservations or limits on anything

       

      [Physical]

      Windows XP 32bit fully patched

      Dual Core Intel (2 x 3.0 GHz)  2 Gigs RAM

      120 Gig SATA C: Drive

       

      When, on a VM, they click on the icon for one of our larger apps  (SPSS v18 install to the C: drive)  it takes like 45 seconds for the app to load and be ready for input.

       

      When, on a Physical PC, they start the same app, it takes like 11 seconds to load and be ready for input.

       

      When I close the app and bring it right back up it take 7 seconds on the VM, and 4 seconds on the physical PC.

       

      During those times, on the VM, I have no great CPU usage.  The VM Host is at 30% RAM usage and 12% CPU usage.

       

      I have my cluster of Hosts connected to an iSCSI datastore running Server 2008R2.  For a simple test I bring up perfornce monitor and look at PhysicalDisk and Disk Reads/sec and Disk Writes/sec (this should be the datastores IOPs).  I do the same thing on the Physical PC.  On the Physical PC as soon as I click the icon I get about 2000 read IOPs.  When I click the icon on the VM and look at the stats on the datastore it's like 400 IOPs.  So to really test my datastore I clone a VM.  During that I get about 3700 read IOPs.

       

      It seems like there's a bottleneck between the VM and it's C: drive.  My hosts have a big pipe to the datastores, but a VM doesn't.  I don't see anything that would suggest any type of contention for resources.  Is there something I can do to get more IOPs out of my VMs?  Any other tests I should try?

       

      Thanks,

        • 1. Re: Sluggish VMs
          Scissor Virtuoso

          First I would try reducing your XP Guest CPU count from 2 to 1 and retest.

           

          If that doesn't help then double-check to see which (virtual) SCSI adapter your XP Guest is using.  You may want to switch your XP Guest from a (virtual) Buslogic SCSI adapter to a (virtual) lsilogic adapter.  See  http://kb.vmware.com/kb/1890 and http://kb.vmware.com/kb/1614 for more details.

           

          Do any other Guests running on the same host report slow disk performance?

          • 2. Re: Sluggish VMs
            DavidMcKnight Novice

            Thanks for the reply, but...

             

            Like I said, I created the XP VM with the default hardware.  So the VM is emulating thru IDE.  I did switch it to an LSI SAS, but that made no real difference.

             

            To me all the VMs feel slugish.

             

            To put this in perspective my Host has 24 logic processors running at 2.4GHz, it's has 96 Gigs of RAM, a 10 Gig Nic, and an iSCSI that, when cloning another VM, does 3500-4000 IOPs and generates 3 - 4 Gigabit of network traffic.  My cluster has 7 of these.

             

            If I set it up so I have my test VM as the only running VM on a particular test Host, and have next to no traffic going to or from the iSCSI datastore, I'd think I'd have a pretty fast VM.  But I don't.  The VM still peaks out around 400 IOPs and there's no great network traffic coming or going to the iSCSI datastore.  I don't know of any settings I can tweek to get a VM to use more resources when there are no resources in contention.  I seem to have this huge pipe that when I tell a Host to do something it can take advantage of it.  But, when I tell a VM on that same host to do something, like load a large statistical app, it doesn't seem to be trying very hard.  I bet if I took the time, and installed XP directly onto the Host's hardware, and set it to talk to my iSCSI datastore, it would bring up that app just as fast or faster than my other physical workstation I compared it to.

             

            I guess I'm wondering if I misunderstood the purpose to VMware.  I thought I could purchase these super servers and when needed a VM would run with the power of a super server.  When it wasn't needed other VMs would have that power.  I have Hosts that have processor utilization at 12%, memory usage at 33%, and 100 megabit of traffic to the datastore.  Yet when I tell a VM to do something it's like the host is asleep.  I understand that an XP VM isn't going to out performing some gaming PC.  But I thought it would be closer to that of a cheap workstation.

             

            So long story short...  What could be causing, what I believe, is such poor performance for all my VMs, yet give me, what I believe, is very good performance of the Hosts themselves (like when cloning a VM)?  What kind of IOPs are other people getting from thier XP VMs?  What kind of performance are other people getting from thier XP VMs?  And perhaps it's nothing to do with IOPs or datastores.

             

            Hoping for more insights...

             

            Thanks,

            • 3. Re: Sluggish VMs
              ThompsG Master

              Hi there,

               

              I guess I'm wondering if I misunderstood the purpose to VMware.  I thought I could purchase these super servers and when needed a VM would run with the power of a super server.  When it wasn't needed other VMs would have that power.  I have Hosts that have processor utilization at 12%, memory usage at 33%, and 100 megabit of traffic to the datastore.  Yet when I tell a VM to do something it's like the host is asleep.  I understand that an XP VM isn't going to out performing some gaming PC.  But I thought it would be closer to that of a cheap workstation


              Sorry if I have misunderstood your above statement but yes it appears you have a misunderstanding. The purpose of running ESX is not to have single VM run with the power of super server but rather to enable hardware (workload) consolidation in order to have the resource utilization on your super server. If we look at processor alone a single VM will only consume as much CPU as the virtual CPU's assigned to the VM.

               

              Now I notice you are using ISCSI target for the datastore so was wondering if you have created a ISCSI disk and attached it to the physical XP machine then run the same test. This will confirm if the problem is in ESX or on the array/network. I realise that you are seeing some impressive numbers from the array when cloning but this is sequential IOs and random IOs could be quite different.

               

              Again sorry if this is covered ground but are you running the latest VMware tools in the virtual machines? Have you run through this guide: http://www.vmware.com/pdf/vsphere4/r40/vsp_40_iscsi_san_cfg.pdf

               

              Thanks and best regards.

              • 4. Re: Sluggish VMs
                idle-jam Champion

                i would suggest runnign ESXtop (http://www.yellow-bricks.com/esxtop/) and that it would be able to pinpoint the root cause of the slowness from the metrics.

                • 5. Re: Sluggish VMs
                  Scissor Virtuoso

                  Please attach the vmware.log file from the directory containing your Guest so we can take a look.  (you will need to "browse" the datastore that your Guest is stored on to locate the file)

                   

                  Also, can you confirm that you redid your tests after configuring your Guest with a single vCPU?