6 Replies Latest reply on Jan 27, 2009 12:58 PM by wila

    What is the expected cost of virtualization?

    IT_Architect Hot Shot

       

      I've run some tests to test the disks, and CPU & Memory. 

      (Intel XEON 5400 Series, 4 cores, 1 processor, 2.66 GHZ, 4 Gigs RAM, local 10,000 rpm 146 gig SCSIs, RAID-1, FreeBSD 7.1 64-bit)

       

       

      - The disk looks OK.  (diskinfo -t  ) It shows virtual faster than native, but then the size of the disk is less than physical, and there may be some read-ahead going on there.  Either way, worst case is livable, and best case it's a lot faster.  I'm not disk-bound anyway.

       

       

      Native:

      Transfer rates:

      outside:       102400 kbytes in   0.740411 sec =   138302 kbytes/sec

      middle:        102400 kbytes in   0.590457 sec =   173425 kbytes/sec

      inside:        102400 kbytes in   0.816407 sec =   125428 kbytes/sec

      Virtual:

      Transfer rates:

      outside:       102400 kbytes in   0.740411 sec =   102478 kbytes/sec

      middle:        102400 kbytes in   0.590457 sec =   225934 kytes/sec

      inside:        102400 kbytes in   0.816407 sec =   232638 kbytes/sec

       

      - Memory & CPU, the CPU lost about 16%, which seems about right.  Memory is down 50% which is not what I would have expected:

      Native

      Ubench CPU: 1346168

      Ubench MEM:  228570

      -


      Ubench AVG:  787369

       

       

      VM with VMware Tools installed:

      Ubench CPU:  1129519 (-16%)

      Ubench MEM:   115477 (-50%)

      -


      Ubench AVG:   622498  (-21%)

       

       

      I played some games with the amount of memory, CPUs, etc.  None of it

      worked. In fact, if I gave the VM all of the memory and CPU, and just

      let VMware do the scheduling and resource management instead of me, it

      worked WAY, WAY, WAY better than me trying to help by reserving CPU or

      RAM for the hypervisor.  It's not even close.  I learned the hypervisor does a far better job of scheduling resources if I don't "help".  I tried para on, but it wouldn't even boot that way.

       

      I've seen several extensive tests done around the web and they consistently put the cost of virtualization at 25% pretty for this type of virtualization where VMs don't share the same kernel, and 10% when they do.  It's possible this combo in real world this would align with their findings of 25% penalty depending on the CPU to memory mix.  I'd just like to catch someone getting ready to set up and try the ubench test and see if yours is coming out about like mine. It would be best if you tested with a 4 core.  I've noticed as you drop cores, the cost of virtualization as a percent of the total goes UP disproportionately.  That's quite the opposite of physical where adding cores normally has a diminishing rate of return.  That might make someone with a 2 core feel like there is something wrong when it's perfectly normal.

       

       

      Thanks!

       

       

        • 1. Re: What is the expected cost of virtualization?
          larstr Virtuoso
          vExpert

          What VMware product are you using?

          1. hdparm, diskinfo and similar tools will not give you very reliable performance results. Timing inside a virtual machine might also not be too reliable when running such tested.

          In an iometer test I got the following results:

           

          SERVER TYPE: Physical Windows 2003R2sp2 (not a virtual machine)

          CPU TYPE / NUMBER: 8 cpu cores, 2 sockets

          HOST TYPE: HP DL360G5, 4GB RAM; 2x XEON E5345, 2,33 GHz, QC

          STORAGE TYPE / DISK NUMBER / RAID LEVEL: P400i 256MB 50% read cache /

          2xSAS 15k rpm / raid 1 / 128KB stripe size / default ntfs block size

          (4096)

           

          TEST                NAME

          Av.                Resp. Time ms

          Av.                IOs/sec

          Av.                MB/sec

          Max                Throughput-100%Read.

          3.18

          18530

          579

          RealLife-60%Rand-65%Read

          78.6

          739

          5.7

          Max                Throughput-50%Read

          3.74

          15579

          486

          Random-8k-70%Read.

          72.7

          787

          6.1

           

          SERVER TYPE: Virtual Windows 2003R2sp2 on ESX 3.0.2. Descheduled time service disabled

          CPU TYPE / NUMBER: VCPU / 1

          HOST TYPE: HP DL360G5, 4 GB RAM; 2x XEON E5345, 2,33 GHz, QC

          STORAGE TYPE / DISK NUMBER / RAID LEVEL: P400i 256MB 50% read cache /

          2xSAS 15k rpm / raid 1 / 128KB stripe size / default vmfs 1MB

           

          TEST                NAME

          Av.                Resp. Time ms

          Av.                IOs/sec

          Av.                MB/sec

          Max                Throughput-100%Read.

          5.3

          9711

          303

          RealLife-60%Rand-65%Read

          43

          786

          6.1

          Max                Throughput-50%Read

          6.4

          8796

          274

          Random-8k-70%Read.

          55

          778

          6

           

          2. What you have observed about memory is probably right since you're using an intel cpu. Newer AMD cpus have faster memory handling since they support RVI.

          3. Even though VMware Workstation and Fusion support directX9, many new games will not run well.

          4. Adding multiple virtual cpus to a guest is also something that you're only adviced to do only when needed, even though ESX3.x handles this better than hosted products.

           

           

          • 2. Re: What is the expected cost of virtualization?
            IT_Architect Hot Shot

             

            What VMware product are you using?

             

             

            Thanks tons for checking this with me.  I'm using ESXi Update 3.

             

            Newer AMD cpus have faster memory handling since they support RVI

             

             

            We have 2 other servers that are AMD.  They've been good for us.  Where we are now though the only esxi-certified machines we can get are the Intels.  As a percent, the penalty isn't much different.

             

             

            Thanks again!

             

             

            • 3. Re: What is the expected cost of virtualization?
              wila Guru
              vExpertUser ModeratorsCommunity Warriors

              Where we are now though the only esxi-certified machines we can get are the Intels.  As a percent, the penalty isn't much different.

              That's probably because 2nd generation intel CPUs support EPT



              --

              Wil

              _____________________________________________________

              Visit the new VMware developers wiki at http://www.vi-toolkit.com

              | Author of Vimalin. The virtual machine Backup app for VMware Desktop Products
              | Vimalin : Automated backups for VMware Fusion and VMware Workstation Professional
              | More info at https://www.vimalin.com
              | Twitter @wilva
              | VMware Wiki at http://www.vi-toolkit.com
              • 4. Re: What is the expected cost of virtualization?
                IT_Architect Hot Shot

                 

                That's probably because 2nd generation intel CPUs support EPT

                 

                 

                Actually, they are that close without the EPT.  The 5500s and 7400s are the ones with EPT.  My hoster has had 7400s for quite awhile, but not the 5500s.  I'd rather have two machines for the time being for failover and maintenance.  The 5500s/7400s have not only the EPT but the also the equivelent of

                hypertransport, plus 4 separate cores on one chip, plus each core able to work on two threads at a

                time.  I haven't seen VMware doing any tests on these yet.  Hopefully vmware doesn't see each thread as a virtual processor.

                 

                 

                • 5. Re: What is the expected cost of virtualization?
                  larstr Virtuoso
                  vExpert

                  Does VMware ESX 3.5u3 support EPT yet? It seems only workstation/fusion/server supports it for the time being.

                   

                  Lars

                  • 6. Re: What is the expected cost of virtualization?
                    wila Guru
                    User ModeratorsCommunity WarriorsvExpert

                    Hi Lars,

                     

                    Ack.. not in ESX3.5 yet according to your document, sorry for the mixup.

                    Very nice document, thanks for pointing me towards it.



                    --

                    Wil

                    _____________________________________________________

                    Visit the new VMware developers wiki at http://www.vi-toolkit.com

                    | Author of Vimalin. The virtual machine Backup app for VMware Desktop Products
                    | Vimalin : Automated backups for VMware Fusion and VMware Workstation Professional
                    | More info at https://www.vimalin.com
                    | Twitter @wilva
                    | VMware Wiki at http://www.vi-toolkit.com