What is the expected cost of virtualization?

[IT_Architect]

I've run some tests to test the disks, and CPU & Memory.

(Intel XEON 5400 Series, 4 cores, 1 processor, 2.66 GHZ, 4 Gigs RAM, local 10,000 rpm 146 gig SCSIs, RAID-1, FreeBSD 7.1 64-bit)

- The disk looks OK. (diskinfo -t ) It shows virtual faster than native, but then the size of the disk is less than physical, and there may be some read-ahead going on there. Either way, worst case is livable, and best case it's a lot faster. I'm not disk-bound anyway.

Native:

Transfer rates:

outside: 102400 kbytes in 0.740411 sec = 138302 kbytes/sec

middle: 102400 kbytes in 0.590457 sec = 173425 kbytes/sec

inside: 102400 kbytes in 0.816407 sec = 125428 kbytes/sec

Virtual:

Transfer rates:

outside: 102400 kbytes in 0.740411 sec = 102478 kbytes/sec

middle: 102400 kbytes in 0.590457 sec = 225934 kytes/sec

inside: 102400 kbytes in 0.816407 sec = 232638 kbytes/sec

- Memory & CPU, the CPU lost about 16%, which seems about right. Memory is down 50% which is not what I would have expected:

Native

Ubench CPU: 1346168

Ubench MEM: 228570

-

---

Ubench AVG: 787369

VM with VMware Tools installed:

Ubench CPU: 1129519 (-16%)

Ubench MEM: 115477 (-50%)

-

---

Ubench AVG: 622498 (-21%)

I played some games with the amount of memory, CPUs, etc. None of it

worked. In fact, if I gave the VM all of the memory and CPU, and just

let VMware do the scheduling and resource management instead of me, it

worked WAY, WAY, WAY better than me trying to help by reserving CPU or

RAM for the hypervisor. It's not even close. I learned the hypervisor does a far better job of scheduling resources if I don't "help". I tried para on, but it wouldn't even boot that way.

I've seen several extensive tests done around the web and they consistently put the cost of virtualization at 25% pretty for this type of virtualization where VMs don't share the same kernel, and 10% when they do. It's possible this combo in real world this would align with their findings of 25% penalty depending on the CPU to memory mix. I'd just like to catch someone getting ready to set up and try the ubench test and see if yours is coming out about like mine. It would be best if you tested with a 4 core. I've noticed as you drop cores, the cost of virtualization as a percent of the total goes UP disproportionately. That's quite the opposite of physical where adding cores normally has a diminishing rate of return. That might make someone with a 2 core feel like there is something wrong when it's perfectly normal.

Thanks!

[larstr · Accepted solution]

What VMware product are you using?

1. hdparm, diskinfo and similar tools will not give you very reliable performance results. Timing inside a virtual machine might also not be too reliable when running such tested.

In an iometer test I got the following results:

SERVER TYPE: Physical Windows 2003R2sp2 (not a virtual machine)

CPU TYPE / NUMBER: 8 cpu cores, 2 sockets

HOST TYPE: HP DL360G5, 4GB RAM; 2x XEON E5345, 2,33 GHz, QC

STORAGE TYPE / DISK NUMBER / RAID LEVEL: P400i 256MB 50% read cache /

2xSAS 15k rpm / raid 1 / 128KB stripe size / default ntfs block size

(4096)

TEST NAME	Av. Resp. Time ms	Av. IOs/sec	Av. MB/sec
Max Throughput-100%Read.	3.18	18530	579
RealLife-60%Rand-65%Read	78.6	739	5.7
Max Throughput-50%Read	3.74	15579	486
Random-8k-70%Read.	72.7	787	6.1

SERVER TYPE: Virtual Windows 2003R2sp2 on ESX 3.0.2. Descheduled time service disabled

CPU TYPE / NUMBER: VCPU / 1

HOST TYPE: HP DL360G5, 4 GB RAM; 2x XEON E5345, 2,33 GHz, QC

STORAGE TYPE / DISK NUMBER / RAID LEVEL: P400i 256MB 50% read cache /

2xSAS 15k rpm / raid 1 / 128KB stripe size / default vmfs 1MB

TEST NAME	Av. Resp. Time ms	Av. IOs/sec	Av. MB/sec
Max Throughput-100%Read.	5.3	9711	303
RealLife-60%Rand-65%Read	43	786	6.1
Max Throughput-50%Read	6.4	8796	274
Random-8k-70%Read.	55	778	6

2. What you have observed about memory is probably right since you're using an intel cpu. Newer AMD cpus have faster memory handling since they support RVI.

3. Even though VMware Workstation and Fusion support directX9, many new games will not run well.

4. Adding multiple virtual cpus to a guest is also something that you're only adviced to do only when needed, even though ESX3.x handles this better than hosted products.

[IT_Architect]

What VMware product are you using?

Thanks tons for checking this with me. I'm using ESXi Update 3.

Newer AMD cpus have faster memory handling since they support RVI

We have 2 other servers that are AMD. They've been good for us. Where we are now though the only esxi-certified machines we can get are the Intels. As a percent, the penalty isn't much different.

Thanks again!

[wila]

Where we are now though the only esxi-certified machines we can get are the Intels. As a percent, the penalty isn't much different.

That's probably because 2nd generation intel CPUs support EPT

--

Wil

_____________________________________________________

Visit the new VMware developers wiki at http://www.vi-toolkit.com

[IT_Architect]

That's probably because 2nd generation intel CPUs support EPT

Actually, they are that close without the EPT. The 5500s and 7400s are the ones with EPT. My hoster has had 7400s for quite awhile, but not the 5500s. I'd rather have two machines for the time being for failover and maintenance. The 5500s/7400s have not only the EPT but the also the equivelent of

hypertransport, plus 4 separate cores on one chip, plus each core able to work on two threads at a

time. I haven't seen VMware doing any tests on these yet. Hopefully vmware doesn't see each thread as a virtual processor.

[larstr]

Does VMware ESX 3.5u3 support EPT yet? It seems only workstation/fusion/server supports it for the time being.

Lars

[wila]

Hi Lars,

Ack.. not in ESX3.5 yet according to your document, sorry for the mixup.

Very nice document, thanks for pointing me towards it.

--

Wil

_____________________________________________________

Visit the new VMware developers wiki at http://www.vi-toolkit.com