Why is RVI not ON by default?

[brugh]

i was told that the first hardware assisted virtualization options like AMD-V where only partly used because the bittranslation was faster than the hardware equivalent. at least for 32-bit guests.

lateron, amd came with the next release and added the extra pagetable layer. now it seems that with RVI enabled this can give you a 0-30% performance gain depending on your workload. but this seems to be only enabled by default on 64-bit guests.. why isn't it on for 32-bit guests by default?? if there's no downside, just enable it and never turn it of.

[depping]

No clue why it's not on by default, but it does indeed give a huge performance boost:

http://www.vmware.com/pdf/RVI_performance.pdf

Duncan

Blogging: http://www.yellow-bricks.com

If you find this information useful, please award points for "correct" or "helpful".

[admin]

It depends on the guest. Current VMware products enable RVI by default for many 32-bit guests, but there are some exceptions. Buggy 32-bit guests for which we have workarounds in the binary translation layer default to binary translation. Also, certain 32-bit Windows guests that hammer the APIC TPR register use binary translation because AMD-V has no equivalent of Intel's FlexPriority, and the high overheads associated with TPR updates outweigh the benefits of RVI. 32-bit Linux and FreeBSD guests default to RVI. Oddball guests such as OS/2 and the generic "Other" also default to RVI.

[brugh]

well if i create a VM it defaults to 'auto' regarding the virtualized MMU. some people say that on all 32-bit guests, this means 'disable RVI' and on 64-bit 'enable'. what you're saying is that on some 32-bit guests, it's enabled by default? i'm perticullarly curious about windows 2003 and 2008 32-bit.

is there a list that shows me exactly what versions of what OS enables RVI when you set it to auto? and what line in the VM's vmware.log tells me wether it's on or not?

[admin]

The list is not very static. Which VMware product and which release are you asking about?

[brugh]

ah yes, that is very important ofcourse

i'm working on ESX 3.5 U3 on a HP DL385G5 that was purchased in october i think.

[admin]

ESX 3.5 is not very sophisticated in this regard. The only 32-bit guests that use RVI by default are Open Server 5, Open Server 6, OS/2 and UnixWare.

The RVI information is also not very obvious in an ESX 3.5 log file. For a 32-bit guest, if you see an extremely long line with the string "vmm64-modules:", then RVI is in use. For a 64-bit guest, you have to actually examine the contents of that line. If the module called "mmu-hwmmu.vmm64" is listed, then RVI is in use.

Most products since ESX 3.5 are much friendlier. They have a line in the log file that says:

HV Settings: virtual exec = 'xxxxx'; virtual mmu = 'yyyyy'

where xxxxx is one of "software", "hardware", or "dynamic", and yyyyy is one of "software" or "hardware".

The virtual exec setting is "software" for binary translation, "hardware" for VT-x or AMD-V, or "dynamic" for 'start with binary translation, but switch to VT-x if the VM ever enters long mode.' The latter case is necessary because binary translation is not supported in long mode on Intel hardware.

The virtual mmu setting is "software" for shadow page tables or "hardware" for EPT or RVI.

As for the guests you are interested in, I believe that RVI is generally preferable for Windows 2003SP2 and for all versions of Windows 2008. BT is generally preferable for older releases of Windows 2003, which hammer the TPR.

[brugh]

ok, that makes sense..

just leaves me with the question that when i select windows 2003 r2 32bit or any 32bit windows 2008 when i create a VM, why doesn't ESX enable RVI if it can benefit from it as i understand it does.

[admin · Accepted solution]

RVI was a new technology when ESX 3.5 hit its code freeze deadline, and the initial decision algorithm was quite simplistic. We are still refining the algorithm today, and I'm sure it's still imperfect. In the particular case of Windows 2003, I don't know if VC has selections representing the different service packs, but by the time the guest OS selection gets to the virtual machine monitor, we only know that it's Windows 2003, which is not sufficient information to make an informed decision. Of course, addressing these issues is a simple matter of programming. The choice of default execution mode is an area that certainly warrants further improvements.

[AWo]

Is there some rule of thumb when to force the use of RVI? As I understood from reading various documents and threads 32 Bit OS (like W2K3 SP2 and later) might gain performance by using RVI.

Some more questions come to my mind (which partially weren't clearly answered from my understandig):

How can you measure if there's a benefit by using it or not?

Do you need specific processors?

Are there any drawbacks regarding VMotion?

AWo

VCP / vEXPERT 2009

!http://communities.vmware.com/servlet/JiveServlet/downloadImage/5441/VMW_vExpert_Q109_200px.jpg|height=45|width=90|src=http://communities.vmware.com/servlet/JiveServlet/downloadImage/5441/VMW_vExpert_Q109_200px.jpg !

[admin]

Is there some rule of thumb when to force the use of RVI? As I understood from reading various documents and threads 32 Bit OS (like W2K3 SP2 and later) might gain performance by using RVI.

As a general rule of thumb, 32-bit non-Windows guests and most 32-bit WIndows guests after Windows 2003 SP2 may benefit from using RVI. Windows XP is the exception. Even Windows XP SP3 continues to hammer the TPR, which generally negates the benefits of using hardware virtualization. The other exception would be a guest workload that exhibits a high rate of TLB misses, as TLB misses are considerably more expensive with RVI. TLB misses are likely to be high for workloads with a large working data set, including database applications and Java VMs.

You may be interested in this paper: Performance Evaluation of AMD RVI Hardware Assist.

How can you measure if there's a benefit by using it or not?

Create a benchmark that characterizes your workload, and measure performance using appropriate metrics (time, transactions per second, etc.) with and without RVI.

Do you need specific processors?

All AMD Family 10H processors support RVI.

Are there any drawbacks regarding VMotion?

VMotion compatibility is not affected by whether or not a particular host supports RVI or whether or not a VM is configured to use RVI.

[AWo]

Thank you for the answer.

As a general rule of thumb, 32-bit non-Windows guests and most 32-bit WIndows guests after Windows 2003 SP2 may benefit from using RVI. Windows XP is the exception. Even Windows XP SP3 continues to hammer the TPR, which generally negates the benefits of using hardware virtualization. The other exception would be a guest workload that exhibits a high rate of TLB misses, as TLB misses are consedirably more expensive with RVI. TLB misses are likely to be high for workloads with a large working data set, including database applications and Java VMs.

Should I set this option manually or does ESX switch it on automatically according to what you wrote?

All AMD Family 10H processors support RVI.

So with Intel you can't use it at all....?

AWo

VCP / vEXPERT 2009

!http://communities.vmware.com/servlet/JiveServlet/downloadImage/5441/VMW_vExpert_Q109_200px.jpg|height=45|width=90|src=http://communities.vmware.com/servlet/JiveServlet/downloadImage/5441/VMW_vExpert_Q109_200px.jpg !

[admin]

Should I set this option manually or does ESX switch it on automatically according to what you wrote?

For ESX 3.5, you should set the option manually. Future ESX products may have better defaults.

So with Intel you can't use it at all....?

Correct. Intel processors do not implement RVI. Intel Core i7 processors implement a similar technology, known as EPT. However, ESX 3.5 has no support for EPT.

[AWo]

Thank you. You answered all my question. Unfortunately I can't award points as I'm not the initiator of this thread...

AWo

VCP / vEXPERT 2009