VMware Cloud Community
iben
Contributor
Contributor

vmxnet3 - features and use information - tips and tricks

Glad to see this has been posted and we can talk about it now... please share your experiences and let us know if these tips work for you and what sort of performance benefits you've noticed when using this new driver.

We've been switching our Windows and Linux VMs to use "VMXNET Enhanced" for some time now and see public information on the new VMXNET3 NIC for guests...

This Thread has been started to help with procedures on the conversion of existing machines from older NIC to newer NIC as it is not 100% straightforward and there are some tricks to remove old hardware and change to new hardware. This would be similar in the physical world to changing from a 100 BaseT PCI Card to a GigE card. The old drivers need to be removed, new drivers installed, and IP Addresses moved over. If you just remove the old NIC and install the new one you may end up with a IP Address Conflict error saying the Address you are trying to use is already in use on another Network Interface. The problem is that when you open Device Manager the old NIC is hidden. See below for steps on how to overcome this.

Question: What is VMXNET3?

Answer: VMXNET3 builds upon VMXNET and Enhanced VMXNET as the third generation paravirtualized virtual networking NIC for guest operating systems.

New VMXNET3 features over previous version of Enhanced VMXNET include:

• MSI/MSI-X support (subject to guest operating system kernel support)

• Receive Side Scaling (supported in Windows 2008 when explicitly enabled through the device's Advanced configuration tab)

• IPv6 checksum and TCP Segmentation Offloading (TSO) over IPv6

• VLAN off-loading

• Large TX/RX ring sizes (configured from within the virtual machine)

What's New in vSphere 4.0

From the Cisco document:VMware vSphere 4 and Cisco Nexus 1000V Series

VMware vNetwork module that encompasses the vDS and VMXNET-3 enables inline monitoring and centralized firewall services and maintains the virtualmachine's network run-time characteristics.

Tech Notes

Flexible shows up in Windows Device Manager as an “VMware

Accelerated AMD PCNet Adapter” and Enhanced vmxnet show up as “VMware

PCI Ethernet Adapter”.

Flexible — The Flexible network adapter

identifies itself as a Vlance adapter when a virtual machine boots, but

initializes itself and functions as either a Vlance or a vmxnet

adapter, depending which driver initializes it. VMware Tools versions

recent enough to know about the Flexible network adapter include the

vmxnet driver but identify it as an updated Vlance driver, so the guest

operating system uses that driver. When using the Flexible network

adapter, you can have vmxnet performance when sufficiently recent

VMware tools are installed. When an older version of VMware Tools is

installed, the Flexible adapter uses the Vlance adapter (with Vlance

performance) rather than giving no network capability at all when it

can’t find the vmxnet adapter.

Enhanced vmxnet — The enhanced vmxnet adapter is

based on the vmxnet adapter but provides some high-performance features

commonly used on modern networks, such as jumbo frames. This virtual

network adapter is the current state-of-the-art device in virtual

network adapter performance, but it is available only for some guest

operating systems on ESX Server 3.5. This network adapter will become

available for additional guest operating systems in the future.

Networking Error, IP Address Already Assigned to Another Adapter

KB Article 1179

Updated Jan. 07, 2009

Why do I see an error message that “The IP address XXX.XXX.XXX.XXX…” is already assigned to another adapter?

Solution

Under certain conditions, you may see the following error message from a Windows guest operating system:

The IP address XXX.XXX.XXX.XXX you have entered for this network

adapter is already assigned to another adapter Name of adapter. Name of

adapter is hidden from the network and Dial-up Connections folder

because it is not physically in the computer or is a legacy adapter

that is not working. If the same address is assigned to both adapters

and they become active, only one of them will use this address. This

may result in incorrect system configuration. Do you want to enter a

different IP address for this adapter in the list of IP addresses in

the advanced dialog box?

In this message, XXX.XXX.XXX.XXX is an IP address that you are

trying to set and Name of adapter is the name of a network adapter that

is present in the registry but hidden in Device Manager.

This can occur when you change a network connection’s TCP/IP configuration from DHCP to a static IP address if:

  • You have upgraded VMware virtual network adapters (for example

when you migrate a virtual machine from an older to a new version of

VMware software.)

  • You have added and removed network adapters multiple times.

The cause of the error is that a network adapter with the same IP

address is in the Windows registry but is hidden in the Device Manager

(My Computer > Properties > Hardware > Device Manager). This

hidden adapter is called a ghosted network adapter.

Using the Show hidden devices option in the Device Manager (View

Show hidden devices) does not always show the old virtual NIC

(ghosted adapter) to which that IP Address is assigned

Microsoft addresses this issue in their Knowledge Base article

269155, which is available at the time of this writing at

.

To resolve this problem, follow these steps to make the ghosted

network adapter visible in the Device Manager and uninstall the ghosted

network adapter from the registry:

1. Select Start > Run.

2. Enter cmd.exe and press Enter.

3. At the command prompt, run this command:

set devmgr_show_nonpresent_devices=1

4. Enter Start DEVMGMT.MSC and press Enter to start Device Manager.

5. Select View > Show Hidden Devices.

6. Expand the Network Adapters tree (select the plus sign next to the Network adapters entry).

7. Right-click the dimmed network adapter, and then select Uninstall.

8. Close Device Manager.

</div>

I b e n +14087824726 IM info: ibenrodriguez - yahoo iben.rodriguez - gmail ibenr - aol ibenrodriguez - skype Follow me on http://twitter.com/iben http://communities.vmware.com/people/iben
21 Replies
ShanVMLand
Expert
Expert

Good one!!! Thanks!

Shan

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!
Reply
0 Kudos
howie
Enthusiast
Enthusiast

vmxnet3 is the first virtual NIC showing 10G speed in the VM at least.:)

Reply
0 Kudos
AndreTheGiant
Immortal
Immortal

Great post Smiley Wink

Andre

Andrew | http://about.me/amauro | http://vinfrastructure.it/ | @Andrea_Mauro
Reply
0 Kudos
rob_nance
Contributor
Contributor

I can't seem to find an answer to this anywhere. Someone asked it once already and received no response. After upgrading the virtual hardware in VM version 7 machines (vmware 4.0) the network adapter, regardless of which type of adapter, shows up as a removeable device in the system tray. This is not optimal as it makes it very easy to kill the network on a VM. If you don't have access to anything but the VM via RDP, this also means you are out of luck the second you click this.

I know you can disable seeing this, and could hack the registry to mark this as non-removeable, but is there a legitimate way to do this, and is there a reason this change was made? I see no compelling reason to change the way this works.

Reply
0 Kudos
Optic_Nerve
Enthusiast
Enthusiast

Hi Rob,

I am only guessing but I suspect this change was made as you can now hot-add and hot-remove network cards in version 7 guests. In order to accomodate this at the OS level, the NIC has to show up as removeable so that you can remove it from the OS before deleting it from the guest.

If I recall correctly the same is true of the SCSI cards in version 7 guests.

Cheers,

David

Reply
0 Kudos
dmadden
VMware Employee
VMware Employee

You can add:

devices.hotplug = "false" to the virtual machine vmx file to prevent the network adapter from being removed.

However, this will also prevent network adapters from being hot added to a VM. Haven't tested it to see if it prevents hard disk hot add, but memory/CPU hotplug will work with this option set to false.

FireDog7881
Contributor
Contributor

I have been looking for a definitive answer to this and I can't find it. It seems this post is a great place to post this question.

My understanding is that when you have your VM NIC set to FLEXIBLE and have the latest VM Tools installed the OS will load the VMXNET drivers, but I haven't been able to find that it will use version 7 will use the VMXNET3 driver specifically. Is this true when you upgrade the virtual hardware to version 7? Will I get the performance benefits of VMXNET3 if my VMs are set to FLEXIBLE after upgrading to the latest VMtools and upgrading the VM to hardware version 7? If so this would be wonderful because I have been removing the FLEXIBLE nic and adding a new VMXNET3 nic which then changes the MAC address and we have software that is licensed to a specific MAC address and I have to reassign the static IP addresses. I have over 350 VMs that I am doing this to (100 so far).

From what I have been reading I don't have to do this and FLEXIBLE will use the VMXNET3 driver and get the same performance. Please tell me this is true.

Reply
0 Kudos
howie
Enthusiast
Enthusiast

Flexible does not use vmknet3 at this point. Depending on the GOS, sometimes it becomes enhanced vmxnet though in vSphere 4.

Reply
0 Kudos
FireDog7881
Contributor
Contributor

I was afraid of that. Is there any way to have the new nic retain the old MAC address? I know that vmware assigns MAC addresses in a specific range and the manual MAC addresses are in another range. Is there a way to force VMware to use the old MAC?

Reply
0 Kudos
LucasAlbers
Expert
Expert

I move machines from one vm server to another. They were configured to have automatically generated mac address. On the new server the auto generated addresses were differernt.

I set the mac addresses manually in the device properites for the network card in windows, to the old addresses.

Reply
0 Kudos
FireDog7881
Contributor
Contributor

That is good to know. Thank you for that information, for some reason it never occurred to me to change it in Windows.

LucasAlbers <communities-emailer@vmware.com>

07/22/2009 10:01 AM

To

<brandon.meyer@quebecorworld.com>

cc

Subject

New message: "vmxnet3 - features and use information - tips and tricks"

,

A new message was posted in the thread "vmxnet3 - features and use information - tips and tricks":

http://communities.vmware.com/message/1317851#1317851

Author : LucasAlbers

Profile : http://communities.vmware.com/people/LucasAlbers

Message:

Reply
0 Kudos
LucasAlbers
Expert
Expert

I have a little goat herd of windows xp desktops running enhanced vmxnet.

Would vmxnet3 improve performance on windows xp?

Reply
0 Kudos
jasoncllsystems
Enthusiast
Enthusiast

Here are my testing result:

2 x vSphere 4 build 175625

2 x Windows Server 2008 Standard Edition running on different host

Qlogic QLE8042 CNA(10G) back to back direct connect from ESX host 1 to ESX host 2

Single 5.63G ISO file transfer between 2 virtual machine sitting on different hosts.

First test using virtual \network adapter = E1000

Second test using virtual network adapter = VMXNET3

See results as attachment.

Regards,

Jas aka Superman

MALAYSIA VMware Communities

http://www.malaysiavm.com

'If you found this or any other answer useful please consider allocating points for helpful or correct answers ***

http://www.malaysiavm.com
Reply
0 Kudos
iben
Contributor
Contributor

Hi Jas,

Thanks for doing these tests. Looks like main result from your tests show less cpu usage doing file transfer. Right?

Few questions...

Were these 32 bit or 64 bit VMs?

What were time difference results during the copies?

Did you repeat the tests a few times to see eliminate caching and other variables?

FTP suposedly has less CPU usage. Could you try that too?

This is great. Thanks again!

I b e n

I b e n +14087824726 IM info: ibenrodriguez - yahoo iben.rodriguez - gmail ibenr - aol ibenrodriguez - skype Follow me on http://twitter.com/iben http://communities.vmware.com/people/iben
Reply
0 Kudos
jasoncllsystems
Enthusiast
Enthusiast

Hi iben,

I have tested on Win2K8 server stadard edition (32bit) and Win2K8 Data Center edition (64bit).

Yes I tried multiple times and the result almost the same.

I didn't try FTP but I tried with Internet Download Manager on HTTP.

See attachment.

IDM_5Gfile_download.png - Internet Download Manager

Win2k8DC_8vcpu_VMXNET3__autotuninglevel_exp_5GFile_Transfer_Test_02.png - Window Server 2008 Data Center edition 8vCPU with VMXNET3 vNIC and TCP tuning enabled.

Win2k8DC_1vcpu_VMXNET3_5GFile_Transfer_Test_01.png - Windows Server 2008 Data Center Edition 1 vCPU with VMXNET3 vNIC and default TCP setup.

Probably you may refer to my previous post @

http://malaysiavm.com/blog/cisco-nexus-5000-poc/

Regards,

Jas aka Superman

MALAYSIA VMware Communities

http://www.malaysiavm.com

'If you found this or any other answer useful please consider allocating points for helpful or correct answers ***

http://www.malaysiavm.com
Reply
0 Kudos
AlbertWT
Virtuoso
Virtuoso

Hi Jason,

Thanks for posting the result here, I can see that by using VMXNet 3, the CPU utilization is lower to transmit large file.

Kind Regards,

AWT

/* Please feel free to provide any comments or input you may have. */
Reply
0 Kudos
seanhsmith
Enthusiast
Enthusiast

Good Stuf!

Reply
0 Kudos
joshuatownsend
Enthusiast
Enthusiast

Good thread, iben.

One thing to point out - the set devmgr_show_nonpresent_devices=1 command does not work with Windows 2008.

Instead, you have to set an environmental variable of "devmgr_show_nonpresent_devices" with a value of "1". Then you can open device manager and click 'View | show hidden devices' to see the ghost NIC.

Josh

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".

Please visit for News, Views and Virtualization How-To's

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful". Please visit http://vmtoday.com for News, Views and Virtualization How-To's Follow me on Twitter - @joshuatownsend
Reply
0 Kudos
godbucket
Enthusiast
Enthusiast

TRUE.DAT

This solution worked for me, big time. Had lots of VM's that were migrated over from Hyper-V and legacy ESX 3.5 hosts. Upon booting into the ESXi 4.0 environment, the Network Adapters/NIC's showed up in the "Safely Remove Hardware" icon in Windows Server 2003.

Solution: with VM powered off, go to Edit Settings, click Options tab at top, click on General, then click the Configuration Parameters button. Scroll all the way down to the bottom of the list and click Add Row. Type "devices.hotplug" without the quotations, and in the next field type "FALSE" without quotations. Power on VM and the "Safely Remove Hardware" icon is gone.

This really helped us because we host hundreds of Citrix and Terminal Servers, so we didnt want users with elevated rights disabling the NIC of the server they were on!

I know this may be rhetorical for the many ubergurus in these forums, but I want to add that I had this behavior not only with the VMXNET3 adapter but the others as well (E1000 and Flexible). Didnt seem to matter which flavor NIC, many VM's had this issue however.

Hope this helps someone!

Reply
0 Kudos