In my efforts to find information about promiscuous mode settings, I've found the Admin Guide to be limited and some of the threads pretty scattered, so I've written this for anyone looking for help with their promiscuous mode settings.
GSTK (Good Stuff To Know) | Promiscuous Mode
Definition: In regards to VMware ESX Server; promiscuous mode is used to allow a host to see all traffic passing on a vswitch.
Uses: Firewalls, Port Sniffers, IDS, etc.
Requirements: When using a vswitch, both the virtual adapter and vswitch must have promiscuous mode enabled. When not using a vswitch (e.g. if you had a dedicated NIC for sniffing and used port mirroring on your physical switch) it just needs to be enabled on the virtual adapter. (The latter is a good config if you dont want to incur the processing overhead associated with promiscuous mode for a sniffer application).
Need to know: vmnet = internal only switch, vmnic = vswitch with only 1 dedicated physical adapter, bond = vswitch with 2 or more dedicated adapters. Familiarity with startup script syntax and precedence. COS file system.
Step 1 | Enable Promiscuous Mode for the VM
Add a line to the configuration file, this probably goes without saying, but substitute n for the adapter (0-4) and remove the brackets.
ethernet[n].noPromisc = FALSE
Youll want to apply this to all VMs you need promiscuous mode enabled on.
Step 2 | Enabling the Virtual Adapter and/or vswitch to Use Promiscuous Mode
For those of you familiar with Ciscos IOS, enabling the virtual adapter for promiscuous mode automatically turns its port on the vswitch into a mirror port of sorts, but it doesnt mirror only selected ports, it mirrors traffic from all ports on the vswitch. You must enable promiscuous mode for the vswitch, in addition to the virtual adapters for this to work. Mirroring can be selectively disabled for specific MAC addresses if needed, but otherwise its a global setting for the vswitch. (Refer to the Admin Guide, a hyperlink is provided in the references section at the bottom, if you want to know about the MAC address piece).
At the service console, logged in as root, type:
echo "PromiscuousAllowed yes" > /proc/vmware/net/[vmnic0]/ config
This allows the guest operating systems in all virtual machines using [vmnic0] to enable promiscuous mode. If the adapter is using a different network, such as vmnet_0, make the appropriate substitution in the command. For a bond, apply this to the individual vmnics that are members of the bond as well as the bond itself. (e.g. vmnic0, vmnic1, and bond0 if bond0 was comprised of members vmnic0 and vmnic1. Look at the config file of the bond to find out which vmnics are members (a.k.a. teaming slaves) of the bond).
*Important Note: This change is applied to the /proc filesystem, which means this setting is applied to the running config only (per say). You must perform Step 3 below to make this setting non-volatile (survive a reboot of the ESX server).
Step 3 | Making the Promiscuous Mode Setting Non-Volatile
In case of an ESX server reboot, this setting needs to be applied to the /proc filesystem vswitch configs before the virtual machines startup sequence begins. This ensures all your sniffer apps or whatever come up clean in the event your ESX server is rebooted.
I would steer away from creating a new script and setting its startup order. Instead I would add a simple entry to the end of the /etc/rc.local script. This applies the change before VMs start powering on. In my opinion, this is the cleanest way to make sure these commands get executed on each system startup and before VMs power on.
Add the following command to the end of the /etc/rc.local script. Repeat for each vmnet, vmnic, or bond you modified in Step 2.
echo "PromiscuousAllowed yes" > /proc/vmware/ net/[vmnic0]/ config
-End of Instructional Steps-
Why Written: The Admin Guide for ESX leaves something to be desired when speaking about Promiscuous Mode settings and all the associated configuration dependencies for it to work.
References: http://www.vmware.com/support/esx21/doc/esx21admin_virtualadapter.html or page 360 in the ESX Server 2.5 Administration Guide PDF and VMware Knowledge Base Answer ID 1557.
All Due Respect: I quoted some pieces from the VMware Admin Guide. Also, the user posts from JMills, lambeth, and ken.cline@hp on the VMTN forums helped me out a lot while I was learning this for one of my environments.
Brandon Sanders, VCP
bsanders@data-strategy.com
Systems Engineer | Data-Strategy
GSTK (Good Stuff To Know) | Promiscuous Mode
Definition: In regards to VMware ESX Server; promiscuous mode is used to allow a host to see all traffic passing on a vswitch.
Uses: Firewalls, Port Sniffers, IDS, etc.
Requirements: When using a vswitch, both the virtual adapter and vswitch must have promiscuous mode enabled. When not using a vswitch (e.g. if you had a dedicated NIC for sniffing and used port mirroring on your physical switch) it just needs to be enabled on the virtual adapter. (The latter is a good config if you dont want to incur the processing overhead associated with promiscuous mode for a sniffer application).
Need to know: vmnet = internal only switch, vmnic = vswitch with only 1 dedicated physical adapter, bond = vswitch with 2 or more dedicated adapters. Familiarity with startup script syntax and precedence. COS file system.
Step 1 | Enable Promiscuous Mode for the VM
Add a line to the configuration file, this probably goes without saying, but substitute n for the adapter (0-4) and remove the brackets.
ethernet[n].noPromisc = FALSE
Youll want to apply this to all VMs you need promiscuous mode enabled on.
Step 2 | Enabling the Virtual Adapter and/or vswitch to Use Promiscuous Mode
For those of you familiar with Ciscos IOS, enabling the virtual adapter for promiscuous mode automatically turns its port on the vswitch into a mirror port of sorts, but it doesnt mirror only selected ports, it mirrors traffic from all ports on the vswitch. You must enable promiscuous mode for the vswitch, in addition to the virtual adapters for this to work. Mirroring can be selectively disabled for specific MAC addresses if needed, but otherwise its a global setting for the vswitch. (Refer to the Admin Guide, a hyperlink is provided in the references section at the bottom, if you want to know about the MAC address piece).
At the service console, logged in as root, type:
echo "PromiscuousAllowed yes" > /proc/vmware/net/[vmnic0]/ config
This allows the guest operating systems in all virtual machines using [vmnic0] to enable promiscuous mode. If the adapter is using a different network, such as vmnet_0, make the appropriate substitution in the command. For a bond, apply this to the individual vmnics that are members of the bond as well as the bond itself. (e.g. vmnic0, vmnic1, and bond0 if bond0 was comprised of members vmnic0 and vmnic1. Look at the config file of the bond to find out which vmnics are members (a.k.a. teaming slaves) of the bond).
*Important Note: This change is applied to the /proc filesystem, which means this setting is applied to the running config only (per say). You must perform Step 3 below to make this setting non-volatile (survive a reboot of the ESX server).
Step 3 | Making the Promiscuous Mode Setting Non-Volatile
In case of an ESX server reboot, this setting needs to be applied to the /proc filesystem vswitch configs before the virtual machines startup sequence begins. This ensures all your sniffer apps or whatever come up clean in the event your ESX server is rebooted.
I would steer away from creating a new script and setting its startup order. Instead I would add a simple entry to the end of the /etc/rc.local script. This applies the change before VMs start powering on. In my opinion, this is the cleanest way to make sure these commands get executed on each system startup and before VMs power on.
Add the following command to the end of the /etc/rc.local script. Repeat for each vmnet, vmnic, or bond you modified in Step 2.
echo "PromiscuousAllowed yes" > /proc/vmware/ net/[vmnic0]/ config
-End of Instructional Steps-
Why Written: The Admin Guide for ESX leaves something to be desired when speaking about Promiscuous Mode settings and all the associated configuration dependencies for it to work.
References: http://www.vmware.com/support/esx21/doc/esx21admin_virtualadapter.html or page 360 in the ESX Server 2.5 Administration Guide PDF and VMware Knowledge Base Answer ID 1557.
All Due Respect: I quoted some pieces from the VMware Admin Guide. Also, the user posts from JMills, lambeth, and ken.cline@hp on the VMTN forums helped me out a lot while I was learning this for one of my environments.
Brandon Sanders, VCP
bsanders@data-strategy.com
Systems Engineer | Data-Strategy
