VMware Communities : Thread List - VMware vSphere™ vNetwork

VM's lose their network connection randomly

macos1984 — Fri, 20 Nov 2009 18:14:49 GMT

Hi everyone:

We recently installed 3 hosts with ESX 4.0 and migrated some of our VI3 VM's to them. We upgraded 3 VM's (one Linux RHEL 5.2 and two Windows 2003) to the virtual hardware version 7, and let the rest on version 4.

After that, the upgraded VM's are missing the network connection and recovering again, without any action from us. That situation doesn't occurs on the version 4 VM's. I have changed the vNics to vmxnet3, and to E1000 and still happens.

The three ESX 4.0 hosts are HP Proliant BL465 g1 in a HP c7000 enclosure, with Virtual Connect Ethernet Modules. We have 13 more servers, on the same enclosure, with ESX 3.5 Update 2 and are working fine.

Hope someone could help me with this situation

Regards

Marcel Alvarez

Cisco Nexus 1000V (separate Host?)

vmproteau — Fri, 20 Nov 2009 15:37:22 GMT

We are looking at this an I don't yet have my head wrapped around it but, my understanding is:

(1) You can have 1 or 2 VSM (primary and secondary)
(2) VEM becomes part of the kernel for ESX 4 Host you want managed by these VSMs
(3) A VSM will manage only one ESX cluster.

My assumption was that these VSM virtual appliance(s)would live within the cluster it manages along side VMs however; this diagram (attached) seems to show the VSM on a sepearate ESX Host that isn't a VEM instance?

It seems like the benefit to this 1000V is lessened if you need to purchase 1 (2 for HA) ESX Host(s) with licensing to support just these virtual appliances.

Am I not understanding this correctly?

HP Virtual Connect versus HP ProCurve (blade switches) in a c7000?

hwangeruk — Thu, 19 Nov 2009 12:26:01 GMT

I'm agonizing whether to go with HP Virtual Connect or HP ProCurve blade switches in my c7000.

Has anyone else been in this position?

Not sure I need the complexity or granularity of Virtual Connect Flex 10.

Is ProCurve the strategic direction for HP?

I can get a 6120XG and just give each server 10Gig, trunk 4 x 10G uplinks, seems simple.
Yet HP are steering me to Virtual Connect.

Has anyone owned both and can give their opinion?

vCenter Linked Mode

pearlyshells — Wed, 18 Nov 2009 18:52:31 GMT

We have VI3 and VirtualCenter2.5 and are about to upgrade. The licensing was previously upgraded for us since we have Platinum support and it shows that our vSphere and vCenters are of Enterprise licensing package. As I review the features that will be supported once we upgrade ( in about 2 months) I see that some key features that our licensing will not have is: Distributed Virtual Switching, host profiles..to name the ones I would like.

However, one question I have is: Does Enterprise license give us vCenter Linked Mode? Is that available in all packages?

vSwitch failures?

namoom105 — Wed, 18 Nov 2009 15:09:48 GMT

Do vSwitches fail? I am trying to figure out how many vswitches I will need for my network and have some serious concerns of how to handle my 8 nics. My current plan is to use 4 vswitches: Management, vmotion, storage (ISCSI), and VM Client Networks. If I assign 2 nics to each of the categories and have a vswitch fail then I will completely lose one of my networks rendering the boxes useless.

Updated MS network load balancing guidelines for vsphere?

foxy1977 — Wed, 18 Nov 2009 11:36:42 GMT

Hi there,

I am trying to find out if there is an updated version for the microsoft network load balancing guidelines for v4. The only one i can find is this one for v3:

http://www.vmware.com/files/pdf/implmenting_ms_network_load_balancing.pdf

I have previously managed to get this up and running on v3 esxi without any problems. I have managed to get a NLB cluster up and running on two servers but i am experiencing weird network connectivity issues. I just want to check my configuration up against the recommendations/guidelines and rule out any NLB configuration problems.

Can't ping VM, but Kernel on it's vSwitch

JoeAKim — Mon, 16 Nov 2009 09:05:46 GMT

Hi.

I'm experiencing some trouble:

I have a "HP DL380" with one NIC on board, and an extra NIC. Each having 2 ports. It's runnging ESXi 4.0 Installable.
When I create a new vSwitch, connecting to the 1st port (Mainboard), everything is fine.
When I create a second vSwitch, connected to either the 2nd (Mainboard) port, 3rd or 4th (extra-Card) port, and connect one of the VMs to it, the VM can't neither see the network, nor can it be pinged from outside.
Now, if I add a VMkernel to that vSwitch, the Kernel can be pinged without problems, but the VM is still isolated.

The on board card is a "NC373i Integrated Multifunctional Gigabit Server Adapter".
The extra card is a "NC380T PCI Express Dual Port Multiunctional Gigabit Server Adapter".
We have the same setup for some other customers, where everything works fine.

I'd really apreciate some help

Thanks a lot,
Joachim.

LinkDiscoveryProtocolConfig error removing physical NIC for migration to dvSwitch

fletch00 — Wed, 22 Jul 2009 18:58:52 GMT

Hi, I am attempting to migrating physical nics to a dvSwitch.

When I attempt to remove a nic from vSwitch0, it throws the error

"A specified parameter was not correct. Vim.Host.VirtualSwitch.LinkDiscoveryProtocolConfig"

How do I check it and resolve?

https://www.vmware.com/support/developer/vc-sdk/visdk400pubs/ReferenceGuide/vim.host.LinkDiscoveryProtocolConfig.html just describes that parameter as new in 4.0

I am on ESX4.0 build 175625

thanks

VCP

Network Failover Issue

dlietz — Tue, 17 Nov 2009 16:40:16 GMT

Working with a new install that consists of an HP c7000 blade enclosre with (3) BL680c's running ESX4 configured as a single cluster. Each server has 4 NIC's, vmnic0 and 1in a vSwitch for vmotion and service console, vmnic2 and 3 in a vswitch for VM's. The NIC's are configured in the active/active failover configuration. I'm using tagged vlans at the vswitches. The enclosure networking consists of (2) HP Virtual Connect 1/10 Gb Ethernet modues in bays 1 and 2. So vmnic0 and 2 map to bay 1 and vmnic1 and 3 map to bay 2. On the interconnects, there are 4 external ports on each switch configured as a shared uplink set carrying all of the VLAN's, each trunk patched in to an etherchannel group on a single Cisco 2960. So both etherchannels are patched to the same 2960.

When testing failover I'm having a slight problem. If I pull the plug on 4 ports from one of the interconnects, I loose maybe one ping from the VM's. However, when I plug the 4 patch cables back in, I will loose up to 6 or 7 pings when the switch fails back. What appears to be happening is that when I plug the 4 patch cables in and a link is established, the ESX servers are trying to fail back, however on the virtual Connect side the 4 ports do not immediately go in to active mode, but rather go to standby first, then active a few seconds later. I think what is happening is that the ESX servers detect the link and start trying to fail back before the ports have actually been made available on the VC switch.

There is a pdf drawing attached of the basic setup.

Does anyone know if there is a way this issue could be resolved, either on the ESX network config or on the HP VC config?

Thanks.

Dan

Multiple VLANs on one vSwitch

kegwell — Tue, 17 Nov 2009 19:41:08 GMT

I need a sanity check for my logic regarding VLANs. I'm not a networking person by any means, so please bear with me. I have an IBM BladeCenter with 4 Cisco 3012 switches, each of which correpsonds directly to a pNIC on a Blade server. In short, this means I have 4 pNICs per ESX server that are each connect to a dedicated Cisco 3012 switch.

pNIC1 --> pSwitch1
pNIC2 --> pSwitch2
pNIC3 --> pSwitch3
pNIC4 --> pSwitch4

The existing network environment has two VLANs, lets call then 5 and 10. My intent is to have two vSwitches which each have two pNICs assigned to them:

vSwitch1 --> pNIC1 & pNIC2
vSwitch2 --> pNIC3 & pNIC4

vSwitch2 will be used for Fault Tolerance and Vmotion, so we can focus entirely on vSwitch1. vSwitch1 will be used for LAN traffic and Service Console. Since we have two VLANs (5 & 10), it's my understanding that I should configure vSwitch1 as follows:

+ vSwitch1 --> pNIC1 & pNIC2
- Virtual Machine Port Group (VLAN 5)
- Virtual Machine Port Group (VLAN 10)
- Service Console Port

So, starting from the VM and working my way to the physical network, I believe it should go as follows:

+ Each VM will then have two vNICs. One vNIC will be assigned to 'Virtual machine Port Group (VLAN 5)' and the second vNIC will be assigned to 'Virtual Machine Port Group (VLAN 10)'.

+ These vNICs will be a port on vSwitch1, which are bound to pNIC1 & pNIC2

+ pNIC1 & pNIC2 have a dedicated connection to pSwitch1 and pSwitch2 (Cisco 3012)

+ pSwitch1 & pSwitch2 are connected to a core switch (Cisco 6500 series I believe) with the two VLANs already configured.

The part I am not certain of is the configuration of pSwitch1 & pSwitch2. My impression is VLAN trunking also needs to be setup on these switches?

Virtual Machine Port Group (VLAN5) --> pSwitch1 (VLAN 5 & 10) --> Core switch (VLAN 5)
Virtual Machine Port Group (VLAN 10) --> pSwitch2 (VLAN 10 & 5) --> Core switch (VLAN 10)

I believe I understand each segment of the configuration, but I am having a difficult time putting the whole picture together in simple terms. Thanks in advance.

Accidentally removed vmnic0 from vSwitch0 - ESX 4.0 & vSphere

norcalwaverider — Tue, 17 Nov 2009 19:32:02 GMT

I am sure no one has done this, but i accidentally removed vmnic0 from vSwitch0 and now I can not access the ESX server with vSphere or via http. Does anyone have any suggestions on how to recover from this erronrous mistake?

Thanks

VLAN Querry

gedi78 — Tue, 17 Nov 2009 13:44:50 GMT

I have two vSwitches that connect to different networks:

vSwitch0 (2 Teamed NICS): Connects to 10.0.0.x

vSwtich1 (2 Teamed NICS): Connects to 192.168.2.x/24

I need to virtualise servers on another network 192.168.1.x/24. Do I have any options besides getting another 2 NICS and configuring a vSwitch2?

Can I implement any form of VLAN on vSwitch1 so that I can house servers that I have on the 192.168.1.x/24 network?

Any help/comments appreciated.

Thanks

Difference Between Standard vSwitch on ESX3.5 and vSphere

number1vspherefan — Tue, 17 Nov 2009 11:52:53 GMT

Hello,

We've all heard about the dvSwitch functionality in vSphere but can somebody clarify to me the difference in functionality available from a standard vSwitch on ESX3.5 and vSphere? Looking at

http://www.vmware.com/products/vnetwork-distributed-switch/features.html

(and excluding the obvious difference in relation to VMsafe compatibility) the table in the above webpage states that the vsphere vswitch supports vlan segmentation whereas the esx3.5 vswitch doesn't. I would appreciate any thoughts / explanations in relation to this as, from what I can see, a standard vswitch in ESX3.5 and vSphere 4 both offer the same functionality in relation to VLANs i.e. (optionally) assigning a VLAN ID to port groups within vSwitches.

Thanks,
number1vspherefan.

VM's can't ping network but can ping host

Bootrix2 — Sat, 14 Nov 2009 00:40:40 GMT

Fresh install of vSphere 4.0

Default configuration, checked my Gateway and it's right.

I can access the ESX hosts through my VI, I convert my Physical Servers to the ESX host no problem.

Once it restarts I get VM tools installed and the network card configured (The physical server, the original is turned off).

I can ping my ESX host no problem, but I can't ping any other device on my network.

Any ideas?

Intel PRO/1000 NICs not working

MarcBouchard — Wed, 10 Jun 2009 17:34:27 GMT

New installation of ESX4 in our lab on "unsupported" hardware. The system has 3 NICs. 2 Intel PRO/1000 and an onboard Broadcom NIC. At installation I noticed something strange but figured I'd run thru it anyway. To summarize the issue, only one NIC works (vmnic2, one of the Intel adapters). On the physical switch, I see the link is up on all 3, but ESX says two of them are down

I need those extra adapters for iSCSI support etc...

Any help would be much appreciated!

Any software based switch that can be used for testing VLAN in ESX in box scenario?

MattG — Thu, 12 Nov 2009 18:24:13 GMT

I am running ESX in a box with Workstation 7 and custom networks (SC, VM, iSCSI). I would like to take the next step and connect those networks to software based switches to simulate/test VLAN configurations to the ESX hosts as well as get comfortable with switch configs. Is this possible? If so what is the recommended software to use?

-Matt

If you find this information useful, please award points for "correct" or "helpful".

Problems with vSphere Install

tnasseth — Thu, 12 Nov 2009 22:28:35 GMT

To the VM experts! :P

I'm very new to VM, and unfortunately am tasked with getting ESX4 vSphere4 running on a blade within our HP C3000 chassis. So, I installed vSphere4 and ran into a few challenges. First off, there are four NIC's available within the chassis (2 embedded, 2 via mezzanine card)

I've configured the Service Console with an IP on the subnet the NIC is attached to (10.100.1.180) and the strange thing is that from the ESX console, I can ping some IP's on the same subnet such as 10.100.1.105 let's say, but not 10.100.1.50. ICMP is enabled on the devices, so it isn't a firewall simply disabling echo replies.

Including a screenshot of the networking screen within vSphere for reference....

Second service console on separate gateway

dean.kirby — Thu, 12 Nov 2009 21:46:42 GMT

Hello,

I was trying to add an additional SC connection to a host today and I've run into some issues. So in my vswif1 in VC I added the new SC connection, with the separate gateway. The 2 SC connections are on the same subnet but I need them to use a different gateway. When I created the 2nd connection, I lost connectivity with my host to the SC. I read KB1008464 and followed that but no joy.

Can anyone assist?

TIA,
Dean

Any way to get an alert when NIC drops to 100MB or disk latency goes above a certain level in vSphere?

badger77 — Thu, 12 Nov 2009 16:00:20 GMT

Hello,

We have 3 NICs connecting to our iSCSI storage, and every so often one of the NICs drops from 1000MB to 100MB, which causes response problems (as you would expect).

I am going to raise a support call with our vendor, although I strongly suspect the Broadcom cards, but in the meantime can anyone think of a way I can actually get an alert when this speed drops in vSphere (or a clever Linuxy way, although I am not an expert) ? As far as I can see it's not actually logged in vSphere.

Disk latency also goes high when connection speed drops, but there does not seem to be an alert for this either.

cheers,

Craig.

comparison of performance between flexible and vmxnet3 nic

LucasAlbers — Thu, 12 Nov 2009 18:33:25 GMT

I have found performance comparisons between vmxnet2 and vmxnet3.

I have not been able to find any performance comparison between flexible and vmxnet3.

this host does not have any virtual machine networks, or you dont haev the permission to access them

storagevmotion — Wed, 11 Nov 2009 15:12:15 GMT

We just migrated all our ESX servers from vcenter 2.5 Update 5 to vcenter 4 as a part of upgrade. when we give access for users to change VLAN for a virtual machine i see the error as above.

i attached the screenshot of the error. all the VLAN configurations (network) are correct and virtual machiens on that VLAN working fine. I gave Rights to change the seettings for virtual machine. Even i gave Administrator rights for testing. Nothing worked out.

Any suggestion much appreciated...

VM disconnect from DVS

Bisti — Thu, 12 Nov 2009 13:37:56 GMT

I am encountering weird (for me atleast) issue - During VCB backup (fullvm, quiesced) of SQL server serving vCenter database (placed on separate vm than vCenter) the SQL vm stops responding and hang resulting in vCenter service stop and that is ok, but when all that happens the SQL vm vnic gets disconnected from dvs (distributed virtual switch) and as vCenter service is down I am unable to recconect that machine back to dvs. The vnic have "connect on startup" option enabled but reboots does not help. What makes thigs more messed up is that this issue affects some vms (getting disconnected after reboot when vcenter is down) and some not.

So if I get this right - if vCenter goes down than all vms will get disconnected after they reboot (On DVS)? If so than putting vcenter and sql on separate vms makes no sense in HA and DVS enviroment.

Virtual ESX 4 under ESX 4 can't ping out or in

djswarm — Wed, 11 Nov 2009 13:09:02 GMT

I'm on a Dell SC430 (pent D w/4gb) and used this nice article to set things up: http://www.vcritical.com/2009/05/vmware-esx-4-can-even-virtualize-itself/

Everything is evaluation copy.

The host esx 4 system works fine.
2 xp guests work fine and can ping each other and the console

But the esx 4 guest get a destination host unreachable when pinging out and don't respond to incoming pings.

It install fine. It can ping internally to the nic and loopback. It can be accessed and manipulated from the console.
Everything looks peaches and cream. NIC says it is up, except no packets out or in.

Nothing in /var/log/messages, nothing seems to be of note in the events, using e1000 virtual nic. Tried opening up the firewall, no joy.

I've set this up at another location under virtual server 2.0.2 and it works like a charm right from the box and looking around the web there seem to be people who say they have got it rolling under esx 4 and esx 3.5.

Granted I'm an esx noob and may not know where to look to find the error message waiting for me, but I admit being stumped at the moment. So since I've heard such good things about the vmware communities I thought I'd let you people get me back on the straight and narrow here.

Why you ask am I loading esx under esx? Well I have a single machine at this location and want to look at some vmotion features. Also keeping everything I can inside the box will keep the IT guys happier.

vCenter and Nated IP

malhareth1982 — Tue, 10 Nov 2009 21:13:00 GMT

I Installed one vCenter 4.0 managing two differnt Zones

First Zone: Data Corp and having same IP of Virtual Center
Second Zone: DMZ Zone having different Subnet and network
the DMZ zone can't be access by vCenter network but I create Nated IP thorugh it vCenter can manage the Service Console and the hosts on DMZ zone
but I faced problem when configured HA Cluster for DMZ hosts it is not configured

Can anyone helps me on this issue

Regards

VMXNET 3 reduces performance

alanrenouf — Thu, 29 Oct 2009 11:09:11 GMT

We have built some Xenapp VMs on a vSphere host, all is working fine and very fast when making a connection to the desktop via the xenapp client, after upgrading the NIC in any of the VMs to a VMXNET 3 adapter the performance of the desktop becomes unusable, it suddenly becomes very slow. If we remove the adapter and go back to the E1000 adapter whilst still being hardware version 7 all is fine again.

CPU/MEM/Network performance through the VIC and the OS look fine.

Anyone else hit this issue or know where i can start looking to resolve it ?

Thanks

Alan Renouf
http://virtu-al.net

Isolating VMotion traffic

Sly — Thu, 02 Jul 2009 21:51:58 GMT

I cannot find any other reason to isolate VMotion traffic, other than keeping the unsecurred information from being exposed to the rest of the network. Is a VLAN for VMotion only a security precaution?

Can't Remove Host from dvSwitch

1ppInc — Thu, 22 Oct 2009 23:21:17 GMT

Hey There,

I had a dvSwitch with two hosts on it and I'm trying to remove it completely. I was able to remove one of the hosts from the dvSwitch with ease... however, the other one just won't budge.

I recieve an error stating: "Cannot complete a Distributed Virtual Switch operation for one or more host members. DVS operation failed on host2, error during the configuration of the host: DVPort "1421" on DVSwitch "DvsPortset-0" is still in use: got (vim.fault.PlatformConfigFault) exception.

Now... There is no port "1421" on the dvSwitch. All port groups have been removed. "Everything" has been removed from the dvSwitch...

Anyone have any ideas? I'm trying to prevent a reboot at all costs, but if necessary I will take it down late evening.

Thanks!

Netflow in ESX4? what happened?

beckhamk — Sat, 27 Jun 2009 01:46:03 GMT

Does anyone know where the experimental support for netflow went in ESX 4?

Currently vmkload_mod netflow no longer finds this module.

thanks in advance

Looking for Best dvSwitch Solution with 8 pNICs per host

chrisaug — Mon, 02 Nov 2009 19:07:02 GMT

We recently implemented vSphere4 on NFS with a NetApp FAS-2050. Currently, our vNetworking is setup with a single dvSwitch with 5 port-groups as follows.

The numbers correspond with the pNICs while the colors represent the port groups (dvUplinks)... does this make sense so far?

1	Service Console
2	Service Console
3	vKernel (storage traffic)
4	vKernel (storage traffic)
5	VM Uplink
6	VM Uplink
7	vMotion
8	Promiscuous Uplink

My main question is, is this an acceptable setup? I've been trying to find out what would be better, one dvSwitch with 5 port groups, 5 dvSwitches each with one port group, or something in the middle. Does anyone have any feedback or recomendations?

Machines on distributed and standard v switches not communicating

firestartah — Wed, 04 Nov 2009 10:35:59 GMT

I have setup a distributed v switch with all the correct vlan numbers names etc. I have moved over one of my esx hosts to the distributed switch and coverted the machines on this host to the matching dvswitch vlans they were on on the standard v switch. Connectivity between machines on differing dvport vlans on the dvswitch seem to work fine as do machines on the same dvswitch vlan port groups and even between machines on differing vlans on the standard and distributed virtual switches, but the communication between machines on the old standard virtual switches and machines on the new distributed that are on the same vlan number won't communicate at all. I have checked and the ports are created correctly and i have recreated the dvports to confirm this.

Has anyone experienced this before? As obviously this means i will have to move over whole vlans to the dvSwitch ones to make sure communication isn't lost, which isn't something i wish to have to do.

Thanks

Gregg Robertson, VCP, MCSE, MCSA, MCTS, MCITP

1 vSwitch 2 Vmnic

vibranze — Mon, 09 Nov 2009 00:22:35 GMT

Dear All,

Under vSphere, is it possible for iSCSI and standard network co-exist under one vSwitch? I understand the for iSCSI jumbo frame, it's turn on from vSwitch, if that is the case what happened to the standard network? If they can't co-exist, what is the recommended solution for this? FYI, the server is a blade with 2 x 10G ethernet port, so instead of using each port for iSCSI and standard network, I'm thinking of combine them together under one port.

Thanks.

Cheers!

Calculate Data Transfer for a VM

mbleske — Fri, 06 Nov 2009 22:16:01 GMT

How would I go about getting statistics on the total data transfered to/from a VM? Specifically for an individual Port Group or even on a per network interface level.

Network performance between two VM's on same ESXi host

pacmantravis — Fri, 06 Nov 2009 16:21:01 GMT

We currently have 8 Vm's running on an ESXi host and we cannot get the network/file transfer speed to break 200Mb/s (Mb not MB)

The esxi host has GigE ports and is connected to the SAN via iSCSI.

Im used to seeing anywhere from 300-500Mb/s on a physical-to-physical transfer and thought it would be the same, if not faster with two VM's on the same host.

What is the limiting factor in this case?

Network-Settings vSphere4

hansis — Wed, 04 Nov 2009 10:29:22 GMT

Hello!

We have got our new vSphere4. I have not worked yet with network-settings and need the "best practice".

We have 3 ESX-Hosts with 4 physical NICS.

Also we have 2 redundant pSwitches.

my Concept:

vSwitch1 (2 NICS): ServiceConsole, vmKernel

vSwitch2 (2 NICS): virtual Machines

is this a well configuration?

and How should I make the settings of the vSwitches?

thx.

hansis

Host only Configuration

anilchaurasia — Sat, 07 Nov 2009 00:43:39 GMT

Hi All,

Can aanyone please tell me how to create a host only network on a host.

I Googled around for some time but could not find any poist that solves my problem.

I am very new to vShere and detailed steps would be appreciated.

Thanks

Distributed Switch - SPOF ?

ascheale — Fri, 06 Nov 2009 06:19:15 GMT

We had quite a long discussion regarding the new dswitch?

What happens when vCenter isn´t available? Will the vSwitch network connectivity be lost for the VMs ?

What if someone has a virtual vCenter and put that network on the dSwitch - i think that will be a problem if vCenter gets rebooted, or ?

Alex

Disconnected Service Console after deleting Virtual Nic

rdiphoorn — Fri, 06 Nov 2009 13:18:51 GMT

My situation:

1 vSwitch -> 2 Service Console portgroups

When I remove one of the 2 Service Consoles then the Service Console is only having network connection again when I issue this command on the console: service network restart

Anybody experienced this before? And how to solve it? It's like the Service Console Default Gateway is removed when I delete the second Service Console. (I have hardcoded the gateway in /etc/sysconfig/network)

MAC Address retention required

mk1968 — Fri, 06 Nov 2009 12:19:07 GMT

Does the virtual machine mac address change with an upgrade of VMware tools and virtual hardware within a Windows Server 2003 environment?
I am upgrading from version ESX Server 3.0.1 to version vSphere 4 and NEED to keep the MAC address of one of my Virtual Machines for licensing purposes.
I have had a look at this article however do not want to reserver MAC addresses manually if they will not change with the upgrade.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=9183481

Any advice or experience of wanting to do similar would be appreciated.

What is the limit of users that can connect simuntaniously to vSphere Infrastructure using the vSphere client

mansof — Thu, 05 Nov 2009 19:53:02 GMT

I have a question that I could not find in the documents.

is there a limit of users that can connect to vSphere Infrastructure, or vSphere ESX using the vSphere Client?

I am going to offere a large number of users to access my vSphere to download some VM's, but I do not want that after a number of connections the vSphere client just replies with a "Sorry no more connections allowed".

Thanks

NIC Configuration on a Dell m1000?

NYSDHCR — Thu, 05 Nov 2009 14:26:20 GMT

We are planning our vSphere4 install. We now have a Dell m1000, with 5 - m610s (half-height blades - 6 physical connections) and 3 - 710s (full height blades - 12 physical connections), and a Cisco 3130 switch on the back. This would give us 6 pnic connections to each blade. We will be running SC, iSCSI, NFS, Fault Tolerant, and 3 seperate vm networks (server lan, dmz, and a 3-tier). NFS will be 10GB. I would like to be as redundant as possible. We have 52 cpu licenses for Enterprise Plus.

What would be the best possible setups? Bonding 2pnics and creating several vlans? The Dell blade serves up internal connections between blades, as was explained to me by the Dell rep. This could serve Fault Tolerant and vmotion?

vSphere Networking

gedi78 — Thu, 05 Nov 2009 13:44:35 GMT

I've just upgraded both my ESX hosts to have 4 physical NICs instead of 2 and was looking for some networking advice.

Current Setup: vmnic0 & vmnic1are both teamed and configured Active/Active for vSwitch0. vSwitch0 contains Virtual Machine Port Group + Service Console Port + VMKernal Port. Both Physical NICs on each host go to different physical switches (No VLAN's are configured on the Physical Switches)

I am keen to add in 2 more physical NICs and setup another vSwitch on
each host so that I can virtualise VM's from our DMZ network that have
different IP addressing. Can anyone offer any advice on whether this is the best option given my setup?

I'm asuming I should also configure another Service Console port on the
second vSwitch for redundancy? - and another VMkernal Port group for
VMotion?

Any comments/queries welcome.

Thanks

Tagged Service Console

HammondC — Tue, 06 Oct 2009 19:01:09 GMT

I reciently moved my Service console to be a tagged in with the rest of my Virtual Machine networks. The attached picture says it all. Is this an acceptable configuration? It was dissapointed to note that the "currently has no management network redundancy" message did not go away. Previously the Service console was setup similar to the VMkernel with only 1 physical adaptor. I moved it to the current config (attached). Thanks.

VMotion Ports

jeffoutwest — Tue, 03 Nov 2009 21:24:56 GMT

For a network design, I am proposing three separate networks with corresponding virtual switches;

There are 8 physical nics available on the ESX host.

Managment Network - 2 nics teamed. One port onboard one port on pci card.

Virtual Machine Network - 2 or more nics teamed. One port onboard one port on pci card.

VMotion Network - 1 active nic, 1 stand-by nic. One port onboard one port on pci card.

A few questions.

Can VMotion take advantage of nic teaming for increase bandwidth?

If I want to design a seperate backup network for the guest OS, should this be part of a port group on the Virtual Machine network, then mapped to a physical port?

Thanks, -Jeff

A few quick network questions:

Can vMotion take advantage of Nic-Teamed ports?

Would you recommend encapsulating the

N1000v lost management access

wponder — Wed, 04 Nov 2009 04:44:32 GMT

I have a single ESXi host that is running a VSM. After setting up a standard config Management/Packet/Control when I was adding the same host to the DVS I lost all connectivity to the management interface. I have rebooted the host several times and tried to restore the standard switch from the physical console but I can not get connectivity back. I have also tried to remove add-on modules from the physical console as well with no luck.

Anyone know anything I can do to remove the VEM or get this host back on line?

Thanks,

IP change is causing problems

robkelley — Wed, 04 Nov 2009 12:48:23 GMT

Hi
I have changed the IP range of my vSphere network from 10.100.n.n to 172.28.0.n and most things are working, but looking in my firewall logs something is still using the old IP range.

The new IP of the vCenter is 172.28.0.11 and its name is vsphere-svr

The firewall reports it as:
OUTGOINGFW:ALLOW:1 TCP (br0) 172.28.0.11:2035 -> 10.100.101.27:9084 (eth1) MAC=00:0c:29:05:80:c0:ff:ff:14:00:03:00 LEN=48 TOS=00 PREC=0x00 TTL=127 ID=22050 DF SEQ=3260876032 ACK=0 WINDOW=65535 SYN URGP=0

Using TCPView on my vCenter server it reports 'tomcat6.exe:2480 - vsphere-svr:1995 - 10.100.101.27:9084'

I have searched the entire VMWare folder for files with this IP in but found nothing.

Any ideas where this IP is stored or how to fix this?

Creating VLANs using one nic on standard vSwitch on esx4.0

malhareth1982 — Tue, 03 Nov 2009 18:55:05 GMT

How I can configure two different VLANs using one NIC to separating VMotion network and service console which implemented on one virtual switch

configuringon virtual switch and physical switch

I need help on this, any one can help me

Regards

Issues with HA and nexus 1000v switches

chrisdp — Mon, 02 Nov 2009 01:33:06 GMT

Hello,

I have recently been testing out the nexus 1000v and have performed the default installations with success. I have kept the service console and vkernel out of the cisco switches and left them on standard switches fo easier management purposes ( I tried to migrate these components but since my vCenter is a VM in the environment I was losing connectivity and having problems so took it out again). Anyway when performing vMotion on my test VM's using the nexus I can easily migrate vm's between hosts. However when I force a crash on an ESX host in the cluster the VM's do not fail over to the other node as expected. All HA settings are configured correctly and what is strange is that when I bring the crashed ESX host back online the VM's power themselves back on despite the fact that I confirmed there are no startup/shutdown rules in place for any VM's in the cluster ( almost as though they are in some sort of standby mode until the host is back online. If anyone has any advice or things to check it would be appreciated. I contacted vmsupport and unfortunately they told me several clients have called about this issue and they currently have no idea what to do about the issue and to wait for a possible fix in the next update of vCenter ( which could be a while) and that because not very many clients have been using the cisco 1000v they didnt know this would be an issue. I found this answer hard to believe that it was never caught in QA testing on their side so I am hoping someone here has a better answer. If it helps I am also using an HP 7000series blade chassis with cisco 3020 switches in the backplane for connectivity so perhaps there is something that needs to be configured there to allow the port transfers during a HA migration? Again any help would be much appreciated.

Thanks in advance

Nexus 1000v Worth it?

cplatt01 — Fri, 16 Oct 2009 13:22:30 GMT

Anyone using this yet? I am curious if it is worth it or not. We are not going to have more than 30 VM's over 4 hosts. Just wanted to hear what others had to say.

CHRIS

Migrating VMs from one cluster (Virtual Switch) to new cluster (vNetwork Distributed Switch)

ChristianWickham — Fri, 09 Oct 2009 01:42:21 GMT

We have just purchased a new blade infrastructure to replace our aging ESX cluster.
The new environment is 4 HP blades, each with 16 CPUs @ 2.93Ghz and 72Gb RAM and 4x 5Gbps networking. These are ESX 4 Enterprise Plus.
The old environment is 4 IBM blades with 4 CPUs @ 1.6Ghz and 16Gb RAM, 2x 1Gbps networking. These are ESX 4 Enterprise, fresh installs from ESX 3.0.2.
All VMs have been upgraded to VM Hardware 7.
All 8 blades are connected to the same Cisco network switches, and to the same SANs. All blades can see all LUNs and VLANs.

We want to be able to migrate machines from the old blades to the new blades. We have been able to do this only if we un-configure the network links prior to migrating. The VMs can be migrated live, but we loose network connectivity as we have to un-check the "Connected" option for the network card. Then, once the VM is on the new blades, we need to select the new dvPortGroup-VLAN130 network before we check the "Connected" option. This causes around 5 minutes of downtime for each VM.

We have tried renaming the dvPortGroup-VLAN130 to "Production network" to match what is on the old blade environment, but this does not work. We have been unable to migrate at all if the network connection is enabled/connected in the VM - we get a warning that "Currently connecteed network interface 'Network adapter 1' uses network 'Production network', which is not accessible" and then refuses to migrate.

Is there a script or method that we can use to migrate machines from the old networking environment and blades to the new blades and Distributed Switch? How can we do this without downtime?

the resource vim.host.port group is in use

iamLinus — Thu, 22 Oct 2009 06:09:32 GMT

Hi Guys,

i have vsphere. two vswitch: vswitch0 and vswitch1. i wan to remove vswitch0 as it not longer needed as we move all service console and vm network to vswitch0.

However, I unable to remove vswitch0. It show error: A specific parameter was not correct.

next, i when to vswitch0 properties and try to remove virtual machine port group. It show error: the resource vim.host.port group is in use

please check my attachment and appreciate help.

thanks

How to remove a dvSwitch with activate dvUplink?

Testdrive2009 — Tue, 27 Oct 2009 19:33:10 GMT

Hi,

i have some horrible problems with the Distributed Switches

I set up a dvSwitch and added one host. Than i tried to migrate the Service Console to the virtual Switch.

The host network crashed completly and i have to reinstall it. Now i tried to remove the damaged dvSwitch but one Uplnk ( the one from

the Servce Console i tried to migrate) is still active. When i tried to delete the portgroup i get:

Distributed vNetwork-Switch remove
TestSwitchThe Ressource
vim.dvs.Dis-
stributedVir-
tualPort 261
is in use

Now my questions: how to remove the Distributed Switch and deactivate the dvUplink? and why i cant migrate the default Service Console?

Migrated virtual machine has lost network connection

HIWJoshua — Fri, 30 Oct 2009 21:34:44 GMT

I've been previously running a few Windows 2003 server machines on VMWare server, and we are migrating that all to a much more solid ESX-based structure. However, converting over my test server has caused issues. It appears to have no network connection. I can't ping the server either externally from the network nor internally in the same subnet. I can't gain access to the server itself to look at the network settings due to having moved hardware, windows is requiring me to re-activate. When I try to authenticate, it can't find the Microsoft authentication server with no networking.

The host itself is active and pingable, and the console on the Virtual Machine is working just fine - it's just that the Virtual Machine doesn't have a working network. It's set to connected via the settings.

Any thoughts?

Edit: My best thought is that the VM's NIC was configured with a specific IP, DNS server and subnet. When I moved it over, it no longer found the NIC that the VM was dependant on, and created a new one. This new one doesn't have the settings, so when it tries to get DHCP, it fails (it's getting an ip, but a nonsensical internal one that isn't part of our 192.168.x.x network). No idea how to get around this, though.

Final Edit: My assumption was right. WIth the new NIC, all the settings vanished. Enabling DHCP allowed for me to authorize windows again and get back on. Excellent.

Two LAN connection in guest OS

balaonline — Sun, 01 Nov 2009 05:41:40 GMT

HI all, Greeetings,

I have laptop in that i installed VMWARE workstaion. In that i created a TEAM of 4 win2003 servers.

When i applied NAT , all server ping perfectly. But i need two lan connection in particular two servers

for making network connection. because i need to install software that need two NIC In each two server.

I was tried by applying bridge connection but in guest OS showing one LAN connection only. I down t want NAT , i want to use my own ip address.

Please help me to solve this problem. Feel free to query if u have doubts.

Thank you,

vSwitch - pNIC - iSCSI - MPIO

Randles — Mon, 26 Oct 2009 18:06:23 GMT

Hi Guys,

I'm about to setup the virtual networking for two new ESX boxes, and I was wondering what the communities throughts were in terms of usage of NICs.

I have 12 NICs on each Host.- I figured I'd assign 8 of them like this for redundancy and increased throughput;

2 NICs - vMotion (Teamed / Port Channel)
2 NICs - service console (Teamed / Port Channel)
4 NICs - vm network (Teamed / Port Channel)

Now the problem comes when I decide how to assign my 4 remaining iSCSI OE NICs - I want to have my Datastores on the SAN but I also want to use the software iSCSI initiators on the VMs to connect to file server volumes etc.

My iSCSI target is on a completely separate physical LAN with redundant switches I was hoping to have 4 iSCSI OE NICs using MPIO to face the SAN. Can I combine the vmkernal iSCSI traffic with the VM iSCSI traffic on the same vSwitch?

Is there a better approach that I am missing? Any advice appreciated

vSwitch Bug? or User Error: Delete & Recreate Port Groups = Dead vSwitch

Randles — Fri, 30 Oct 2009 13:18:04 GMT

Has anyone else run into a problem like this?

I had two ESX boxes between which I wanted to set up HA and DRS - so I set about reconfiguring ESXBOX-2's vSwitches to match the configuration of ESXBOX-1's exactly. (Now we can forget HA / DRS etc this is just how I arrived at the vSwitch Reconfig)

I had 4 vSwitches and I wanted to move some Port Groups around - for example I wanted to use vSwitch2 for iSCSI which was currently configured on vSwitch3 - so I deleted some port groups and recreated them in the relevant places. I then by error deleted a good port group on vSwitch2; and promptly recreated it with exactly the same configuration as before.

Here's where I ran into a problem;

I was unable to ping the vmkernal / service console I had created on vSwitch2, and I was unable to ping to that subnet from the CLI "Destination Host Unreachable" on my pSwitches I noted that the Links were up / and aggregated but no packets were flowing. I tried refreshing the network from the CLI - nothing. I was completely stumped. I spent quite a lot of time trying to solve the issue, painstakingly checking my Cisco configs and ESX configs only to be 100% sure that configuration of ESXBOX-2s vSwitch2 exactly matched that of ESXBOX-1s vSwitch2 both on the ESX and Cisco side; which was puzzling as ESXBOX-1 was working as expected.

Then I decided to delete the vSwitch and recreate it instantly my problems were gone ! Is this a bug or a feature that others have encountered? Or is it simply user error

2 nics in different VLANs failing over for each other?????

w00005414 — Wed, 28 Oct 2009 20:23:15 GMT

Hi all,

We have three ESX hosts with 8 NICs each and we wanted to assign them in the following way

2 for iSCSI

1 for a private LAN for Linux VMs for file level backups through NetBackup Enterprise Server(we'll use VCB over iSCSI network for Windows VM backups)

3 for vmNetwork

1 for vmotion

1 for service console

Our consultant said we can use just 1 nic for vmotion and 1 for the service console (on different VLANs) with each being a failover for each other. Is this possible? I can't wrap my head around how this works if they are on different network 'segments'.

Any help would be greatly appreciated.

%EC-SP-5-CANNOT_BUNDLE2: error with ESX4, Catalyst 6509, Dell R710, & link aggregation

itoperationsgrhc — Thu, 29 Oct 2009 23:18:56 GMT

Using KB Articles, 1001938, 1004048, 1003806, & the VMware Virtual Networking Concepts guide. We set up etherchannels on our ESX4 farm using dell R710 servers with 3x Broadcom dual 1GB NIC cards added. 2 of the interfaces are aggregated for the VM Network & the VMkernel. 8 of the interfaces are aggregated as a virtual switchport on a single VLAN.
On the VMware config the VLAN is not set.

The issue is that we are receiving errors on the Catalyst 6509 containing the following error "%EC-SP-5-CANNOT_BUNDLE2:" . The interfaces are being suspended by the IOS because the speed is dropping from 1GB to 100MB. We perform a shut, no shut and the interface reconnects at 1GB.

When these interfaces drop out of the etherchannel we incur random routing misfires, I.e. the physical machine at 172.16.5.75/23 cannot ping the virtual machine at 10.4.32.82/21 but can ping 10.4.32.80/21 & 10.4.32.81/21. the machine at 10.4.32.82/21 cannot ping 172.16.5.78/23 but can ping 172.16.5.81/23 etc. When we do the shut no shut on the interfaces that have the issue; the interfaces are no longer suspended from the port channel the issue disappears.

The virtual interface was registering variable line transmission speeds on the NICs and this is causing the port-channel suspensions. We set all of the NIC interfaces in esx4 to 1000 Full instead of Auto Negotiate, leaving the Cisco 1GB interfaces at the default auto negotiate.

Why does having an interface in a port channel suspended cause this apparent routing error?

Any body else have a similar issue

I spent some time checking DNS, DHCP, WINS and making sure that the routing was working properly

Sample error log:

Oct 23 06:31:44 MST: %EC-SP-5-COMPATIBLE: Gi2/13 is compatible with port-channel members
Oct 23 06:31:44 MST: %EC-SP-5-COMPATIBLE: Gi2/23 is compatible with port-channel members
Oct 23 06:31:44 MST: %EC-SP-5-COMPATIBLE: Gi2/37 is compatible with port-channel members
Oct 23 06:31:44 MST: %EC-SP-5-COMPATIBLE: Gi3/13 is compatible with port-channel members
Oct 23 06:31:45 MST: %EC-SP-5-COMPATIBLE: Gi3/23 is compatible with port-channel members
Oct 23 06:31:45 MST: %EC-SP-5-COMPATIBLE: Gi3/37 is compatible with port-channel members
Oct 23 06:31:49 MST: %EC-SP-5-CANNOT_BUNDLE2: Gi3/35 is not compatible with Gi2/13 and will be suspended (flow control send of Gi3/35 is off, Gi2/13 is on)
Oct 23 06:31:49 MST: %EC-SP-5-CANNOT_BUNDLE2: Gi2/35 is not compatible with Gi2/13 and will be suspended (flow control send of Gi2/35 is off, Gi2/13 is on)
Oct 23 06:33:54 MST: %EC-SP-5-COMPATIBLE: Gi2/35 is compatible with port-channel members
Oct 23 06:33:53 MST: %EC-SP-5-COMPATIBLE: Gi3/35 is compatible with port-channel members
Oct 23 06:34:10 MST: %EC-SP-5-CANNOT_BUNDLE2: Gi2/23 is not compatible with Gi2/13 and will be suspended (speed of Gi2/23 is 100M, Gi2/13 is 1000M)
000290: Oct 29 14:55:22 MST: %SYS-5-CONFIG_I: Configured from console by admin on vty0

Lost Management connection

EQNish — Thu, 29 Oct 2009 15:47:41 GMT

For some crazy reason I have lost my management connection, I have no access via vSphear Client, according to the Console my IP address is assigned (I can not Ping). Is there anyway to "reset" the Management intereface/port/card My host is essentually down, none of my VMs will come up and I have no host access/controll what so ever, please help!!

82571EB Full/Half duplex issues

callee — Thu, 29 Oct 2009 13:46:25 GMT

Hello everyone,

I have a bit of a peculiar issue after freshly installing a bunch of vSphere hosts (with all available patches applied).

After each reboot the interfaces which are supposed to be set at 100Mbit/full reverts to 100Mbit/half instead. Switches are configured to be 100Mbit full and not auto-neg. And this issue only seems to apply to my pci-e controllers. The onboard NICS are fine (NetXtreme II BCM5708, hp dl380 g5).

Just to be sure I confirmed the settings in esx.conf:

/net/pnic/child0002/duplex = "full"
/net/pnic/child0002/mac = "00:15:17:33:fa:c8"
/net/pnic/child0002/name = "vmnic2"
/net/pnic/child0002/speed = "100"
/net/pnic/child0002/virtualMac = "00:50:56:53:fa:c8"

but running esxcfg-nics -l gives me the following output

vmnic2 10:00.00 e1000e Up 100Mbps Half 00:15:17:33:fa:c8 1500 Intel Corporation 82571EB Gigabit Ethernet Controller

GUI also displays the same. Setting the nics to 100Mbit/full during operation works fine and continues to run this way until I reboot the host again.

I could just write a script to reconfigure the nics post boot, but surely this shouldn't be needed.

Anyone with any brilliant ideas?

Thanks.

Best Practice in setting up vNetwork Distributed Switches

felix107 — Thu, 22 Oct 2009 18:44:41 GMT

Hi Everyone,

I'm in the process of setting up vSphere ESX 4 into our environment. I wanted in incroporate the new vNetwork Distributed Switch into our brand new Dell R610 servers. There is a total of 12 network port in each server. I'm not limited to the number of ports to setup. We do not have vlan setup in our network environment.

After carefully reading up on most of the documents on line in the VMware website. I still not sure how I should proceed. But here is a quick stab at how I was planning and designing the new vSphere Enterprise Plus for the vNetwork Distributed Switch.

I have one dvSwitch setup and 6 dvPortGroup added onto the the dvSwitch. Here are the list of all the dvPortGroups:

1. LAN

2. SAN (b/c we are using NFS and possible iSCSI)

3. Private Network (This is where private local traffic is done, this includes Management Network/Service Console for the ESX host)

4. vMotion

5. Fault Tolerance

6. Public LAN 2 (we are connect to another lan network in another building)

Note the following

Each dvPortGroup has two connection each of them are setup to be live.

There is no trunking/aggreation done yet, which I need to look into later.

I plan on purchasing the nexus1000 switch soon, is this how I should set things up?

Each dvPortGroup is uplinked to two network port on the Dell R610 servers to proivde redundancy and they are each connected to different switches for each service.

Currently we have 2 servers setup this way and everything is working accordingly. Is there anything that I might be missing that anyone can add. Please feel free to ask me any quesitons with my currently setup.

Thanks in advance.

help with a dvSwitch setup

jpd4save9 — Thu, 22 Oct 2009 16:06:50 GMT

I have created a dvSwitch with one uplink port, and have one nic from each node of a two node cluster attached to it.

Problem I have got is that it will not ping across the dvSwitch - so some thing is up with the uplink ports.

The uplink ports are connected to a switch, the ports are set to tagged to vlan 4.

PortGroup is set to the vlan of 4.

So i am not sure on what to do next, as most the docs suggest this should work.

Any help would be great.

esx 4 + vCenter 4.

VM are not able to ping with Gateway through another NIC (except console nic).

mallinath — Mon, 26 Oct 2009 18:24:33 GMT

I have configured one of the ESXi 4 on DL380 G6 server.
This server has 8 port NIC card like VMNIC0,VMNIC1,VMNIC2,VMNIC3,VMNIC4,VMNIC5,VMNIC6 & VMNIC7.

VMNIC0 is configured as console cable & assigned IP address 192.168.1.210 (VLAN 11) in a same way we have configured other NIC

VMNIC3 & VMNIC5 (VLAN 10)
VMNIC2 & VMNIC6 (VLAN 11)
VMNIC1 & VMNIC7 (VLAN 12)

We have created 3 VM's all are in VLAN 11.

Whenever we move this VM to a Vswitch connected to VMNIC2 & VMNIC6 we are not able to ping with the gateway but when i move these VM to a console switch (VMNIC0 which is in same VLAN 11) i can able to ping with the gateway through these VM's.

I have checked the NIC card 2 & 6 there is no any problem I have checked the switch port of NIC 2 & 6 ther are absolutely good.
I have checked the VLAN of port 2 & 6 they are correct (VLAN 11 as console VLAN)

What is the reason of this strange behavioure?
Is there anything i need to chek from ESX side?
Have you came across of this kind of behaviore from any another customer?

Please let me know your input on the same.

Thanks

How can I change vswif from a dVS to another and associate it with a vmnic (from CLI) ?

condor02 — Tue, 27 Oct 2009 06:17:37 GMT

Hello,
I have 2 dVS : dvSwitch01 and dvSwitch02. I wanted to migrate the service console from dvSwitch02 to dvSwitch01. When I did it from the GUI it didn't work.
I lost the communication with the ESX on the VCS.
When I typed esxcfg-vswitch -l I found that all vmnic are assoicated with the dvSwitch01. But the vswif is under dvSwitch02
dvSwitch01
vmnic0 ....
vmnic1 ....
vmnic2 .....
vmnic3
vmnic4
vmnic5

dvSwitch02
vswif ........

How can I remove vswif from dvSwitch02 and associate it with vmnic4 under dvSwitch01from CLI commands ?
How can I delete dvSwitch02 from CLI ?

does network communication among guests under same ESXi host go to physical/real network???

stevemebius — Fri, 23 Oct 2009 20:50:35 GMT

I am evaluating ESXi (was using vmware server 2.0 in the past).

I have guests under the same ESXi host communicating with each other. One host is the NSF server for the others.

The performance seems to be a lot slower than my previous vmware server setup.

I hope the network communication does not go out to the real network. Is there a way for me to verify that??

I know in the vSphere client. there is tab called "performance", but I am not sure if I can get what I want from there.

When I setup ESXi, I took all the defaults.

Many thanks.

Steve

Delegated networks with a dvSwitch

purduecjs — Mon, 26 Oct 2009 16:12:28 GMT

As a follow-on to my previous post related to Department role/permissions problem solved! Virtual Center vCenter , I thought it would be a good idea to elaborate on the new permissions available for datastores and networking. However, I'm having a bit of a problem with delegated networking - has anyone else attempted this?

Assume we have an external dvSwitch that includes multiple VLANs (VLAN 10, VLAN 11, VLAN 12, VLAN 13, VLAN 14) and I have two departments : department A should have access to VLAN 10, and 11 ... and department B should have access to VLAN 12, 13, and 14.

So I create a role called "Network Manager" with the permission of:

Network -> Assign network

and I assign this role for Department A to VLAN 10 and VLAN 11. I assign this role for Department B to VLAN 12, 13, and 14.

I also have a similar permission set on the VM (or folder containing the VM) for "Network -> Assign network".

So I login as Department A, 'edit settings' for a VM, add device - enet adapter, and I can choose among the VLANs that I have given perms on ... specifically VLAN 10 and 11. I finish adding the device and click ok to reconfigure the machine, then 'edit settings' again to change the VLAN from 10 to 11 and I get the error:

"This host does not have any virtual machine networks, or you don't have the permission to access them."

A work-around that I've discovered is to add the dvSwitch in question to a folder. Then on this folder add the "Read only" role for all of your deparmental groups (Department A and Department B, from my example). When they go to "Edit settings", behavior is as expected and they will only be shown those networks that have been granted "Network -> Assign network"

I've filed a request with support and they are looking into it, but I wanted to reach out to see if anyone has thoughts on a better way to delegate or if anyone else can replicate this issue.

Thanks very much,

Cameron J. Smith
System Administrator, Purdue University

Slow network and packets drops when one VM (vmxnet3) is under heavy traffic

Docent — Mon, 26 Oct 2009 11:12:44 GMT

Hi,

I have an ESXi 4 installation with three VMs running on top and two physical network adapters. One VM is used as a router machine for my network, it's Debian and has open-vm-tools installed. It has two vmxnet3 network adapters. One adapter is connected to a dedicated vSwitch ("External") and the other is connected to the second vSwitch ("Internal") with all other VMs. Virtual OS is configured with VLANs, so ESXi passes all VLAN IDs on the External vSwitch. On the other ESXi host there is an identical VM with Debian and the same configuration. They're both working in a failover cluster using linux-ha.

There is no problem when the machine is passive, e.g. not routing any traffic. However there is an issue with the active node: it's nearly impossible to connect to the services (web, remote desktop, etc.) hosted on the other VMs running on top of the host with active router node. The connection frequently drops and the file transfer via HTTP is just a few KB/s. What is interesting, the ESXi host is affected itself as the download of vSphere Client goes around 5 KB/s as well. When I shutdown the primary router node (thus causing failover to another), everything goes back to normal on that host but the problem moves to the secondary node. Linux itself is unaffected and it can handle heavy traffic of several hundreds of mbit/s.

The problem is gone when I switch vmxnet3 adapters on Linux to e1000. However e1000 doesn't work well with heavy traffic like >100-200 mbit/s. So I'm stuck with vmxnet3 and would really appreciate any suggestions.

Thanks,
Docent

configuring diff. IP space

sanj2976 — Mon, 26 Oct 2009 07:41:10 GMT

Hi,

I have a single series of IP say i.e. 172. 16. 0.0 for all the things i.e. vkernal, esxhost , vc server, gateway and all the vmimages.

Now i have a requirement to migrate 4 servers on the same virtual infrastructutre which has Ip of 192.168.0.0 series and i cannot change the Ip of this servers.

So, is this possible. As per my knowledge may be possible through VLAN.

If yes can any one help how can i goahead configuring.

thnks,
Sanj

Network drivers for WinPE

Nico2212 — Wed, 21 Oct 2009 13:40:05 GMT

Hi!

Im creating a custom WinPE image for use with sccm and VMWare. I need to put the right nic drivers in my PE image so that I can boot it up and get a network connection. My question is; what drivers do I need to install to get e1000 and flexible adapters to work with WinPE?

Hope someone can help =)

Question on best practice for NIC cards on ESX host.

ssd@bhcs — Fri, 23 Oct 2009 15:45:56 GMT

Hello all,

i have a question: Im trying to figure out what the best practice is for multiple NICS on an ESX host and how they should be uplinked to the physical Switch.

So, in a nutshell i have a host that has 6 coppers and 3 switches that are stacked.

Should i split these 6 coppers over the 3 switches for redundancy? in other words 2 coppers on each switch? whats the typical setup ?

any info is appreciated.

francisco-

ESX4i - 2 servers and 1 HP Switch - VLAN / Trunking ???

mabsoft — Sat, 24 Oct 2009 10:06:01 GMT

Hello

I wonder if anyone can confirm if my thoughts on building a lab with 2 esx4i servers are correct. ??

Suppose I have 2 ESX4i servers both with 3 nics.

1 nic on both ESX servers is used for an External WAN link
1 nic is used for potential storage network /management.

The final nic used for the internal network on both ESX4i servers is connected to an HP 1800G switch that supports trunking and 802.1q vlans.

This means I can power off a vm move it with vmotion to the other ESX server and then power it up - no problem since both servers nics are set up the same the vm just works.

I now have a friend who wants to use a couple of VM's for his project so I aim to split the internal nic such that his stuff works on vlan 90 and my stuff works on the default with no vlan, eg. this hopefully provides separation.

If I setup a vlan on both esx servers say vlan90 no problem, however I want to have vm's in both 90 vlans on either server to be able to talk to each other. Currently the two vlan 90's are marooned and will not communicate with each other. Can I achieve my aim using a vlan / trunking with the VST method from both ESX servers through a single HP 1800G physical switch ??.

Any advice or help would be really appreciated.

Thanks in advance.

Mike

To etherchannel or not?

AllBlack — Tue, 20 Oct 2009 23:09:49 GMT

Hi guys,

We are changing our ESX configuration as we bought a new storage solution.
I was wondering if someone could give me some advice whether to use etherchannel or not.
I don't have much knowledge of advanced networking so excuse my ignorance.

We started with enabling etherchannel on the switch ports that connect to our ESX NFS nics.
I have two nics dedicated to NFS IP storage (both on different chipsets) and I set both of them to active.
I left the Load balancing policy to the default originating port ID.
In this scenario, I am able to connect to some volumes but not others. Once we removed etherchanneling
on the switch it works fine. I have done some reading and could it be that it is because I am using the wrong load balance policy and it should be IP hash?

That leads to the following question, should we etherchannel at all? In our "old" setup, I have a NIC team of two active NICS on my data vswitch (not etherchanneled and default settings)
This seem to provide me with good failover.

Cheers

Please consider marking my answer as "helpful" or "correct"

2003 R2 x64 vNIC - Flexible vs. E1000?

hutchingsp — Fri, 23 Oct 2009 09:40:30 GMT

I've got a running 2003 R2 x64 VM. I noticed the vNIC is showing as an E1000 whilst all my other VMs (32bit Windows or 64bit linux) are showing as Flexible or Enhanced VMXNET.

I've read Available Network Adapters but I'm not clear what the "real world" differences are i.e. is one faster than the other, does one impose more CPU load than the other?

I ask as this VM has several Tb of file LUNs attached which have to be backed up over the network...

Problems with network traffic, using VLANs and trunks.

ThomasKristensen — Fri, 23 Oct 2009 12:16:18 GMT

I have set up a Vmware environment and have problems with the network to the virtuale machines.

I have a vm with the ip 192.168.0.90, it can ping itself and 192.168.0.1 as its default gateway, but everything else is unreachable.

The system is set up with 2x1gbit trunks to a HP2910al switch running VLAN. Port 11 and 12 are trunked so
Trunk group 1 is in VLAN ID 20, tagged and port 19 is also in VLAN ID 20, untagged, as the uplink to the 192.168.0.0/24 network.
Each ESX is running teaming of the 2 network cards with 1Gbit on each.

I can see, if I make mirrorport on my trunks that the machine i ping sends a reply, it just doesn't reach the vm.

Switch's address table looks rights, it can see where all the MAC addresses belong to.

The VM is also plug'ed into the right vSwitch with the VLAN ID 20

What could be wrong? I am missing something?

VMWare and VLAN w/virtual firewall

itbegary — Thu, 15 Oct 2009 19:44:31 GMT

As the subject states, we have 3 physical servers, one of which has a firewall. We would like to create some vlan zones for some of the virtual servers. Here is what we currently have:

Each server has:

vlan a: 0
vlan b: 1
vlan c: 2
vlan d: 3

Now, with that said, is there an easy was for the firewall on vlan a to be able to handle all traffic for all vlans? I know I can add the other vlans to the firewall as virtual interfaces but for testing, we will be adding lots of interfaces to this. So, I don't what to have to reboot the firewall each time we add an interface to it. I understand the concept of vlans from a switch perspective, but I just don't know how to tie it all together.

Unfortunately, these boxes are hosted offsite and we only have the 6u's of space where they are, otherwise we would opt for a real firewall. The switch connecting them does support vlans, if that helps (it's a cheaper web based switch).

Note, this is a development environment as I know most will say "Don't do this for mission critical stuff."

Message was edited by: itbegary

Test Environment

GRama — Thu, 22 Oct 2009 15:23:45 GMT

If I have two distributed switch 1 productionvm and the other for the testvm. Both is uplink to a different physical switch which doesn't have communication with each other. would they still be able to communicate inside my VMware environment? Thanks in advance.

Network performance suddenly slowed to a halt

itbegary — Thu, 22 Oct 2009 14:06:26 GMT

I have 3 servers, 2 are Dell 1425's running ESXi 3 (box A and B), another is a SuperMicro running ESXi 4 (box C). Everything was running fine until about 4 days ago. I noticed that one of the guests on box A was getting a mind blowing 40kb/s file transfer rate when doing a yum update against a repository on box B. Testing showed that any instance on box B or C can access the repo just fine and get various speeds well above the 5MB range when doing file transfers.

Box A had both Windows and Linux guests on it. It doesn't seem to matter which guest OS is installed, I still get this slowness. Sometimes on the Windows box, it's fine. Other times, it's not. I suspect that the same is true for the Linux boxes.

The only change I made about 10 days ago was to add a virtual switch on a seperate VLAN and put a single server on that one. I created this VLAN on all ESXi hosts though (in the same way with the same VLAN ID's). I would suspect that if this change is the root cause that I would be seeing this problem across the board. scp'ing a file from box B to/from box A is about 5mb/s so the physical network seems fine.

Realtime:
CPU on the affected host is about 30% (10 guests), memory is about 73% (6GB of 8GB allocated and used), disk is averaging 100kb/sec, network 17kb/sec.

Any ideas on how to approach this problem?

Move physical adapters to DVS

mcvosi — Tue, 20 Oct 2009 20:15:46 GMT

I initially setup my vCenter to host connectivity using vSwitches. I now want to move those adapters to the DVS where my other adapters are already. Will I lose connectivity to a host when I remove the adapter from the vSwitch to move it to the DVS? When I added the hosts to vCenter originally I did it by FQDN, not IP address, but was not sure if it would matter or not. Is it simply just adding the host back in to vCenter if connectivity drops?

Thanks.

Multi-gigabit uplinks to VMs with Flex-10 LACP

ChristianWickham — Fri, 09 Oct 2009 01:30:03 GMT

I have the following hardware;
HP BL 490 G6 with Flex-10 (Firmware 2.12) connected to Cisco 3750 core switches (4 in a stack)
ESX 4.0 with Virtual Center 4, vNetwork Distributed Switch.
BL 490 blades have 2x LOM which have been set to 4x 5Gbps links.
Both Flex-10 modules have 3 uplinks to the Cisco 3750 switches, each at 1Gbps, all in one vNet. These are assigned to LOM1-a and LOM2-a (we have other links assigned to LOM1-b and LOM2-b).

We are trying to get multi-Gigabit connectivity to Virtual Machines in ESX4, using the above infrastructure. In HP Virtual Connect, we can see that 3 links are active (and 3 are standby) - so we would expect at least 3Gbps. These links have been configured on the Cisco swich with

interface Port-channel17
description HP Blades
switchport trunk encapsulation dot1q
switchport mode trunk
interface GigabitEthernet1/0/49
description HP Blades
switchport trunk encapsulation dot1q
switchport mode trunk
channel-protocol lacp
channel-group 17 mode active

In the ESX console, we can see that the network cards are operating at 5000 Full Duplex, but none of the virtual machines are getting more than 1Gbps.
We have two network cards in a team in the dvPortGroup, each network card is set to 5Gbps and has 6x1Gbps links (3 active, 3 have gone into standby mode) and the network adapters are configured for "Route based on IP hash" (we also tried "Route based on originating port")

Can anyone help us to get more than 1Gbps to each VM?

Problem with VM on DVS

mcvosi — Thu, 08 Oct 2009 21:34:53 GMT

I have one VM that refuses to operate on a distributed virtual switch. It works fine on a host-based virtual switch. Strangely, egress traffic from the VM works, but nothing can connect to it. I've tried creating new port group, but that hasn't worked. It's not a host issue as I have other VMs that work without any issues. I've also tried re-installing the Tools, just to be safe.

Anyone have any ideas?

intermittent network loss for single clients on shared vSwitch

macallan — Fri, 09 Oct 2009 18:21:39 GMT

Various RHEL and CentOS clients, in ones and twos, lose access to the uplink out of the vSwitch either during vMotion transfer, or at semi-random times thereafter. All other VM clients can see the network and the outside world, and the problem VM can still ping other clients on the same vSwitch. Migrating the problem VM to another server, and back, often clears the problem. Running "service network stop|start" or rebooting the client does not help.

The host environment is VMware ESX, 3.5.0, build 163429, on IBM x3850 hardware connected to the same Cisco switch stack. The switch stack doesn't have trunking turned on, so all of our ports get the same, single VLAN from the provider's network. The three servers in the cluster have identical vSwitch configurations, and nearly identical hardware (one has three physical NICs connected) but all of the hosts only use one NIC for uplink from their vSwitches, presumably due to low load. Network staff report all relevant physical switch ports are configured identically, and there are no logged events on the switch stack.

Any VM transferred with vMotion connects to the same NIC as other, running VMs on the same vSwitch. The client's current adapter is "Flexible" and the same version of VMware tools are installed on the working and non-working VMs. Intermittently, a vMotion'ed VM appears to transfer correctly but the VM loses network connectivity. It cannot ping or be pinged, but all settings appear normal and unchanged otherwise. Using vMotion to return it to the first server gives it network connectivity again. Sometimes the transfer works, and connectivity drops minutes or hours afterwards. The VM still appears to be in exactly the same state, but the VM cannot see the outside network.

The time on all hosts is within 2 seconds, and they have been getting NTP updates. I'm getting rational output from
esxtop
esxcfg-vswitch -l
The oddity is occurring on one physical server, but the symptoms are never server-wide.

I'm as interested in diagnostic tool/technique recommendations as I am in fixing this. I've only worked with VMware for half a year, out of my 14 years as a unix admin. Thanks!

NLB - Two VMs can't ping eachother when on seperate hosts

bramvermeulen — Tue, 13 Oct 2009 12:37:09 GMT

Greetings,

I've been running a few tests with NLB in the different set-ups as adviced here:

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1006558&sliceId=1&docTypeID=DT_KB_1_1&dialogID=42912404&stateId=0%200%2041454115
http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1006778&sliceId=1&docTypeID=DT_KB_1_1&dialogID=42912404&stateId=0%200%2041454115

I've found that Unicast works best in our network environment, however there is still an issue:

In http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1006580&sliceId=1&docTypeID=DT_KB_1_1&dialogID=42912690&stateId=0%200%2041454807
it says "Since all hosts in
the cluster have the same IP Address and the same MAC Address, there is
no inter-host communication possible between the hosts configured in
Unicast mode. A second NIC is therefore needed for other host
communication."

However, even with a second NIC the VMs can't ping eachother when they're on a different esx host. When they are on the same esx host the VMs can ping eachother. Also, when they're on different hosts, while one VM can't reach the other on their dedicated IP, the VMs can reach other when pinging the second NIC (the one without a gateway filled in).

For example the next 2 hosts are connected in unicast:

VM1 has 2 NICs, NIC1 = 172.18.18.41 with gateway (dedicated IP) and NIC2 = 172.18.18.43 without gateway and WLBS enabled.
VM2 has 2 NICs, NIC1 = 172.18.18.42 with gateway (dedicated IP) and NIC2 = 172.18.18.44 without gateway and WLBS enabled.

When on the same ESX host VM1 can reach VM2 on both 172.18.18.42 and 172.18.18.44 and vise versa.
When on different ESX hosts VM1 can't reach VM2 on 172.18.18.42 but he can reach VM2 through 172.18.18.44. In this case VM2 can't reach VM1 on 172.18.18.41 but can reach VM1 on 172.18.18.43.

I've followed and applied the settings adviced in this KB article: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1556&sliceId=1&docTypeID=DT_KB_1_1&dialogID=42912404&stateId=0%200%2041454115

I even tried the microsoft kb as stated here: http://support.microsoft.com/kb/898867 eventhough the VMs have 2 NICs but of course that didn't help either.

Does anybody have an idea what is going on here?

The fact that it works on the same host but not on different hosts kind of points to a network issue inside VMware.

Thanks in advance,

Bram

Understanding vSphere™ IP Pools

nicholas1982 — Sat, 11 Jul 2009 12:33:47 GMT

I see when you click on a vSphere™ datacenter there is a IP Pool tab. Could someone kindly explain what it is used for, maybe an example of when you would need to use it, I see you can associate it with a virtual network too.

Thanks

Nicholas

ESX 3.5 and 4.0 LACP support

OlafB — Wed, 23 Sep 2009 19:51:35 GMT

Hi,

The documentation is a bit fuzzy about this. Docs state that VMware supports static LACP but not dynamic LACP. The IEEE has different wording for this: active LACP and passive LACP. The question is: What does VMware really support? Here is an extract from the IEEE:

Passive LACP indicates the port's preference for not transmitting LACPDUs unless its partner's control vlaue is active LACP (i.e., a preference not to speak unless spoken to). Active LACP indicates the ports preference to participate in the protocolo regardless of the partner's control value (i.g, a preference to speak regardless).

In VMware terms, does static LACP mean passive? If that is the case, you still have to send LACPDUs if requested from the link partner. VMware doesn't seem to be able to send LACPDUs. Or does static mean: Force the "trunk or link-aggregation" without supporting the LACPDUs? Then VMware shouldn't claim that they support LACP at all (only static link aggregation, not LACP).

Any clarification would be greatly appreciated.

ESXi 4 - I have 2 vmkernels on two separate networks, why only allowed 1 gateway?

ThePitViper — Thu, 27 Aug 2009 18:43:45 GMT

I have not had this issue with ESX Classic because I can set the Service Console for Management traffic and have a separate gateway.

My issue is ESXi has 2 completely different networks....1 for Management, and 1 for SAN. The Management address is public address with the correct gateway, but the SAN is a private address, but it automatically gives the second Kernel the Managements Gateway. Why cant I change the SAN gateway without messing with the Management gateway.

Setting up VMDq on ESX 4.0

bookbinder — Fri, 25 Sep 2009 20:53:23 GMT

I have a Intel 10 Gigabit AF DA Dual Port Server Adapter that I recently installed on ESX 4.0 host. I am trying to setup VMDq on the sever so that it is optimized for jumbo frames. I found a document on how to do this in ESX 3.5 but nothing for ESX 4.0. Any ideas where I can find a updated document for ESX 4.0?

Thanks,
Eric

Easy way to remove a vDS?

burdweiser — Mon, 21 Sep 2009 18:50:47 GMT

Is there an easy way to remove a vDS other than applying a host profile?

The reason I ask is because I'm not sure at the moment how a host profile would affect some of my hosts in a test cluster. Some hosts have 2 physical nics assigned to a vSS and some have 1. Would I be safe with using host profiles in that type of configuration?

ESX host disconnects from vCenter when adding nic to form team on switch with service console

Zacnutz — Wed, 23 Sep 2009 08:03:27 GMT

OK Guys and Gals, let me lay the scene....

I have a HP C3000 blade system with 6 x BL460c G1 blades. This is all connected to a HP EVA SAN via fibre. Each blade has 4 x physical nics. This setup has been running ESX 3.5 for about 12 months without any issues. About 8 weeks ago, I upgraded the environment to vSphere. All ESX hosts have been upgraded using update manager to ESX 4.0.0, 175625. Each and every VM has been upgraded to version 7. This has been running fine for 8 weeks or so. From day 1 (even when running ESX 3.5), I have had the following standard switch configuration;

vSwitch0 - 1 x uplink, 1 x "VM Network" port group (which all VM's hang off), 1 x "Service Console" port group.

vSwitch1 - 1 x uplink, 1 x "VMOTION" port group.

I have no distributed vSwitch in this environment. I run HA, and DRS. Also from day 1, I have been receiveing the alert "Host xxxx currently has no management network redundancy". This I understand is due to no uplink redundancy on the vSwitch.

Today, I have finally pulled my finger out and tried to rectify this on each host. By my understanding, adding a second uplink to each vSwitch will get rid of the alert, as well as give me reundancy on both vSwitches, which is what I want to achieve. Here's the steps I follow performed on my vCenter server (which happens to be a VM runnign on a different host to the one I am configuring);

1. Select the Properties of my vSwitch.

2. Select the Network Adapters tab.

3. Select Add.

4. Select one of the Unclaimed Adapters by ticking the check box, then selecting Next.

5. Select Next again, leaving the Failover Order as determined by the vCenter.

6. Select Finish.

This is where I run into problems..... Basically, the vCenter sits there for a minute or so, and then errors out telling me it took to long and host hasn't responded. I then loose connectivity to the ESX host that I was configuring. vCenter shows it as disconnected. I can't view any of its properties or access any part of it via vCenter. The funny thing is that any VM on that host can still ping everything.

Turning my attention to the host, if I try to reconnect...I can't. The only way I can get it back is to connect via the ILO2, and issue a reboot. Now, when it reboots, and I reconnect through the vCenter console and look at that hosts network config, it has two uplinks on that switch. However, None of the vm's on that system can connect to anything (even if they were before the reboot when the host was showing disconnected). If I immediately remove the uplink that has been added, the vm's start talking again without any issues whatsoever.

I have tried this with VM's on the host and also with no VM's on the host. Each time, I get the same result.

I have tried this on two of the hosts so far, and I get this same error/reboot/no-ping issue on each of them.

Any ideas? I haven't hit the Knowledge base too hard as yet....as I was planning to do that tonight.

vDS network problem

iNik0 — Thu, 20 Aug 2009 17:53:18 GMT

Hi all,
i've migrated my Standard Switch to the new vDS but i've many problems with vms in port groups..
i've created all my port group with respective vlan Id and the migrate the vm networking .. but something strange occur.. some machines in the same port group (and the same vlan)
cannot reach the gateway and so cannot exit on internet while other can.. i cannot understand what's the difference ..
how can i check ?
thank you so much for help,

N.

vSphere and Cisco Nexus

fish6288 — Fri, 16 Oct 2009 14:54:41 GMT

Just wondering if anyone out there has seen this kind of behavoir with the Nexus 1000v and vSphere.

We currently have 4 vSphere host running with Cisco Nexus 1000v. We have multiple vlans going thru the Nexus and everyting has been working just fine untill 2 days ago. Now all the sudden some of the hosts are having issues connecting to certain Vlans for the VM nics.

We have 4 different vlans going to each Host...we will use vlan 2, 3, 4, 5 just as example numbers.

Host 1 - VM conenctivity works on all vlans listed

Host 2 - VM connectivity only works with vlans 2, 3, and 4 now

Host 3 - VM connectivity only works with vlans 2, 3, and 4 now

Host 4 - VM connectivity onlt works with vlan 2 now

Before 2 days ago all hosts could use every vlan for VM nics...now only 1 can use them all and the others can not. Two of the hosts can see all but 1 of the vlans and the last host can only see 1 vlan.

As a note...I can the all the VLANs when assigning the Virtual NIC to the VM...its just that the connectivity within the VM does not work for those particular VMs. Example: I can assign vlan 3 to a VM Nic on host 4 and cant ping anything from that vm on that vlan BUT if i vmotion it over to host 2 i can ping things just fine in that vlan.

Very strange to me. We have rebooted the vCenter server, restarted the Nexus 1000v Switch and even failed over to the standby, and have rebooted the Hosts. Our network guy says nothing has changed on the switch side and that all the ports are configure the exact same for each host.

Im stuck...Just curious if anyone has run into porblems where hosts start loseing vlans when using the nexus 1000v switch.

Thanks

Strange issue with ESXi 4.0 and routing

kenwardc — Sat, 17 Oct 2009 17:38:19 GMT

Hi Folks

I'm new to VMWare. I have installed ESXI 4.0 on a DELL 2950 server with two NICs. When I originally installed the software and started out on my learning curve, I did it at home and used my ADSL router and ADSL line as the network. The only machine on the network was the DELL 2950 and the VMs I created inside it. Everything worked 100% to and from everywhere. I could see the DNS servers and surf the web from any of the created VMs (combination of Windows 2008 Server and CentOS 5.x) so just assumed everything was set up correctly.

Now, I've moved the box to our rack space at the datacenter and the people there have given us a brand new network range to use with the machine, because basically we weren't given enough IP addresses to grow to the number being used on the old range, let's call it 12.13.14.x. The new IP range has been put into our Netscreen firewall routing and the Management Console was used to change the network IP addresses, subnet mask and gateway to the new address range before we moved the DELL unit to the rack.

The rack has an HP Procurve network switch as the interface between the Datacenter switch (the outside world) and our servers. So... it's switch, firewall, servers.

The address range is: 2.3.4.192 - 2.3.4.255 with .193 being the first available address and .253 the last. The .192 and .255 IPs are used for broadcast addresses and the gateway address set to .254. The subnet is /26 or .192

The host is set up as follows:
NIC number 1: Set up with address 2.3.4.193, subnet mask .192 and gateway of .254
NIC number 2: Set up with address 2.3.4.194, subnet mask .192 and gateway of .254

On advice from the datacenter support team, I separated the VMkernel Port (using vSwitch 0) and made it the managementnetwork. It is attached to physical adaptor vmnic0 and the IP address is set to and appears as 2.3.4.193. As I can see the machine from remote, I'm assuming this is working OK. I can attach to the host using the vSphere client and manage everything as expected.

The VMs are all attached to the second network, called vSwitch1 (Virtual Machine Port Group) and appears to be using adaptor vmnic1. There is no mention of IP address so I have no idea whether this network is using the .194 address as above or not.

Just to throw you all a little curved ball, looking at the "Observed IP Ranges" in network properties from the vSphere client, it says that the observed IP ranges (on both NICs) are 12.13.14.x and there is no mention at all of the new range i.e. 2.3.4.x

Here's the problem. Although I can see the host from the Internet, I cannot see any of the VMs contained inside it from the Internet. If I go to the host using vSphere Client, then choose a VM, open the console, login and open a terminal window, I can ping the host but I cannot ping anything outside the 12.13.14.x range and I cannot ping any of the other Vms. Here's an example of what's happening:

Our DNS servers are both inside the 12.13.14.x range, on 12.13.14.65 and 12.13.14.42

If I ping "google.co.uk" I see an instant lookup on DNS, the IP is resolved correctly, but then there is nothing. No replies.

If I ping ourdomain.co.uk, which is hosted within our 12.13.14.x range, I get the same instant lookup and then lots of ping replies from that host.

If I ping the VM on 2.3.4.196 from the VM on 2.3.4.195 I get absolutely nothing so there is a disconnect there, too.

From where I'm sitting and taking into account my inexperience with VMWare, it seems as though the "Observed IP Ranges" are telling the host to only use that range. How do I fix this issue? Any help would be gratefully accepted.

Regards and thanks
Chris

Intel 10Gbe AF DA (82598EB) dual-port adapter experience?

max.inglis — Tue, 30 Jun 2009 16:48:45 GMT

Hey - I was wondering if anyone else has tried these cards out with ESX4 yet. I'm having what is either an incompatibility or an incredible failure rate. Known-good switch ports and 10Gbe cables have been used to test, and I went through 5 cards to get 2 working (4 ports total). They show up in ESX fine but the ports show as down both on the switch and in ESX. I had 4 working before the weekend and now today another is showing as down, and I went and tested it with a known-good cable and port, and its not working. Any thoughts?

Thanks!

Max.

http://ITangst.blogspot.com

Network segments & virtual switches

emaredskin — Sat, 17 Oct 2009 19:43:39 GMT

Hello. I have a question about associating physical NIC ports to one virtual switch. I have two network segments. The first owns a group of VLAN ID's. The second owns a network isolated from the VLAN ID's just mentioned. Can I associate the physical NIC ports, which are connected through the physical switches to the two network segments, on the same virtual switch defined in VMware ESX 3.5/ vSphere ? I suppose I'd better associate the two network segments to two virtual switches , but I'd like to hear from you.

Thanks in advance for your help

vmxnet3 - features and use information - tips and tricks

iben — Mon, 04 May 2009 16:07:11 GMT

Glad to see this has been posted and we can talk about it now... please share your experiences and let us know if these tips work for you and what sort of performance benefits you've noticed when using this new driver.

We've been switching our Windows and Linux VMs to use "VMXNET Enhanced" for some time now and see public information on the new VMXNET3 NIC for guests...

This Thread has been started to help with procedures on the conversion of existing machines from older NIC to newer NIC as it is not 100% straightforward and there are some tricks to remove old hardware and change to new hardware. This would be similar in the physical world to changing from a 100 BaseT PCI Card to a GigE card. The old drivers need to be removed, new drivers installed, and IP Addresses moved over. If you just remove the old NIC and install the new one you may end up with a IP Address Conflict error saying the Address you are trying to use is already in use on another Network Interface. The problem is that when you open Device Manager the old NIC is hidden. See below for steps on how to overcome this.

Question: What is VMXNET3?

Answer: VMXNET3 builds upon VMXNET and Enhanced VMXNET as the third generation paravirtualized virtual networking NIC for guest operating systems.

New VMXNET3 features over previous version of Enhanced VMXNET include:
• MSI/MSI-X support (subject to guest operating system kernel support)
• Receive Side Scaling (supported in Windows 2008 when explicitly enabled through the device's Advanced configuration tab)
• IPv6 checksum and TCP Segmentation Offloading (TSO) over IPv6
• VLAN off-loading
• Large TX/RX ring sizes (configured from within the virtual machine)

What's New in vSphere 4.0

http://communities.vmware.com/viewwebdoc.jspa?documentID=DOC-9225&communityID=2701

http://www.vmware.com/files/pdf/VMW_09Q1_WP_vSphereNetworking_P8_R1.pdf

From the Cisco document:VMware vSphere 4 and Cisco Nexus 1000V Series

VMware vNetwork module that encompasses the vDS and VMXNET-3 enables inline monitoring and centralized firewall services and maintains the virtualmachine's network run-time characteristics.

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9902/solution_overview_c22-529767-00.pdf

Tech Notes

Flexible shows up in Windows Device Manager as an “VMware
Accelerated AMD PCNet Adapter” and Enhanced vmxnet show up as “VMware
PCI Ethernet Adapter”.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1001805

Flexible — The Flexible network adapter
identifies itself as a Vlance adapter when a virtual machine boots, but
initializes itself and functions as either a Vlance or a vmxnet
adapter, depending which driver initializes it. VMware Tools versions
recent enough to know about the Flexible network adapter include the
vmxnet driver but identify it as an updated Vlance driver, so the guest
operating system uses that driver. When using the Flexible network
adapter, you can have vmxnet performance when sufficiently recent
VMware tools are installed. When an older version of VMware Tools is
installed, the Flexible adapter uses the Vlance adapter (with Vlance
performance) rather than giving no network capability at all when it
can’t find the vmxnet adapter.

Enhanced vmxnet — The enhanced vmxnet adapter is
based on the vmxnet adapter but provides some high-performance features
commonly used on modern networks, such as jumbo frames. This virtual
network adapter is the current state-of-the-art device in virtual
network adapter performance, but it is available only for some guest
operating systems on ESX Server 3.5. This network adapter will become
available for additional guest operating systems in the future.

Networking Error, IP Address Already Assigned to Another Adapter
KB Article 1179
Updated Jan. 07, 2009

Why do I see an error message that “The IP address XXX.XXX.XXX.XXX…” is already assigned to another adapter?

Solution

Under certain conditions, you may see the following error message from a Windows guest operating system:

The IP address XXX.XXX.XXX.XXX you have entered for this network
adapter is already assigned to another adapter Name of adapter. Name of
adapter is hidden from the network and Dial-up Connections folder
because it is not physically in the computer or is a legacy adapter
that is not working. If the same address is assigned to both adapters
and they become active, only one of them will use this address. This
may result in incorrect system configuration. Do you want to enter a
different IP address for this adapter in the list of IP addresses in
the advanced dialog box?

In this message, XXX.XXX.XXX.XXX is an IP address that you are
trying to set and Name of adapter is the name of a network adapter that
is present in the registry but hidden in Device Manager.

This can occur when you change a network connection’s TCP/IP configuration from DHCP to a static IP address if:

You have upgraded VMware virtual network adapters (for example

when you migrate a virtual machine from an older to a new version of
VMware software.)

You have added and removed network adapters multiple times.

The cause of the error is that a network adapter with the same IP
address is in the Windows registry but is hidden in the Device Manager
(My Computer > Properties > Hardware > Device Manager). This
hidden adapter is called a ghosted network adapter.

Using the Show hidden devices option in the Device Manager (View

Show hidden devices) does not always show the old virtual NIC

(ghosted adapter) to which that IP Address is assigned

Microsoft addresses this issue in their Knowledge Base article
269155, which is available at the time of this writing at
http://support.microsoft.com/?kbid=269155.

To resolve this problem, follow these steps to make the ghosted
network adapter visible in the Device Manager and uninstall the ghosted
network adapter from the registry:

1. Select Start > Run.
2. Enter cmd.exe and press Enter.
3. At the command prompt, run this command:

set devmgr_show_nonpresent_devices=1
4. Enter Start DEVMGMT.MSC and press Enter to start Device Manager.
5. Select View > Show Hidden Devices.
6. Expand the Network Adapters tree (select the plus sign next to the Network adapters entry).
7. Right-click the dimmed network adapter, and then select Uninstall.
8. Close Device Manager.

What does VLAN off-loading is doing?

differentthinking — Thu, 18 Jun 2009 05:31:22 GMT

Good morning (Germany) everyone,

I just read trough the enhencments of vmxnet3 within vSphere4 but I didn't find any further information about VLAN off-loading.

I just have an idea what it could do - but I've no clue if I'm right. So hopefully somebody could explain what VLAN off-loading is doing.

Thanks in advance!

Robert

How to define subnets

curiousagain — Thu, 15 Oct 2009 09:17:31 GMT

Hi,

I have 2 Hosts with HA, SAN and 2 Switches. Each host has 4 NICs and the SAN has 1 NIC. (See attached scheme)

I would like to put the VMs under subnet '192.168.3.X'

The hosts & SAN with subnet ''192.168.0.X'' to switch 2

The hosts with subnet ''192.168.2.X'' to switch 1 and the exit to the outside world with subnet '''192.168.2.X '

All of this is to avoid access from VM's to Hosts and SAN.

My problem is, when I'm trying to connect from the outside world (''192.168.2.X') I can see only the hosts but not the VMs or SAN.

How do I need to configure it to make it work?

I'll appreciate your response.

Thanks.

network design suggestions

w00005414 — Tue, 13 Oct 2009 21:09:44 GMT

Hi all,

We are designing a new VMware environment with three IBM ESX 4 hosts, a physical server acting as the VCenter server, and the vmdk files for the VMs will live in a Dell Equallogic PS6000E connected via iSCSI. Our IBM consultant said we should have 4 networks, they are

Service Console

Vmotion

vmNetwork

iSCSI Storage

The switches we are looking to use are Nortel 4548GT-PWR. A couple questions are (and bare with me, I am just getting my head wrapped around all this).....

1.) Has anyone used these switches? Any pros and/or cons or suggestions?

2.) Of the above 4 networks, which should be redundant (across 2 or more physical switches)? I would think the vmNetwork and the iSCSI Storage network are a must.

3.) How many physical switches should we get? 2, 3, maybe 4? I know the Equallogic takes up 6 ports (3 across each switch) and each ESX hosts will have up to possibly 8 connections I believe (4 across 2 different switches) so thats a total of 24 ports + whatever ports our NetBackup Enterprise server and the VCenter need to use to connect..

4.) What network would be used for backups? We use NetBackup Enterprise 6.5.3 . As some background, in the past we have tested VCB, for some VMs we would just install an agent.... the new Data Recovery option from VMWare looks interesting too as a replacement for VCB..... not sure which is best.

Any suggestions you could offer would be greatly appreciated

Take care,

Brian

Restricting Access to vNetwork Distributed Switch Port Groups?

CayceWill — Thu, 15 Oct 2009 22:16:40 GMT

We have a vDS configured with many port groups to seperate out our VLANs. Our VM cluster will have a small set of 5 to 10 groups with a few users a peice that will have VM power user role. How do we restrict which port groups a set of power users can connect their VMs to? The vSphere Basic System Administration doc does state that you cannot set permissions directly on a vNetwork Distributed Switch. Does this mean we have to create multiple distributed switches with only the specific VLANs each group can access? Is there a folder structure that allows us to grant permissions on a subset of port groups in a single vDS?

Ultimately we don't want admins from one group being able to hook up a VM on someone else's VLAN.

Thanks,

Cayce

Seperate Management Network?

mattrgee — Thu, 15 Oct 2009 09:32:11 GMT

Hi all,

We are currently redesigning our network in preperation for ESX4 and a SAN, I've started to consider the subnets we need and the use of a seperate management network.

My quetsion is: is the use of a seperate management network required and if so, is physical network segmentation a requirement or is VLAN'ing acceptable? We plan to HA between hosts and will more than likely move to DRS and vMotion at some point after.

Thanks.

Trying to install VMWare ESXI 4.0

Edoh — Mon, 10 Aug 2009 00:18:11 GMT

Hi Guys,

I have decided to try to play with the VMWare ESXI due to it's attractive features. I downloaded the installation iso and burned it on cd. After a while of running it, I got the famous "lvmdriver" error and through searching the community found out it's a NIC issue.

The system I am trying to install it on is Dell E521 desktop. Apparently ESXI doesn't like the integrated NIC. I have found a suggestion by a member that Intel Pro 1000gt would work but my E521 only has PCI-Express Ports. I am willing to take out the sound card which is on a PCI-E port (the short one).

I tried to look for a list of cards from the compatibility list but I cannot search by whether or not the card is a PCI or PCI Express.

Which NIC should I get to make this work?

Does anyone know if I get the NIC to work, will ESXI install on my system?

If I were to buy DELL PowerEdge T300(listed on the VMWare list as compatible system), does it mean that the NIC card will work too? Or will I still have to worry about each individual components compatibility?

Thank you for reading.

Interesting Networking Error

KevinMC — Mon, 12 Oct 2009 15:54:21 GMT

Hi all,

One of my clients is having a very interesting error related to vswif networking. I have a case open with VMware, but I thought I would put this out there and see if anyone has come across it. The error we are getting happens when I either create or try to enable the vswif interface. It says "Cannot update management NIC. No suitable network interface found"

So, some setup...

While attempting to transition his service console from a vSwitch to a dvSwitch, the customer accidentally attached his only service console to a dvPortGroup with a downed NIC. So of course, he lost connection to the host from vCenter and could not connect to or ping the host directly. He got into the host by logging on directly at the machine and we attempted to recreate the service console networking using KB http://kb.vmware.com/kb/1000266. Everything seemed fine up to step 7 which is when the error appears. It is strange though because when we run a esxcfg-vswif, it shows the vswif was created and is up. We went through the rest of the KB noting that GATEWAYDEV=vswif0 was missing from /etc/sysconfig/network so we added that and restarted networking but were still unable to ping in or out. After looking through the logs, I found that the line /adv/Net/ManagementIface="vswif0" was also missing from /etc/sysconfig/esx.conf. Again we added the values and restarted networking. Then we noticed that both of the entries disappear after restarting networking.

We have tried to create multiple vSwitches, vswifs and have used multiple up vmnics and even multiple IP addresses with no success. If anyone out there has seen the error "Cannot update management NIC. No suitable network interface found" and can tell me how to work around it or why it is happening I would appreciate it.

Regards,
Kevin