ESXI Networking Concepts

[stevendt]

Hi, VMWare newby here!

I'm really hoping that someone can help me get to grips with networking in ESXi please. I have read various VMWare docs and other help files, but I'm still a little lost on how ESXi should be configured to tie hosts into my physical network infrastructure.

In particular, I am looking to virtualize a couple of domain controllers (on separate hosts) and have some questions . . . though they are probably as much about networking in general as the specifics of ESXi.

In the current setup, each domain controller has a single network connection that connects to a separate physical switch to provide some redundancy. (Ideally, I would have set up these machines with dual NICs, but Windows Server does not like that for Domain Controllers.) If I virtualize the servers, I initially thought the I'd be able to provide increased redundancy, i.e., I'd like to have both hosts with virtualized domain controllers connected to both switches but I think that this may not be possible?

On the ESXi host, I have built one server (which will eventually become a DC) and another couple of VMs. It has four NICs, I have only configured two so far, which are connected to a VMWare virtual switch to which the VMs are connected. (At the moment, the Management Network is connected to the same physical adapter as the VMs, but I will probably put it on a dedicated NIC port eventually.) Only one physical port is currently connected to one of the physical switches.

I had planned to connect one of the host's physical ports to each of my physical switches, but I think that this is probably a bad idea! I'm thinking that I would end up with two paths to the VM network interface which would end up sending/receiving multiple packets for every network transaction?

As you can probably tell, I am pretty new to all this, but would really appreciate someone more knowledgeable than I commenting on what I'm trying to do here, whether it is possible, and if there is any particular ESXi network configuration that would support what I am trying to achieve. Would, say, NIC teaming allow me to do this?

regards
Dave

[IRIX201110141]

If youre are ESXi beginner for sure the networking part can be challenging but to make it short..... its way simpler as you think about. If we leave NSX out of the game than think about the same as when you do it in a complete physical world. A ESXi Host will not route, firewalled or touch your traffic in any way(its not the entire true but just think that way when you start).

So the vNetworking part is only needed to tell the ESXi how a packet from a vNIC leave on which pNIC port. I a very smart manner VMware decouple the vNIC from the pNIC because without this a VM will become unmovable and would be bind on that Host and its particular pNIC. Thats why the 2 abstraction layer named vSwitch and Portgroup exists.

A DC never needs more than one vNIC. Avoid multi home system when ever possible especially in a windows driven environment ! Ony Systems like a Router/Firewall need more than one vNIC and maybe Application Cluster when needs a Heardbeat network. But even when dealing with the later its best practice to use only one vNIC and a sec. subnet of course. (have something todo how a update of VMware tools effect the system/vnic.

For me the simple network reduncancy of a ESXi is one of the best things. Just connect as many cable as you like to your 2 pSwitches and thats without configuring anything on the pSwitches. On ESXi just be sure that you pick up the right vmnic as uplink to a given vSwitch. No configuration on vSwitch side needed because all the default settings do the magic out of the box.

The vNICs are not limited to 1 or 10G when it comes to west/east traffic or when the traffic stays inside the Host. With a simple round robin the ESX will assign a VM to use a uplink for outgoing traffic so with 2 or 4 1G uplinks all your VMs will separate most likely. There is no need for thinking about a LAG or vNIC Teaming inside a VM. When 1G is not enough spend some bucks and get a 10G nic and pSwitch.

Your concerns about a VM may send packets twice because the ESXi have multiple connection to one or more of your pSwitch at same time are not true.  ESXi will assign one temporary way (aka pNIC) when you power on a VM an as long not pNIC port, Cable or pSwitch went down the traffic will flow this route. If the Host have to change to another pNIC of the same Uplink group this will happen in no time and will be transparent for the VM Guest... your pSwitch will need to update the arp cache but thats it.

Regards,
Joerg

[stevendt]

Hi Joerg,

thanks a lot for the reply - yes, a definite ESXi beginner here 🙂

Yes, I know not to multi-home the DC, I was just looking for a way to provide some protection against the physical switch going down and cutting off client access to the DC. (As things stand at the moment, protection for the domain would be that the second DC remains accessible over its connection to the other switch. Slightly off topic, but for some background, part of the reason for trying to create a path to a second physical switch is due to the previous limitations in the Windows DHCP server role that I have configured on the DC. Up until now, I have had a split scope arrangement between the two DCs as a work around for Windows Server not supporting redundant DHCP servers. I had an occasion where one switch went down but there was not enough available addresses in the other machine's 50% of the scope to allocate addresses to all of the clients. That issue will go away when I configure DHCP failover that's available in Windows Server now, but I thought that protection against a physical switch failure was still worth doing.)

Based on what you said though, it seems that I may be overthinking it anyway.

You said "Just connect as many cable as you like to your 2 pSwitches and thats without configuring anything on the pSwitches."

But you go on to say "On ESXi just be sure that you pick up the right vmnic as uplink to a given vSwitch." Sorry for being so dumb, but I just don't get this bit?

As I said, at the moment, I just have a single vSwitch configured. The VM network adapter is configured to connect to what I have called my "VM Network". It consists of a single vSwitch connected to the VMs and has two uplinks, one of which is currently connected. Are you saying that I can just connect my second physical switch to the other uplink port and all will be good, or do I have to add a second vSwitch?

You also said "ESXi will assign one temporary way (aka pNIC) when you power on a VM an as long not pNIC port, Cable or pSwitch went down the traffic will flow this route. If the Host have to change to another pNIC of the same Uplink group this will happen in no time and will be transparent for the VM Guest." Are you saying that the host will know to use the other uplink port (and hence the VMs will be unaffacted) automatically if I just plug a cable in?

regards
Dave

[IRIX201110141 · Accepted solution]

We stay at your example with 2 pNICs, one pSwitch, one vSwitch0 and a single Portgroup named "VM Network".

The ESXi numbered the pyhs. pNICs as device  vmnic0, vmnic1. These devices you have to assign as uplink(s) to a vSwitch. A vmnic can only be one uplink which means a vmnic is only assigned to one vSwitch at a time.
If you configure more uplinks to a vSwitch and hook up 2 cables from ESXi to your single pSwitch you will receive some kind of network redundancy (bad cable or vmnic) and in some way more bandwidth. No special configuration is needed.

If you place one of your two cables to a 2nd. pSwitch which maybe you will add later to the environment you increase your network redundancy in case of a pSwitch failure. Again not special configuration is needed and it will work out of the box... even with dump switches. ( i have to admit that some times a dumb switch is easier to deal with as a smart one 😉

So whats happend when you start a VM?

Start VM 1 and ESXi will use Uplink(vmnic0) to send and receive Packets for that VM1
Start VM 2 and ESXi will use Uplink(vmnic1) to send and receive Packets for that VM2
Start VM 3 and ESXi will use Uplink(vmnic0) to send and receive Packets for that VM3

... so a simple round robin for load balacing..  If you add more vmnics as uplinks to the same vSwitch the VMs will be spread over all awailable uplinks over time and when powered on. As long as nothing will happens to your pNetwork the Host and the VMs will use these "routes".

If a cable or pSwitch falied and the vmnic will get a status of disonnected the Host will choose another vmnic from the uplinks for the effected VMs to send out the traffic. This will be fully transparent for the VM and no action is needed from the human Admin.

If you just plugin your 2nd. cable it will not harm your setup but IIRC the Host will not automatically reassign or re-route the Traffic of one or more VMs through the new uplink.

Yes there are other load balancing methods available within ESXi but your'e in class vNetworking 101 so we wont discussions these yet.

Old... but true
https://kensvirtualreality.wordpress.com/2009/03/29/the-great-vswitch-debate-part-1/

Regards,
Joerg

[stevendt]

Hi Joerg,

thanks a lot for the explanation, and the link to that very helpful post - it is much appreciated!

regards

Dave