Advice needed for 10 port NIC

[tractng]

Guys,

We have a dell 2950 with 10 port nics (2 cards with 4 ports each and built in 2 ports on the dell).

We are planning to have redundancy with 5 ports going to one physical switch and other other 5 ports going to another physical switch.

For now, we are looking at just 5 vm machines on each host machine. In terms of building the vm machine, do I have to create a seperate virtual switch for each machine if I want to utilize 2 ports per one vm machine?

TIA,

tnt

[kjb007 · Accepted solution]

You can't really use 2 ports per one vm machine, unless the vm has two virtual NICs. At that point, they shouldn't be on the same network, and it's not really a good practice for servers, unless you will be using them as routers.

In order to use multiple physical NICs from a vSwitch perspective, your physical NICs that are uplinks to the vSwitch have to run to the same physical switch. Interfaces going to separate switches provide redundancy only, and can not be aggregated unless the switches are "stacked" together, effectively making them one logical switch.

So, with that many ports, I would use one pair for service console, one pair for vMotion , another pair for storage traffic (if using iSCSI/NFS), and then you have 2 pairs remaining for vm network traffic. For 5 vm's, that is a huge amount of bandwidth, but 4 GigE physical NICs should go a long way towards many more vm traffic needs.

-KjB

VMware vExpert

[tractng]

It is going to a gig network. Unfortunately, I don't have too much experience on the network side, but the network guy is doing the configuration on the switches.

Network guy is doing based on this concept:

We are looking for redundancy. So 2 physical nics (one from each physical switch) cannot go into a virtual switch on the EXSi host? The idea is if one physcial switch goes down, we will be on the second physical switch.

tnt

[kjb007]

I didn't say that. You can certainly have multiple NICs added to virtual switches, just not a good idea to have those nics to the vm's directly. Anyway, you can't currently add a nic to a vm directly.

What I stated still holds. Since you're using ESXi, create one vSwitch for management, and add two physical NICs from separate switches to it. Use a 2nd vSwitch for storage, etc. as posted previously. This will give you the redundancy/failover you want, and still leave you plenty of bandwidth for your vm's.

-KjB

VMware vExpert

[tractng]

Thanks so much.

tnt

[tractng]

Kjb007,

I had a talk with the network guy and this is what he is going to do. The two physical switches are "stacked. In his words, "aggregate" into one, making a large pipe and redudancy.

From that stacked switch, the cat cables will be connected to the 8 physical nics on the host (ESXi machine). From there I will create a virtual switch by adding these 8 nic adapters into it (Is this how it is done?)

For the remaining 2 ports, we will have a different virutal switch for management.

Btw, we are hosting the images locally for now.

Does this sound right?

Thanks again,

tnt

[kjb007]

For the most part, this will be fine, but I would not put all 8 physical NICs into one vSwitch. Most of the time, this ends up being fine. You will notice, however, that if you have network problems, it is very difficult to figure out which interface/nic is related to the problem. I would instead create multiple vSwitch's, each with two physical NICs. Since they are stacked, you can aggregate the two, and modify the teaming policy to ip hash. Only use ip hash if your switch ports are in an etherchannel. Since they are stacked, you can etherchannel a port on each switch together to create an aggregate. Use src-dst ip on the channel config, channel mode on, and you should be good to go. That way, you can use one vSwitch for each vm network you will create or use. If you are only using one vm network, then you can put all 8 NICs into a vSwitch, but remember, when it comes time to troubleshoot, be ready for a tough time pinning it down.

-KjB

VMware vExpert

[s1xth]

I just have a quick question, I have a similiar setup that I am going to be moving to. If I have 4 nic ports on a server, should I have a dedicated management vswitch and a dedicated VM vswitch for the vm's? One NIC for management and one nic for all the vm traffic, on separate vswitches?

[kjb007]

You always want to make sure your setup is redundant. I would say one vSwitch for management, with 2 NICs, and 1 vSwitch for vm's, 2 NICs.

-KjB

VMware vExpert

[tractng]

kjb007 ,

In terms of creating a virtual switch, and adding the 8 NICS to the virtual switch, do I have to do anything else? Just making sure there is no network loops.

tnt

[kjb007]

There is nothing else to do. ESX vSwitch is a flat architecture, and performs its own validation and loop checking, and does not run STP. No loops are possible in a vSwitch.

-KjB

VMware vExpert

[tractng]

Sorry to dig this up from the dead. Is there anything on the esxi that I have to do in terms of NIC teaming.

I know the process of adding the vSwitch and choosing the physical NICs. Anything that i have to configure with NIC teaming?

Thanks,

tnt

[mclark]

Under my "NIC Teaming" settings I have 2 NICS, both in the "Active" setting. That way both are serving data and in use, giving some more throughput. You can do whatever suits you in that tab. For instance, you could have one active and one on standby if you wanted.

[tractng]

Since the networking guy already configure the physical switch as an aggregate (stacked), selecting all the NICS for that vSwitch as "active" would still provide redundant?

Sorry for the newibe question.

Edit: nevermind, I think I know the answer.

Tnt