VMware Cloud Community
dborgill
Enthusiast
Enthusiast
Jump to solution

NIC overkill?

I have searched and only come up with basic answers in the forums about general networking design. I'd like some feedback from the people who have had experience with high capacity servers (We are using IBM x3850 with 8 way and 32 gigs of RAM).

We plan to have as many as 20 VMs on one box or 10 VMs with lots of traffic. Right now, we have 1 NIC dedicated to service console, 1 to VMotion and 8 (2 quad Intels) to the VMs.

Has anyone actually done any testing or monitoring to see how many NICs dedicated to VMs is "enough""?

Thanks.

Reply
0 Kudos
1 Solution

Accepted Solutions
BenConrad
Expert
Expert
Jump to solution

We have some servers that run ~20 VM's per server. We're using 2 Gb nics in a vSwitch and could not be happier. If we need more bandwidth we'll add another NIC.

A common practice is :

vSwitch for Vmotion and SC (2 nics)

vSwitch for VMs (2+ nics)

and if you can't do 802.1q:

vswitch for vmotion

vswitch for sc

vswitch for lan

vswitch for DMZ

etc, etc

View solution in original post

Reply
0 Kudos
6 Replies
VirtualNoitall
Virtuoso
Virtuoso
Jump to solution

Hello,

I would look into getting redundancy for your service console and Vmotion. You could even set the two up in active/standby on the two nic you have already set aside. During normal operations the traffic would be separate but during a failure you would have both on the same nic.

How much network traffic will you be pumping through your hosts? That is the real question. No one is going to be able to give you the definitive answer on this.

The suitability of your 8 nic config depends on your plans. Do you plan to have redundancy across switches? 4 nics to one switch and 4 nics to another? 8 nics to one switch?

Reply
0 Kudos
esiebert7625
Immortal
Immortal
Jump to solution

8 gigabit NIC's for 20 VM's is probably more then enough bandwidth. You can monitor your NIC's on the performance tab for the ESX server, select the Network option and you can check out the usage for the whole server and each NIC. Are you seeing any network performance degradation with your VM's or are you just looking for other users opinions?

jbp00007
Contributor
Contributor
Jump to solution

I have 15 running VMs on ESX 3.x box which has 16GB RAM, 2 Dual core Xeon 3.6Ghz, 2 HBA, 3 GbNICs.

1 NIC for SC, and 2 other for VMs. I don't have other ESX so no need for VMotion. Apart form 15 VMs, 5 VMs are runing high traffic application and SQL database server but so far not having any network output problem but sometimes higher usage of RAM.

I am not finding any problem in your configuration infact 8 NICs are more then required. You can check performance on Perforamance tab as well report each VM performance under VM menu in VM Console.

Good Luck,

JB

RParker
Immortal
Immortal
Jump to solution

If you have Gig, and 20 VM's per host 1 NIC is PLENTY

20 VM's / 1000 is 50 meg PER VM, and that's only IF they are ALL running something that takes up a lot of bandwidth.

You can check the performance of the NIC on each host via the performance tab, you say you have a lot of traffic, but you can see exactly how much. My guess is, you mean remote desktops, and maybe a few programs that contact other systems across the network. Even if they were all day, this isn't as much bandwidth as you think.

If you only have 100 MB NIc as in our case, it still would be plenty if you have 4 NICs. 2 dedicated for VM's bonded together. This will be plenty.

First check out your performance, and see where you stand now. I bet you would be surprised by how much bandwidth you have left.

Reply
0 Kudos
BenConrad
Expert
Expert
Jump to solution

We have some servers that run ~20 VM's per server. We're using 2 Gb nics in a vSwitch and could not be happier. If we need more bandwidth we'll add another NIC.

A common practice is :

vSwitch for Vmotion and SC (2 nics)

vSwitch for VMs (2+ nics)

and if you can't do 802.1q:

vswitch for vmotion

vswitch for sc

vswitch for lan

vswitch for DMZ

etc, etc

Reply
0 Kudos
dborgill
Enthusiast
Enthusiast
Jump to solution

Thanks for all your help everyone. I was mainly just looking for real world experiences and suggestions.

You have been most helpful and sorry I couldn't give everyone points!

Reply
0 Kudos