System network resource pools & NIOC

[akshunj]

Hello,

I am concerned about system network resource pools and their impact on the user-defined network resource pools. For example; if I create three user-defined network resource pools for virtual machines and assigned them to portgroups, so all virtual machines are covered by user-defined resource pools.

It appears that I must take into account the shares assigned to the system resource pool for virtual machine traffic in order to accurately calculate the bandwidth allocated to my user-defined resource pools;

resource pool bandwidth = (shares / total shares) * 10Gb

Aside from including the shares allocated to the system network resource pools, is their presence a don't care scenario that never comes into play?

I suppose it would be nice if VMware would allow us to remove the system network resource pools.

Thx.

[rickardnobel]

akshunj wrote:

resource pool bandwidth = (shares / total shares) * 10Gb

It is not really the "bandwidth", but the "worst case bandwidth", which is a small but important point, since the bandwidth is not physically limited to the percent of the shares, but instead rules how to deal with the rare situations where the uplinks are overloaded.

Aside from including the shares allocated to the system network resource pools, is their presence a don't care scenario that never comes into play?

Do you have the same physical links for both VM traffic and Management and vMotion and perhaps IP storage? If so, then the system resource pool settings come into play. If they do not share the same links you could ignore them in the sense of the VM network traffic share.

[akshunj]

Correct, I was trying to calculate a worst-case scenario with the understanding that my throughput levels can exceed these values and also that LBT will move the traffic as well.

"Do you have the same physical links for both VM traffic and Management  and vMotion and perhaps IP storage? If so, then the system resource pool  settings come into play. If they do not share the same links you could  ignore them in the sense of the VM network traffic share."

No not mixing traffic types on physical adapters. Now that you mention it, what if, for example, if I created a user-defined network resource pool for iscsi traffic and applied it to my vmkernel portgroups? Does the system resource pool take a back-seat to my user-defined network pools?

[rickardnobel]

akshunj wrote:

No not mixing traffic types on physical adapters.

If so the you could just setup the various VM portgroups shares level to suitable levels between them.

Now that you mention it, what if, for example, if I created a user-defined network resource pool for iscsi traffic and applied it to my vmkernel portgroups?

I am not sure you could attach user-defined resource pools to the iSCSI network for example, since there are already different "system" resource pools for vMotion, NFS, iSCSI and other, and I belive you must (should) use them to priorize the vmkernel traffic.

[akshunj]

See that's why I wasn't sure, because I am able to assign user-defined resource pools to vm portgroups, iscsi, vmotion etc. That made me wonder if there were conflicts that could arise, or if the user-defined pools override the system resource pools. Working with 10Gb it's tough generate enough traffic to see NIOC take effect.

[rickardnobel]

It does seems strange to be able to attach both system and user defined policies to the vmkernel traffic types. To me it does not really make sense, since you should be able to configure the shares, limits and 802.1p settings on the system groups if needed.

If indeed having both types at the same time then it would be very hard to determine in advance what would happen, so I would assume it should not be done.

[akshunj]

Thanks Rickard. I'll probably avoid trying the user-defined resource pools for vmkernel for now due to the uncertainty.

On another note, I couldn't find much documentation regarding vmkernel port-binding and dvs. I see vmkernel port-binding works in much the same fashion on a dvs as it did on vss. I've been testing and decided to adandon LBT and use MPIO and RR PSP for iSCSI and configure multi-NIC vMotion.

Have you ever come across any documention for vmk port-binding on a dvs? I've combed all the usual blogs and documentation and haven't found any information or best practices.

[rickardnobel]

I agree that the documenation and best practices for using vmkernel interfaces on Distributed vSwitches are must less common than information of standard vSwitches.

Personally I am a bit old-school and think the most value from Distributed vSwitches are for VM portgroups. These portgroups are often changed, added or deleted and it is great to have the distributed ability to "push" out such configuration to all hosts.

For the Vmkernel interfaces almost all configuration would still be per host, such as IP addresses and VLANs for vMotion, iSCSI and Management, so there is not much administration gain as you would have to setup the "virtual adapters" ( = vmk) per host anyway. As you say the better teaming options with LBT and even LACP does not really make sense on the vmkernel ports which pretty much need active / passive / unused configuration anyway.

Since it is very possible to combine Standard and Distributed vSwitches it could be reasonable to have the vmkernel adapters on host based standard vSwitches and VM portgroups on Distributed.

If however sharing all traffic (vmkernel and VM) on the same, for example, two 10 Gbit interfaces then the Distributed would be a good choice, allowing the NIOC feature to make sure for example iSCSI does get a certain minimum level of traffic and perhaps lower the vMotion shares to not let it fill all available bandwidth for everything else.

"It depends."