normal you have own NICs for ISCSI, Mgmt and also for VM traffic... To simplefy the cabling and reduce also the number of needed switches (bigger setup) Is it possible to have only 1 NIC card (e.g. 10G or 25G) with 2 ports.
Example Standard configuration:
NIC0 (vmnic0) & NIC1 (vmnic1) - 1G - (Active/Active or Active/Stdby)
Management - vlan 1000 (10.10.10.0/24) - vSwitch1
NIC2 (vmnic2) & NIC4 (vmnic3) - 10G - (Active/Active or Active/Stdby)
- vMotion - vlan 1100 (10.10.20.0/24) vSwitch2
- VM traffic - vlan 1200 (10.20.10.0/24), vlan 1201 (10.20.11.0/24) vSwitch3, vSwitch3
NIC3 (vmnic4) - 10G - iSCSI1 (Storage Controller A) - vlan 1300 (10.30.10.0/24) vSwitch 4
NIC5 (vmnic5) - 10G - iSCSI2 (Storage Controller B) - vlan 1301 (10.30.10.0/24) vSwitch 5
In a bigger set need a lot of cables and switches therefore the idea to use only 2 port NIC card e.g. with 25G (I know only 1 NIC card is SPoF)
If I put all the configuration on only a 2 port NIC card I have to put several IP addresses to one NIC and in the upper examples I am using teaming for NIC0/NIC1 and NIC2/NIC4 but not for the iSCSI ports (and Jumbo frames are enabled on ISCSI ports but not on the other ports).
Has anyone already expirencense with such configuration to put all traffic on only 2 NIC ports (and each traffic type is configured on the 2 ports?
Means Mgmt is running on NIC0 and NIC1 also vMotion, VM traffic and iSCSI
It would work but is strongly discouraged.
Attached is the official document and the link of the Network Best Practices
...Isolate from one another the networks for host management, vSphere vMotion, vSphere FT, and so on, to improve security and performance.