Ok, I've been racking my brains out trying to get this working. I've read multiple blogs, forum posts and youtube to death; to the point of bloodshot eyes. I've read several of William Lam's blogs. I am actually using his nested ESXI library images. And despite all efforts, not able to get full network connectivity for my nested vHosts.
Most, if not all examples in the wild describe creating multiple nested ESXI Vhosts on 1 x Phost. I have 3 x Phosts and therefore need to get all vmkernel nics communicating through physical switch. Today, I finally got the vhost management working. Yes, I know about the Promiscuous mode and Forged transmits set to Accept.
What I'm having an issue with is getting my vMotion and Storage VNICs (separate from vhosts) to traverse physical switch. These two additional vnics have the exact same security profile as the management, which again do communicate across hosts. Here is a breakout of my network:
All the physical servers/NAS have dual 10Gb nics (no LACP/Teaming)
- 3 x ESXi 7.0.2 phosts (mtu 9000).
- 3 x Nested ESXi 7.0.2 vhosts (built from William Lam's library) one on each phost.
- 3 x Port Groups (vDSwitch / vlan 105-110 trunking / Promiscuous / Forged Transmits traffic / mtu 9000)
- pg-nested-host (vlan105)
- pg-nested-vmotion (vlan 107)
- pg-nested-iscsi-a (vlan 108)
- pg-nested-iscsi-b (vlan 109)
- pg-nested-nfs (vlan 110)
- 1 x Unifi16XG switch (vlan 1 native / 105-110 trunked / mtu 9000)
vlan1 = switch management
vlan 1 = hosts management 1Gb nics (separate vDSwitch)
vlan 105 = nested esxi host network
vlan 107 = nested vmotion network
vlan 108 = nested iscsi channel-a network
vlan 109 = nested iscsi channel-b network
vlan 110 = nested NFS network
- 1 x TrueNAS physical (VIF for 108-110 / mtu 9000) hosting iSCSI LUNs and NFS share.
Teaming and failover are properly setup.
Since the vlan105 is a routable network, I traceroute to confirm pings are not going out native vlan. I'm attempting to get my nested environment to look/work just like my physical (different vlans) for segregation, learning and testing: lab in a LAB. I upgraded primary VCSA yesterday to 7.0.2.00100 and it didn't go over smoothly.
I'm open to any assistance offered or to answer further questions that get me to the goal...lab in a LAB! I hope to get this worked out so I can eventually put my VMUG licenses to use and build Tansu and VCF labs.
PS: Is there a way to turn on auto-save or save draft for posts? I did a write-up yesterday about my VCSA upgrade experience and lost it all.
UPDATE: So, I copied this content before resigning in. It turns out if you return to original community it's autosaved and you can be reloaded. Wished I knew that yesterday.
