MrJazze
Enthusiast
Enthusiast

Nested ESXI 7.0.x vHosts on Multiple Physical Hosts

Jump to solution

Ok, I've been racking my brains out trying to get this working. I've read multiple blogs, forum posts and youtube to death; to the point of bloodshot eyes. I've read several of William Lam's blogs. I am actually using his nested ESXI library images. And despite all efforts, not able to get full network connectivity for my nested vHosts.

 

Most, if not all examples in the wild describe creating multiple nested ESXI Vhosts on 1 x Phost. I have 3 x Phosts and therefore need to get all vmkernel nics communicating through physical switch. Today, I finally got the vhost management working. Yes, I know about the Promiscuous mode and Forged transmits set to Accept.

 

What I'm having an issue with is getting my vMotion and Storage VNICs (separate from vhosts) to traverse physical switch. These two additional vnics have the exact same security profile as the management, which again do communicate across hosts. Here is a breakout of my network:

 

All the physical servers/NAS have dual 10Gb nics (no LACP/Teaming)
 - 3 x ESXi 7.0.2 phosts (mtu 9000).
 - 3 x Nested ESXi 7.0.2 vhosts (built from William Lam's library) one on each phost.
 - 3 x Port Groups (vDSwitch / vlan 105-110 trunking / Promiscuous / Forged Transmits traffic / mtu 9000)
       - pg-nested-host (vlan105)
       - pg-nested-vmotion (vlan 107)
       - pg-nested-iscsi-a (vlan 108)
       - pg-nested-iscsi-b (vlan 109)
       - pg-nested-nfs (vlan 110)
 - 1 x Unifi16XG switch (vlan 1 native / 105-110 trunked / mtu 9000)
       vlan1 = switch management
       vlan 1 = hosts management 1Gb nics (separate vDSwitch)
       vlan 105 = nested esxi host network
       vlan 107 = nested vmotion network
       vlan 108 = nested iscsi channel-a network
       vlan 109 = nested iscsi channel-b network
       vlan 110 = nested NFS network
 - 1 x TrueNAS physical (VIF for 108-110 / mtu 9000) hosting iSCSI LUNs and NFS share.
 Teaming and failover are properly setup.

 

Since the vlan105 is a routable network, I traceroute to confirm pings are not going out native vlan. I'm attempting to get my nested environment to look/work just like my physical (different vlans) for segregation, learning and testing: lab in a LAB. I upgraded primary VCSA yesterday to 7.0.2.00100 and it didn't go over smoothly.

 

I'm open to any assistance offered or to answer further questions that get me to the goal...lab in a LAB! I hope to get this worked out so I can eventually put my VMUG licenses to use and build Tansu and VCF labs.

 

PS: Is there a way to turn on auto-save or save draft for posts? I did a write-up yesterday about my VCSA upgrade experience and lost it all.

 

UPDATE: So, I copied this content before resigning in. It turns out if you return to original community it's autosaved and you can be reloaded. Wished I knew that yesterday.

 

0 Kudos
1 Solution

Accepted Solutions
MrJazze
Enthusiast
Enthusiast

I FIXED IT!

It turns out every vswitch/dvswitch and portgroups at the physical hypervisor level carries trunking and vlan assignments. The nested vswitch/dsvswitch is assigned trunking vlans (default) and don't forget MTU. At the nested level, use vlan "None (0)" for every portgroup and assign the appropriate nested portgroup to it's intended traffic using Teaming and failover.

 

Which all makes since now given ports at the hypervisor level are "vlan access" ports; much the same as with physical switches.

 

Also, I did enable the "maclean dvfilter" William Lam posts about to the additional network interfaces I added. His nested nested ESXI image only ships with two interfaces.

 

I truly hope this helps out the next guy/gal in search of a nested lab.

View solution in original post

2 Replies
MrJazze
Enthusiast
Enthusiast

I FIXED IT!

It turns out every vswitch/dvswitch and portgroups at the physical hypervisor level carries trunking and vlan assignments. The nested vswitch/dsvswitch is assigned trunking vlans (default) and don't forget MTU. At the nested level, use vlan "None (0)" for every portgroup and assign the appropriate nested portgroup to it's intended traffic using Teaming and failover.

 

Which all makes since now given ports at the hypervisor level are "vlan access" ports; much the same as with physical switches.

 

Also, I did enable the "maclean dvfilter" William Lam posts about to the additional network interfaces I added. His nested nested ESXI image only ships with two interfaces.

 

I truly hope this helps out the next guy/gal in search of a nested lab.

rickdida
Contributor
Contributor

HI Mr Jazze,

 

What Server are you using for your physical server?

 

Thank you

0 Kudos