Hello
I have question regarding to communication with VTEP and NSX controller.
Will communication with VTEP and NSX controller flows within management vmkernel network?
Or will it flows within VTEP network?(VXLAN network?)
I believe it is via management vmkernel network, because ESXi host with NSX controller do not always need to have
VTEP(or logical switches) segment.
I thought it might flows within NSX controller segment if it is independent from normal(vCenter and ESXi host communication)
management vmkernel, but I think there is restrictions which controller network must be in same layer 2 segment.
I tried to find document but none of document clearly states which network port group will transmit VTEP and NSX controller flows.
I'll be very happy if you experts give me answer.
BR
As far as I'm aware you're correct - this traffic goes over the host Management network and a routed connection between Controllers is not currently supported with NSX-v.
As far as I'm aware you're correct - this traffic goes over the host Management network and a routed connection between Controllers is not currently supported with NSX-v.
Thank you for your response.
I want to have environment where I can packet trace and see the flow, but unfortunately I don't have it.
Hi pcparts001,
This is actually covered by a note in the documentation. "The IP address of the controller must be reachable from the NSX Manager and the management network of the vSphere hosts communicating with the controller."
Reference the NSX 6 Documentation Center
NSX Installation and Upgrade Guide > Installing NSX Components > Prepare and Enable Clusters for Logical Switches
The VTEP is a component of the VXLAN data plane. There is absolutely no requirement for the network used by the VTEP VMKernel interfaces to be able to reach the NSX Controllers. For a flow from ESXi Management network to NSX-Controller network, you need IP reachability.
" I think there is restrictions which controller network must be in same layer 2 segment. "
#correction
For communications from NSX Controller to NSX Controller, they do not have to be in the same L2 segment. It's probably not a bad idea for the cluster to have adjacent members, but the configuration UI does not impose restriction on the Port Group or IP Pool selection.
That is not correct - NSX Controllers do not need to be in the same L2 segment and can use different dvPortgroups and IP Pools if required.
grosas
Thank you for your response.
OK. So, VTP-VTP flow includes "only" dataplane data.
And control plane data flows between controller/manager/vsphere host flow.
I'm going for VCIX-NV and your post helped me alot.
rbudavari
Well the document "NSXvSphereDesignGuidev2.1.pdf" page 72 states
The ESXi hosts part of the management cluster do not normally require to be provisioned for VXLAN. If they are
deployed across two racks (to survive a rack failure scenario) they usually require extending L2 connectivity
(VLANs) across those racks for management workloads such as vCenter Server, NSX Controllers, NSX Manager
and IP Storage. Figure 87 highlights the recommended (even if not the only) way to provide L2 connectivity across
the management racks.
The section you have quoted from the NSX Design Guide is making a different point. As the VMs in the management cluster are on VLAN backed dvPortgroups, if this management cluster is extended across two racks for greater resiliency then L2 needs to span those racks. This is because VMs could vMotion between ESXi hosts in different racks, or vSphere HA can recover VMs in a different rack so the same VLAN needs to be available.
It is not the same as all 3 NSX Controller nodes needing to be in the same L2 segment, which is not a requirement (particularly as it is possible to deploy NSX Controllers in the edge cluster also).
regards,
Ray
rbudavari and grosas
Thank you for your great support.
But, I'm still confused.. (Last month I got CCIE certification and currently I'm aiming for VCIX, and my current hurdle is that thre is so many management segment that I need to be aware...)
I don't want to make thing complex, so I'd like to divide the discussion based on IP segment as below.
1.Controller management IP addresses
Where I'm reffereng:[Network & Security] -> [Installation] -> [NSX Controller (+ sign)] -> IP Pool
All 3 controller should be in same L2 segment?(should use same IP Pool?)
->No, it can be in different L2 segment, and can use different IP Pool.
Is this IP segment must be in same segment as NSX Manager IP?
->No, but it must have IP reachablilty with NSX Manager IP. If I have different requirement other than Controller<->Controller communication
such as vMotion between the rack, then I may need same L2 segment design, but if I focus just for Controller<->Controller communication
Contrllers do not need to be in same IP subnet.
2.VTEP IP addresses
Where I'm reffereng:[Network & Security] -> [Installation] ->[Host Preparation] ->[Cluster Configure] ->[Configure VXLAN networking]
All VTEP should be in same L2 segment?(should use same IP Pool?)
->Ofcourse no, that is the main reason why we use VXLAN.
Is this IP segment must be in same segment as NSX Manager IP?
->No, there is no relation between VTEP IP segment and NSX Manager IP or Controller IP. Only dataplane will flow betwen VTEP<->VTEP.
3.NSX Manager IP address
Where I'm reffereng:[While installing NSX Manager OVF] -> [Network 1 IPV4 address]
All NSX Manager should be in same L2 segment?(should use same IP Pool?)
->Only one NSX Manager is supported so this discussion is not relevant.
Is this IP segment must be in same segment as NSX Manager IP?
->No, there is no relation between VTEP IP segment and NSX Manager IP or Controller IP. Only dataplane will flow.
If I want to, I can make [Controller management IP addresses],[NSX Manger IP address],[vCenter IP address],[ESXi host management vkernel IP address] in same IP segment.
is above information correct?
(I didn't mention Edge IP address and DLR IP address because it might make confusion even more...)
That looks like a good summary to me but would be interested to hear others input. Apologies for confusing the issue previously as I wasn't aware that it was supported to place the different Controller nodes in separate subnets.
pcparts001pcparts001
That reads pretty accurate to me. Sounds like you're understanding it more than most. :smileysilly:
Thank you for your response! Now I'm pretty confident.
No need for apologies. In fact,,,,, to be honest,,, my confusion and misunderstanding started exactly from the site you mentioned.
"His" explanation certainly lead misunderstanding to newfaces of NSX...
Well, my concern have been solved so I would like to close this issue.
Again, thank you guys for helping me out for long time.
Actually you can do exactly this - individual Controller nodes from the same NSX Controller cluster can be in different L2 Domains. Hope this is clearer
This is perfect, thank you!