The below design:
- 1 vSphere Cluster with 5 ESXi hosts
- 1 center, 1 NSX Manager, 1 Active Edge , 1 Standby Edge , 1 Control VM active , 1 Control VM standby & 3 NSX controllers.
Is this design acceptable.
Is it good to run all the NSX components in 1 vSphere cluster.
What is the advantage & disadvantage of this cluster.
One other point may be redundancy of the Blade Chassis, if all the ESXi hosts are on a single Blade enclosure, then a chassis failure would make all ESX hosts to be lost. So if possible, chassis redundancy may provide better redundancy.
Additionally there could be additional resource pools needed if the Cluster CPU and memory resources would densely be used:
Decision ID | Design Decision | Design Justification | Design Implication |
|---|---|---|---|
CSDDC-VI-VC-008 | Create a consolidated cluster with a minimum of 4 hosts. | Three hosts are used to provide n+1 redundancy for the vSAN cluster. The fourth host is used to guarantee n+1 for vSAN redundancy during maintenance operations. You can add ESXi hosts to the cluster as needed. NSX deploys 3 Controllers with anti-affinity rules. the forth host is used to guarantee controller distribution across 3 hosts during maintenance operation. | ESXi hosts are limited to 200 virtual machines when using vSAN. Additional hosts are required for redundancy and scale. |
CSDDC-VI-VC-009 | Configure Admission Control for 1 host failure and percentage based failover capacity. | Using the percentage-based reservation works well in situations where virtual machines have varying and sometime significant CPU or memory reservations. vSphere 6.5 automatically calculates the reserved percentage based on host failures to tolerate and the number of hosts in the cluster. | In a four host cluster only the resources of three hosts are available for use. |
CSDDC-VI-VC-010 | Create a host profile for the consolidated Cluster. | Utilizing host profiles simplifies configuration of hosts and ensures settings are uniform across the cluster. | Anytime an authorized change to a host is made the host profile must be updated to reflect the change or the status will show non-compliant. |
CSDDC-VI-VC-011 | Set up VLAN-backed port groups for external and management access. | Edge gateways need access to the external network in addition to the management network. | VLAN-backed port groups must be configured with the correct number of ports, or with elastic port allocation. |
CSDDC-VI-VC-012 | Create a resource pool for the required management virtual machines with a CPU share level of High, a memory share level of normal, and a 146 GB memory reservation. | These virtual machines perform management and monitoring of the SDDC. In a contention situation it is imperative that these virtual machines receive all the resources required. | During contention management components receive more resources then user workloads as such monitoring and capacity management must be a proactive activity. |
CSDDC-VI-VC-013 | Create a resource pool for the required NSX Controllers and edge appliances with a CPU share level of High, a memory share of normal, and a 17 GB memory reservation. | The NSX components control all network traffic in and out of the SDDC as well as update route information for inter-SDDC communication. In a contention situation it is imperative that these virtual machines receive all the resources required. | During contention NSX components receive more resources then user workloads as such monitoring and capacity management must be a proactive activity. |
CSDDC-VI-VC-014 | Create a resource pool for all user NSX Edge devices with a CPU share value of Normal and a memory share value of Normal. | vRealize Automation can be used to create on-demand NSX Edges to support functions such as load balancing for user workloads. These Edge devices do not support the entire SDDC, and as such they receive a lower amount of resources during contention. | During contention, these NSX Edges devices will receive fewer resources than the SDDC Edge devices. As a result, monitoring and capacity management must be a proactive activity. |
CSDDC-VI-VC-015 | Create a resource pool for all user virtual machines with a CPU share value of Normal and a memory share value of Normal. | Creating virtual machines outside of a resource pool will have a negative impact on all other virtual machines during contention. In a consolidated cluster the SDDC edge devices must be guaranteed resources above all other workloads as to not impact network connectivity. Setting the share values to normal gives the SDDC edges more shares of resources during contention ensuring network traffic is not impacted. | During contention, user workload virtual machines could be starved for resources and experience poor performance. It is critical that monitoring and capacity management must be a proactive activity and that capacity is added before contention occurs. Some workloads cannot be directly deployed to a resource pool, as such additional administrative overhead may be required to move workloads into resource pools. |
CSDDC-VI-VC-016 | Create a DRS VM to Host rule that runs vCenter Server and the Platform Services Controller on the first four hosts in the cluster. | In the event of an emergency vCenter Server and the Platform Services Controller is easier to find and bring up. | Limits DRS ability to place vCenter Server and the Platform Services Controller on any available host in the cluster. |
In general, Edges Northbound are connected to the Physical Routers through Vlans. If the Management and Edge Clusters are collapsed instead of keepeing them in seperate clusters or racks, then these vlans should be extended to every cluster member that may host Management or Edge NSX Components. As the current physical DC switching architectures are based on keeping the Failure domains (such as Spanning Tree with Vlans), each vlan as a best practice should reside in its own rack. So this design does not very clearly seperate failure domains and a failure in STP could effect whole NSX Infrastructure. For Large environments thus recommended to seperate Management and Edge.
The tradeoff of this design is to seperate ESX hosts for a single function and leaves less hosts for Compute - Payload hosts. Thus for small environments consisting of 10 hosts without NSX, migrating to NSX requires seperation of 5 hosts for non-compute workloads, even this means half of the current workloads for Mangement and Control Planes(Plus edges).
If the environment consists of 100 hosts and growing rapidly, istead of doing design change in the future, seperation of Clusters at the beginnning may be better choice as in this case only %5 of hosts are used for non-payload clusters. Thus it may depend on the scale of the Data Center and use cases.
In general Small, Medium and Large designs could be as below:
http://stretch-cloud.info/2015/03/a-separate-nsx-edge-cluster-or-not/
- For small environments there is no need for management and edge clusters.
- For medium environnements, management and edge clusters might be combined.
- For large environments, there is absolutely a need to separate out the cluster based on their purpose otherwise managing VM sprawl in a single vCenter will be nightmare for vSphere/Network Admin.
You may please check -> NSX Small and Medium Business (SMB) Data Center Design Guide . Usually a host count between 10-100 with less north-south bandwidth requirement (<10g) is considered as a medium design. Below would be the minimum specs for NSX components.
