New NSX deployment Sizing and design

[tyoungbauer]

Hi All,

First, thank you for your time and response.

We are looking to move a medium size VM environment from using DvSwitches to using NSX.

I am looking for some recommendations on sizing and hardware.

We are trying to decide if we can, should or should not deploy a collapsed Management and Edge cluster or if we should deploy a full stack.  If we deploy a full stack, I am looking to see if we have enough hosts.  We currently have 12 ESXi hosts.  If I do a full stack, I am looking at 3 hosts in Management, 2 hosts in Edge and 7 hosts in compute.

Is that correct?  If it is, can I also run any VMs in the Management / Edge cluster?  We host clients and would run all of them in the compute cluster but not sure I have enough resources in the compute cluster to run all my internal servers and I will have extra resources available in the Management and Edge cluster.  So can I run VMs in any cluster or do they all need to be in the compute except for vCenter and NSX manager and controllers etc?

Next question, what are the networking requirements?  Do I need to run 10Gb or can I run multiple 1Gb?  We currently run multiple 1Gb interfaces in Dvswitches.  I see a lot of talk about North / South and East / West traffic.  Currently we already run a zero trust network and all traffic runs North / South to ingress and egress.  I guess we have some East / West traffic between servers on the same VLAN running on different hosts.

Any other comments are helpful.

Thank you again.

Todd

[Sreec]

Assuming you are going with NSX-v , you can certainly go with options what you mentioned in this thread , also below options are possible.

1. Dedicated Management Cluster  or Management + Edge

2. Dedicated Edge Cluster or Edge + Compute

3. Dedicated Compute cluster   or Edge+ Management+Compute

If you think about failures and upgrade scenarios , you will have a straightforward answer on what is best keeping the budget factor aside.  However i'm against having just 2 host for Edges , that is way too less. From an uplink perspective 10gb will be perfect for majority of the workloads , better go for 10Gb or more .

Please do go through ->https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/nsx/vmw-nsx-network-virtu... 

NSX Small and Medium Business (SMB) Data Center Design Guide .

[MostafaElSayedF]

"Next question, what are the networking requirements?  Do I need to run 10Gb or can I run multiple 1Gb?  We currently run multiple 1Gb interfaces in Dvswitches.  I see a lot of talk about North / South and East / West traffic.  Currently we already run a zero trust network and all traffic runs North / South to ingress and egress.  I guess we have some East / West traffic between servers on the same VLAN running on different hosts."

The most important thing in the network requirement is the jumbo frames you need to configure your network to support 1600 MTU in case of NSX-V. As it is mandatory. 

NSX-V will provide you the more flexibility to run the north/south traffic or the VLANs to be routed in the same host without going to the physical network. but if you want to run the "zero trust network" as physical module so i think the nsx will not give you very high value more than the additional layer for the micro segmentation and in this case you don't want edges or VXLANs.

I hope this answer your question and i hope that this become answer or helpful comment for you. also, for More details and more information just follow my blog http://www.syncgates.com.

[tgrayatshi]

Hi Todd,

A collapsed Management/Edge cluster would be appropriate (3-4 hosts) and 8-9 hosts for compute would be fine.   I would not look at implementing NSX-V - it is EOL.   NSX-T 3.0+ would be the best direction for anything you do.   10GB is the best option for all VM networking   I know the Management/Edge cluster seems like you are wasting resources, but it lets your production VMs get dedicated resources.   

[MostafaElSayedF]

You are totally right and i agree with you. But it will be very challenging to build nsx-t on 1G network, he will require to at least upgrade the network cards and check the network supportability.

[HassanAlKak88]

In addition to all the above and regarding the following

"Is that correct?  If it is, can I also run any VMs in the Management / Edge cluster?  We host clients and would run all of them in the compute cluster but not sure I have enough resources in the compute cluster to run all my internal servers and I will have extra resources available in the Management and Edge cluster.  So can I run VMs in any cluster or do they all need to be in the compute except for vCenter and NSX manager and controllers etc?"

A. you can run VMs on Management cluster but taking on consideration if these VMs should use logical switches you have to prepare this management cluster for NSX, which means that you have to add 3 NSX licenses for this cluster. on almost deployment we didn't prepare the management cluster for NSX, but if you need that you have to pay for three licenses for three hosts under the Mgmt cluster.

Regarding the second question, sure it is better to go with 10GB. but there is a lot of deployment with multiple 1GB and when you split your clusters lets say Edge cluster and Workload cluster, all N/S traffic go though the edge hosts and all E/W traffic use the workload hosts so in that case if almost of your traffic is N/S you can use 10 GB on the edge nodes only.

[tyoungbauer]

Hi,

Thanks for the reply.  So I am already into the deployment with NSX-V 6.4  I assume I can migrate later if needed?

I deployed, NSX, Controllers and hosts in my LAB cluster.  I plan to tear down NSX from the LAB cluster once I have it working and understand it and can deploy to the production compute cluster.

So I went with the following:

1 - vCenter

2 - Clusters - one for Management and Edge - One for Compute.

1 - Lab cluster

I have 5 hosts in the Management/Edge Cluster and 7 hosts in the compute cluster and 3 hosts in the Lab cluster with some older Dell Servers running ESXi 6.0.  All run DvSwitches.

I put in the management cluster my vCenter, NSX Controllers, NSX Manager etc.  I also have a number of my internal servers on that cluster and they live in a separate VLAN and subnet.

I put in my Compute Cluster all of the client servers.  They are currently separated by VLAN and firewalls.

So far it seems to be working and all status lights are green.

I do have a new question.

I am not sure I understand how I would need to deploy VXLAN?  Since I have multi-tenant and need zero trusts.  When I set up the VXLAN on the hosts it asks for a VLAN number.  Do I leave it as 0 or do I use my tagged management VLAN number that all my management devices are running on or do I need one for each client network?  Also, it is asking for an IP pool or DHCP.  What subnet should that use?  I new one that doesn't route anywhere or my VM management subnet??  Any help to deploy VXLAN would be helpful.  Remember I need multi-tenant and microsegmetation and security.

Any help is greatly appreciated.

Thanks

[MostafaElSayedF]

VXLAN mainly work on the virtual environment only between the VM's and the virtual routers (distributed logical router) and the edge gateway is only converting the vxlan traffic to the vlan. so the only interface need to be connected to the VLAN traffic is the edge gateway uplink interface.

All the traffic of the VXLAN transferred between the esxi hosts transferred in a tunnel named (VTEP) and this is the VMK created for the NSX per host and you can dedecate a vlan for it or put it in the management vlan and better for security is to but it in isolated vlan / subnet (and this is the ip pool which you are asking about).

And for the micro-segmentation is done by diffault in the nsx after you install the nsx manager and add the vcenter and install the nsx vib on the hosts the firewall service on the host start by default all what you need is to configure the firewall. needless to say or to remind you that the better design for the firewall always the consolidated rules do not but multiple roles to serve same service (design it in smart way and use the grouping to save the processing of the roles) .

I hope this answer your question and i hope that this become answer or helpful comment for you. also, for More details and more information just follow my blog http://www.syncgates.com.

[tyoungbauer]

Thanks for the reply.

So to get the load balancer working for 2 web servers I only need an edge gateway and no need to setup VXLAN?  If I do or want to setup VXLAN, you are recommending setting up a new subnet, VLAN ID on my switch core?  When I setup VXLAN it want an IP Pool and that asks for a gateway.  Do I just enter an IP or does it need to route out? So if it does I need to create an L3 interface with an IP on my switch core so it can route out.

Hope the question makes sense.

Thanks

[MostafaElSayedF]

Load balancer is one of the network services and the edge, which require  the VXLAN setup. {only micro-segmentation not requiring the VXLAN}.

If you will put all the VTEPs in the same switch without different subnet with no multisite or multi subnet setup, you can isolate it.  but the Gateway is must in the VTEP Pool configuration. so, you can configure it on the switch and no need for it to reach the core switch.

I hope this answer your question and i hope that this become answer or helpful comment for you. also, for More details and more information just follow my blog http://www.syncgates.com.