Minimum NSX Install - number of ESXi Host

[stanj]

We are setting up new project that will include a Datacenter.

The datacenter needs to be secure and we only need the DFW and Micro-segmentation components of NSX.

We also need to have ESXi failover / HA working in case of an ESXi Server issue, ESXi Updates, etc.

I have seen a config with 3-4 servers and multiple clusters.  I believe we only need one cluster of ESXi Servers.

What number of ESXi Servers is required for this base NSX setup for the Datacenter?

If 3-4 ESXi Servers are in the cluster, do I install NSX Manager, vCenter, AD, NSX controller on one of the ESXi Servers? 

thanks

[lhoffer]

The NSX Reference Design Version 3.0.pdf​ calls for a minimum of 3 hosts in the "small" design referenced on page 144.  Even that number is mainly to allow for physical separation of the controller VMs, (which you won't have if you're just deploying DFW and not VXLAN) but I'd still recommend at least 3 to avoid any potential support issues either way.  These hosts can all be in a single cluster, just take care when creating DFW rules because while NSX is smart enough not to block communications inside of its own infrastructure with DFW (i.e. for the NSX Manager) it doesn't automatically allow the network traffic needed by other VMs (i.e. vCenter, your domain controller, etc.) so there's a definitely possibility of blocking things you didn't intend to and breaking parts of the infrastructure if you aren't careful about it.

[Techstarts]

3 Node cluster is strongly recommended as it gives 30% fail over capacity compared to two Node (here you get only 50%). In two node you have to disable Strict Admission control while doing maintenance. In the maintenance window you have just one host which could be risk in certain situations

I have seen a config with 3-4 servers and multiple clusters.  I believe we only need one cluster of ESXi Servers.

Yes, you just need one cluster. It is more than sufficient

If 3-4 ESXi Servers are in the cluster, do I install NSX Manager, vCenter, AD, NSX controller on one of the ESXi Servers?

NSX Manager and vCenter is all you need. NSX Controller is not required.

AD yes but only if you don't have existing. AD is required for IDFW.

You have to exclude vCenter and Partner AV solution from DFW. This is not done automatically.

[stanj]

Pardon my rudimentary drawing attached.

I put this together based on the research and comments provided here.

We have not decided on the hardware, however; hardware selected will
need to allow for growth and expansion.

The drawing shows no NSX Controllers at  this time.

Follow up:

• Does it matter where the VMs are placed in the design?
• I will assume HA / DRS are enabled and allow for VMs to fail over and
  ESXi Update vi Update Mgr?
• Cost of NSX in this design – 3 Dual Core Servers?
• Give that this will use DFW and Micro-Segmentation at this point in
  time, will we be able to Expand to full NSX down the road?

You will note a Physical Tap in the drawing.

This was based on the need to save and analyze data coming into the datacenter.

If NSX is used with Flow Monitoring, along with Endpoint Monitoring, is a Physical Tap needed or is it overkill?

Any other requirements for using Flow Monitoring / Endpoint Monitoring?

thanks

[Techstarts]

Does it matter where the VMs are placed in the design?

No, it doesn't matter. You need following VMs

1. NSX Manager

2. vCenter (vCenter Appliance embedded with PSC + VUM)

NB: NSX Manager is automatically configured for 16 GB RAM Reservation

I will assume HA / DRS are enabled and allow for VMs to fail over and
ESXi Update vi Update Mgr?

Yes, ensure AD, vCenter and NSX Manager restart priority is high.

Cost of NSX in this design – 3 Dual Core Servers?

Try to go with 12 cores minimum

Give that this will use DFW and Micro-Segmentation at this point in
time, will we be able to Expand to full NSX down the road?

Yes, it is possible. Needs some changes but they are not disruptive

If NSX is used with Flow Monitoring, along with Endpoint Monitoring, is a Physical Tap needed or is it overkill?
Any other requirements for using Flow Monitoring / Endpoint Monitoring?

In my opinion, it is overkill as the solution is fully virtualized. Are you using IDS/IPS services then these should come through the vendor?

Starting NSX 6.3, you are entitlement for vRealize LogInsight which might address your requirements

[stanj]

Try to go with 12 cores minimum -

Do you mean sockets?

I have a Dell r430 with 2 Sockets (CPUs) with 6 cores per socket for 12 logical processors.

The ESXi Enterprise + license is based on the 2 CPU sockets.

How is NSX priced for using DFW / Micro-seg?

Are you using IDS/IPS services then these should come through the vendor?

We may use IDS like FireEye, but it has not been determined.

We also are looking at HyTrust and TrendMicro on the virtual side.

Log Insight is for looking into the ESXi logs and can be used to alert if there is some issue (perf, network, hardware)

I do know content packs can be used in conjunction with it.

We would like to somehow filter events that are outside of hardware/network/perf and push them to a logging process so an alert can be raised to notify an operator.

Like if something is out of the ordinary is occurring (intrusion attempt, etc.)

[Techstarts]

Try to go with 12 cores minimum -

Do you mean sockets?

I have a Dell r430 with 2 Sockets (CPUs) with 6 cores per socket for 12 logical processors.

The ESXi Enterprise + license is based on the 2 CPU sockets.

How is NSX priced for using DFW / Micro-seg?

I meant cores. More core better as DFW is CPU intensive.

NSX -Advance is the minimum you have to go as Microsegmentation is included starting from advanced edition

Are you using IDS/IPS services then these should come through the vendor?

We may use IDS like FireEye, but it has not been determined.

We also are looking at HyTrust and TrendMicro on the virtual side.

Just check these are compatible with NSX Edition you are planning to deploy.

We would like to somehow filter events that are outside of hardware/network/perf and push them to a logging process so an alert can be raised to notify an operator.

Like if something is out of the ordinary is occurring (intrusion attempt, etc.)

I'm unsure what use cases you are looking for. I'm not expert on it.

Hope this help

[stanj]

thanks for the great details

For the cores, in the example I gave (Dell r430 with 2 Sockets (CPUs) with 6 cores per socket), then the r430 has a total of 12 cores.

Comparing to a (Dell r930 2 Socket Server with 8 cores per socket), has a total of 16 cores.

Are you saying three of the above (either r430 or r930) would work for our NSX setup - since the total cores are 12 or16 depending on the server selected ?

Would you happen to have an estimate for NSX -Advanced for a 2 CPU server (like the above servers)?

For the event filtering, the use case is to monitor incoming traffic to the data center and act upon it if there is some malice event and also log the traffic

[Techstarts]

For the cores, in the example I gave (Dell r430 with 2 Sockets (CPUs) with 6 cores per socket), then the r430 has a total of 12 cores.

Comparing to a (Dell r930 2 Socket Server with 8 cores per socket), has a total of 16 cores.

Are you saying three of the above (either r430 or r930) would work for our NSX setup - since the total cores are 12 or16 depending on the server selected ?

Sorry for the confusion. I meant to look for minimum 12 cores per socket. But if you don't have the budget I think total 12-16 cores should fine. Do note here I'm only considering vCenter (2 CPU, 8 GB RAM), NSX Manager(4 CPU, 16 GB RAM) if you have another VMs please review sizing of the cluster

Would you happen to have an estimate for NSX -Advanced for a 2 CPU server (like the above servers)?

Unfortunately not. It varies with region and Sales Manager's appetite

For the event filtering, the use case is to monitor incoming traffic to the data center and act upon it if there is some malice event and also log the traffic

I can speak of workload inside this cluster. As long as you put right rules in place and control you don't need the physical TAP. But if you are referring event coming inside your data center (Physical), the short answer is YES.

[RaymundoEC]

Hey there sir,

check this document, please

https://communities.vmware.com/servlet/JiveServlet/download/33244-2-172474/NSX_Samll_DC_Design_Guide...

so basically will help you understand mode design desitions about small data centers, things like the number of ESXi hosts or other reqs.

Also feel free to ping me if you have more questions.

regards

[stanj]

Thanks for the information and guide.

It appears we are looking at the section labeled as "Security Focused Deployment Mode with Micro-Segmentation (DFW) showing Figure 7.

We are still determining the hardware for the Datacenter.

The first diagram I posted is shows 3 host.  We are still leaning that way.

Cost may be an issue.  We are considering an EqualLogic SAN and Dell Servers.

For instance, the PE r730 with 2 Intel Xeon E5-2697 each with 12 cores looks promising.

We also may look at a converged architecture as there is a requirement to allow for growth in compute, storage, networking.

Using NSX will provide the secure Datacenter we are looking for.

I am getting a price for the Advanced version as this appears to have what we need.

I also see there was a release of a new product called VMware AppDefense that also look promising to have in Datacenter

[RaymundoEC]

Glad to help,

Just a couple of things,  take this guide as a reference,  so for example, if you have a budget constraint it is possible to have at least two ESXi hosts in management cluster, in this way you have a minimum for HA, it is not ideal but it works, other can be having three ESXi hosts and use resource pools for specific management NSX VM's so for the use case of Microsegmentation you are not required to have controllers, but you will have enough room to host vCenter itself or two vCenters schemas used in security deployments (one to control the management suite of VMware and the other for the ESXi and the compute resources), so again may be this kind of desitions are not in the frame of a good practices but the background of justification should be good enough to take it and always taking in account what you what to achieve, I have seen many collapsed deployments of NSX working fine and been a seed for next advance complex designs.

regards

[stanj]

For NSX Adv, I got a price quote of 3k per CPU.

I assume this would then be 6k for a dual CPU and would then be 18k total to install NSX across the three ESXi Server?

[RaymundoEC]

Kind of, check this link on what you can get for that X price

NSXv License Versions - Support Insider - VMware Blogs

regards