My client is doing a complete redesign of all the ESXi Hosts at all their datacenters across the world (currently 8x DCs in Americas, Europe and Asia).
Requirements:
- ESXi Hosts are commodities.
- All the same CPU, Memory and Brand
- Must be Four Socket CPU
- Only needs to support single host failure in clusters
- 20% free resources when in full production
- ESXi 5.5
- Supports each DC that hosts 2,000+ VMs (company total of 16,000 VMs for those counting)
- Memory dependent is usually the case. Not CPU
- Windows and Linux Mixed
- 10G network backbone
- Production Systems will be hosted on different clusters than DEV and TEST
Nice to have:
- Support HBA and NFS
- ESXi 6
- Support VMs that have high CPU and Memory requirements (HPC situtations) when needed
- They want vSphere to be the default platform for everything
Now, the clients infrastructure is completely internal. The customers are employees. This is not an transaction company like eCommerce or Search Indexing. They are however big data.. BIG DATA.
Also the budget is.. unlimited is not fair, but have plenty to spend 
Soooooo the existing environment is a mess. They have hundreds of hosts that dont match. Mixed clusters of hardware specs and brands. Also the hardware itself is very low spec with the number of hosts:
- CPU: 2x Socket with 4 to 8 cores average
- Memory: 64GB - 128GB with a few 192GB here and there
- Storage: this company is BIG BIG DATA.. storage is really a non issue. Well into the 10's of petabytes (not 100s yet.. but..)
NEW HOST DESIGN:
- 2U Server
- 4x Socket CPU with 18 Cores each
- 1TB Memory
Because the current design is so old and lack of direction I have an interesting problem. The new systems I have designed appears too small. Let take a real case:
America Datacenter has 2,000 VMs averaging 1,000 in Prod and 1,000 in DEV/TEST. This is sitting on 100 Hosts. I can replace all 100 Hosts with 10 of the new Hosts...
From a host review of pure CPU and Memory.. what am I missing.. I cant find anything. It really is a 10 to 1 return on the hardware.
Looking for thoughts, questions and review.
Thanks,
Boston TechGuy