VMware Cloud Community
pirx666
Contributor
Contributor

All Flash Server/VSAN Configuration for Kubernetes on OpenStack

Hi,

for each (internal) customer we are deploying dedicated k8s clusters in OpenStack tenants. Currently our payload cluster size is 8 DL360g9 hosts with 2 x 14 cores and 512GB + Hybrid VSAN (1x800 + 7+900). We have to build new payload clusters and think about using All Flash. Currently neither the performance nor the amount of storage the customers will need is clear. We started with projects that demanded only little persitent storage, now they are growing and new projects are testing the platform.

I thought about the follwing config:

8 x HPE DL360Gen10 with 10 disk bays

2 x Xeon 6154, 2 x 18 Cores @3.0GHz

768 GB RAM

P816i-a Array Controller

2 x 800 SAS WI SSD

4 x 3,84TB RI SSD

3 x 562SFP NIC

The SSD's would the be configured as 2 disk groups, with ~120 TB RAW capacity and ~45TB usable with FTT=1 and 30% slack space. If needed we can add 2 additional disks to each disk group.

Does this look sane, any ideas for improvement or lessons learned?

Reply
0 Kudos
2 Replies
depping
Leadership
Leadership

looks like a decent configuration to me. two disk groups.If you have 8 hosts you could consider going RAID-5 with vSAN Advanced, that would provide some significant capacity returns (much lower overhead). Of course without understanding the workload it is difficult to say how it will behave. But if it is hybrid now then this looks like a good step up!

Reply
0 Kudos
pirx666
Contributor
Contributor

Hi Duncan,

seems that somehow I missed you feedback. In the meantime our scope changed a little bit. We'll still start 2 DG's, each with 1 x 1,6TB NVMe (HPE can not deliver 800GB or they are more expensive for us), and 3 x 3,84 TB RI SSD. The server will be DL380Gen10 with 3 disk cages, so that we can use max. 6 NVMe's and 18 SFF SSD's if needed. HPE uses one P408i-a controller + SAS Riser for this setup. I had some doubts regarding this controller - even if the controller is only used on pass-through - but HPE confirmed that this controller is sufficient, even with 3 DG's with 3 NVME's and 18 SSDs. HPE vSAN ready nodes I found also use only one controller. Can you confirm this?

Reply
0 Kudos