Two SAS HBA in Single VSAN node

[hishamrixs]

Hi,

Has anyone implemented/installed 2 x SAS HBA in a single VSAN node?

[TheBobkin]

Having multiple HBAs for vSAN is supported and there is no issue with this. That being said (but depending on the devices and the controller) the controller may not be a significant bottleneck and thus you may not get improved performance by having multiple (vs single and SAS-expander/backplane).

[hishamrixs]

Hi @TheBobkin ,

Thank you for your reply.  The plan is to use Fujitsu RX 2540 M5 with 12 x 4/6TB NL-SAS + 4 x 800GB SSD on the rear. To accomplish this we need to use 2 x Fujitsu PSAS CP403i(in VSAN HCL). 1 SAS card for the 12 disks in front and another for the rear. What do you think?

[TheBobkin]

@hishamrixs, No, I wouldn't advise putting all Cache-tier devices on 1 controller and all Capacity-tier devices on the other controller for various reasons including (but not limited to) the following:
- A single controller failure will take out all Disk-Groups on a node.
- Potentially severe imbalance of IO on the controllers (this is potentially debatable but dependent on cache-usage).
- Any destage operations (e.g. data moving from cache to capacity) will need to go through 2 controllers.

A more advisable configuration would be to put all devices of 2 Disk-Groups on each controller (or just use a single controller and a SAS-Expander, but do note that SAS-Expanders are technically only supported on vSAN ReadyNodes).

Regarding the Cache-tier and Capacity-tier sizing - while I am aware that NL-SAS will typically outperform comparable SATA disks, 6TB is a bit on the large side and this combined with relatively low cache:capacity ratio means this cluster may not be able to gracefully handle more intensive workloads with acceptable latency (that being said, having 4 Disk-Groups is a good choice and certainly better than just 2 here).

[hishamrixs]

Hi @TheBobkin ,

Thank you again for the reply. Thank you for the reply. Fujitsu RX 2540 M5 is VSAN Ready Node. But the configuration can add more disk than stated in ready node. I think we can handle putting all disks on single controller but we need to change to Fujitsu EP420i(VSAN HCL) and configure each this as RAID 0. I agree with you with the cache capacity being low, we will put 4 x 1.92TB + 12 x 4TB NL SAS. Thank you again for the suggestion to use 4 Disk-Group. We will certainly follow that if the above configuration permit.

[TheBobkin]

@hishamrixs, Please excuse my ignorance (I deal almost exclusively with vSAN/vSphere troubleshooting as job and not so much hardware other than compliance/trending/layout/functional and finding a way to fix it when the 'hardware guy' can't fix it 😅),why would it be necessary/beneficial to use a EP420i vs CP403i/CP400i? They are both 8-port controllers and thus I don't see how this would benefit in any way (and maybe only deficit as while supported for certain controllers RAID is becoming a thing of the past and passthrough is preferred).

Not to admonish but I am seeing a steadily increasing number of All-Flash clusters being the main choice of vSAN customers and for very good reasons as prices of SSDs comparative to HDDS is very good now (even considering comparable speed/TBW/lifespan).

Putting a more beefier (1.92TB vs 800GB) Cache-tier may help with more space for both write and read-cache but it won't necessarily make this cluster 'fly like the wind' - I have seen Hybrid clusters that out-perform All-Flash clusters (of the same size) but really how well any cluster performs is based on whether it is suited and capable of managing the workload it is running (and the extra potentially storage-IO-intensive stuff that may be needed e.g. backups, migrations, storage policy reconfigurations etc.) so you would want to make sure you have this sized and specced right for the intended workload.

[hishamrixs]

Hi @TheBobkin ,

I really appreciate all your feedback. Thank you for helping so far. What we are trying to achieve here is high density. The physical rack space vs capacity that we require is ridiculous, but we are trying to find a working solution. Performance is not a big concern because the workloads are not high IOPS/high throughput. But we are worried about the configuration (number of disks) will not work with VSAN because it is not in Ready Nodes reference. So I'm here to seek advice and help if we can make the configuration works. So far I think you have been very helpful in clarifying my doubts.

[depping]

Just to be clear, the Ready Node Configurations are suggestions to make life easier for customers and vendors when buying/selling vSAN. if you feel comfortable, you can pick any of the vSAN certified components from this list (vmwa.re/vsanhclc) and build your own configuration. As long as you follow our HCL and our documented requirements/limits it will work. We have a lot of customers who start with a Ready Node Configuration when they want to buy vSAN and they tweak that configuration to what their apps require and to what fits their budget, again, as long as you follow our HCL and guidelines that is okay!