VMware Cloud Community
AmazingTed
Enthusiast
Enthusiast
Jump to solution

vSAN and Host Server RAID

Hi All, i wondered if i could get some clarification here?

I have been researching a vSAN solution and howt o arrange my disks RAID vs JBOD and the general consensus i am getting is that vSAN is recommended to run on Non RAID volumes only.

Now from what i understand this is becuase the vSAN software uses pFTT to basically replace the need for RAID so if i have 3 hosts in a vSAN cluster and select a DS policy that has a pFTT+1 that means my vmdk file that is 100GB in size is written twice to my vSAN datastore now whilst this consumes 200GB of space it means my VMDK file is basically written to 2 seperate hosts (Disk Groups).

Looking for 3 things

1. Confirmation that the above understanding is correct.

2. Confirmation that the RAID vs non RAID recommendation i have summised is correct

3. Understanding of the following:

'If the vmdk is written on 2 host disk groups through pFTT+1, are they synchronised and every bit of changed data is written to both files to ensure if 1 copy is lost, the other copy is exactly the same'

Additionally, can anyone advise what happens if a disk is lost, on a host? is it a case of hot swapping it and then vSAN automatically imports it into the disk group or is intervention required to add it into the disk group manually?

Thanks in Advnace for everyones help!

Reply
0 Kudos
1 Solution

Accepted Solutions
TheBobkin
Champion
Champion
Jump to solution

Hello AmazingTed

"when replacing the physical disks, do i have to then claim them within the VSAN settings to ensure they are added back into the Disk Group on the host or is this done automatically?"

You have to add them to an existing Disk-Group (e.g. back to the old DG the dead disk was in) - in older versions Auto-claim could be enabled to automatically claim disks but not in later versions (6.5 0d and later e.g vSAN 6.6/6.7).

"Also with the Witness Appliance, this only needs deploying in a 2 node cluster right? if there are more than 2 hosts in a cluster the witnessing is done intelligently right?"

Correct, a Witness Appliance is only required in 2-node Direct-Connect (2+1) or stretched clusters (1+1+1 to 15+15+1).

In standard configuration clusters (3-64 nodes) Witness components are placed on data-nodes alongside data-components and should end up relatively fairly distributed across nodes.

"Out of interest, if i configured my 3 hosts to have a 10 disk RAID 10 and then the hosts saw this as an available location for a datastore, would the VSAN configuration wizard allow me to add these RAIDed volumes as members of the disk groups or would i have to have individual disks to allocate"

If the RAID devices were locally attached and no filesystem partitions on it, it may actually allow you to claim this as a single disk, however as I said above vSAN isn't intended to function with devices RAIDed across disks and this is unsupported - devices need to be exposed as single whole devices and with no partitions. Do ensure the devices you are using/re-using are certified on the vSAN HCL and for the specific purpose (e.g. certified for All-Flash Cache-tier, All-Flash Capacity-tier, Hybrid Cache-tier, Hybrid Capacity-tier).

"is there any dis/advantage of using a hardware RAID at host level in conjunction with the software based storage policies over just using the VSAN to perform the RAID/RAIN Config?"

It is unsupported, don't do this. There is no benefit, only downsides (e.g. having non-functional Disk-Groups and inaccessible data).

Bob

View solution in original post

Reply
0 Kudos
3 Replies
TheBobkin
Champion
Champion
Jump to solution

Hello AmazingTed

"1. Confirmation that the above understanding is correct."

"2. Confirmation that the RAID vs non RAID recommendation i have summised is correct"

Somewhat. Firstly "RAID vs JBOD" for how disks are presented is defined by the compatibility of the controller+driver/firmware of the controllers in use for vSAN (e.g. HBA330 and the majority on the vSAN HCL are only supported in passthrough), also it's not RAIDed as it would be for redundancy/performance but just individual RAID0 VDs for each disk.

We essentially apply the equivalent of RAID0/RAID1/RAID10/RAID5/RAID50/RAID6/RAID60 to data Objects e.g. vmdk/vswap/namespace(but distributed across nodes not just disks so technically 'RAIN') using SPBM(Storage Policy Based Management) defined rules which are not only for redundancy(FTT)/striping but also thinks like IOPS limiting and local-affinity in stretched clusters.

"3. Understanding of the following:

'If the vmdk is written on 2 host disk groups through pFTT+1, are they synchronised and every bit of changed data is written to both files to ensure if 1 copy is lost, the other copy is exactly the same'"

Yes, vSAN operates on the basis of synchronous IO commit (when it hits cache) and thus why a solid low-latency network interconnect between nodes is essential.

"Additionally, can anyone advise what happens if a disk is lost, on a host? is it a case of hot swapping it and then vSAN automatically imports it into the disk group or is intervention required to add it into the disk group manually?"

Actually 'what happens' depends on a number of factors, e.g. if you lose a node or all disks in 1 node in a 3-node cluster then no data gets resynced as component-placement requires 3 Fault-Domains (nodes here) for data+data+witness. However if you just lost 1 of 10 capacity disks on a node in a 3-node cluster it would just rebuild the missing data on one of the other 9/10 available disks in this Fault Domain. Replacing physically failed disks can then be done at a later point but obviously one shouldn't leave it long with less capacity in one node.

Bob

Reply
0 Kudos
AmazingTed
Enthusiast
Enthusiast
Jump to solution

Thanks Bob, to confirm when replacing the physical disks, do i have to then claim them within the VSAN settings to ensure they are added back into the Disk Group on the host or is this done automatically?

Also with the Witness Appliance, this only needs deploying in a 2 node cluster right? if there are more than 2 hosts in a cluster the witnessing is done intelligently right?

Out of interest, if i configured my 3 hosts to have a 10 disk RAID 10 and then the hosts saw this as an available location for a datastore, would the VSAN configuration wizard allow me to add these RAIDed volumes as members of the disk groups or would i have to have individual disks to allocate, and is there any dis/advantage of using a hardware RAID at host level in conjunction with the software based storage policies over just using the VSAN to perform the RAID/RAIN Config?

I did notice that when i had configured the disks in a RAID, the VSAN disk manager couldn't see any storage to add to disk groups, i could only see disks that weren't a RAID Member.

Thanks for your help!

Reply
0 Kudos
TheBobkin
Champion
Champion
Jump to solution

Hello AmazingTed

"when replacing the physical disks, do i have to then claim them within the VSAN settings to ensure they are added back into the Disk Group on the host or is this done automatically?"

You have to add them to an existing Disk-Group (e.g. back to the old DG the dead disk was in) - in older versions Auto-claim could be enabled to automatically claim disks but not in later versions (6.5 0d and later e.g vSAN 6.6/6.7).

"Also with the Witness Appliance, this only needs deploying in a 2 node cluster right? if there are more than 2 hosts in a cluster the witnessing is done intelligently right?"

Correct, a Witness Appliance is only required in 2-node Direct-Connect (2+1) or stretched clusters (1+1+1 to 15+15+1).

In standard configuration clusters (3-64 nodes) Witness components are placed on data-nodes alongside data-components and should end up relatively fairly distributed across nodes.

"Out of interest, if i configured my 3 hosts to have a 10 disk RAID 10 and then the hosts saw this as an available location for a datastore, would the VSAN configuration wizard allow me to add these RAIDed volumes as members of the disk groups or would i have to have individual disks to allocate"

If the RAID devices were locally attached and no filesystem partitions on it, it may actually allow you to claim this as a single disk, however as I said above vSAN isn't intended to function with devices RAIDed across disks and this is unsupported - devices need to be exposed as single whole devices and with no partitions. Do ensure the devices you are using/re-using are certified on the vSAN HCL and for the specific purpose (e.g. certified for All-Flash Cache-tier, All-Flash Capacity-tier, Hybrid Cache-tier, Hybrid Capacity-tier).

"is there any dis/advantage of using a hardware RAID at host level in conjunction with the software based storage policies over just using the VSAN to perform the RAID/RAIN Config?"

It is unsupported, don't do this. There is no benefit, only downsides (e.g. having non-functional Disk-Groups and inaccessible data).

Bob

Reply
0 Kudos