VSAN cluster configuration

[ChevUribe]

I would like to verify some questions regarding VSAN:

1. How many is the maximum diskgroups that I can create regarding of the cluster size?

Scenario 1:

Server 1 : 16 disks (2 SSD 1TB, 2 SATA 300GB, 12 SATA 1TB)

Server 2: 16 disks (2 SSD 1TB, 14 SATA 1 TB)

In my example in scenario 1, which is the ideal server configuration for VSAN? Do I need to install the ESXi on the 300GB disk and the rest is used for disk group like in Server 1? Or I can have a server 2 configuration since after installation, whether if any of the 14 disks I install ESXI, it will still be included during disk group creation?

Scenario 2:

If I have to implement VSAN with SRM, will I create a seperate placeholder datastore that is not included in the VSAN disk group or I can create inside VSAN as a seperated disk group (this I think is tedious since a diskgroup will consist of 1 SSD used for a placeholder datastore)?

[zdickinson]

Good morning, I will try to answer your questions and then some comments.

How many is the maximum diskgroups that I can create regarding of the cluster size?

Assuming version 6.  You can have 64 nodes in a hybrid cluster and 5 disk groups per node.  Max 320 disk groups per cluster.  It's half that, 160, for all flash.

Which is the ideal server configuration for VSAN?

I'm not sure you can do an ideal configuration with what you have.  With 2 nodes you will need a third witness appliance that should be located in another datacenter.  My assumption is that you do not have another datacenter to host the witness appliance.  Also, the 2 node with witness is best for ROBO deployments.  It's not recommended to have varying types/sizes of drives either.  I hope this is for test.

Do I need to install the ESXi on the 300GB disk and the rest is used for disk group like in Server 1? Or I can have a server 2 configuration since after installation, whether if any of the 14 disks I install ESXI, it will still be included during disk group creation?

If I understand correctly you are wondering if it matters where you install ESX because after the install the disk will be part of a group.  The disk in which you install ESX cannot be part of a group.  Each disk needs to be partition free before it can be consumed by vSAN.

If I have to implement VSAN with SRM, will I create a seperate placeholder datastore that is not included in the VSAN disk group or I can create inside VSAN as a seperated disk group (this I think is tedious since a diskgroup will consist of 1 SSD used for a placeholder datastore)?

If a disk is being used for vSAN it cannot be used for any other purpose.  SRM is supported with vSAN, but that is just for orchestration.  You need some other means of SRM supported replication, what is that?

Thank you, Zach.

[ChevUribe]

HI Zach,

Thanks for the swift response.

I would like to add follow questions for my inquiries

"

How many is the maximum diskgroups that I can create regarding of the cluster size?

Assuming version 6.  You can have 64 nodes in a hybrid cluster and 5 disk groups per node.  Max 320 disk groups per cluster.  It's half that, 160, for all flash."

- Can I create diskgroups with SSD and SATA disks that are comprised from different servers. Like 3 SSDs (1 each for server 1-3) and 6 SATA disks (2 each for server 1-3) but are grouped as 1 diskgroup?

Do I need to install the ESXi on the 300GB disk and the rest is used for disk group like in Server 1? Or I can have a server 2 configuration since after installation, whether if any of the 14 disks I install ESXI, it will still be included during disk group creation?

If I understand correctly you are wondering if it matters where you install ESX because after the install the disk will be part of a group.  The disk in which you install ESX cannot be part of a group.  Each disk needs to be partition free before it can be consumed by vSAN.

- I see so If I installed the ESXi on that disk, it will be a separate disk (i assume it will be the local datastore default seen first by the host) and not be included if you create the VSAN disk groups. AM i correct? So it is best to have a separate disk for ESXI installer

- I will still do RAID on the disks on the hardware level on both SSD and SATA disks, this is still necessary right? If yes, what would VMware recommend RAID level? Can we raid the SATA disks as RAID 0, RAID 1 or RAID 6? Or no RAID is necessary because VSAN will be pooling it in one large datastore (I assume will be like a software based RAID)?

If I have to implement VSAN with SRM, will I create a seperate placeholder datastore that is not included in the VSAN disk group or I can create inside VSAN as a seperated disk group (this I think is tedious since a diskgroup will consist of 1 SSD used for a placeholder datastore)?

If a disk is being used for vSAN it cannot be used for any other purpose.  SRM is supported with vSAN, but that is just for orchestration.  You need some other means of SRM supported replication, what is that?

- I assume that I can create a a local placeholder datastore (a separate disk on the local server) to the the vsphere replication.

[zdickinson · Accepted solution]

- Can I create diskgroups with SSD and SATA disks that are comprised from different servers. Like 3 SSDs (1 each for server 1-3) and 6 SATA disks (2 each for server 1-3) but are grouped as 1 diskgroup?

No you cannot.  The SSDs and the backing HDs must be on the same host to form a disk group.

I see so If I installed the ESXi on that disk, it will be a separate disk (i assume it will be the local datastore default seen first by the host) and not be included if you create the VSAN disk groups. AM i correct? So it is best to have a separate disk for ESXI installer.

You are correct.  A common practice is to install ESX on an SD card or USB stick, preferably mirrored.  Careful with the size though.  The logs should not reside on the vSAN storage and logging for vSAN will quickly fill up a 8 GB or 16 GB SD/USB pretty quickly.

- I will still do RAID on the disks on the hardware level on both SSD and SATA disks, this is still necessary right? If yes, what would VMware recommend RAID level? Can we raid the SATA disks as RAID 0, RAID 1 or RAID 6? Or no RAID is necessary because VSAN will be pooling it in one large datastore (I assume will be like a software based RAID)?

No, you do not do any RAID.  OK you might do RAID 0 if you cannot do passthrough.  vSAN needs to see the disks directly.

- I assume that I can create a a local placeholder datastore (a separate disk on the local server) to the the vsphere replication.

Yes you could.

Thank you, Zach.

[ChevUribe]

Hi Zach!

Thanks for the helpful response!

I would like to add follow up questions:

Can I create diskgroups with SSD and SATA disks that are comprised from different servers. Like 3 SSDs (1 each for server 1-3) and 6 SATA disks (2 each for server 1-3) but are grouped as 1 diskgroup?

No you cannot.  The SSDs and the backing HDs must be on the same host to form a disk group.

- If diskgroups are tied to the host, will it still do HA if that host goes down (I saw that different diskgroups, VSAN will still see it as a single pool of resources. Is this combined within the cluster or one VSAN datastore per host?). Can I still do vMotion between diskgroups?

I will still do RAID on the disks on the hardware level on both SSD and SATA disks, this is still necessary right? If yes, what would VMware recommend RAID level? Can we raid the SATA disks as RAID 0, RAID 1 or RAID 6? Or no RAID is necessary because VSAN will be pooling it in one large datastore (I assume will be like a software based RAID)?

No, you do not do any RAID.  OK you might do RAID 0 if you cannot do passthrough.  vSAN needs to see the disks directly.

- what would be the data protection from VSAN if one disk on the host goes down or one of the disk in one of the diskgroup encountered a problem? Since no RAID will be done on the hardware level or even if RAID 0 is configured.

- I assume that I can create a a local placeholder datastore (a separate disk on the local server) to the the vsphere replication.

Yes you could.

- If this will be the case, it will be applicable only to the host where it is configured, right? Would you still recommend an external storage that will be used as placeholder datastore to be seen in VSAN cluster for vsphere replication and SRM?

Thanks!

[zdickinson]

- If diskgroups are tied to the host, will it still do HA if that host goes down (I saw that different diskgroups, VSAN will still see it as a single pool of resources. Is this combined within the cluster or one VSAN datastore per host?). Can I still do vMotion between diskgroups?

HA, vMotion, DRS, etc... are all fully functional.  There is one vSAN datastore per cluster, it's the aggregate of all storage contributed by hosts.  Let's say you had a 5 node cluster and VM with fault tolerance of 1 node failure.  That VM's data will be on three of the nodes.  It's possible that the VM will be running on one of the other 2 nodes.  The VM is does not need to run on a node where it's data lives.

- what would be the data protection from VSAN if one disk on the host goes down or one of the disk in one of the diskgroup encountered a problem? Since no RAID will be done on the hardware level or even if RAID 0 is configured.

There is no RAID in vSAN.  In it's simplest form there will be two exact copies of the data and then a witness object.  Each of these will be located on separate nodes in a cluster.  As long as 2 of the 3 are up and communicating, you can achieve quorum and the VM will remain up.

- If this will be the case, it will be applicable only to the host where it is configured, right? Would you still recommend an external storage that will be used as placeholder datastore to be seen in VSAN cluster for vsphere replication and SRM?

Correct, if you used local storage for the place holder VMs you would need to do a failover and the VMs could only power on the node where the storage is located.  You would then need to storage motion to the vSAN datastore.  If you're replicating from another storage, then I would use the vSAN datastore as the placeholder location.  It is fully supported.

Thank you, Zach.

[ChevUribe]

- what would be the data protection from VSAN if one disk on the host goes down or one of the disk in one of the diskgroup encountered a problem? Since no RAID will be done on the hardware level or even if RAID 0 is configured.

There is no RAID in vSAN.  In it's simplest form there will be two exact copies of the data and then a witness object.  Each of these will be located on separate nodes in a cluster.  As long as 2 of the 3 are up and communicating, you can achieve quorum and the VM will remain up.

--> will VSAN is the one doing the creation of the duplicate copies of each VM? how many maximum copies will be created per VM? Is this configurable?

- If this will be the case, it will be applicable only to the host where it is configured, right? Would you still recommend an external storage that will be used as placeholder datastore to be seen in VSAN cluster for vsphere replication and SRM?

Correct, if you used local storage for the place holder VMs you would need to do a failover and the VMs could only power on the node where the storage is located.  You would then need to storage motion to the vSAN datastore.  If you're replicating from another storage, then I would use the vSAN datastore as the placeholder location.  It is fully supported.

--> SDRS works with VSAN? thats between local datatstore and VSAN datastore only right? If VSAN can be the placeholder datastore, would it be an empty diskgroup or the VSAN datastore of the entire cluster? Can I have VSAN datastore as my replicating storage from primary to recovery site and have a placeholder VM datastore attached on a SAN storage that is seen by the VSAN cluster? If not or external SAN storage is not applicable or available, where do you suggest that we create the placeholder VM? 

Thanks!

[zdickinson]

--> will VSAN is the one doing the creation of the duplicate copies of each VM? how many maximum copies will be created per VM? Is this configurable?

Yes, vSAN does this.  It's called FTT (Failures to Tolerate) in the storage profile you assign to a VM.  Default is FTT = 1 and is the most you can do with 3 nodes.  FTT = 2 would have three copies of the data and you need 5 nodes to do that.

--> SDRS works with VSAN? thats between local datatstore and VSAN datastore only right? If VSAN can be the placeholder datastore, would it be an empty diskgroup or the VSAN datastore of the entire cluster? Can I have VSAN datastore as my replicating storage from primary to recovery site and have a placeholder VM datastore attached on a SAN storage that is seen by the VSAN cluster? If not or external SAN storage is not applicable or available, where do you suggest that we create the placeholder VM?

I don't think I can comment further, a lot of those questions are answered by "it depends".  My apologies, but I don't want to lead you astray.

Thank you, Zach.

[ChevUribe]

Thanks ZACH!

You have been very helpful to me with my inquiries.

Now I can approach our VSAN deployment with much ease (I have to think a way to get around with SRM)

I hope someone will join in with my discussion to further answer my questions, but you are one patient guy out there to assist me.