We are running a 2 node cluster for our VDI deployment and want to use VSAN. I understand from reading that a 2 node cluster with a witness is supported in 6.2. However, our cluster is in the same data center. I was told we could "fool" vsan by deploying a witness host however I don't quite understand how to do that. Especially with what I'm reading in the documentation. Here is our current layout.
2 Dell R730p servers
Dual 10gb nics
each has 2 180gb flash drives - cache tier
each has 4 10k 400gb drives - storage tier
They are physically in the same data center
I think I might be over complicating it but I would covet any suggestions or even examples for deployment.
Thank you in advance,
Tom
Greetings!
Yes, you can do this. This is known as 2+1 Stretched Cluster configuration.
Here are some of the points which should help you:
1) You need to have an ESXi VM OR physical ESXi host OR download VMware Virtual SAN Witness Appliance. Any of these can act as "Witness Host" for your 2 Node Cluster. The only requirement is that Witness Host must NOT be part of the Cluster where you want to configure VSAN and this witness host needs to contribute at least 1 SSD to Witness Cache Tier and 1 HDD to Witness Capacity Tier for VSAN Stretched Cluster. These drives would only be used for Witness purpose and would not contirbute any capacity to VSAN Datastore.
2) You need to have vmk port for Virtual SAN traffic or you can enable Virtual SAN traffic on existing vmk port. It is recommended to use dedicated NIC port for Virtual SAN traffic for a better performance.
3) You need Virtual SAN Enterprise license. (https://www.vmware.com/files/pdf/products/vsan/vmware-vsan-62-licensing-guide.pdf)
4) Here is the doc which explains in detail about VSAN Stretched Cluster. Please have a look at page 13 - Networking and latency requirements. Virtual SAN traffic between data sites is multicast. Witness traffic between a data site and the witness site is unicast. You can refer this doc for entire configuration.
Example:
Host A - 192.168.10.11 (Data Host -- Part of Cluster -- Multicast network with Host B)
Host B - 192.168.10.12 (Data Host -- Part of Cluster -- Multicast network with Host A)
Host C - 192.168.10.13 (Witness Host -- Not Part of Cluster -- Unicast network with Host A and Host B)
Hope this helps you acheiving your requirement. Let me know how it goes.
_________________________
Was your question answered correctly? If so, please remember to mark your question as answered when you get the correct answer and award points to the person providing the answer. This helps others searching for a similar issue.
Cheers!
-Shivam
Greetings!
Yes, you can do this. This is known as 2+1 Stretched Cluster configuration.
Here are some of the points which should help you:
1) You need to have an ESXi VM OR physical ESXi host OR download VMware Virtual SAN Witness Appliance. Any of these can act as "Witness Host" for your 2 Node Cluster. The only requirement is that Witness Host must NOT be part of the Cluster where you want to configure VSAN and this witness host needs to contribute at least 1 SSD to Witness Cache Tier and 1 HDD to Witness Capacity Tier for VSAN Stretched Cluster. These drives would only be used for Witness purpose and would not contirbute any capacity to VSAN Datastore.
2) You need to have vmk port for Virtual SAN traffic or you can enable Virtual SAN traffic on existing vmk port. It is recommended to use dedicated NIC port for Virtual SAN traffic for a better performance.
3) You need Virtual SAN Enterprise license. (https://www.vmware.com/files/pdf/products/vsan/vmware-vsan-62-licensing-guide.pdf)
4) Here is the doc which explains in detail about VSAN Stretched Cluster. Please have a look at page 13 - Networking and latency requirements. Virtual SAN traffic between data sites is multicast. Witness traffic between a data site and the witness site is unicast. You can refer this doc for entire configuration.
Example:
Host A - 192.168.10.11 (Data Host -- Part of Cluster -- Multicast network with Host B)
Host B - 192.168.10.12 (Data Host -- Part of Cluster -- Multicast network with Host A)
Host C - 192.168.10.13 (Witness Host -- Not Part of Cluster -- Unicast network with Host A and Host B)
Hope this helps you acheiving your requirement. Let me know how it goes.
_________________________
Was your question answered correctly? If so, please remember to mark your question as answered when you get the correct answer and award points to the person providing the answer. This helps others searching for a similar issue.
Cheers!
-Shivam
Thank you so much for the reply. Very good information. I've read most of those documents but your comments helped pull it together for me.
1 more question though. If I use the Witness appliance you said it requires 1 ssd and 1 hdd to participate in the vsan. Do they have to be physically ssd or is there a way to tell a vdmk to be ssd? Confused again...sorry.
Tom
I assuming you are referring to "this witness host needs to contribute at least 1 SSD to Cache Tier and 1 HDD to Storage Tier for VSAN Stretched Cluster." I think there maybe a bit of confusion, but in general most people don't deploy an actual host to do the witness. It's just a witness appliance that runs in a different cluster than your 2 node vSAN cluster. The appliance is actually a nested ESXi. Thank you, Zach.
No problem, It's an open community for all kind of queries and I love to share the little knowledge I have.
VMware suggests to use Virtual SAN Witness Appliance to avoid the license cost for Witness Host as the witness appliance comes with its own license. You do not need to consume vSphere licenses for the witness appliance. A witness host/VM cannot be shared between multiple Virtual SAN Stretched Clusters. This Witness Appliance is actually an ESXi VM (Nested ESXi concept).
The answer to your query - If you use Witness Appliance, the OVA deployment for Witness Appliance automatically creates the required SSD and HDD. You don't need to configure it manually. Yes, It will be vmdk only at the backend datastore but ESXi VM will treat the vmdk as SSD.
Again you can refer the same document - http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/vsan/vmware-virtual-san-6....
Page 50 to 68 should help you with the step-by-step guide on Witness Appliance deployment and configuration.
Hope this answers your query.
Cheers!
-Shivam
1) You need to have an ESXi VM OR physical ESXi host OR download VMware Virtual SAN Witness Appliance. Any of these can act as "Witness Host" for your 2 Node Cluster. The only requirement is that Witness Host must NOT be part of the Cluster where you want to configure VSAN and this witness host needs to contribute at least 1 SSD to Cache Tier and 1 HDD to Storage Tier for VSAN Stretched Cluster.
Is there no way to use a 3rd ESXi host as a witness without it contributing storage at all? I have ESXi hosts in a cluster and only 2 of them have storage (1 cache SSD, 2 capacity SSDs each) and I'd really love to be able to use my 3rd/4th host as my vSAN witness. The problem with the witness appliance is it requires configuring static routes on the ESXi hosts and I can't find any good documentation/examples of how to do this.
The witness appliance does not contribute storage. It is just ESXi running in a VM with hard drives attached. 1 of the hard drives is tagged as SSD and another as HDD, so it doesn't contribute any storage to the datastore, only acts as a witness. Thank you, Zach.
Right, but to use the appliance I need to configure static routing.
I'm trying to use a 3rd physical ESXi host as the witness but to do that (stretched cluster) it makes you add cache and capacity drives.
Does your 3rd physical ESXi host has two extra physical drives? If yes, you can mark one disk as Flash and use it for Witness Cache Tier and use another disk for Witness Capacity Tier.
If you don't have extra physical disks on 3rd ESXi host then you have to use Witness Appliance or ESXi VM for Witness Host. If you deploy Witness Appliance, It will have Flash and HDD disks in it to be used for Witness Cache Tier and Witness Capacity Tier. If you use ESXi VM then you need to add extra disks (vmdk) to the the ESXi VM in order to have free disks for Witness Cache Tier and Witness Capacity Tier.
Static routes are only required when your Data Hosts and Witness Host are on different network. If your hosts are in same network then I don't think so static routes would be required.
Example (No Static routes required as all hosts are in same network)
Host A - 192.168.10.11 / 255.255.255.0 (Data Host -- Part of Cluster -- Multicast network with Host B)
Host B - 192.168.10.12 / 255.255.255.0 (Data Host -- Part of Cluster -- Multicast network with Host A)
Host C - 192.168.10.13 / 255.255.255.0 (Witness Host -- Not Part of Cluster -- Unicast network with Host A and Host B)
Example (Static routes required as Data Hosts and Witness Host are in different networks)
Host A - 192.168.10.11 / 255.255.255.0 (Data Host -- Part of Cluster -- Multicast network with Host B)
Host B - 192.168.10.12 / 255.255.255.0 (Data Host -- Part of Cluster -- Multicast network with Host A)
Host C - 10.130.12.15 / 255.255.255.0 (Witness Host -- Not Part of Cluster -- Unicast network with Host A and Host B)
If your Network Configuration needs static routes between Data Hosts and Witness Host, please see Page 69-70 for static routes configuration. It has example screenshots for static routes configuration.
The esxcli commands used to add a static route is:
# esxcli network ip route ipv4 add –n <remote network> -g <gateway to use>
Hope this helps you achieving your requirement.
_________________________
Was your question answered correctly? If so, please remember to mark your question as answered when you get the correct answer and award points to the person providing the answer. This helps others searching for a similar issue.
Cheers!
-Shivam
Good morning, I'm a bit confused are you trying to do a 2 node ROBO config or a stretched cluster? Thank you, Zach.
Thank you all for the information. I've got a lot to digest. I will keep this thread updated as I get through some of the ideas you all have given.
2 node stretched cluster
My 3rd and 4th host do not have any local storage no so that answers the question thank you. I will deploy the witness appliance.
On a related note, since I have 4 hosts and 1 of them has to be out of the vSAN cluster in order to host the witness appliance, can I still have the 3rd non-storage contributing node in the vSAN cluster so it can make use of the vSAN datastore? Also, once I've added the witness appliance ESXi host to my inventory, I assume I should just leave that host as a standalone and not part of any clusters?
Honestly zdickinson, whichever will allow me to have at least 3 of my 4 hosts in the vSAN cluster being able to make use of the vSAN datastore. From what I've read it seems I need to do make use of the stretched cluster if all 3 hosts (2 storage contributing + witness) are on the same local network.
On a related note, since I have 4 hosts and 1 of them has to be out of the vSAN cluster in order to host the witness appliance, can I still have the 3rd non-storage contributing node in the vSAN cluster so it can make use of the vSAN datastore?
Yes, your 3rd host (which will not contribute storage to VSAN Datastore) can be part of VSAN Cluster and can use VSAN Datastore.
Also, once I've added the witness appliance ESXi host to my inventory, I assume I should just leave that host as a standalone and not part of any clusters?
Yes, you should leave that host as a standalone host and not part of any cluster. In case of any failure to Witness Appliance, the witness components will go in an “absent” state. This will not impact the VMs running on VSAN Datastore as the VMs will continue to have a full copy of the data available from the data sites. If the witness host is unrecoverable, and a new witness host is deployed, the stretched cluster configuration can be recreated.
Hope this answers your queries
Cheers!
-Shivam
Thanks so much for answering those questions, that clears everything up.
On a related note, since the witness appliance requires at least 2 CPU's for even small environments, that means the corresponding ESXi hosts needs a 2CPU vSphere license. Is there no way around that?
Sorry, the license thing cannot be tweaked for the ESXi host. However, you do not need to consume vSphere licenses for the witness appliance as the witness appliance comes with its own license.
Cheers!
-Shivam
Where does one find the included license for the witness appliance? Because upon adding the witness appliance esxi host to my inventory it is asking for a 2 CPU license.
It should show something like this while adding the Witness Appliance to vCenter Server inventory:
You can refer to Page 50 to 68 of document - http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/vsan/vmware-virtual-san-6....
This should help you with the step-by-step guide on Witness Appliance deployment and configuration.
Cheers!
-Shivam
Unfortunately it did not and I already selected 2 available licenses I had in my environment. Is there anyway to add the correct license without re-adding the witness to the vSAN cluster?