VMware Cloud Community
morantl
Enthusiast
Enthusiast
Jump to solution

2 node vsan deployment

We are running a 2 node cluster for our VDI deployment and want to use VSAN.  I understand from reading that a 2 node cluster with a witness is supported in 6.2.  However, our cluster is in the same data center.  I was told we could "fool" vsan by deploying a witness host however I don't quite understand how to do that.  Especially with what I'm reading in the documentation.  Here is our current layout.

2 Dell R730p servers

Dual 10gb nics

each has 2 180gb flash drives - cache tier

each has 4 10k 400gb drives - storage tier

They are physically in the same data center

I think I might be over complicating it but I would covet any suggestions or even examples for deployment.

Thank you in advance,

Tom

Reply
0 Kudos
1 Solution

Accepted Solutions
admin
Immortal
Immortal
Jump to solution

Greetings!

Yes, you can do this. This is known as 2+1 Stretched Cluster configuration.

Here are some of the points which should help you:

1) You need to have an ESXi VM OR physical ESXi host OR download VMware Virtual SAN Witness Appliance. Any of these can act as "Witness Host" for your 2 Node Cluster. The only requirement is that Witness Host must NOT be part of the Cluster where you want to configure VSAN and this witness host needs to contribute at least 1 SSD to Witness Cache Tier and 1 HDD to Witness Capacity Tier for VSAN Stretched Cluster. These drives would only be used for Witness purpose and would not contirbute any capacity to VSAN Datastore.

2) You need to have vmk port for Virtual SAN traffic or you can enable Virtual SAN traffic on existing vmk port. It is recommended to use dedicated NIC port for Virtual SAN traffic for a better performance.

3) You need Virtual SAN Enterprise license. (https://www.vmware.com/files/pdf/products/vsan/vmware-vsan-62-licensing-guide.pdf)


4) Here is the doc which explains in detail about VSAN Stretched Cluster. Please have a look at page 13 - Networking and latency requirements. Virtual SAN traffic between data sites is multicast. Witness traffic between a data site and the witness site is unicast. You can refer this doc for entire configuration.

http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/vsan/vmware-virtual-san-6....

Example:

Host A - 192.168.10.11 (Data Host -- Part of Cluster -- Multicast network with Host B)

Host B - 192.168.10.12 (Data Host -- Part of Cluster -- Multicast network with Host A)

Host C - 192.168.10.13 (Witness Host -- Not Part of Cluster -- Unicast network with Host A and Host B)

Hope this helps you acheiving your requirement. Let me know how it goes.

_________________________

Was your question answered correctly? If so, please remember to mark your question as answered when you get the correct answer and award points to the person providing the answer. This helps others searching for a similar issue.


Cheers!

-Shivam

View solution in original post

Reply
0 Kudos
29 Replies
admin
Immortal
Immortal
Jump to solution

Greetings!

Yes, you can do this. This is known as 2+1 Stretched Cluster configuration.

Here are some of the points which should help you:

1) You need to have an ESXi VM OR physical ESXi host OR download VMware Virtual SAN Witness Appliance. Any of these can act as "Witness Host" for your 2 Node Cluster. The only requirement is that Witness Host must NOT be part of the Cluster where you want to configure VSAN and this witness host needs to contribute at least 1 SSD to Witness Cache Tier and 1 HDD to Witness Capacity Tier for VSAN Stretched Cluster. These drives would only be used for Witness purpose and would not contirbute any capacity to VSAN Datastore.

2) You need to have vmk port for Virtual SAN traffic or you can enable Virtual SAN traffic on existing vmk port. It is recommended to use dedicated NIC port for Virtual SAN traffic for a better performance.

3) You need Virtual SAN Enterprise license. (https://www.vmware.com/files/pdf/products/vsan/vmware-vsan-62-licensing-guide.pdf)


4) Here is the doc which explains in detail about VSAN Stretched Cluster. Please have a look at page 13 - Networking and latency requirements. Virtual SAN traffic between data sites is multicast. Witness traffic between a data site and the witness site is unicast. You can refer this doc for entire configuration.

http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/vsan/vmware-virtual-san-6....

Example:

Host A - 192.168.10.11 (Data Host -- Part of Cluster -- Multicast network with Host B)

Host B - 192.168.10.12 (Data Host -- Part of Cluster -- Multicast network with Host A)

Host C - 192.168.10.13 (Witness Host -- Not Part of Cluster -- Unicast network with Host A and Host B)

Hope this helps you acheiving your requirement. Let me know how it goes.

_________________________

Was your question answered correctly? If so, please remember to mark your question as answered when you get the correct answer and award points to the person providing the answer. This helps others searching for a similar issue.


Cheers!

-Shivam

Reply
0 Kudos
morantl
Enthusiast
Enthusiast
Jump to solution

Thank you so much for the reply.  Very good information.  I've read most of those documents but your comments helped pull it together for me.

1 more question though.  If I use the Witness appliance you said it requires 1 ssd and 1 hdd to participate in the vsan.  Do they have to be physically ssd or is there a way to tell a vdmk to be ssd?  Confused again...sorry.

Tom

Reply
0 Kudos
zdickinson
Expert
Expert
Jump to solution

I assuming  you are referring to "this witness host needs to contribute at least 1 SSD to Cache Tier and 1 HDD to Storage Tier for VSAN Stretched Cluster."  I think there maybe a bit of confusion, but in general most people don't deploy an actual host to do the witness.  It's just a witness appliance that runs in a different cluster than your 2 node vSAN cluster.  The appliance is actually a nested ESXi.  Thank you, Zach.

Reply
0 Kudos
admin
Immortal
Immortal
Jump to solution

No problem, It's an open community for all kind of queries and I love to share the little knowledge I have.


VMware suggests to use Virtual SAN Witness Appliance to avoid the license cost for Witness Host as the witness appliance comes with its own license. You do not need to consume vSphere licenses for the witness appliance. A witness host/VM cannot be shared between multiple Virtual SAN Stretched Clusters. This Witness Appliance is actually an ESXi VM (Nested ESXi concept).


The answer to your query - If you use Witness Appliance, the OVA deployment for Witness Appliance automatically creates the required SSD and HDD. You don't need to configure it manually. Yes, It will be vmdk only at the backend datastore but ESXi VM will treat the vmdk as SSD.


Again you can refer the same document - http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/vsan/vmware-virtual-san-6....

Page 50 to 68 should help you with the step-by-step guide on Witness Appliance deployment and configuration.


Hope this answers your query. Smiley Happy


Cheers!

-Shivam

Reply
0 Kudos
JimPhreak
Contributor
Contributor
Jump to solution

1) You need to have an ESXi VM OR physical ESXi host OR download VMware Virtual SAN Witness Appliance. Any of these can act as "Witness Host" for your 2 Node Cluster. The only requirement is that Witness Host must NOT be part of the Cluster where you want to configure VSAN and this witness host needs to contribute at least 1 SSD to Cache Tier and 1 HDD to Storage Tier for VSAN Stretched Cluster.

Is there no way to use a 3rd ESXi host as a witness without it contributing storage at all?  I have ESXi hosts in a cluster and only 2 of them have storage (1 cache SSD, 2 capacity SSDs each) and I'd really love to be able to use my 3rd/4th host as my vSAN witness.  The problem with the witness appliance is it requires configuring static routes on the ESXi hosts and I can't find any good documentation/examples of how to do this.

Reply
0 Kudos
zdickinson
Expert
Expert
Jump to solution

The witness appliance does not contribute storage.  It is just ESXi running in a VM with hard drives attached.  1 of the hard drives is tagged as SSD and another as HDD, so it doesn't contribute any storage to the datastore, only acts as a witness.  Thank you, Zach.

Reply
0 Kudos
JimPhreak
Contributor
Contributor
Jump to solution

Right, but to use the appliance I need to configure static routing.

I'm trying to use a 3rd physical ESXi host as the witness but to do that (stretched cluster) it makes you add cache and capacity drives.

Reply
0 Kudos
admin
Immortal
Immortal
Jump to solution

Does your 3rd physical ESXi host has two extra physical drives? If yes, you can mark one disk as Flash and use it for Witness Cache Tier and use another disk for Witness Capacity Tier.

If you don't have extra physical disks on 3rd ESXi host then you have to use Witness Appliance or ESXi VM for Witness Host. If you deploy Witness Appliance, It will have Flash and HDD disks in it to be used for Witness Cache Tier and Witness Capacity Tier. If you use ESXi VM then you need to add extra disks (vmdk) to the the ESXi VM in order to have free disks for Witness Cache Tier and Witness Capacity Tier.

Static routes are only required when your Data Hosts and Witness Host are on different network. If your hosts are in same network then I don't think so static routes would be required.

Example (No Static routes required as all hosts are in same network)

Host A - 192.168.10.11 / 255.255.255.0 (Data Host -- Part of Cluster -- Multicast network with Host B)

Host B - 192.168.10.12 / 255.255.255.0 (Data Host -- Part of Cluster -- Multicast network with Host A)

Host C - 192.168.10.13 / 255.255.255.0 (Witness Host -- Not Part of Cluster -- Unicast network with Host A and Host B)

Example (Static routes required as Data Hosts and Witness Host are in different networks)

Host A - 192.168.10.11 / 255.255.255.0 (Data Host -- Part of Cluster -- Multicast network with Host B)

Host B - 192.168.10.12 / 255.255.255.0 (Data Host -- Part of Cluster -- Multicast network with Host A)

Host C - 10.130.12.15 / 255.255.255.0 (Witness Host -- Not Part of Cluster -- Unicast network with Host A and Host B)

If your Network Configuration needs static routes between Data Hosts and Witness Host, please see Page 69-70 for static routes configuration. It has example screenshots for static routes configuration.

http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/vsan/vmware-virtual-san-6....

The esxcli commands used to add a static route is:

# esxcli network ip route ipv4 add –n <remote network> -g <gateway to use>

Hope this helps you achieving your requirement.

_________________________

Was your question answered correctly? If so, please remember to mark your question as answered when you get the correct answer and award points to the person providing the answer. This helps others searching for a similar issue.


Cheers!

-Shivam

Reply
0 Kudos
zdickinson
Expert
Expert
Jump to solution

Good morning, I'm a bit confused are you trying to do a 2 node ROBO config or a stretched cluster?  Thank you, Zach.

Reply
0 Kudos
morantl
Enthusiast
Enthusiast
Jump to solution

Thank you all for the information.  I've got a lot to digest.  I will keep this thread updated as I get through some of the ideas you all have given.

Reply
0 Kudos
morantl
Enthusiast
Enthusiast
Jump to solution

2 node stretched cluster

Reply
0 Kudos
JimPhreak
Contributor
Contributor
Jump to solution

My 3rd and 4th host do not have any local storage no so that answers the question thank you.  I will deploy the witness appliance.

On a related note, since I have 4 hosts and 1 of them has to be out of the vSAN cluster in order to host the witness appliance, can I still have the 3rd non-storage contributing node in the vSAN cluster so it can make use of the vSAN datastore?  Also, once I've added the witness appliance ESXi host to my inventory, I assume I should just leave that host as a standalone and not part of any clusters?

Reply
0 Kudos
JimPhreak
Contributor
Contributor
Jump to solution

Honestly zdickinson‌, whichever will allow me to have at least 3 of my 4 hosts in the vSAN cluster being able to make use of the vSAN datastore.  From what I've read it seems I need to do make use of the stretched cluster if all 3 hosts (2 storage contributing + witness) are on the same local network.

Reply
0 Kudos
admin
Immortal
Immortal
Jump to solution

On a related note, since I have 4 hosts and 1 of them has to be out of the vSAN cluster in order to host the witness appliance, can I still have the 3rd non-storage contributing node in the vSAN cluster so it can make use of the vSAN datastore? 

Yes, your 3rd host (which will not contribute storage to VSAN Datastore) can be part of VSAN Cluster and can use VSAN Datastore.

Also, once I've added the witness appliance ESXi host to my inventory, I assume I should just leave that host as a standalone and not part of any clusters?

Yes, you should leave that host as a standalone host and not part of any cluster. In case of any failure to Witness Appliance, the witness components will go in an “absent” state. This will not impact the VMs running on VSAN Datastore as the VMs will continue to have a full copy of the data available from the data sites. If the witness host is unrecoverable, and a new witness host is deployed, the stretched cluster configuration can be recreated.

Hope this answers your queries Smiley Happy

Cheers!

-Shivam

Reply
0 Kudos
JimPhreak
Contributor
Contributor
Jump to solution

Thanks so much for answering those questions, that clears everything up.

On a related note, since the witness appliance requires at least 2 CPU's for even small environments, that means the corresponding ESXi hosts needs a 2CPU vSphere license.  Is there no way around that?

Reply
0 Kudos
admin
Immortal
Immortal
Jump to solution

Sorry, the license thing cannot be tweaked for the ESXi host. However, you do not need to consume vSphere licenses for the witness appliance as the witness appliance comes with its own license.

Cheers!

-Shivam

Reply
0 Kudos
JimPhreak
Contributor
Contributor
Jump to solution

Where does one find the included license for the witness appliance?  Because upon adding the witness appliance esxi host to my inventory it is asking for a 2 CPU license.

Reply
0 Kudos
admin
Immortal
Immortal
Jump to solution

It should show something like this while adding the Witness Appliance to vCenter Server inventory:

Witness_License.JPG

You can refer to Page 50 to 68 of document - http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/vsan/vmware-virtual-san-6....

This should help you with the step-by-step guide on Witness Appliance deployment and configuration.


Cheers!

-Shivam

Reply
0 Kudos
JimPhreak
Contributor
Contributor
Jump to solution

Unfortunately it did not and I already selected 2 available licenses I had in my environment.  Is there anyway to add the correct license without re-adding the witness to the vSAN cluster?

Reply
0 Kudos