Groups, components, witnesses, resilience, performance, concepts ?

[TronAr]

Hi,

I'm trying to nail down concepts about the VSAN architecture and resilience implementation.

So far I've been reading a lot, but still don't quite get some facts about the architecture.

Just to recap, VSAN provides storage with flexible services as of resilience (to failures of disks/hosts/network), performance (via paralel stripes on different groups, % of SSD) and safety (via % of allocation) based on ESXi "native" services (i.e. no appliance involved) and resources (SSDs, disks, 1Gbps nic).

Services are defined via policies on objects.

The implementation requires at least 3 hosts providing disk resources, and a minimum disk resource (group) is composed of 1 SSD and at least 1 disk.

Now, things start less firm to me.

i) Resilience is stated as number of failures to tolerate. That is any of group/host/network ?

Most of the examples I've seen are for FTT = 1, interesting things are with FTT = 2 or 3

ii) Given that it tolerates network failures, an involved scheme using quorum is used. I guess that because of that, the # of hosts required goes from n+1 to 2n+1, but I have been not able to find a document that explains how it works. It would seem that n+1 replicas on different hosts are needed and the rest to 2n+1 is just enough to go with witnesses (on different hosts) but again, I have not seen a document that clearly states so. I have not seen any requirements to network redundancy either.

iii) Were this to be correct, a FTT = 2 would need 3 replicas and 2 witnesses. In this scenario, a network failure could create a partition, say 2/3 hosts. Given that votes seem to have the same weight, it could be that majority is obtained in a 3 component partition. I assume that the 2 component partition will safely shutdown. But the 3 comp. side could be just 2 witnesses and 1 replica... then if THAT replica fails... ???

I'd love some insight.

TIA,

-Carlos

[cmiller78]

I think you've got it.  When host 1 goes down,  host 2 if a witness knows host 3 has the active mirror.  If the mirror is reestablished on host 2 it knows its mirror copy is good and if not it knows host 2 has the most recent.  Then host 2 stops l drops and subsequently host 3. When the first two boot the cluster comes online there is still awareness that host 3 has the recent copy and HA waits to restart.

To your point the document doesn't describe what happens if host 3 never comes back.  I will assume based on other descriptions of functionality that when the 60 minute timer expires then the mirror on host 1 becomes the most recent copy and there is data loss. I will try to test this when I get time in my lab,  you've made me think about this differently and for that thank you!

[TronAr]

That could be, but is not what I would expect. I would rather request a manual intervention before accepting a continue with data loss.

Given that you had FTT=1, and you had already 2 failures, VSAN is free to do whichever way vmware decides

I would expect a stuck situation.

Again host 1 & 2 have no way of telling that host 3 survived host 2. In fact, host 3 should have shut itself down as soon as it lost connection to host 2, because he was alone (with no quorum). But I digress ...

This is a very interesting topic. Nice chat, sorry for my roughness

[cmiller78]

no worries conversation is good

I think you are right,  once quorum is lost the datastore goes offline.  I recall this when I was blowing up my lab intentionally,  I just didn't have anything on it