Can VSAN run over "campus distances"?

[hennish]

Hi. My customer is planning on running a couple of mission critical (but not performance heavy) VMs on a four-node HA cluster spread (2*2) across a "campus" (around 500 m between the two server rooms).

To make sure their applications get HA/DR across both these "sites", they would need either a replicated SAN + SRM or a vMSC certified storage. Both of these are relatively expensive solutions.

Would they instead be able to run this reliably on VSAN, if they set all VM policies to FTT=3? This would mean that all objects are spread across all four ESXi hosts, which in theory should be enough to make sure that HA would work regardless of whether one host or an entire "site" fails.

What do you think? Doable or not supported?

[depping · Accepted solution]

Not supported:

http://www.yellow-bricks.com/2013/10/31/vsphere-metro-storage-cluster-using-virtual-san-can/

Also, with FTT=3 they will need more hosts:

4 is the minimum number of hosts for VSAN if you ask me

[hennish]

Hi. Thanks for the quick reply.

I understand that VSAN doesn't qualify as a vMSC solution, but what is the actual difference between a traditional setup and a vMSC setup when it comes to VSAN? Doesn't the line get a little blurred since there is no clear way to see whether the storage solution resides in one site or multiple sites?

If we count this customer's architecture as traditional LAN setup because of the "well-connectedness", I guess a "site" failure would count as a HA/VSAN partition event, or perhaps just as a failure of two of the four hosts?

Regarding the FTT=3 I didn't think about the number of data mirrors vs. witnesses, so when reading the table in your second blog post it makes sense. That issue itself invalidates the whole point of my idea, since the theory was to make sure that all VSAN nodes had a copy of the data.

Back to the drawing board for me.

[depping]

Yes the line gets blurred for sure. The main difference is: vMSC is officially tested for this scenario and VSAN never has been... It is an area of interest though.

problem today with VSAN for this use case: you cannot control where the components of your objects go. So all DATA and Witness could reside in 1 side.

[joergriether]

I talked to some engineers on vmworld europe and they also did mention that. As of now, you can not tell VSAN any affinity rules, for example to always place one physical copy on exactly host x at side a and one on host y at side b. But they however said that the devs are already talking about making that possible. So there is hope 😉

But to make this possible i guess the devs also have to implement routing capabilities for VSAN, because the chances to have another ip subnet at another side are high.

Best regards,

Joerg