MartinKerr
Enthusiast
Enthusiast

vSAN - Fault Domains Error

Jump to solution

Hi,

I have created a new 3 host cluster within vCenter 6 and enabled vSAN, added the correct storage (1 x Flash, 3 x HDD). They are all mounted and healthy. vSAN Switch has been set up and is connected via 10GB switches, MTU sizes on the virtual adapters are set at 9000 and set at 9240 on the physical switches. Also IGMP has also been enabled on the switches.

So the problem i am having is that i cannot migrate any VM storage to the vSAN Storage, the error i receive when trying to migrate is:

"Cannot complete file creation operation. This storage policy requires at least 3 fault domains with hosts contributing storage but only 1 were found. Failed to create object."

A bit strange as i have 3 fault domains with one host in each.

I currently don't have any leads on any resolution. Any help would be very grateful.

Many Thanks

0 Kudos
1 Solution

Accepted Solutions
MartinKerr
Enthusiast
Enthusiast

An update!

I have fixed the issue, it seems the MTU Size of the Virtual Switch did not match the Virtual Adapters (MTU Size 9000).

Now all is working fine and dandy! Thanks for your help.

Thanks

View solution in original post

0 Kudos
16 Replies
zdickinson
Expert
Expert

Good afternoon, I don't think fault domains are what you think they are.  With three hosts you can only have one fault domain.  A fault domain would be if you had three hosts in one rack and then three hosts in another.  You would then create two fault domains so you could lose an entire rack and maintain data availability.  With three hosts you want, and can only have, one fault domain.

Please verify and if someone can confirm, that would be great.  Thank you, Zach.

MartinKerr
Enthusiast
Enthusiast

Hi Zach,

Thanks for your comments and you are correct in what you are saying.

So how am I able to over come this issue?

Do I need to add all 3 hosts into one fault domain? Change the vSAN storage policy to only need one fault domain?

Any help is appreciated.

Thanks

0 Kudos
marcelo_soares
Champion
Champion

Hi MartinKerr,

Fault domais, as mentioned before, are only used if you have mutiple racks/chassis and want to separate your ESXi servers on this kind of organization - if you loose one entire rack, VMs are still available on the other. Fault domains must have at least 3 hosts each (or 2 hosts and a witness host in case of VSAN 6.1).

In your case, having only 3 ESX server in the cluster, you CAN'T use fault domains. You must remove all domains you created,leaving your ESX servers not associated with any domain. That way each one of them will be considered a fault domain themselves.

Let me know if this helps you on your configuration.

Marcelo

Marcelo Soares - VCP 3/4/5, VCI 5, VCAP5-DCA, vExpert 2012/2013 - Twitter: @mtsoares42
0 Kudos
MartinKerr
Enthusiast
Enthusiast

Hi Marcelo,

I have removed all hosts from their fault domain, leaving all 3 hosts not in a fault domain.

I still receive the same error regarding needing 3 fault domains.

Do i need to change the vSAN storage policy so it doesn't require 3 fault domain, if so where, as i have already removed the line 'Number of failures to tolerate' but i still receive the same error.

vSAN Fault Domains.PNG

vSAN Storage Policy.PNG

Thanks

0 Kudos
CHogan
VMware Employee
VMware Employee

Which version of VSAN is this?

If it is 6.0, can you install the health check?

If it is 6.1 (with vSphere 6.0U1), the health check is pre-installed.

Run the health check and see if there are any errors on the cluster.

http://cormachogan.com
0 Kudos
MartinKerr
Enthusiast
Enthusiast

I have downloaded, installed and restarted windows services for the Health Plug-In.

But, after install i cannot do anything on the cluster, it does not even show any status? Is this a common thing where it is currently doing a job which would prevent me seeing any information?

HealthPlugIn.PNG

Thanks

0 Kudos
CHogan
VMware Employee
VMware Employee

There are a few issues.

For Windows VC, make sure you install in a window opened with "Run as Administrator". This is in the health check guide - http://www.vmware.com/files/pdf/products/vsan/VMware-Virtual-SAN-Health-Check-Guide-6.1.pdf

There is a more recent issue in the VCSA appliance - it seems that certification permissions can affect the health check. See this thread: Problem with VSAN Health Check Plugin installation

http://cormachogan.com
0 Kudos
MartinKerr
Enthusiast
Enthusiast

There is no option to run the MSI as Administrator, so i logged in as the local administrator and installed.

Currently restarting the services and will update on the outcome, although not holding out much hope!

Thanks

0 Kudos
MartinKerr
Enthusiast
Enthusiast

I have also checked the MultiCast network settings and have the below:

MultiCast.PNG

Do these settings seem ok?

Thanks

0 Kudos
MartinKerr
Enthusiast
Enthusiast

So after overcoming the Plug-In installation by installing via local administrator, i now have the below:

Health.PNG

So...... i cannot enable DRS as i only have Essentials Plus Licenses?! Any way around this or any other method to check the health of vSAN?

Not going well at the moment Smiley Sad

Thanks

0 Kudos
CHogan
VMware Employee
VMware Employee

‌Download the VIB and install it manually on each of the hosts in the cluster. The link to the VIB is in the health check guide.

Keep persevering. The health check should reveal why you cannot deploy your VMs. I suspect the cluster is partitioned. The health check will tell you why.

http://cormachogan.com
0 Kudos
MartinKerr
Enthusiast
Enthusiast

And I don't need DRS enabled if I use the VIB?

0 Kudos
MartinKerr
Enthusiast
Enthusiast

So... within the documentation it states that DRS in needed for vSAN Health Check but i am not licensed for DRS.

Any other ways i can check the health of the vSAN and resolve the issues?

Thanks

0 Kudos
MartinKerr
Enthusiast
Enthusiast

I have a suspicion that it is to do the Network Partition Groups as i believe that they should be the same?

NetworkPartitionGroup.png

Is someone able to confirm this? If true, how do i overcome this?

Thanks

0 Kudos
CHogan
VMware Employee
VMware Employee

DRS enables a rolling upgrade of the VIB from vCenter.

Without DRS, you will have to manually install the VIB. Health check needs this software on the ESXi hosts so that it can check their state.

And yes - that partition is your problem.  You need to figure out why host 61 is not communicating correctly to host 62 and 63 over the VSAN network.

If you don't install the health check, use the networking section of the VSAN troubleshooting guide to figure it out. Remember, multicast is a requirement, so if you have IGMP, make sure its working properly. This is usually the problem.

http://www.vmware.com/files/pdf/products/vsan/VSAN-Troubleshooting-Reference-Manual.pdf

http://cormachogan.com
0 Kudos
MartinKerr
Enthusiast
Enthusiast

An update!

I have fixed the issue, it seems the MTU Size of the Virtual Switch did not match the Virtual Adapters (MTU Size 9000).

Now all is working fine and dandy! Thanks for your help.

Thanks

View solution in original post

0 Kudos