Rebuild VSAN node procedure

[jim33boy]

Hi All,

We purchased a 3 node VSAN cluster from Dell. The hardware was ordered with no mirrored SD cards. So the hypervisor is running on an non mirrored installation.

I have since ordered 3 additional SD cards and will mirror them.

What I need to do is to take the members down one node at a time and mirror the SD cards, then restore the ESXi configuration.

What I have done so far is to backup the ESXi config from powercli with the following command.

Get-VMHostFirmware -VMHost ESXi_host_IP_address -BackupConfiguration -DestinationPath output_directory

So my need now is to take down my first node and proceed with mirroring. Can someone provide me the correct procedure to restore a VSAN node? Is it as simple as this...

• Reinstall the same version of ESXi.
• Run Set-VMHost -VMHost ESXi_host_IP_address -State 'Maintenance'
• Set-VMHostFirmware -VMHost ESXi_host_IP_address -Restore -SourcePath backup_file -HostUser username -HostPassword password

Will this get me back to the state I was before?

Before i take down the host, is there a way to move all components to the other 2 VSAN members first?

Thanks in advance.

[Jasemccarty]

jim33boy‌,

You can put each host in maintenance mode (obviously one at a time) and rebuild your boot media.

Depending on what policy you have in place, different Virtual SAN maintenance modes could be chosen.

You've got the following to pick from:

• Ensure Accessibility
• Full Migration
• No Data Migration

Ensure Accessibility will make sure that you have objects on remaining available hosts to meet the requirement of >50% components available. With a storage policy including FTT=1 (min 3 nodes), that would then ensure all objects have a (at least) working copy & witness on the remaining hosts (2 in your situation). In a storage policy including FTT=0, components would be recreated on the remaining hosts. FTT=2/FTT=3 is not possible in a 3 node configuration, so I won't go into it.

Full Migration, evacuates data from the disk group(s) on the host being put in maintenance mode. You would have to have enough resources to satisfy policies. In your case of 2 remaining hosts, FTT=1 (or 2 or 3) can not be satisfied. If a policy cannot be satisfied, then this method may not be chosen.

No data migration does nothing to the data. This would be treated the same as if the host went offline (absent). VMs with components that have a FTT=0 policy residing on this host, would have a disruption, but components with FTT=1 would not be affected.

The question would really be how long do you think it will take for you the make the correction/change? I'd suggest Ensure Accessibility in most cases.

Thanks,

Jase

Jase McCarty
VMware Storage & Availability Technical Marketing.

[jim33boy]

Thank you for the response Jase. We are currently set to the default of FTT=1. I think I will choose Full Evacuation.

So can you help me with the proper steps to restore the config? I mean with VSAN, is it just like a typical ESXi host where I can restore the config and go with set-firmware or vicfg-cfgbackup or would I be better off doing a fresh install, keeping VSAN partitions intact and following this link below?

Thanks again

Adding a host back to a Virtual SAN cluster after an ESXi host rebuild (2059091)

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=205909...

[zdickinson]

Can you not just put the host in maintenance mode, down it, put the card in, go into the BIOS and say mirror?  Obviously you want to mirror from the existing to the new and not vice versa.  Thank you, Zach.

[jim33boy]

In my experience when you create a virtual disk from existing disks, there is an initialize process.

For my DR purposes I would like to know the correct process to build from the ground up so I can document it.

While I do appreciate this response it does not answer my question.

I am looking for guidance on how to recover a VSAN ESXi host ground up.

Thanks again

[elerium · Accepted solution]

As Zach has mentioned, your mirroring of SD cards should not involve the need to rebuild a VSAN node. Maintenance mode with either Ensure Accessibility (okay for maintenance under 1 hour) or Full Migration would likely be enough for what you are doing. Depending how much data you have and how fast your drives are, Full Migration can take a while but is the safest option.

However if you needed to rebuild a node because of host hardware failure as an example, you would remove the disk group for the failed host from VSAN and then remove the failed host from the cluster. Then bring up a new working host, reinstall ESXi, configure network/vmkernels etc, re-add it to the VSAN cluster and lasly re-add the disk group for the new host. I have also rebuilt hosts on my clusters from RAID0 to HBA as an example with this procedure and it works fine.