VMware Cloud Community
1MarkM
Enthusiast
Enthusiast
Jump to solution

ESXi configuration backups - not

I have 6 ESXi hosts divided into 2 clusters, 1 cluster per Vcenter. The Vcenters are in embedded linked mode. We use SRM hence the two Vcenters - one onsite and the other at our Colo.

Hosts are running VMware ESXi, 7.0.1, 17168206 or newer (based off of the Dell 7.0.0U1 OEM image)

Vcenters are Version:7.0.1 Build:17327586

All of ESXi hosts are configured for a "persistent" scratch location via a SAN LUN/Datastore. Each cluster maps to different SAN.

3 of my hosts have no problem generating a config backup either from CLI or PowerCLI. The other 3 hosts nada. When the backup attempt is made on the bad hosts a "GUID" subdirectory in the /scratch/downloads folder is created, but no config file is in the created subdirectory (an empty directory basically). 

PowerCLI errors with- get-vmhostfirmware : 1/7/2021 12:07:57 PM Get-VMHostFirmware A general system error occurred: Internal error
At line:1 char:15
+ ... et-vmhost | get-vmhostfirmware -BackupConfiguration -DestinationPath ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Get-VMHostFirmware], SystemError
+ FullyQualifiedErrorId : Client20_SystemManagementServiceImpl_BackupVmHostFirmware_ViError,VMware.VimAutomation.ViCore.Cmdlets.Commands.Host.GetVMHostFirmware

Yeah I know what you are saying - dude you got 3 that work, what's the difference between them? That's the problem I don't see any difference. It is not Vcenter/cluster based because 2 work on 1 cluster, and 1 works on the other Vcenter cluster. Each of the hosts on the cluster have their /scratch directory map to the same datastore (different directories for each), so it isn't the LUN/Datastore. 

I've seen plenty of issues regarding the /scratch/"downloads" directory missing because of driver conflicts (HP/Emulex/ESXi 6.5). But the download directory is there (as a "GUID" subdirectory gets created with each config backup attempt). 

Any thoughts?

Reply
0 Kudos
1 Solution

Accepted Solutions
1MarkM
Enthusiast
Enthusiast
Jump to solution

I'll answer my own question. Our boot media is based on SD Flash cards (mirrored) used in the Dell R730. Dell calls it IDSDM. So the flash cards are 4-5 years old by now on our 6 R730's. Each was migrated from 6.0 to 6.7 to 7.0.1 plus quarterly updates.

As it turned out the 3 R730's that didn't have the problem backing up the configuration file, also had recently been updated with new flash cards (long story). I then found that each of the problem child ESXi servers (that couldn't backup the configuration) also actually could not boot anymore due to corrupted SD Flash cards. After replacing the cards (and having to hand rebuild the configuration (STD license here), voila - the configuration backups worked.

I suspect the SD Flash cards ran out or "write" cycles.... that's my story and I'm sticking to it.

View solution in original post

Reply
0 Kudos
1 Reply
1MarkM
Enthusiast
Enthusiast
Jump to solution

I'll answer my own question. Our boot media is based on SD Flash cards (mirrored) used in the Dell R730. Dell calls it IDSDM. So the flash cards are 4-5 years old by now on our 6 R730's. Each was migrated from 6.0 to 6.7 to 7.0.1 plus quarterly updates.

As it turned out the 3 R730's that didn't have the problem backing up the configuration file, also had recently been updated with new flash cards (long story). I then found that each of the problem child ESXi servers (that couldn't backup the configuration) also actually could not boot anymore due to corrupted SD Flash cards. After replacing the cards (and having to hand rebuild the configuration (STD license here), voila - the configuration backups worked.

I suspect the SD Flash cards ran out or "write" cycles.... that's my story and I'm sticking to it.

Reply
0 Kudos