VMware Cloud Community
BlackMoonWolf
Enthusiast
Enthusiast

VCSA - Cyclical generation of vCLS nodes failing

VCSA Versions:
VCSA VC-7.0U2a  --> VCSA VC-7.0U2d

ESXi Host Versions: 
(Updated) ESXi-7.0U2a-17867351-standard (Dell Inc.)
Host servers: Dell PE R630
--------------------------------------------
I backed up my VCSA to upgrade to the current version. The upgrade was successful. Ever since I did that, the cluster is repeatedly trying to spin up new vCLS nodes. However, they don't seem to be able to find a bootable Photon installer. They immediately report "Operating system not found".

The nodes usually delete themselves afterward as well as their files in the datastores. (Not always it seems, though. I think interacting with the node prevents that.)

This is causing issues as I now have no vCLS nodes for my cluster. I've tried completely removing my ESXi nodes and deleting the entire cluster then rebuilding it. The issue reappears and persists. I've tried to see if it's just missing a bootable image or OVF according to the config files for the VM (as found in the corresponding datastore) but I'm not seeing anything noteworthy.

This is also causing issues for my network inventory software as it's creating NUMEROUS node entries that don't actually exist and causing the software to hit it's license limit... I have to clean it out repeatedly.

I'm not sure what to do at this point other than delete this seasoned VCSA install and start all over again...and I really DO NOT want to do that if I don't have to.

Anyone have any thoughts as to why the cluster seems to have lost it's ability to generate it's vCLS nodes?

Please see the screenshots attached!

 
 
Reply
0 Kudos
3 Replies
dvandelaar
Enthusiast
Enthusiast

Hi,

First thing notable is the naming of the vCLS vm's. In your added pictures this is 'vCLS-some-uuid'
This is introduced in vSphere 7.0U3. According to your information you upgraded to vSphere 7.0U2d. That version still should use the 'vCLS(##)' format. 

See also vSphere 7 Update 3: vCLS Updates | VMware

There are options to retreat and reactivate, the document I found for this is quite long though.
Hope it is of any help. Look for 'Retreat Mode'. vSphere Cluster Services (vCLS) in vSphere 7.0 Update 1 and newer versions (80472) (vmware.com)

Be careful though! Good luck.

BlackMoonWolf
Enthusiast
Enthusiast

Hey. Thanks for the response. I'm afraid I've been far too busy to come back to this of late. I'll review the documentation that you've included and see if I can work something out from it.

If I thought that fragging the vCenter install all together and just building a new one from the ground up would be a better investment of time and less frustrating I would just do that at this point. However... In all of my reading so far, it seems that this is a newer function of sorts that I assume I would just experience again in a new installation anyway. (Just that, in the case of this older installation, it's somehow broken? I just can't be sure...)

I'll keep looking into this and continue keeping an eye out here for responses. Thanks again! 🙂

Reply
0 Kudos
BlackMoonWolf
Enthusiast
Enthusiast

Found that there was another update for vCenter this morning so I ran it.

Version shown in Server Management GUI: v7.0.3.00300 Build number: 19234570

Version shown in GUI: "Version:7.0.3 Build:19234570"

This behavior still continues. 

 

Reply
0 Kudos