BHagenSPI
Enthusiast
Enthusiast

How to remove Absent VSAN disks?

Jump to solution

I'm seeing a couple "absent" disks in my new VSAN install. But I can't remove them because they don't exist. How do I get rid of them? In the screenshots below, I've set the host into maintenance mode, clicked the disk group, then clicked "Remove the disk group" icon. It errors out with " A general system error occurred: Disk is not mounted on this host". I then SSH into the host and use #esxcli vsan storage remove -d naa.5000c5008cd8af28, but it says the disk is not mounted. Yeah; I knew that...

Also, that specific host (anesxi03) should look like all the others; but a) it's saying "Disks in Use" is 6 of 6, even though it only has 4 physical disks like the other 4 hosts (1x SSD, 3x HDD), and b) the 2 unmounted disk groups have completely different looking UUIDs.

vsan1.jpg

1 Solution

Accepted Solutions
BHagenSPI
Enthusiast
Enthusiast

The answer for me, in this case, was to go into the configuration of the host that has the absent disks and remove the partitions from all 4 drives. Then claim them for VSAN again, and we're good to go.

View solution in original post

12 Replies
elerium
Hot Shot
Hot Shot

Do the absent disks appear when you run esxcli vsan storage list?

I haven't seen your issue before, but maybe try rescanning the host for storage devices (Storage->Rescan Storage in webclient) or rebooting the host.

0 Kudos
BHagenSPI
Enthusiast
Enthusiast

No; they don't appear when I run that command. I've rescanned and rebooted several times; still no joy...

0 Kudos
elerium
Hot Shot
Hot Shot

From all the indications, it looks like the host itself doesn't recognize those disks (because they're really not there!) but vcenter does. If you're able to, maybe worth restarting vCenter, it's possible it's a webclient bug.

0 Kudos
admin
Immortal
Immortal

Greetings!

I think this is happening because vCenter Server still believes that these disks are part of Disk Groups of your ESXi Host. Please try restarting vmware-vsan-health service on vCenter Server and see if that helps to resolve this issue.


From vCenter Server Appliance command line interface, run below command to restart vmware-vsan-health service:

# service vmware-vsan-health restart

_________________________

Was your question answered correctly? If so, please remember to mark your question as "Correct" or "Helpful" when you get the correct answer. This helps others searching for a similar issue.


Cheers!

-Shivam

0 Kudos
BHagenSPI
Enthusiast
Enthusiast

Thanks; I ran the command service vmware-vsan-health restart on that host, and I get the response: Failed to restart vmware-vsan-health.service: Unit vmware-vsan-health.service is masked.

I ran the command service vmware-vsan-health status, and I get:

vmware-vsan-health.service

     Loaded: masked (/dev/null; bad)

     Active: inactive (dead)

I got desperate and rebuilt the host, from scratch...and I get the exact same responses and the same disks are still not found in the web interface.

0 Kudos
admin
Immortal
Immortal

You mentioned that you ran the command on the problematic host but I am assuming you ran it on vCenter Server Appliance as this service is not available at ESXi host level. This is a service at vCenter Server level.

Looks like there is some issue with the "vmware-vsan-health" service in vCenter Server which is causing this issue. Can you please try stopping it and starting it back using service-control as shown below:

Stopping "vmware-vsan-health" service with service-control command:

# service-control --stop vmware-vsan-health

Perform stop operation. vmon_profile=None, svc_names=['vmware-vsan-health'], include_coreossvcs=False, include_leafossvcs=False

Successfully stopped service vsan-health

Starting "vmware-vsan-health" service with service-control command:

# service-control --start vmware-vsan-health

Perform start operation. vmon_profile=None, svc_names=['vmware-vsan-health'], include_coreossvcs=False, include_leafossvcs=False

2016-12-14T20:17:50.216Z   Service vsan-health state STOPPED

Successfully started service vsan-health


_________________________

Cheers!

-Shivam

0 Kudos
BHagenSPI
Enthusiast
Enthusiast

I ran those two commands. At first, the --stop command didn't work at all; I'm completely unfamiliar with RVC, so I futzed around for a while...not really sure what all I tried. Then I tried the --stop command again and it actually worked. Did the --start command and it worked as well.

Went back into vcenter to see that the two unmounted disk groups were still there. Refreshed, logged out of the web interface, logged back in...no change. Tried to delete one of them but couldn't; same error: "A general system error occurred: Disk is not mounted on this host."

I noticed that each of the unmounted groups had a valid hdd and an invalid sdd. So for some reason I though to delete the MOUNTED disk group. I did that and it worked, but still couldn't delete the UNMOUNTED groups.

Then for some reason I though to go into the configuration of the host itself and remove the partitions from all 4 drives. That went fine, and when I went back to Datastore / Configure / Device backing, and poof...the unmounted disk groups were gone!

I claimed the disks, and a valid and correct disk group was created.

Fantastic!

But...I went back to RVC just now and ran the service vmware-vsan-health status command...and I get the exact same result as before:

vmware-vsan-health.service

     Loaded: masked (/dev/null; bad)

     Active: inactive (dead)

Thoughts?

0 Kudos
BHagenSPI
Enthusiast
Enthusiast

The answer for me, in this case, was to go into the configuration of the host that has the absent disks and remove the partitions from all 4 drives. Then claim them for VSAN again, and we're good to go.

RAJ_RAJ
Expert
Expert

I also tried all the options and finally i have to remove the drives from ESXi host and add it again it worked .

RAJESH RADHAKRISHNAN VCA -DCV/WM/Cloud,VCP 5 - DCV/DT/CLOUD, ,VCP6-DCV, EMCISA,EMCSA,MCTS,MCPS,BCFA https://ae.linkedin.com/in/rajesh-radhakrishnan-76269335 Mark my post as "helpful" or "correct" if I've helped resolve or answered your query!
0 Kudos
Athar_1
Contributor
Contributor

Login to ESXi host and type below command to list read VSAN UUID for the absent disk

#esxcli vsan storage list

Now you have VSAN UUID of absent disk, Type below command to remove absent disk 

#esxcli vsan storage remove --uuid <VSAN UUID of absent disk>

Now absent disk is removed, you can verify with below command.

#esxcli vsan storage list

Refresh vCenter and re-test skyline health for vSAN cluster.

 

Now cheers once issue is respved.

Jazcat
Contributor
Contributor

@Athar_1 wrote:

Login to ESXi host and type below command to list read VSAN UUID for the absent disk

#esxcli vsan storage list

Now you have VSAN UUID of absent disk, Type below command to remove absent disk 

#esxcli vsan storage remove --uuid <VSAN UUID of absent disk>

Now absent disk is removed, you can verify with below command.

#esxcli vsan storage list

Refresh vCenter and re-test skyline health for vSAN cluster.

 

Now cheers once issue is respved.


This worked for me thank you