Unable to delete ghost volume

[GAPCABIV]

naa.6006016003c02800dc0bc7f0b6f0e311 is a fibre channel LUN from our VNX5500 presented to our 7 vSphere 5.1U2 hosts.  It is identified as LUN 0 when viewed through the vSphere Client on all 7 hosts an is an empty datastore.

When I unmount the datastore and detach the LUN, it greys out as expected.  I then go the the VNX and remove that LUN from the storage pool, which in turn removes it from the hosts when a "Rescan All" is performed.

At that time naa.6006016003c02800dc0bc7f0b6f0e311 disappears however, a new LUN 0 naa.50060160bde00e5850060160bde00e58 appears and there is no size associated and there is no new datastore.  If I choose "Add Storage" this "ghost" volume is not available and I can unmount it and mount it, but there is nothing there and it really is not related to any actual storage.  It is just a ghost LUN in vSphere.

If I re-add the real naa.600... LUN to the storage pool, naa.500... disappears and LUN 0 reflects correctly again with naa.600... identifier.  I remove it again and LUN 0 becomes naa.500... identifier again.

I want them both gone but I cannot get rid of naa.500... no matter what I try.

There is no sDRS or SIOC in play.

I restarted management agents as well as a services.sh restart and when neither of those worked I rebooted 1 host but it still didn't remove the ghost LUN.

I have done all the esxcli trickery I can find related to deleting volumes.  Funny thing is when I run the commands to unmount it, set it to "off" etc, the naa.500... volume grays out in the client and the commands are successful.  Then I run the actual delete command and it runs successfully within the CLI but if you watch the GUI, naa.500... that was grayed out becomes active (not grayed out) again within a couple of second after deleting from the CLI.

How whack is that?

I suspect it is something left behind from a volume that no longer exists and probably has not existed for a very long time, but that was not properly removed when it was removed.

Does anyone have any suggestions for how to get rid of this mysterious ghost volume?

[Anjani_Kumar]

You stucked in All paths dowh (APD) condition. to know more and fix this please follow this link. hope it will help you out .

Best Practice: How to correctly remove a LUN from an ESX host | VMware vSphere Blog - VMware Blogs

[GAPCABIV]

Thanks for the reply Anjani.  Unfortunately I have previously in the day tried the steps in that as well as several other articles with unsatisfactory results.

Anyone else have any suggestions?

[a_p_ · Accepted solution]

That's by design with several EMC storage systems. Unless you present a LUN with a host ID 0, you will see the "Management LUN" (which is not a disk device) as LUN 0 on the host.

André

[GAPCABIV]

Thanks Andre.  I have always admired your posts here.  Does it make any sense to attempt to mask the LUN using the steps in this KB > VMware KB: Masking a LUN from ESX and ESXi using the MASK_PATH plug-in < or, in your opinion, is it best to just leave it be?

[a_p_]

Rather than making things complex with e.g. LUN masking, I usually present a LUN with ID 0 to the hosts, even if it is just a small one for storing e.g. .iso images, scratch data, ... This way the LUN is in use and nobody will be confused with what you currently see.

André