VMware Cloud Community
DickyDck
Contributor
Contributor

iSCSI object in vCenter remains even after SAN is no longer connected to the network

Hello!

I am running vCenter 6.5 with 2 hosts. I was running iSCSI Software adapters to connect to 2 different storage units, with 1 Datastore on each. 1 SAN died completely. No big deal as I have plenty of storage on my other SAN. But I cannot get rid of the object in vCenter. I have logged in directly to both Hosts and have been able to remove the SAN from them no issues, but it still shows up in vCenter. No option to delete, and even appears in the tree as active???? Tried the rename then delete trick, tried maintenance mode but that never goes anywhere. I "could" leave it there since it isn't harming anything but if it isn't even connected to the network shouldn't vCenter at least see it as down? I would really like to remove it, there has to be a reference to it somewhere in the file system that I can just SSH into and remove the entry from a configuration file somewhere I would think?

Reply
0 Kudos
11 Replies
Lalegre
Virtuoso
Virtuoso

Hey @DickyDck,

Try to run a rescan of the VMFS and Storage Adapters, if the LUN is no more available the host will not populate that information as a result.

If that does not work maybe you can try detaching the Datastore and then deleting it. Also give it a try and run services.sh restart command as it will maybe recognize that is no more available.

Reply
0 Kudos
DickyDck
Contributor
Contributor

Thanks, I have tried rescanning the adapter already.

 

All the normal options for graceful removal are greyed out, I have rebooted every host and vcenter itself. Nothing has worked so far.

 

Other parts of vCenter are aware that the Datastore is no longer accessible, since it is no longer accessible there should be a way to force remove it from inventory?

Reply
0 Kudos
Lalegre
Virtuoso
Virtuoso

What do you mean by other parts of vCenter? Also what if you remove the datastore from every host individually and then login back into vCenter?

Reply
0 Kudos
DickyDck
Contributor
Contributor

So generally if a Datastore is inaccessible in any way, it may be greyed out, or a red ! mark, or possibly a "?" on the icon. 

 

Attached Picture DS.PNG shows 3 Datastores, the 3rd literally is in my trashcan, it is not attached to any cables.

Attached Picture "Normal.png" shows all 3 Datastores, and as you can see the status is "Normal" but how..... the storage unit is outside in my trashcan! lol

Attached picture "options.png" shows I have no available options to do a graceful removal of the device.

I can only attach one pic per post, so more posts with the screenshots

Reply
0 Kudos
DickyDck
Contributor
Contributor

 
Reply
0 Kudos
DickyDck
Contributor
Contributor

 
Reply
0 Kudos
a_p_
Leadership
Leadership

Does one (or more) of your VMs have active snapshots which were created while the VM somehow pointed to that datastore? Even a not attached ISO image in a VM's configuration may cause this.

After deleting the snapshot(s), or removing existing references, the "ghost" datastore should disappear.

André

Reply
0 Kudos
DickyDck
Contributor
Contributor

Confirmed no snapshots, never made any in this environment.

There were ISO images on that datastore, but presently since the SAN is a pile of scrap metal there are earthworms on it.  😛 

I've verified manually on each VM if there were any ISOs or anything pointing to that old datastore, and to the best that I can see none.

Even stranger, if I try to add an ISO to a VM, it only shows the 2 active Datastores, and doesn't show the rogue one. it only shows up in vCenter, but individually on the 2 hosts it is nowhere to be found.

 

This is really strange wouldn't you agree? Leave it to me to have the strange snowflake environment! 

Reply
0 Kudos
a_p_
Leadership
Leadership

Strange, because what you explain points to a stale reference.
So no snapshots, no ISOs, or other images, neither in VMs nor in templates?

Depending on the size of your environment it may be worth verifying this again using RVTools.

André

Reply
0 Kudos
DickyDck
Contributor
Contributor

Good plan! RVTools! let me see what that shows me

Reply
0 Kudos
DickyDck
Contributor
Contributor

Downloaded RVTools, and it definitely still sees the rogue Datastore, really confused now..

Reply
0 Kudos