"Orphan" datastore

[melevy]

I have a datastore that I cannot remove or use. It's essentially "stuck." It shows up in the VSphere Client (connected to vCenter) in the datastore view, but it does not show up under configuration - storage of any host.

If I select a host and go to configuration - storage and try to add a datastore, it will "find" the identifier but will throw an error.

The vmkernel log has this to say:

Oct 11 00:41:04 VMVirt02 vmkernel: 0:13:25:50.628 cpu6:4110)LVM: 6740: Initialized naa.600805f3000c1c40a0be7d3eb04a000b:1, devID 4e93d6f0-28f4347d-5564-0015170bbcd9
Oct 11 00:41:04 VMVirt02 vmkernel: 0:13:25:50.638 cpu4:4504)NMP: nmp_CompleteCommandForPath: Command 0x16 (0x4102bfa80840) to NMP device "naa.600805f3000c1c40a0be7d3eb04a000b" failed on physical path "vmhba1:C0:T1:L1" H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x55 0x0.
Oct 11 00:41:04 VMVirt02 vmkernel: 0:13:25:50.638 cpu4:4504)ScsiDeviceIO: 1685: Command 0x16 to device "naa.600805f3000c1c40a0be7d3eb04a000b" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x55 0x0.
Oct 11 00:41:04 VMVirt02 vmkernel: 0:13:25:50.638 cpu6:4110)LVM: 6880: Error "Not supported" creating volume 4e93d6f0-c51fc343-2028-0015170bbcd9 on device naa.600805f3000c1c40a0be7d3eb04a000b:1.

It seems to be locked somehow, but I've rebooted both hosts, so I'm at a loss. I've googled high & low and I can't find anything similiar to this.

Any pointers are greatly appreciated!

-Mark

[Virtualinfra · Accepted solution]

1. Is this datastore is from SAN, If so did you request the storage team to remove visibility for this Lun to the server.

2.  Did you try rescanning on the HBA and Datastores- If not please try rescanning and try to add the LUN

3. Also the error say failed on physical path - so please check with storage team.

Refer the below KB:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=102903...

[melevy]

We're a small operation, so I'm also the "storage team." It is an FC SAN. I'll see if removing the lun and recreating it helps, I didn't think of that. I have rescanned, many times, both hosts have been rebooted. I did see the article you posted,

[Virtualinfra]

Thats great and you play both the roles will help a lot with VMware virtualization......

[melevy]

Well, some progress. I found this article that got rid of the "orphan":

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=102666...

I think that there is a problem with the SAN which is why I can't create a new datastore on that LUN. I see the following in the vmkernel log at the time that I tried to create a datastore:

Oct 11  07:55:31 VMVirt02 vmkernel: 0:20:40:16.575 cpu4:4111)WARNING:  Partition: 801: Partition table read from device  naa.600805f3000c1c40aa8439c64d65000c failed: I/O error

Other datastores on that SAN are working fine and there are LUNs for non-VMware hosts (yes, they are masked) that are working. Possibly a bad drive, but there's nothing in the SAN logs. I'm going to try resetting it tonight (uptime is 647 days) and see if that helps.

[melevy]

The problem was the SAN. One of the controllers failed in such a way as to not trigger failover. I reset both controllers and now the ESX hosts don't see a path through that controller. I was able to create a datastore on that LUN that was giving me the errors previously. Time to call service...

Thanks for your help!