Problem with a ghost LUN in ISCSI which is driving me crazy

[Albatros66]

Hello everyone !

I really need your help with a problem that I haven't been able to solve for several days...

If anyone has an idea and finds a solution, I'm willing to pay for a beer anywhere in the world!

The context: I have a small infrastructure with 2 esxi V7u3, a Synology 1215 NAS, a Cisco 3850 in the middle. The connection is made via ISCSI fiber.

After a restart, impossible to find my datastore even though the devices are clearly seen in the ESXi.

I can clearly see my LUNs on the NAS side and on the ESX side with an esxcli iscsi adapter target lun list

If I create a new LUN on the NAS, I see this devices listed ESX and I can create my datastore, so the path seems OK.

I also cloned my LUN to storage to try to reconnect but it doesn't work.

I feel like I have a lock or hang on my LUN.

My 2 LUNS are:
naa.60014053b90ac9ddd622d463edae0dd1 (heartbeat)
naa.6001405aa947686d3ca2d481ed8c85d0 (my vms)

My VMKernel.log is not easy to read, but here is an excerpt in attachment (i can share other log if needed) 

Thank you veryyyyyyy  much in advance if you can help me

[a_p_ · Accepted solution]

According to the log file, the LUNs have been detected as "Spanshot LUNs".

2023-09-25T16:16:16.289Z cpu31:1718527)LVM: 11764: Device naa.60014053b90ac9ddd622d463edae0dd1:1 detected to be a snapshot:
2023-09-25T16:16:16.289Z cpu31:1718527)LVM: 11770: queried disk ID: <type 2, len 22, lun 1, devType 0, scsi 0, h(id) 2259623917972509401>
2023-09-25T16:16:16.289Z cpu31:1718527)LVM: 11777: on-disk disk ID: <type 2, len 22, lun 2, devType 0, scsi 0, h(id) 10461069855284865661>
2023-09-25T16:16:16.289Z cpu31:1718527)LVM: 11794: If Device naa.60014053b90ac9ddd622d463edae0dd1:1 is OsData:0 status:Success
2023-09-25T16:16:16.290Z cpu31:1718527)LVM: 11764: Device naa.6001405aa947686d3ca2d481ed8c85d0:1 detected to be a snapshot:
2023-09-25T16:16:16.290Z cpu31:1718527)LVM: 11770: queried disk ID: <type 2, len 22, lun 2, devType 0, scsi 0, h(id) 9153303937128236436>
2023-09-25T16:16:16.290Z cpu31:1718527)LVM: 11777: on-disk disk ID: <type 2, len 22, lun 1, devType 0, scsi 0, h(id) 13470468055773502861>
2023-09-25T16:16:16.290Z cpu31:1718527)LVM: 11794: If Device naa.6001405aa947686d3ca2d481ed8c85d0:1 is OsData:0 status:Success

Please follow the steps (I recommend the command line approach) in https://kb.vmware.com/s/article/1011387 to verify this, and to possible mount the LUNs again.

André

[battybishop · Accepted solution]

SSH onto the host and run this command

"esxcli storage vmfs snapshot list"

This will return the Volume name and a VMFS UUID 

Then using the UUID you can mount it using this command

esxcfg-volume -M "vmfs uuid"

Your volumes should then be available

[Albatros66]

Thank you a_p_ and battybishop !!!

It's work ! my datastore is working now

Send me a private message if i can send you some $ for drink a thanks beer !

Vmware is amazing product, thank you to the community !

[battybishop]

great to hear your back up and running, no need for beer I'm just pleased I can help.