topic Re: One of the extents LUN was missing from the datastore and cause some VMs cannot power-on in vSphere Hypervisor Discussions

One of the extents LUN was missing from the datastore and cause some VMs cannot power-on

PHONGSATHORN — Tue, 23 May 2017 04:07:44 GMT

Hi,

I have a problem regarding datastore extents issue.

Normally, I have one datastore provisioned from the local disk (300GB x 4 with RAID5), ESXi was installed on this datastore.

It's working fine for a while and last week I decided to increase the datastore capacity using "increase" button on the datastore, I added another iSCSI LUN into the datastore so the total capacity has increased, now the datastore combined with two storages (Local Disk + iSCSI LUN).

Unfortunately, there was a power outage yesterday which cause SAN storage down for a few minutes. I found the error in ESXi that our VM has a problem with VMDK file. In the datastore properties, iSCSI LUN was missing from the datastore, it used to has two storages (Local + iSCSI) but now there is only local disk as below.

Total Capacity = 1.79TB but there is only local disk which is 833.22GB

If I try to increase the capacity again, it shows the storage which is missing from the Extents datastore.

I'm not sure that if I go ahead next -> next -> finish. All my data will lose?

I use the command vmkfstools -P /vmfs/volumes/xxxx , it shows that device naa.6exxxx might be offline

How can I bring it online or manually put it back to the datastore?

Here is the VMkernel.log

2017-05-22T12:21:51.423Z cpu3:2839)WARNING: LVM: 13178: An attached device went offline. naa.6e843b63fd07072d0790d467cdac56d7:2 file system [datastore1, 50d07035-651a6a7f-10a2-2c768a52e739]

2017-05-22T12:21:53.106Z cpu5:2883)Config: 346: "SIOControlFlag1" = 0, Old Value: 0, (Status: 0x0)

2017-05-22T12:21:54.618Z cpu0:2839)Config: 346: "SIOControlFlag2" = 0, Old Value: 0, (Status: 0x0)

2017-05-22T12:21:58.589Z cpu2:2839)Config: 346: "VMOverheadGrowthLimit" = -1, Old Value: -1, (Status: 0x0)

2017-05-22T12:21:59.006Z cpu1:2839)WARNING: LVM: 13178: An attached device went offline. naa.6e843b63fd07072d0790d467cdac56d7:2 file system [datastore1, 50d07035-651a6a7f-10a2-2c768a52e739]

2017-05-22T12:22:02.790Z cpu11:3158)Loading module hpilo ...

2017-05-22T12:22:02.791Z cpu11:3158)Elf: 1862: module hpilo has license GPLv2

2017-05-22T12:22:02.791Z cpu11:3158)module heap: Initial heap size: 102400, max heap size: 5562368

2017-05-22T12:22:02.791Z cpu11:3158)vmklnx_module_mempool_init: Mempool max 5562368 being used for module: 58

Does somebody used to has this kind of problem?

Re: One of the extents LUN was missing from the datastore and cause some VMs cannot power-on

Jitu211003 — Tue, 23 May 2017 06:55:13 GMT

Hi,

I came to the same situation in my environment recently where ESXi attached HP MSA storage restarted due to sudden power loss.

2 out of 4 LUNs were disappeared from the ESXi host and becomes visible in available list.

Run the command

"esxcli storage filesystem list" if it show an unknown vmfs version for the LUNs which are not visible on the esxi host.

Below are the steps to resolve this issue.

esxcli storage filesystem unmount -u "UUID" (where write the impacted/invisible LUN uuid).

Now you will be able to see this in inactive condition in the esxi host.

but still you are not ok, you can not mount this it will give you an error.

Get a graceful restart of the storage once.

Then, once storage comes online, try to mount it from Vcenter and if still fails, try to unmount from ESXi directly by logging on vi client then try to mount it from vcenter.

It resolved mine issue, hope it will resolve yours too.

If find this helpful or solves your query, do not forget to mark it.

Thanks,

Re: One of the extents LUN was missing from the datastore and cause some VMs cannot power-on

PHONGSATHORN — Tue, 23 May 2017 07:08:41 GMT

Thanks for your recommendation.

Here is the result of esxcli storage filesystem list

datastore1 is the datastore that has an issue.

I'm not what I have to do next?

Re: One of the extents LUN was missing from the datastore and cause some VMs cannot power-on

Jitu211003 — Wed, 24 May 2017 04:41:50 GMT

Hi,

As per your scenario, as i said, i came across same situation where mine storage went down suddenly due to power outage.

Now the only option you have to take the storage down gracefully once and power on again. I am 100% sure, once storage will come up, your esxi will start reflecting the disappeared lun.

It is known with HP MSA storage as far as I observed. and if you go with storage vendor, they gonna suggest you to upgrade firmware first.

Take the storage down gracefully in off business hours.

Re: One of the extents LUN was missing from the datastore and cause some VMs cannot power-on

Dee006 — Wed, 24 May 2017 10:21:02 GMT

Hi Jitu,

From screenshot 1,it says HP SAS disk and Screenshot 2 shows as IQN number.Are you they from same storage system?

Re: One of the extents LUN was missing from the datastore and cause some VMs cannot power-on

deepaksel — Wed, 24 May 2017 16:06:12 GMT

Try to restart Esxi Host it may fix your issue.