Datastore connection lost

[WilliamFor]

I am using ESXi 6.0. I have IO problem in datastores and lost connectivity occurs. Very short-term access to the datastore. Therefore, virtual machines are locked or hanged. How can I solve this problem?

[daphnissov]

Open a support case with GSS.

[tayfundeger]

I don't know exactly which ESXi build you are using. However, in ESXi 6.0, I had problems as I mentioned. What is the model of storage you have used? Such problems can be caused by firmware and driver. Have you done ESXi update / upgrade recently?

[continuum]

One of the recent cases I looked into had I/O errors and GSS could not help.

They later asked the customer how I  did it ...

If the datastore disappears after a few minutes and the data is valuable there is only one way to proceed:

unmount that datastore and clone it to a new disk / lun with Linux ddrescue.

Yes - with Linux ddrescue

Option 1:

power off the ESXi and boot into Linux LiveCD

clone problem device to new disk or file with ddrescue

ddrescue /dev/sdx /dev/sdy ddrescue.log

Option 2 - if ESXi can not be powered off or if the disk/LUN can not be detected by Linux

create new VM with http://sanbarrow.com/livecds/moa64-nogui/MOA64-nogui-incl-src-111014-efi.iso​

assign a small vmdk - 1gb is enough

assign 4gb ram, 2vcpus

assign a network that can reach esxi

boot into the VM - root password is sanbarrow

configure network, format disk so you can use it for the ddrescue log

then you can access the problem disk by connecting to the esxi via sshfs

...

Ulli

[WilliamFor]

I am using the ESXi 6.0 2494585. ESXi upgrade process was done 1 month ago. There is no active interruption at the moment, but I'm having this problem when the backup is started. IO Latency problems are followed by lost connectivity warnings.

[continuum]

Ok - check firmware  / driver issues first.

[tayfundeger · Accepted solution]

Did you upgrade with custom iso? If so, you are unlikely to come across a driver bug. But you are using ESXi version 6.0 GA. Driver may be in trouble.

Could you please try the steps in the following KB? ESXi 6.0 Update 2 and earlier had serious problems with ATS.

https://kb.vmware.com/s/article/2113956

If your problem doesn't improve, we'il check it again.

[a_p_]

Are you using local, or shared storage?

In case of local storage, do you use a RAID controller (vendor/model), and does it have write-back cache (BBU/BBWC/FBWC)?

In case of shared storage, which storage system (vendor/model), and protocol (iSCSI/FC/SAS) do you use?

André

[WilliamFor]

I am using Sun ZFS storage. Connected to ESXi hosts via fiber cable.

[Maio312]

review the IOPS utilization from the storage and i would suggest to contact VMware support

[WilliamFor]

Thank you very much, my problem is solved. There are no problems right now.

[tayfundeger]

I'm glad the problem was solved. Be sure to check your drivers and firmware as soon as possible.