I'm experiencing a weird problem today, the VMs on one of the datastore -which is backed by a LUN on NETAPP FS8040- are neither able to be vmoted, storage vmoted, or even powering them on (if I power off any of them). And when I try to create a folder (for testing) on that datastore via SSH, the operation fails with the error "input/output locked". VMware support team have investigated the issue, but nothing is locking the VM files. Only one host has succeeded in creating and deleting a test folder on that datastore, but it's holding one of the victim VMs and I won't be able to reboot it (assuming that it's locking that datastore).
I'm using vSphere 6.5, and VMFS6
Note: other datastores no the same SAN are healthy.
What do you think guys?
Without further detail one can only guess.
How are the datastores connected to the hosts? iSCSI, FC, or NFS?
Are the datastores only presented to the ESXi hosts, or also to backup servers?
In case they are presented to backup servers, check whether there was an issue with backups.
I'm sorry for the delayed response, I've been busy fixing this, however, we've failed to fix the datastore issue. Both NETAPP and VMware support couldn't fix it, and thanks to our backup solution. So, we've recovered the VMs to another datastore, and recreated the LUN with issues. I guess that the VMFS itself got corrupted.