VMware Cloud Community
kheath74
Contributor
Contributor

SAN Crash and cannot move or copy vmdk-flat file

We had a SAN controller fail today and the secondary controller then failed so all our VMs lost access to their datastores, we got the one controller back online when we reseated it but one of our VMs was frozen and we had to power it off.


When we tried to restart it we got the error "Invalid Argument 0" and now we cannot copy or move this vmdk-flat file or do anything with it.  I have checked the vmdk file and the descriptor reference is all correct and I rebuilt the vmdk file and it did not fix the issue.

I have a feeling that this file is corrupt for good as when I try to fix it on CLI I get this error

DiskLib_Check() failed for source disk Invalid argument (1441801).
If a file is corrupt would it be correct that I could not move or copy it, in fact all I could do would be to delete it or is there a way to get this file off the datastore to another store?
Thanks Kyle
Reply
0 Kudos
8 Replies
continuum
Immortal
Immortal

can you get SAN access with a Linux LiveCD ?
If yes - try to copy the flat.vmdk with dd_rescue after mounting the VMFS-volume with Linux vmfs-tools


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
kheath74
Contributor
Contributor

Thanks, I will have to ask how do we map linux to the datastore so it can see the vmdk files?

Reply
0 Kudos
continuum
Immortal
Immortal

you need to map the whole VMFS volume to the Linux LiveCD
or - create a full dd-image of the problem LUN - that could be done with ESXi itself


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
kheath74
Contributor
Contributor

So if I boot to Ubuntu with the ESXi Server then iscsi the SAN, I can mount the datastore with the vmfstools and copy the vmdk-flat file off?

Reply
0 Kudos
continuum
Immortal
Immortal

you can not simply install the vmfstools in a UbuntuLiveCD - you have to compile it - use the ISO I made for such tasks

http://sanbarrow.com/mcs-esxi5-recovery-X-001.iso

that CD may not work with broadcom network device 5708

whats the size of the datastore that is affected ?


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
kheath74
Contributor
Contributor

OK thanks, I am downloading the ISO now.  The datastore is 1.2TB in size, I have migrated every other VM off the datastore so we just have the problem VM left.  What happened was that our controller B failed in our HP P2000 iscsi array this morning and caused controller A to crash too.  The result was when we removed controller B and reattached A we got all the VMs back bar this one.  It has been stuck and we cannot copy, move or seem to do anything to it except rename it.

Reply
0 Kudos
continuum
Immortal
Immortal

lets continue via skype


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
akha666
Contributor
Contributor

Did you fix this issue ?? I've the same in  my Esxi 5.5 and openfiler storage on raid 10

Reply
0 Kudos