VMware Cloud Community
ben_was_here
Contributor
Contributor

Recover vmdk from failed datastore with extents

Hi All,

New account on the forum, but have been a lurker for some time. Here's what has brought me to posting today:

  • Had a failure of a datastore on ESXi 3.5 that was made up of 6 x HP 73GB drives.
  • ESXi would not boot successfully.
  • Through examination of the drives at a low level using a linux boot disk, it was only a few sectors on the first extent and i was able to use ddrescue to make a clone of this first drive, replace it in the enclosure and ESXi would successfully boot
  • However, after booting, no VMs on that datastore were active, and the datastore itself would not show in ESXi
  • Looking at logs, it appeared the datastore GUID changed with the clone of the drive.
  • Using the rescan (and trying with LVM advanced option flags changed) ESXi would not find the file system
  • Research here led me to advise from continuum and others to use vmfs-tools and the vmfs-fuse to browse the datastore in a linux environment.
  • There were a few hiccups figuring out how to do this with extents, but i have this working now

I think the few sectors that had failed on the first extent have messed up the file allocation table (i could be using wrong terminology), because when i access the mount folder that i have connected the LVM extent drives to, and do an ls-l, heres what happens (note the italicized lines):

/mnt/vmfs2# ls -l

ls: cannot access GCSTS01: No such file or directory

ls: cannot access GCSSQL00: No such file or directory

total 384

d????????? ? ?    ?       ?            ? GCSSQL00

drwxr-xr-x 2 root root  560 Oct 19  2012 GCSTS00

drwxr-xr-x 2 root root 3220 Mar  9 05:56 GCSTS00_1

d????????? ? ?    ?       ?            ? GCSTS01

drwxr-xr-x 2 root root 2520 Mar  9 05:59 GCSTS02

drwxr-xr-x 2 root root 3080 Mar  9 05:36 GCSWEB00

drwxr-xr-x 2 root root 1400 Jun 19  2013 New Virtual Machine

drwxr-xr-x 2 root root 1400 Jun 21  2013 New Virtual Machine 1

Just as an example to extend this a little further, if i cd into GCSTS02, and do another ls -l, i get this (again, note the italicized lines):

/mnt/vmfs2/GCSTS02# ls -l

ls: cannot access GCSTS02.vmxf: No such file or directory

ls: cannot access GCSTS02.vmsd: No such file or directory

ls: cannot access GCSTS02-flat.vmdk: No such file or directory

ls: cannot access GCSTS02.vmdk: No such file or directory

ls: cannot access GCSTS02_1-flat.vmdk: No such file or directory

ls: cannot access GCSTS00.nvram: No such file or directory

ls: cannot access GCSTS02-6d4aebf2.vswp: No such file or directory

total 576

-????????? ? ?    ?        ?            ? GCSTS00.nvram

-????????? ? ?    ?        ?            ? GCSTS02_1-flat.vmdk

-rw------- 1 root root   587 Mar  9 06:01 GCSTS02_1.vmdk

-????????? ? ?    ?        ?            ? GCSTS02-6d4aebf2.vswp

-????????? ? ?    ?        ?            ? GCSTS02-flat.vmdk

-????????? ? ?    ?        ?            ? GCSTS02.vmdk

-????????? ? ?    ?        ?            ? GCSTS02.vmsd

-rwxr-xr-x 1 root root  2390 Mar  9 05:59 GCSTS02.vmx

-????????? ? ?    ?        ?            ? GCSTS02.vmxf

-rw-r--r-- 1 root root 28553 Mar  6 08:58 vmware-0.log

-rw-r--r-- 1 root root 27888 Feb 25 11:11 vmware-1.log

-rw-r--r-- 1 root root 26905 Feb  6 06:35 vmware-2.log

-rw-r--r-- 1 root root 27885 Nov 12  2014 vmware-3.log

-rw-r--r-- 1 root root 26903 Nov 10  2014 vmware-4.log

-rw-r--r-- 1 root root 26903 Nov  2  2014 vmware-5.log

-rw-r--r-- 1 root root 28141 Apr 10 08:03 vmware.log

I know beyond any doubt these files are all in the drives 2 - 6 in the LVM extents, and are completely undamaged. What should i do next to try to properly extract them?

Thanks in advance for any help.

Ben

0 Kudos
0 Replies