0 Replies Latest reply on Jun 15, 2015 11:28 AM by ben_was_here

    Recover vmdk from failed datastore with extents

    ben_was_here Lurker

      Hi All,

       

      New account on the forum, but have been a lurker for some time. Here's what has brought me to posting today:

       

      • Had a failure of a datastore on ESXi 3.5 that was made up of 6 x HP 73GB drives.
      • ESXi would not boot successfully.
      • Through examination of the drives at a low level using a linux boot disk, it was only a few sectors on the first extent and i was able to use ddrescue to make a clone of this first drive, replace it in the enclosure and ESXi would successfully boot
      • However, after booting, no VMs on that datastore were active, and the datastore itself would not show in ESXi
      • Looking at logs, it appeared the datastore GUID changed with the clone of the drive.
      • Using the rescan (and trying with LVM advanced option flags changed) ESXi would not find the file system
      • Research here led me to advise from continuum and others to use vmfs-tools and the vmfs-fuse to browse the datastore in a linux environment.
      • There were a few hiccups figuring out how to do this with extents, but i have this working now

       

      I think the few sectors that had failed on the first extent have messed up the file allocation table (i could be using wrong terminology), because when i access the mount folder that i have connected the LVM extent drives to, and do an ls-l, heres what happens (note the italicized lines):

      /mnt/vmfs2# ls -l

      ls: cannot access GCSTS01: No such file or directory

      ls: cannot access GCSSQL00: No such file or directory

      total 384

      d????????? ? ?    ?       ?            ? GCSSQL00

      drwxr-xr-x 2 root root  560 Oct 19  2012 GCSTS00

      drwxr-xr-x 2 root root 3220 Mar  9 05:56 GCSTS00_1

      d????????? ? ?    ?       ?            ? GCSTS01

      drwxr-xr-x 2 root root 2520 Mar  9 05:59 GCSTS02

      drwxr-xr-x 2 root root 3080 Mar  9 05:36 GCSWEB00

      drwxr-xr-x 2 root root 1400 Jun 19  2013 New Virtual Machine

      drwxr-xr-x 2 root root 1400 Jun 21  2013 New Virtual Machine 1

       

      Just as an example to extend this a little further, if i cd into GCSTS02, and do another ls -l, i get this (again, note the italicized lines):

      /mnt/vmfs2/GCSTS02# ls -l

      ls: cannot access GCSTS02.vmxf: No such file or directory

      ls: cannot access GCSTS02.vmsd: No such file or directory

      ls: cannot access GCSTS02-flat.vmdk: No such file or directory

      ls: cannot access GCSTS02.vmdk: No such file or directory

      ls: cannot access GCSTS02_1-flat.vmdk: No such file or directory

      ls: cannot access GCSTS00.nvram: No such file or directory

      ls: cannot access GCSTS02-6d4aebf2.vswp: No such file or directory

      total 576

      -????????? ? ?    ?        ?            ? GCSTS00.nvram

      -????????? ? ?    ?        ?            ? GCSTS02_1-flat.vmdk

      -rw------- 1 root root   587 Mar  9 06:01 GCSTS02_1.vmdk

      -????????? ? ?    ?        ?            ? GCSTS02-6d4aebf2.vswp

      -????????? ? ?    ?        ?            ? GCSTS02-flat.vmdk

      -????????? ? ?    ?        ?            ? GCSTS02.vmdk

      -????????? ? ?    ?        ?            ? GCSTS02.vmsd

      -rwxr-xr-x 1 root root  2390 Mar  9 05:59 GCSTS02.vmx

      -????????? ? ?    ?        ?            ? GCSTS02.vmxf

      -rw-r--r-- 1 root root 28553 Mar  6 08:58 vmware-0.log

      -rw-r--r-- 1 root root 27888 Feb 25 11:11 vmware-1.log

      -rw-r--r-- 1 root root 26905 Feb  6 06:35 vmware-2.log

      -rw-r--r-- 1 root root 27885 Nov 12  2014 vmware-3.log

      -rw-r--r-- 1 root root 26903 Nov 10  2014 vmware-4.log

      -rw-r--r-- 1 root root 26903 Nov  2  2014 vmware-5.log

      -rw-r--r-- 1 root root 28141 Apr 10 08:03 vmware.log

       

      I know beyond any doubt these files are all in the drives 2 - 6 in the LVM extents, and are completely undamaged. What should i do next to try to properly extract them?

       

      Thanks in advance for any help.

       

      Ben