ESXi

 View Only
Expand all | Collapse all

Repair / Recover VMFS Volume

jagdish_rana

jagdish_ranaAug 29, 2015 05:57 AM

  • 1.  Repair / Recover VMFS Volume

    Posted Jul 31, 2012 05:01 AM

    I have a situation where someone had an older vSphere 4.x cluster. The VMs were hosted off of a iSCSI SAN controller to the nodes in the cluster.  The volume was 260GB originally when I made it a few years back.   Subsequently they have added enough VMs that they needed to expand.  They expanded the LUN capacity to 3TB, but was never able to expand the partition.   With VMFS3 this is an issue.

    Seems if you epand the LUN, EVEN IF you NEVER expand the partition, your systems, if they unmount the volume can never mount it again.

    This apparently went on for some time where over time more systems would not "access" the LUN.... and after a power outage, no system could then access the dozens of VMs.

    Now I need to find a way to mount that 3TB iSCSI LUN, even if read only, long enough to mount the 260 of VMs off of it to NFS share i have ready.

    Goal: map iSCSI LUN to Windows / Linux system.  Mount the volume RO.  Copy data to NFS export. Destroy and rebuild the data on new iSCI VMFS5 lun.

    I can mount the LUN to CENTOS system and parted shows the 3TB LUN but primary partition only 260GB.  So I see the data as expected.   Now I tried to compile 'vmfs-tools-0.2.5"  but get compiler errors.  Nothing of any help or note. I tried ubuntu and CENTOS48 also and errors not much help.

    Is their another, more simplistic way to get this accomplished?

    Thanks,



  • 2.  RE: Repair / Recover VMFS Volume

    Posted Jul 31, 2012 05:25 AM

    Was this by chance a LUN that was expanded to over the 2Tb limit on an ESXi 4.x host?



  • 3.  RE: Repair / Recover VMFS Volume

    Posted Jul 31, 2012 08:14 AM

    Hi there
    I made a LiveCD with vmfs-tools 0.2.5

    You can download it from here:

    http://sanbarrow.com/mcs-esxi5-recovery-X-001.iso

    Open-iscsi is installed as well - so you should be able to use it without needing to add anything after boot.

    Let me hear about your results please

    Ulli



  • 4.  RE: Repair / Recover VMFS Volume

    Posted Jul 31, 2012 12:23 PM

    I was up late last night and fubmled my way through getting the data off the volume.  Below is my notes.

    I am not an ubuntu guru.... so my comments may reflect ignorance vs actual issues and has been a few years sense I tried that distro... but my fond memories of that distro no longer exist.  The below notes are based on the mount of a local ext3, because my goal of export via NFS was not possible due to various apt-get faults and issues....

    I will try the supplied ISO boot image that previous poster provided once it finishes download such that I can let them know how that works.  Goal was simple, mount the vmfs volume,  export via nfs.. copy via NFS mount to new iSCSI volume.

    *******************************

    login as: ibm

    ibm@172.20.13.28's password:

    Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic x86_64)

    * Documentation:  https://help.ubuntu.com/

      System information as of Tue Jul 31 01:09:23 EDT 2012

      System load:  0.16              Processes:           77

      Usage of /:   8.5% of 14.69GB   Users logged in:     0

      Memory usage: 3%                IP address for eth0: 172.20.13.28

      Swap usage:   0%

      Graph this data and manage this system at https://landscape.canonical.com/

    Last login: Mon Jul 30 23:39:47 2012 from 192.168.59.31

    su ibm@ubuntu12:~$ su -

    Password:

    su: Authentication failure

    ibm@ubuntu12:~$ su -

    Password:

    su: Authentication failure

    ibm@ubuntu12:~$ vmfs-fuse

    VMFS: Unable to read FS information

    Unable to open volume.

    ibm@ubuntu12:~$

    ibm@ubuntu12:~$

    ibm@ubuntu12:~$ mount

    /dev/mapper/ubuntu12-root on / type ext4 (rw,errors=remount-ro)

    proc on /proc type proc (rw,noexec,nosuid,nodev)

    sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)

    none on /sys/fs/fuse/connections type fusectl (rw)

    none on /sys/kernel/debug type debugfs (rw)

    none on /sys/kernel/security type securityfs (rw)

    udev on /dev type devtmpfs (rw,mode=0755)

    devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)

    tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)

    none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)

    none on /run/shm type tmpfs (rw,nosuid,nodev)

    /dev/sda1 on /boot type ext2 (rw)

    ibm@ubuntu12:~$ sudo iscsiadm  -m discovery -t st -p 172.20.31.40

    [sudo] password for ibm:

    172.20.31.40:3260,1 iqn.2003-10.com.lefthandnetworks:atllhn1:2686:bl490-ss-2

    ibm@ubuntu12:~$ sudo iscsiadm --m node --targetname iqn.2003-10.com.lefthandnetworks:atllhn1:2686:bl490-ss-2 --portal 172.20.31.40 --login

    Logging in to [iface: default, target: iqn.2003-10.com.lefthandnetworks:atllhn1:2686:bl490-ss-2, portal: 172.20.31.40,3260]

    Login to [iface: default, target: iqn.2003-10.com.lefthandnetworks:atllhn1:2686:bl490-ss-2, portal: 172.20.31.40,3260]: successful

    ibm@ubuntu12:~$ sudo iscsiadm -m session

    tcp: [1] 172.20.31.40:3260,1 iqn.2003-10.com.lefthandnetworks:atllhn1:2686:bl490-ss-2

    ibm@ubuntu12:~$ sudo  fdisk -l

    Disk /dev/sda: 17.2 GB, 17179869184 bytes

    255 heads, 63 sectors/track, 2088 cylinders, total 33554432 sectors

    Units = sectors of 1 * 512 = 512 bytes

    Sector size (logical/physical): 512 bytes / 512 bytes

    I/O size (minimum/optimal): 512 bytes / 512 bytes

    Disk identifier: 0x00062394

       Device Boot      Start         End      Blocks   Id  System

    /dev/sda1   *        2048      499711      248832   83  Linux

    /dev/sda2          501758    33552383    16525313    5  Extended

    /dev/sda5          501760    33552383    16525312   8e  Linux LVM

    Disk /dev/mapper/ubuntu12-root: 15.8 GB, 15825108992 bytes

    255 heads, 63 sectors/track, 1923 cylinders, total 30908416 sectors

    Units = sectors of 1 * 512 = 512 bytes

    Sector size (logical/physical): 512 bytes / 512 bytes

    I/O size (minimum/optimal): 512 bytes / 512 bytes

    Disk identifier: 0x00000000

    Disk /dev/mapper/ubuntu12-root doesn't contain a valid partition table

    Disk /dev/mapper/ubuntu12-swap_1: 1069 MB, 1069547520 bytes

    255 heads, 63 sectors/track, 130 cylinders, total 2088960 sectors

    Units = sectors of 1 * 512 = 512 bytes

    Sector size (logical/physical): 512 bytes / 512 bytes

    I/O size (minimum/optimal): 512 bytes / 512 bytes

    Disk identifier: 0x00000000

    Disk /dev/mapper/ubuntu12-swap_1 doesn't contain a valid partition table

    Disk /dev/sdb: 279.2 GB, 279172874240 bytes

    255 heads, 63 sectors/track, 33940 cylinders, total 545259520 sectors

    Units = sectors of 1 * 512 = 512 bytes

    Sector size (logical/physical): 512 bytes / 512 bytes

    I/O size (minimum/optimal): 512 bytes / 512 bytes

    Disk identifier: 0x000927cf

       Device Boot      Start         End      Blocks   Id  System

    /dev/sdb1             128   524281274   262140573+  fb  VMware VMFS

    ibm@ubuntu12:~$ sudo vmfs-fuse /dev/sdb /media/cdrom/

    VMFS VolInfo: invalid magic number 0x00000000

    VMFS: Unable to read volume information

    Trying to find partitions

    ibm@ubuntu12:~$

    ibm@ubuntu12:~$ sudo parted /dev/sdb

    GNU Parted 2.3

    Using /dev/sdb

    Welcome to GNU Parted! Type 'help' to view a list of commands.

    (parted) p

    Model: LEFTHAND iSCSIDisk (scsi)

    Disk /dev/sdb: 279GB

    Sector size (logical/physical): 512B/512B

    Partition Table: msdos

    Number  Start   End    Size   Type     File system  Flags

    1      65.5kB  268GB  268GB  primary

    ibm@ubuntu12:~$ sudo vmfs-fuse /dev/sdb1 /media/vmfs/

    ibm@ubuntu12:~$ sudo ls /media/vmfs -lah

    total 4.0K

    drwxr-xr-t 36 root root 5.7K Jun 29 11:10 .

    drwxr-xr-x  5 root root 4.0K Jul 31 01:30 ..

    ……

    -r--------  1 root root 4.0M Jun 26  2010 .vh.sf

    drwxr-xr-x  2 root root 1.6K Nov 19  2010 w2003r2entx86

    drwxr-xr-x  2 root root 1.6K Oct 25  2011 w2008r2ent64bit

    ibm@ubuntu12:~$ sudo cp -a /media/vmfs/hpdc01 /media/hp

    *********************



  • 5.  RE: Repair / Recover VMFS Volume

    Posted Jul 31, 2012 03:41 PM

    which version of vmfs-tools do you use ?
    AFAIK only the latest has experimental support for extends

    do you see all .*.sf files or just the .vh.sf ?

    before you give up ... I am testing a procedure to cut vmdks out of raw filesystems - if you want to serve as a guinea pig let me know ;-)



  • 6.  RE: Repair / Recover VMFS Volume

    Posted Nov 21, 2012 05:53 AM

    I was wondering if you ever had any luck with a procedure to extract vmdks out of a raw system.

    I've got some VM's which were mounted via iSCSI. Something got messed up during a reboot of the NAS, I can still see all of the iSCSI mounts, and partitions. However ESXi does not see anything on them.

    I'm hoping to extract the VMDK's if possible. If their is anything I can try, would be happy to give it a spin.

    Thanks.

    4. Jul 31, 2012 8:40 AM in response to: ArrowSIVAC

    before you give up ... I am testing a procedure to cut vmdks out of raw filesystems - if you want to serve as a guinea pig let me know ;-)



  • 7.  RE: Repair / Recover VMFS Volume
    Best Answer

    Posted Nov 21, 2012 09:34 AM

    If ESXi can not see any content of the VMFS-volume - try Linux vmfs-tools - it often sees more.
    Try the LiveCD I make - it has latest vmfs-tools buildin.

    Carving out flat.vmdks is possible but only is worth a try if:

    1. you have good reason to assume that the flat.vmdk was written in one fragment
    2. the vmdk was thick provsioned (eager zeroed works best)
    3. you can provide exact details on the disk layout of the lost flat.vmdk



  • 8.  RE: Repair / Recover VMFS Volume

    Posted Nov 21, 2012 12:05 PM

    I have been working on other projects the last few months but the volume I did save.  I did a DD of the volume over to another controller and just rebuilt the environment from scratch.  I would be glad to try this for a few reasons.   1)  Because... what good nerd would not want to aspire to the status of guinee pig...  2) Their are two VMs which if we could get back would save us lots of hours rebuilding demos.

    Can you point me to the URL for the live DVD.  I fully realize this is "beta" and I have copies of copies of copies of this data set so it is not a big deal if it mucks things up.



  • 9.  RE: Repair / Recover VMFS Volume

    Posted Nov 21, 2012 07:23 PM


  • 10.  RE: Repair / Recover VMFS Volume

    Posted Nov 28, 2012 06:43 PM

    @continuum

    Thanks for the ISO, but I am not seeing anything on my local VMFS datastore. I've installed this fresh ESXi 5.0 server to test your ISO, but cannot see anything of the disk.

    fdisk -l only list /dev/sda1 and the system is GPT and the Id is ee, no VMFS, not FAT16, no Linux partitions.

    It seems that the total area of the physical disk is in that /dev/sda1.

    I am also getting warning that GPT is on /dev/sda.

    Am I missing something?

    BTW: I tried installing ESXi on the same disk or on a USB flash then create a datastore on the disk, but still nothing.

    I've tested it on both 5.0 and 5.1. In all my tests, I've created a number of VMs on the disk.

    This is a local SATA disk.

    I have also tried installing ESXi as a VM inside Workstation, did not see any of the VMFS5 datastores, but saw a VMFS3 datastore that I have added to make sure. I am sure, I downloaded the correct ISO mcs-esxi5-recovery-X-001.iso

    Thanks again.



  • 11.  RE: Repair / Recover VMFS Volume

    Posted Nov 29, 2012 01:29 AM

    if the vmfs volume is lets say /dev/sdc1 run
    vmfs-fuse /dev/sdc1 /mnt



  • 12.  RE: Repair / Recover VMFS Volume

    Posted Nov 29, 2012 07:59 PM

    For some odd reason this works on /dev/sdc1 but fails on /dev/sda1 !!

    It gives

    VMFS VolINFO: Invalid magic number 0x00000000

    VMFS: Unable to read volume information

    Trying to find partitions

    Unable to open device/file "/dev/sda1"

    Unable to open filesystem

    It seems to me that this is because this is the disk where ESXi is also installed.

    BTW: I saw posts for you saying that you have included some recovery tools with the ISO, is there a list of those tools?

    Thanks again for this great ISO :smileyhappy:

    Tomorrow, I should be trying it to recover a missing server, whish me luck :smileywink:



  • 13.  RE: Repair / Recover VMFS Volume

    Posted Nov 29, 2012 08:07 PM

    Run gparted first - there you will see which partitions are used for VMFS.

    /dev/sd*3 = thats typical for a LUN with an ESXi installation and the default VMFS datastore

    /dev/sd*1 = thats typical for a VMFS only LUN

    I included tools that I often use for remote recovery jobs: ddrescue , scalpel , open-iscsi , testdisk .... some more mount-tools for vmdks.


    In case you get "Invalid magic number" with vmfs-fuse for a partition that is listed as VMFS call me - this is serious but can be fixed in lucky conditions



  • 14.  RE: Repair / Recover VMFS Volume

    Posted Nov 29, 2012 08:34 PM

    mounting /dev/sda3 worked without issue :smileyblush:

    Tomorow is the real issues.

    We have an iSCSI LUN that had a number of VMs, the LUN was having issues and suddenly the folder containing one VM is no longer accessible.

    If you CD to it, you get an I/O error message after a while. But the data should be still there somewhere/somehow as the free space on the VMFS datastore did not change. Does your ISO has tools that can help retrieve the data??

    I was hopping that we will be lucky, and be able to see it using your ISO and copy it off. Maybe we will be, but in the likely case that we will not be that lucky, I may need to use some recovery tools.

    To make my life simpler I will be attaching the LUN to a VM directly using RDM (this way, I avoid using iSCSI and other remote tools). Yet, I probably will need to use other tools to recover the data :smileysad:



  • 15.  RE: Repair / Recover VMFS Volume

    Posted Dec 03, 2012 10:49 AM

    To recover vmdks with I/O errors I often use the LiveCD with vmfs-fuse and then use ddrescue to copy out the files



  • 16.  RE: Repair / Recover VMFS Volume

    Posted Dec 20, 2016 01:35 PM

    How do you copy out the vmdk file after vmfs-fuse mount with ddrescue?



  • 17.  RE: Repair / Recover VMFS Volume

    Posted Dec 20, 2016 10:40 PM

    using the MOA-iso from vm-sickbay.com

    sudo su
    mkdir /esxi

    mkdir /vmfs-out

    mkdir /vmfs-in

    vmfs-fuse <DEVICE> /vmfs-in

    sshfs -o ro root@esxi1:/ /esxi

    sshfs root@esxi2:/vmfs/volumes/datastoreRecoveryOUT /vmfs-out

    cd /vmfs-out

    mkdir out

    ddrescue /vmfs-in/directory/name-flat.vmdk  out/name-flat.vmdk out/name-flat.vmdk.log

    DEVICE maybe something /dev/sdc1 or /esxi/dev/disks/naa.*:1

    esxi1 maybe the same host as esxi2 - but can also be another host

    By the way ....
    With ESXi 5.5 or 6 I noticed that I only rarely can use vmfs-fuse.
    Most of the times it boils down to using scripts with
    a bunch of dd-commands per flat.vmdk

    Ulli



  • 18.  RE: Repair / Recover VMFS Volume

    Posted Dec 20, 2016 11:46 PM

    sshfs -o ro root@esxi1:/ /esxi

    sshfs root@esxi2:/vmfs/volumes/datastoreRecoveryOUT /vmfs-out

    I presume that this above is when you want to 'chroot' to enviroment?

    I was trying to ddrescue file from datastore to another datastore and it failed. By the way when I used vmfs-fuse to mount datastore which was the outfile (destination path) it was mountend only RO and I i couldn't save the file into it.

    I made
    1.  vmfs-fuse /dev/sda1 (broken driver with important vmdk file) /mnt/in

    2. vmfs-fuse /dev/sdb1 (destination datastore) /mnt/out

    after that ddrescue /mnt/in/file.vmdk /mnt/out/file.vmdk


    And received error I/O


    So now trying to recover but with -d and -r1 as raw file if it works I will be able to mount the raw file in loop to pull the vmdk file.

    So Basically  pulling file to file doesn't work all the time.


    Update:


    After the ddrescue finished It just cloned the disk to file with the same error. So when I tried to fsck.vmfs datastore.raw file I get lots


    Pointer Block 0x20025843 is lost.

    Pointer Block 0x30025843 is lost.

    Pointer Block 0x40025843 is lost.

    Pointer Block 0x50025843 is lost.

    Pointer Block 0x60025843 is lost.

    with

    File Block 0x02c6f4c1 is lost.

    File Block 0x02c6f501 is lost.

    File Block 0x02c6f541 is lost.

    File Block 0x02c6f581 is lost.

    File Block 0x02c6f5c1 is lost.

    File Block 0x02c6f601 is lost.

    And when I mount that file with

    vmfs-fuse

    It gone with no problem but

    When I wanted to rsync vmdk file from that clone image to another location


    I get in 99%


    receiving incremental file list

    ./

    file_3-flat.vmdk

    536,763,109,088  99%   53.57MB/s    0:00:01  ad

    rsync: read errors mapping "/mnt/restore/file/file_3-flat.vmdk": Input/output error (5)

    536,870,912,000  99%   50.27MB/s    2:49:45 (xfr#1, to-chk=1/3)

    WARNING: file_3-flat.vmdk failed verification -- update discarded (will try again).

    file_3.vmdk

    536,870,912,502 100%   50.26MB/s    2:49:46 (xfr#2, to-chk=0/3)



  • 19.  RE: Repair / Recover VMFS Volume

    Posted Dec 21, 2016 08:39 PM

    :> I presume that this above is when you want to 'chroot' to enviroment?

    Not sure what you mean by that ...
    I usually work remotely - I ask my customer to create a 64-bit Linux VM that has network access to the esxi network.
    Then I can do my work even when the esxi-host is still in production.
    And I also use vmfs-fuse really rarely these days - the more terrabyte-size vmdks you work with the less useful it is.
    I also noticed that is very useful to access VMFS-volumes with readonly mode - helps to workaround I/O-errors or stale locks.
    Writing to VMFS via vmfs-fuse is something I never do - instead I just mount one directory of a VMFS-volume in writeable mode via sshfs.
    This way I never get complaints about bad writes from a dangerously out-dated vmfs-fuse tool.
    Only advantage of using vmfs-fuse and directly booting the affected esxi-host into Linux is the higher read-rate that you can get that way.

    > So when I tried to fsck.vmfs datastore.raw file I get lots


    Pointer Block 0x20025843 is lost.

    Pointer Block 0x30025843 is lost.

    Pointer Block 0x40025843 is lost.

    Pointer Block 0x50025843 is lost.

    Which vmfs-fuse version do you use ?
    Can you read flat-vmdks larger than 256gb ?
    If not you can expect errors like that

    > So Basically  pulling file to file doesn't work all the time.

    Of course not.
    The key factor for recovery is the corruption-rate of the vmfs-metadata.

    vmfs-fuse only helps with lightly damaged VMFS-Metadata.
    If the hidden sf-files in the first GB of the volume are damaged, overformatted or missing vmfs-fuse does not help at all


    Ulli




  • 20.  RE: Repair / Recover VMFS Volume

    Posted Dec 21, 2016 10:48 PM

    Which vmfs-fuse version do you use ?

    Can you read flat-vmdks larger than 256gb ?

    If not you can expect errors like that

    I Don't know, but I downloaded the latest MON-iso so it should be the latest.

    I also noticed that is very useful to access VMFS-volumes with readonly mode - helps to workaround I/O-errors or stale locks.

    Writing to VMFS via vmfs-fuse is something I never do - instead I just mount one directory of a VMFS-volume in writeable mode via sshfs.

    This way I never get complaints about bad writes from a dangerously out-dated vmfs-fuse tool.

    Only advantage of using vmfs-fuse and directly booting the affected esxi-host into Linux is the higher read-rate that you can get that way.

    Oh ok so thats why you use sshfs .. I didn't know what it was till now.

    I bought new disk 2TB and put into the server.

    The Damaged one has vmdk file. Thos file is an image of /dev/sdd disk in the VM. It's added to LVM in VolumeGroup.

    ddrescuce to image didn't work and I was furious. The last Idea (beside yours) was try to run the VM again with faulty disk. Maybe it gone error only one time but maybe it can run some time with it. Guess what, The VM started. So it's good. I decide to make some operations inside the GuestOS luckily, like I said, it's LVM. I added new disk from the new disk to VM with 550GB storage amount. Now In GuestOS I added this disk to Physical Volume and extend the VolumeGroup. After That I ran pvmove from the /dev/sdd to new disk /dev/sde (about 400GB to move) it's running slow but it works like RAID 1 - sync. So I think it will work. After that I will be able to remove the /dev/sdd from VolumeGroup and then from PV and finally disconnect that disk from Server.



  • 21.  RE: Repair / Recover VMFS Volume

    Posted Dec 22, 2016 05:59 PM

    > I Don't know, but I downloaded the latest MON-iso so it should be the latest.
    I dont know MON-isos - if you mean the latest MOA-iso from  vm-sickbay.com - that iso has the latest official Ubuntu-build vmfs-tools which is NOT good enough for working with VMFS from ESXi 5.5 or later.
    Call me via skype - dont want to post how to use unoffical builds of vmfs-tools in public.
    The risk of damaging VMFS-volumes using the writeable flag is too big for my taste.
    Ulli



  • 22.  RE: Repair / Recover VMFS Volume

    Posted Dec 24, 2017 05:01 AM

    I have a similar problem with ESXI 5. Caused by several things.

    It started with failure of the UPS which caused a HDD failure of a RAID1 configuration and it damaged my external attached backup device.

    Before I received a spare HDD to rebuild the RAID1 The Ups failed again and corrupted somehow the last good working HDD. Which resulted that the ESXI5 doesn't see VMFS store anymore.

    Is there any change to retrieve the VM's which where stored on the missing VMFS volume?



  • 23.  RE: Repair / Recover VMFS Volume

    Posted May 23, 2016 03:43 PM

    Hi.

    I'm having a similar problem. After failed RAID member and RAID rebuild, datastore is no longer visible. The volume is visible in devices, but in datastores nothing is visible.

    I tried rebuilding the partition with partedUtil, vmfs-tools - failed to mount because of invalid magic number 0x00000000, tried cloning volume from the RAID to a single disk and recover data with UFS Explorer. UFS identifies the partition as VMFS (as well as parted). I see all the files and managed to copy all of them except for the flat.vmdk and swap file.In the kernel log I see no FS driver claimed this partition. It has no label. The partition is VMFS5 and I'm using ESXi 5.1. I tried VOMA in ESXi 6 with the fix option, but it failed as well. Any help appreciated.



  • 24.  RE: Repair / Recover VMFS Volume

    Posted May 23, 2016 08:12 PM

    Hi Marek
    if you create a dump of the vmfs-meta data - thats the first 1536mb of the VMFS-partition - and provide a download -then I can assist.
    Call me via skype - see my signature.
    Ulli



  • 25.  RE: Repair / Recover VMFS Volume

    Posted May 24, 2016 06:26 AM

    Hi.

    Thanks for the offer, I sent you the link to the dump via Skype.

    Rgds

    Marek



  • 26.  RE: Repair / Recover VMFS Volume

    Posted May 24, 2016 01:06 PM

    We looked at the case together and agreed to give up.
    Diagnosis: Raid 5 rebuild gone terribly wrong - this is a case for Ontrack :smileycry:
    Ulli



  • 27.  RE: Repair / Recover VMFS Volume

    Posted May 24, 2016 04:05 PM

    Hi Ulli.


    Thank you very much for your time, help and valuable tips today. I'm really grateful that you tried to help me recover the data and showed me some very interesting techniques. Let me know if you find something in the dump I sent you earlier.

    Marek



  • 28.  RE: Repair / Recover VMFS Volume

    Posted Jul 22, 2021 10:25 AM

    Dear support

    I have the same problem. invalid  magic number 0x00000000 VMFS. please help



  • 29.  RE: Repair / Recover VMFS Volume

    Posted Jun 16, 2013 04:13 PM

    This link is not working, can you please send me new link if it's still available...



  • 30.  RE: Repair / Recover VMFS Volume

    Posted Aug 28, 2014 04:55 PM

    hello

    can i have the link of the iso file because i have a similar problem.

    Like that :

    Running in esxi 3.0.2 Raid5 (4 disc) and the first disc is not in the member raid5 because it's damage.

    One of my virtual machine stop working and i was trying to extend virtual disk to creating another disk bigger than the first one to link the vmdk file on it when i saw that not handle  my problem i delete the disk & the virtual disk and the big problem start at this moment because the vmdk file goes .

    Now i'm in the situation where i have lost one vmdk file

    I have try to recover using diskinternal VMFS recover 1.5 , UFS explorer professional nothing !!!!!
    You have did a best work

    Thanks & Enjoy the rest of the day



  • 31.  RE: Repair / Recover VMFS Volume

    Posted Apr 20, 2015 04:34 PM

    thanks for the link but this Fallen

    you can upload it again

    thanks



  • 32.  RE: Repair / Recover VMFS Volume

    Posted Jul 31, 2015 08:59 AM

    Hello,

    I have the same issue and i'm unable to connect on your ftp to download the iso file.

    Please Help :smileycry:



  • 33.  RE: Repair / Recover VMFS Volume

    Posted Jul 31, 2015 01:02 PM

    Hi
    latest commandline-only version see http://vm-sickbay.com
    earlier version with graphical interface http://sanbarrow.com/iscsiworkshop/moa64dvd.iso

    Please dont use any older versions - I update them every now and then for good reasons.

    Ulli



  • 34.  RE: Repair / Recover VMFS Volume

    Posted Jan 18, 2016 09:34 PM

    Hi, moa64dvd.iso dont boot

    help please



  • 35.  RE: Repair / Recover VMFS Volume

    Posted Jan 26, 2016 07:10 PM

    Hi Jose
    no need to reply here - just wanted to update this post for other users.In case someone finds this thread via google : do not try to download any of those old versions that are mentioned here.
    Even the latest vmds-tools build are outdated nowadays.
    vmfs-fuse is still very very useful - but with ESXi 5.5 and later it has problems.
    In some conditions the max size of a vmdk that you can extract with vmfs-fuse is 256gb.
    Unfortunately I dont expect to see an updated version of the vmfs-tools - the project seems to be dead since a good while.
    That does not mean that we are running out of options.
    On the contrary - nowadays I usually get better results than 2 or 3 years ago.
    But it was sure easier in the days of ESXi 5.1 and earlier.
    You will find the current ISO-files I use on http://vm-sickbay.com
    If they dont boot - please complain and call me on skype
    Ulli



  • 36.  RE: Repair / Recover VMFS Volume

    Posted Aug 28, 2015 03:40 PM

    I know this is an old thread, but I used it a lot to fix my own issue. If you are getting similar symptoms and the "magic number" error, please check to make sure you haven't taken a snapshot on your SAN. I'm running ZFS and took a snapshot of the zvol containing my datastore. This is the output I got in the vmkernel log when trying to rescan the storage adapter:

    [root@ESXI:~] tail /var/log/vmkernel.log

    2015-08-28T02:35:57.160Z cpu4:33159)NMP: nmp_ResetDeviceLogThrottling:3345: last error status from device mpx.vmhba32:C0:T0:L0 repeated 19 times

    2015-08-28T02:37:40.649Z cpu6:32797)NMP: nmp_ThrottleLogForDevice:3178: Cmd 0x9e (0x439d806e13c0, 0) to dev "mpx.vmhba32:C0:T0:L0" on path "vmhba32:C0:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE

    2015-08-28T02:37:40.717Z cpu0:43827)LVM: 10060: Device naa.6589cfc000000903f337f92db5f049d3:1 detected to be a snapshot:

    2015-08-28T02:37:40.717Z cpu0:43827)LVM: 10067:   queried disk ID: <type 1, len 21, lun 0, devType 0, scsi 0, h(id) 13996138677875135599>

    2015-08-28T02:37:40.717Z cpu0:43827)LVM: 10074:   on-disk disk ID: <type 1, len 21, lun 0, devType 0, scsi 0, h(id) 1294898510287430916>

    2015-08-28T02:37:40.822Z cpu6:32797)NMP: nmp_ThrottleLogForDevice:3130: last error status from device mpx.vmhba32:C0:T0:L0 repeated 10 times

    2015-08-28T02:37:40.922Z cpu0:43827)FSS: 5327: No FS driver claimed device 'control': No filesystem on the device

    2015-08-28T02:37:40.923Z cpu0:43827)VC: 3551: Device rescan time 77 msec (total number of devices 6)

    2015-08-28T02:37:40.923Z cpu0:43827)VC: 3554: Filesystem probe time 205 msec (devices probed 5 of 6)

    2015-08-28T02:37:40.923Z cpu0:43827)VC: 3556: Refresh open volume time 1 msec

    If you see this, you need to resignature the datastore. I didn't lose anything when I did the resig using the "Keep existing signature" option under "Add storage".

    If you took a snapshot, and are not seeing the "detected to be a snapshot", let me know. There were a couple other settings I messed with on the FreeNAS extent like the compatibility setting, but I'm not sure whether or not it had an effect. Also, I'm in the middle of a recovery from a messed up pool configuration, so the proper fix is probably to delete the snapshots. I just need to access them again to back them up.



  • 37.  RE: Repair / Recover VMFS Volume

    Posted Aug 29, 2014 12:46 AM

    Hello

    I think that i found where you can download the iso live CD & hope it's the good one , keep me post if not

    link: ftp://recovery:recovery@ftp.mightycare.de/mcs-esxi5-recovery-X-001.iso

    Best regard

    Ballaflex



  • 38.  RE: Repair / Recover VMFS Volume

    Posted Aug 29, 2015 05:57 AM

    Hi,

    Thanks for feedback.