Hello,
I would like to advice with you.
I had datastore on RAID 0. I lost it after re-installation of my ESXi hypervisor. I made a mistake and recreated a new datastore on the place of old datastore. No data was not written on this datastore. I can say that I see it in ls vmfs/volumes..but it is empty.
Do you think that I have lost data definitely?
thank you very much, Olda
Try to contact continuum, who may be able to help.
André
Hi,
Check out this blogpost regards recovering a VMFS datastore.
https://vmwaremine.com/2013/09/20/recover-vmfs-datastore-with-all-data/#sthash.RsrXGDl8.dpbs
Do not follow the advice from adgate !!!!
By formatting the datastore you very likely overwrote the existing VMFS metadata for the vmdks stored on that volume.
It still maybe possible to recover files - read my instructions for creating a VMFS header-dump:
Create a VMFS-Header-dump using an ESXi-Host in production | VM-Sickbay
If the VMFS metadata has been overwritten only thick provisioned vmdks that were written in one piece can be recovered.
Anyway - with a headerdump I can give you a better evaluation ....
Call me via skype ...
Ulli
continuum what is wrong with my advice to recover VMFS3 datastore?
Am I assuming right that the VMware Knowledge Base article would not help to recover a lost partition table on a VMFS volume?
adgate
In this case the datastore has been re-formatted, which means that the VMFS metadata within the partition has been overwritten, so it's not actually a matter of recovering a partition table.
André
Hello Ulli,
I prepared header-dump for you https://www.dropbox.com/s/igekeq9zlf9wer9/array.2048.gz?dl=0
When I checked it via trialware of VMFS recovery utils, I saw my data (VMs).
thank you very much for help, Olda
Hi Olda
just downloaded your dump - unfortunately it is truncated: only 1.21 gb instead of the expected 2 GB.
But even with the truncated dump it looks promising. I found several vmdks and I expect to be able to restore full VMs including latest snapshots.
Did you watch the command execution while you created the dump ?
Looks like you received a "short write" message while using gzip ...
Anyway - I suggest to either buy the full version or call me via Skype.
With a full dump I am sure I can help you
Ulli
Hi Ulli,
yes, i see in log "short write":
[root@HPG8:/dev/disks] ls -lisa | grep -v vml
total 13187275106
4 0 drwxr-xr-x 2 root root 512 Jun 14 20:53 .
1 0 drwxr-xr-x 13 root root 512 Jun 14 20:53 ..
198 3922432 -rw------- 1 root root 4016570368 Jun 14 20:53 mpx.vmhba32:C0:T0:L0
186 4064 -rw------- 1 root root 4161536 Jun 14 20:53 mpx.vmhba32:C0:T0:L0:1
188 255984 -rw------- 1 root root 262127616 Jun 14 20:53 mpx.vmhba32:C0:T0:L0:5
190 255984 -rw------- 1 root root 262127616 Jun 14 20:53 mpx.vmhba32:C0:T0:L0:6
192 112624 -rw------- 1 root root 115326976 Jun 14 20:53 mpx.vmhba32:C0:T0:L0:7
194 292848 -rw------- 1 root root 299876352 Jun 14 20:53 mpx.vmhba32:C0:T0:L0:8
196 2621440 -rw------- 1 root root 2684354560 Jun 14 20:53 mpx.vmhba32:C0:T0:L0:9
183 262144 -rw------- 1 root root 268435456 Jun 14 20:53 mpx.vmhba32:C0:T0:L1
181 257008 -rw------- 1 root root 263176704 Jun 14 20:53 mpx.vmhba32:C0:T0:L1:1
178 143338560 -rw------- 1 root root 146778685440 Jun 14 20:53 naa.600508b1001c3a79a4f4a61826ef8e55
174 358400 -rw------- 1 root root 367001600 Jun 14 20:53 naa.600508b1001c3a79a4f4a61826ef8e55:1
176 142978048 -rw------- 1 root root 146409521152 Jun 14 20:53 naa.600508b1001c3a79a4f4a61826ef8e55:2
164 2930141144 -rw------- 1 root root 3000464531456 Jun 14 20:53 naa.600508b1001c5afa370b08de8580cd9c
162 2930138112 -rw------- 1 root root 3000461427200 Jun 14 20:53 naa.600508b1001c5afa370b08de8580cd9c:1
171 3516169176 -rw------- 1 root root 3600557236224 Jun 14 20:53 naa.600508b1001cfd5e14e8b8e5264c73eb
167 4064 -rw------- 1 root root 4161536 Jun 14 20:53 naa.600508b1001cfd5e14e8b8e5264c73eb:1
169 3516163072 -rw------- 1 root root 3600550985728 Jun 14 20:53 naa.600508b1001cfd5e14e8b8e5264c73eb:2
[root@HPG8:/dev/disks] esxcfg-scsidevs -m
naa.600508b1001c5afa370b08de8580cd9c:1 /vmfs/devices/disks/naa.600508b1001c5afa370b08de8580cd9c:1 5edcea68-9140a3f7-5a0f-d89d672d7 a30 0 IBM_Array_01_OLD
[root@HPG8:/dev/disks] dd if=/dev/disks/Device:1 bs=1M count=2048 of=/tmp/replace with your name.2048
[root@HPG8:/dev/disks] dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=2048 of=/tmp/replace.2048
dd: writing '/tmp/replace.2048': No space left on device
259+0 records in
258+0 records out
[root@HPG8:/dev/disks] dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=2048 | gzip -c > /tmp/array.2048.gz
gzip: short write
[root@HPG8:/dev/disks] dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=2048 | gzip -c > /tmp/array.2048.gz
gzip: short write
[root@HPG8:/dev/disks]
My partition is 2700 GB large (5 x 600Gb SAS hard drives). ESXi is on 4GB SD card.
How can I do full dump? I used your guide...
thank you, Olda
PS: it is my a private server, unfortuantely it doesnt make sence buy license for almost 1000 dolars for me 😞
Dump file is necessary to recover data or only for an analyses?
Do you have another - unaffected datastore ?
If yes - store the dump there.
If not :
- clean up /tmp
- then create dumps in smaller pieces like for example
dd if=DEVICE of=/tmp/DUMP_1.600 bs=1M count=600 skip=0
dd if=DEVICE of=/tmp/DUMP_2.600 bs=1M count=600 skip=600
dd if=DEVICE of=/tmp/DUMP_3.600 bs=1M count=600 skip=1200
Hello continuum,
I have added USB drive to ESXi. Path to drive is /vmfs/devices/disks/mpx.vmhba33:C0:T0:L0 but I am not able to store dump to this drive. I didnt find out how to do it. Do you help me?
I modified command
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=2048 of=/vmfs/devices/disks/mpx.vmhba33:C0:T0:L0/array.2048
but it doesn work. thank you Olda
That will not work like that - and it is not worth the effort.
Instead create a partial dump.
Connect to the ESXi with WinSCP
Create an empty directory on your adminhost - call it Olda-dump
Clean up /tmp
Then run the following commands via putty one by one.
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=500 skip=0 | gzip -c > /tmp/partdump500-0.gz
Download generated file into Olda-dump and clean up /tmp
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=500 skip=500 | gzip -c > /tmp/partdump500-1.gz
Download generated file into Olda-dump and clean up /tmp
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=500 skip=1000 | gzip -c > /tmp/partdump500-2.gz
Download generated file into Olda-dump and clean up /tmp
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=500 skip=1500 | gzip -c > /tmp/partdump500-3.gz
Download generated file into Olda-dump and clean up /tmp
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=500 skip=2000 | gzip -c > /tmp/partdump500-4.gz
Download generated file into Olda-dump and clean up /tmp
You now should have 5 gz-files in Olda-dump
Zip the complete directory Olda-dump and provide a download link to the zip-file.
Hello continuum
I did it and there is link: Dropbox - Dump.7z - Simplify your life
The file has over 800 MB, but there was "short write" in log too :
[root@HPG8:~] dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=500 skip=0 | | |
gzip -c > /tmp/partdump500-0.gz
500+0 records in
500+0 records out
[root@HPG8:~] dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=500 skip=500 |
gzip -c > /tmp/partdump500-1.gz
500+0 records in
500+0 records out
[root@HPG8:~] dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=500 skip=1000 |
gzip -c > /tmp/partdump500-2.gz
gzip: short write
[root@HPG8:~] dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=500 skip=1000 |
gzip -c > /tmp/partdump500-2.gz
gzip: short write
[root@HPG8:~] dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=500 skip=1500 |
gzip -c > /tmp/partdump500-3.gz
gzip: short write
[root@HPG8:~] dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=500 skip=2000 |
gzip -c > /tmp/partdump500-4.gz
gzip: short write
thank you, Olda
BTW: for modification USB drive I used this guide:
Unfortuanately it didnt work.
Did you really clean up /tmp after each dd-command ?
So far we only have dumped the first 1000mb - thats not enough.
continuum There are only a couple of 1-2 KB files in temp directory. It doesnt help to delete them. When I delete it, i save 50-60 KB 🙂
There is probably around 260 MB free space on SD card. I dont know If I can clean up other directories
So we must make sure that none of the dump-parts ever gets larger than 250 MB ...
This should do ...
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=250 skip=0 | gzip -c > /tmp/partdump-0.gz
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=250 skip=250 | gzip -c > /tmp/partdump-250.gz
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=250 skip=500 | gzip -c > /tmp/partdump-500.gz
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=250 skip=750 | gzip -c > /tmp/partdump-750.gz
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=250 skip=1000 | gzip -c > /tmp/partdump-1000.gz
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=250 skip=1250 | gzip -c > /tmp/partdump-1250.gz
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=250 skip=1500 | gzip -c > /tmp/partdump-1500.gz
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=250 skip=1750 | gzip -c > /tmp/partdump-1750.gz
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=250 skip=2000 | gzip -c > /tmp/partdump-2000.gz
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=250 skip=2250 | gzip -c > /tmp/partdump-2250.gz
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=250 skip=2500 | gzip -c > /tmp/partdump-2500.gz
Do the same as before - but please give the archive you upload a new name so that I immediatly see which is the newer one.
Wait a minute ...
Splitting the dump in several parts is so anoying ... lets try a workaround I have not used before : lets try to store the dump in a ramdisk.
Again clean up /tmp first.
Then connect via putty and run the following commands:
mkdir /moa
esxcli system visorfs ramdisk add -m 1000 -M 2000 -n moa -t /moa -p 0755
dd if=/dev/disks/naa.600508b1001c5afa370b08de8580cd9c:1 bs=1M count=2048 | gzip -c > /moa/olda.gz
If that works without error message you can download a one-piece dump from the /moa directory.