Hi. I was "trying" to move some virtual machines around between servers and SANs. On one ESXi 3.5 server, I "had" a VMFS volume on an iSCSI target on a basic SAN device.
The volume "was" listed on the server/configuration tab under storage. I removed it thinking it would just be removing the reference to the volume from that host's perspective but it appears to have deleted the volume from the iSCSI target.
Is that correct - is my volume gone? Am I screwed? Is there anyway to recover the volume?
Any help or guidance would be really appreciated. Thanks in advance.
You have to thank Edward Haletky, not me
In the VMware Communities his nick is .
Andre
Did you try to rescan your adapters to see if the volume is still available? What happens if you go into Storage and click Add? Do you still see the volume?
I should have mentioned in my first post...
I did try a rescan for volumes but nothing showes up.
When I went to add a storage volume, it says the LUN has no volumes on it and all the space is free.
It sounds like there are some options to recover based on the research but I don't see any clear step-by-step instructions.
After the rescan operation, what is the output of this command?
fdisk -l
Andre
Thank you for your help.
The first /dev entry is the iSCSI LUN I believe.
Disk /dev/disks/vmhba32:1:0:0: 214.7 GB, 214748364800 bytes
255 heads, 63 sectors/track, 26108 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
Disk /dev/disks/vmhba1:0:0:0: 146.6 GB, 146695782400 bytes
64 heads, 32 sectors/track, 139900 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/disks/vmhba1:0:0:1 5 750 763904 5 Extended
/dev/disks/vmhba1:0:0:2 751 4845 4193280 6 FAT16
/dev/disks/vmhba1:0:0:3 4846 139900 138296320 fb VMFS
/dev/disks/vmhba1:0:0:4 * 1 4 4080 4 FAT16 <32M
/dev/disks/vmhba1:0:0:5 5 52 49136 6 FAT16
/dev/disks/vmhba1:0:0:6 53 100 49136 6 FAT16
/dev/disks/vmhba1:0:0:7 101 210 112624 fc VMKcore
/dev/disks/vmhba1:0:0:8 211 750 552944 6 FAT16
Partition table entries are not in disk order
Bad news... I do not see the VMFS partition in your LUN (it was a 214G LUN?).
Call VMware to try to restore the partition (if possible).
Andre
Thanks for your help. After searching, I finally came upon some instructions that worked and it ended up being pretty easy. I lifted the following and pasted it here for others. I also uploaded the PDF they mention although their version of the instructions worked great. Whew!!!
Recovering VMFS partitions with VMware ESX troubleshootingEdward Haletky, Contributor 02.19.2009 Rating: --- (out of 5) | ||||
[Enterprise IT tips and expert advice\ | http://digg.com/submit?phase=2&url=http%3A%2F%2FsearchVMware%2Etechtarget%2Ecom%2Ftip%2F0%2C289483%2...VMFSpartitionswithVMwareESXtroubleshooting&topic=tech_news&bodytext=Reinstalling%20servers%20is%20a%20common%20task%20for%20system%20administrators%2C%20but%20when%20a%20typical%20process%20goes%20awry%2C%20it%20can%20wreak%20havoc%2E%20In%20this%20two%2Dpart%20series%20on%20troubleshooting%20VMware%20ESX%2C%20a%20VMware%20expert%20mistakenly%20deletes%20additional%20partitions%20that%20show%20up%20after%20he%20performed%20a%20VMware%20Consolidated%20Backup%20proxy%20host%20reinstall%2C%20which%20nearly%20destroys%20the%20Virtual%20Machine%20File%20System%20and%20RDM%20data%2E] Digg This!\ [Digg This!\ | http://www.stumbleupon.com/submit?url=http%3A%2F%2FsearchVMware%2Etechtarget%2Ecom%2Ftip%2F0%2C28948...VMFSpartitionswithVMwareESXtroubleshooting] StumbleUpon\ [StumbleUpon\ | http://del.icio.us/post?v=4&noui&jump=close&url=http%3A%2F%2FsearchVMware%2Etechtarget%2Ecom%2Ftip%2...VMFSpartitionswithVMwareESXtroubleshooting] Del.icio.us\ [Del.icio.us\ | http://fusion.google.com/add?feedurl=http://rss.techtarget.com/520.xml] |
+In the course of reinstalling a VMware Consolidated Backup (VCB) proxy host, our expert encountered two problems that can be major headaches for accessing files and data in the Virtual Machine File System (VMFS). First, he deleted partitions from his ESX logical unit numbers (LUNs), which could have meant losing access to all data in the VMFS. The following article features his workaround to that problem. He then discovered he couldn't restore virtual raw device mapping (RDM) in the same way, which also meant potentially losing a massive amount of valuable historical data. Part two of this series features his workaround to that issue.
+ As a VMware expert, I am supposed to know that I should disconnect my storage area network (SAN) from a server before reinstalling the server. The server in question was a backup host instead of an ESX host, so I mistakenly thought that Windows would disconnect the SAN. Deleting mystery partitions is a bad idea After the reinstall, my server had more partitions than before the reinstall. Without too much thought, I deleted them -- bad idea. The partitions I deleted happened to be those used by my VMware ESX LUNs. So if I were to reboot my ESX servers, they would permanently lose access to their data. At this point, I knew that the prior day at least one part of the irreplaceable data had been backed up and taken off-site. But I still needed to restore the Virtual Machine File System (VMFS) and raw device mapping (RDM). VMware ESX troubleshooting 101: Check the VMware ESX hosts The first step in troubleshooting a VMware ESX problem is checking the ESX hosts. I discovered that the virtual machines (VMs) were still running. Apparently, ESX is robust enough to continue operation without its physical partitions. Since ESX was still running, and wanting to discover the damage, I went to the service console and ran the command fdisk -l. I ended up with a lot of blank partitions, as you can see in this file\ (click on link to see text). I expected to see something like this\ instead (click on link to see text). The difference between the two was that the first three partitions (highlighted in the second file) were missing. Restoring the VMFS After some research and some help from the tech folks who follow me on Twitter\, I discovered a link to a presentation on . The PDF proved invaluable. To summarize the information in that document, to restore the VMFS you need to use the command fdisk to rebuild the partition table. Then move the start block to the proper alignment. Then refresh and re-scan to access the partitions. Using fdisk, however, can be dangerous and requires caution. Instructions for using fdisk:
This is what those instructions translated into for /dev/sda and /dev/sdc within my ESX host's service console:
I tested my handiwork by rebooting the single ESX host that wasn't running any VMs and rescanned its LUNs. Thankfully, the VMFS had been successfully restored. |
|
You have to thank Edward Haletky, not me
In the VMware Communities his nick is .
Andre
Edward AKA Texiwill - you saved me - thank you, thank you!
And what iSCSI target do you use?
I use a QNAP appliance. Does that make a difference? I had deleted the volume through the Virtual Infrastructure Client.
Just interesting, cause I also one day have accidientally deleted my disk from target, but then connected it back without any problem (i use starwind`s offer)
Hi All,
After reading thru all the possible solutions I decided to ask for help, since none of these scenarios apply to me perfectly.
In my case a storage partition "fall out" from vmware host because of a faulty FC switch. Yes, it is a 1T mirrored disk attached via FC from a storage subsystem.
FC Switch was replaced, and the vmware host could see the FC storage pratition, unfortunately the guest OS's were gone. I have choosen to restart the vmware host, just in case.., hoping it might clear things up before I try to run the guest VMs. It might have been a bad decision, since it cannot attach the disk to the datastore anymore. However it can see it under storage controllers, but when I do a rescan nothing happens. It shows the correct size, LUN, etc however...
When I try to add as a storage it wants to reformat it, which I obviously don't want to happen. It also shows that the disk currently contains a VMFS partition...
fdisk output:
:~# fdisk -l /dev/sdd
Disk /dev/sdd: 892.5 GB, 892553134080 bytes
255 heads, 63 sectors/track, 108513 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x2324134e
Device Boot Start End Blocks Id System
/dev/sdd1 1 108513 871630608+ fb Unknown
Seems like the partition is okay, but how can I remount it to be a datastore again? I might be able to restore data from it.
Thanks much,
Peter