Last weekend, I can in to do a BIOS update on my last of four ESX host. When it rebooted, it couldnt see our datastore anymore. Ouch!
Fortunatly, the three other host that wasn't shut down still has access to our one an only DataStore. So my Virtual machines are still up.
From the VMWare infrastructure client, the host can see all luns, but not the DataStore. All other hosts can.
This is an output of fdisk.
fdisk -l
Disk /dev/sda: 1198.2 GB, 1198295875584 bytes
255 heads, 63 sectors/track, 145684 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sda doesn't contain a valid partition table
Disk /dev/sdb: 1198.2 GB, 1198295875584 bytes
255 heads, 63 sectors/track, 145684 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 481.0 GB, 481036337152 bytes
255 heads, 63 sectors/track, 58482 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdc doesn't contain a valid partition table
Disk /dev/sdd: 481.0 GB, 481036337152 bytes
255 heads, 63 sectors/track, 58482 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdd doesn't contain a valid partition table
Disk /dev/cciss/c0d0: 36.3 GB, 36385505280 bytes
64 heads, 32 sectors/track, 34699 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/cciss/c0d0p1 * 1 100 102384 83 Linux
/dev/cciss/c0d0p2 101 5100 5120000 83 Linux
/dev/cciss/c0d0p3 5101 32055 27601920 fb Unknown
/dev/cciss/c0d0p4 32056 34699 2707456 f Win95 Ext'd (LBA)
/dev/cciss/c0d0p5 32056 32599 557040 82 Linux swap
/dev/cciss/c0d0p6 32600 34599 2047984 83 Linux
/dev/cciss/c0d0p7 34600 34699 102384 fc Unknown
Please advise!
If all your VMs are up and running on the other hosts can you not reinstall ESXi and reconfigure?
But from the look of fdisk, my partitions are lost no?
Are the /dev/sd* local or shared?
You stated that the VM's are up. Up on the same shared storage? Are you looking at the same storage from the other hosts?
First, thanks for your time !
/dev/sd* is shared storage, wich comes from our SAN. For some reason, I still can browse the datastore from the three other host, even if fdisk mentionned theres no partitions.
Here's two screen shots, one from host2, wich sees the datastore, and one from host1, wich can't. I've tried to rescan in the storage adapter section, no way to see the missing Datastore .
At this point the only problem is the local install of the ESXi host. It has no VM's running on it. The SAN is iSCS?
No it's not IScsi. If I do a fdisk -l from working ESX host, partions are gone as well. I a reboot one other host, I think they won't be able to acces the VMDKs.
If you have support contracts with hardware vendors and or VMware I would exercise them now. If you feel you are experiencing failures don't wait.
Yes, they found out that the Windows VCB proxy server initialized the SAN disk, thus making the partition unreadable. They inserted some Hex code right into the partitions, and everything works fine now!!
Yipee! :smileygrin:
So happy all went well.
I wanted to send a quick update to let everyone reading this post in the future know that there are two (common) reasons you can end up with missing partition tables - and it has nothing to do with the BIOS update, this is just the reboot of an ESX host that makes you notice there is a problem accessing the VMFS volumes. If you are missing partitions, do not power off your virtual machines until you have a scheduled maintenance window (this would be an urgent maintenance window, don't wait till the weekend), then power off the virtual machines and recreate the partitions for the VMFS volumes using this KB article: http://kb.vmware.com/kb/1002168
The first way of causing all partition tables to go missing is to install ESX from a CD with the fibre plugged in and the existing VMFS volume LUNs presented to the HBA's. This will usually result in an admin not paying attention to the installation questions and answering 'yes' when prompted to 'initialize' them during installation.
The second is by not disabling automount when installing VCB on a proxy server and presenting LUNs to it's HBA (the problem Mike ran into on this post). The VM Backup Configuration Guide page 35 describes how to do this on the proxy: http://www.vmware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_vm_backup.pdf