VMware Cloud Community
meecect
Contributor
Contributor

upgrade gone awry

Hi,

We had a two node cluster running ESX 2.5.3. It was SAN attached an there were probably 8 or so LUNS zoned to each server. We VMotioned the guests to the secondary server and began an install on the primary. I wasn't doing the install, so I'm not sure of the exact steps taken, but at some point through the install, the luns disappeared from the secondary host. We have been unable to get any of the servers back and now the the luns look like unformatted volumes from both hosts.

Does anyone have a clue as to what may have happened? The tesch swears that nothing was done during the install,and indeed, I don't recall even an option to wipe out attached SAN volumes during the install process.

Any hints as to tools or steps that one might investigate what may be 'hidden' (if anything) on these volumes?

thanks in advance for any help you may provide,

meecect

0 Kudos
3 Replies
meecect
Contributor
Contributor

sorry, I should have mentioned we were upgrading to 3.0.1m but using a method suggested to us from vmware and others, ie, vmotion the hosts and do a fresh install one one node., then vmotion back, doing an upgrade to the FS and guests, then upgrade the second. We have lost ~35 machines.

0 Kudos
copkilla
Contributor
Contributor

This is typical situation when you install esx on server which is connected to SAN and vmfs luns. Before you install esx disconnect server from SAN. I think that it will be difficult to repair this vmfs lun and get back vm.

Regards

Daniel

0 Kudos
meecect
Contributor
Contributor

firstly, I was able to determine that yes, the partitions were removed. I found this in anaconda-ks.cfg:

clearpart --all

So I verified that the install technician did make an error during the install, and since the server was still cabled and zoned to the san, that the install likely wiped the partitions in question.

Not stopping there, I did find a very helpful thread at http://communities.vmware.com/message/258927

which showed how to put back the partition table.

After doing that, I seem to have recovered all the mising vmdk's.

0 Kudos