Kasraeian
Expert
Expert

Problem With Datastore - All VMs Gone (Urgent)

Hi,

Sorry for my writing but I'm in hurry

Wh have data center which we running ESX 3.5 on it as Infrastructure. after 9 months of working, today suddenly our datastores goes and disappearded.

all our VMs are still working and answering to all commands, but Hosts start generating alarm for datastore and asking for for adding new datastore.

I have searched and find this KB (9453805)language=en_US&cmd=displayKC&externalId=9453805] and did it without shuting down VMs (big mistake as I'm see it until now).

Now, all VMs gone from datastore but the are still working fine, these are screenshots from the state

I'm really appreciate all your helps and support.

we need solution soon because we have to deliver these services soon.

Again thanks for all your helps



------

-= If you found this note/reply useful, please consider awarding points for "Correct" or "Helpful" =-

-= If there's any mistake in my notes, please correct me! =-

-= Thanks =-

Sohrab Kasraeian Fard

If you found this note/reply useful, please consider awarding points for "Correct" or "Helpful" If there's any mistake in my notes, please correct me! Sohrab Kasraeianfard | http://www.kasraeian.com | @Kasraeian
0 Kudos
22 Replies
moniquevmb
Contributor
Contributor

Now this is purely speculating, since I don't have enough information, but I had this type of thing happen to me twice. First time was when I added a ESX4 server to a cluster with ESX 3.5 machines. Resignaturing did the trick for me then. After wich I was left with aparent "snapshot" luns. Wich were actually the same luns, but seen twice by the vCenter server. I had to shut down all vm´s, remove them from the inventory and add them back from the right vmfs volume, after wich I could rename the `snapshot` vmfs volumes....I´m not certain this is the case here.

Second time, we had a VCB server wich was wrongly configured. This only came to light the first test backups were made with it. It had `automount`enabled. And windows wrote its signature on the disks.

All vmfs volumes dissapeared on the ESX servers where no vm´s were running, since the ESXes no longer recognized the disks as VMFS volumes.

More information about this can be found here±

http://www.yellow-bricks.com/2008/02/11/vcb-i-forgot-all-about-automount-disable-what-now/

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=100216...

http://www.virtualizationteam.com/virtualization-vmware/vmware-vi3-virtualization-vmware/vmware-esx-...

I hope this helps and if not, I hope you find the correct answer soon.

0 Kudos
AlexNG_
Enthusiast
Enthusiast

Hi Kasraeian,

Maybe sounds stupid, but in these cases I use to start from the basics... AS someone suggested, can you see on the afected hosts the mapped LUNs (vdf)? If not, check SAN zonning and mapping. Also as a side sugestion, poweroff all VMs and do a full backup (scp, veeam... )

AlexNG

If you find this information useful, please award points for "correct" / "helpful".
0 Kudos
Kasraeian
Expert
Expert

Hello everyone,

First of all thanks all of you, because of your help and I don't know how can I explain my state in that night and latter please accept my apologize for being late in responding to all your notes and helps, as I was really busy in this two weeks.

Good or bad, the problem solved by decision which made by our project manager and our team leader respectively and that was "starting from the scratch" (not every thing but most of it).

I know, most of you must be shucked by seeing this but this is what happened and thanks to god, we made it on time on delivering the service to our customer.

I will post more information about this on my site and I'd be really happy to hearing your idea and comments on it.

Again thanks everyone for your help and support, those post mean a lot to me as I didn't have any other place for asking for help.

Cheers


------

-= If you found this note/reply useful, please consider awarding points for "Correct" or "Helpful" =-

-= If there's any mistake in my notes, please correct me! =-

-= Thanks =-

Sohrab Kasraeian Fard

If you found this note/reply useful, please consider awarding points for "Correct" or "Helpful" If there's any mistake in my notes, please correct me! Sohrab Kasraeianfard | http://www.kasraeian.com | @Kasraeian
0 Kudos