Datastores Disappeared!

[manfriday]

Yikes.

So, yesterday I upgrade my vcenter server to version 4 update 1.

I log in and it is upgradeing the vpxa agents on my hosts.

two of the hosts work fine. one fails.

I cant seem to get the agent to install, and the host shows up as "disconnected"

connecting it wont work.

All of the VM's are running, so I decide to wait until the weekened (we are a 9-5 shop) to reboot that host.

I reboot the host this morning, and the agent installs fine.

However all of the VM's on that host are "inaccessible"

I look and see that the datastores are all gone.

The LUNS appear OK in the iSCSI software controller.

SO, I go to another host to add the offline VM's to a working box.. I right click the datastore and click 'browse'

I get a message saying something along the lines of "datastore1 \ does not exist"

then all the datastores from THAT host disappear.

Any ideas?

[AndreTheGiant]

Can you send the output of this command?

fdisk -l

Have you tried to use add datastore to see if you are able to import the old datastore (warning that you can also delete your old datastores).

Andre

[manfriday]

I rebooted all the servers and once again tried to 'add storage'

This time I saw all of the available luns but they are blank.

IM officially freaking out.

Apparently we only have 'gold' support, so I cant get any of the lovely people at VMWare to help me out here, either.

[manfriday]

SOmeone was 'working' on our backup server last night, and I suspect that they may have turned on automounting, despite my warnings not to.

I am trying to run fdisk /lu to see if maybe the signatures got screwed up..

however, there are so many errors currently being spammed to the console that I cant see what is going on.

Any way to disable the logging temporarily so I can see what I am doing?

edit :

ok i got the output of fdisk:

Disk /dev/disks/naa.6090a05830b2024ea5ad2402000060d8: 322.1 GB, 322122547200 bytes

255 heads, 63 sectors/track, 39162 cylinders, total 629145600 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

Disk /dev/disks/naa.6090a05860f202cf24ac54010000000d: 536.8 GB, 536881397760 bytes

255 heads, 63 sectors/track, 65272 cylinders, total 1048596480 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

Disk /dev/disks/naa.6090a05830b2727e5fad0402000060c2: 214.7 GB, 214758850560 bytes

255 heads, 63 sectors/track, 26109 cylinders, total 419450880 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

/dev/disks/naa.6090a05830b2727e5fad0402000060c2p1 128 419441084 209720478+ fb VMFS

Disk /dev/disks/naa.6090a05860f2520b16ac34010000b063: 547.6 GB, 547608330240 bytes

255 heads, 63 sectors/track, 66576 cylinders, total 1069547520 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

Disk /dev/disks/naa.6090a05860f2e21da0ac8402000090cc: 322.1 GB, 322122547200 bytes

255 heads, 63 sectors/track, 39162 cylinders, total 629145600 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

Disk /dev/disks/naa.6090a05860f2e20d11ac04010000b056: 322.1 GB, 322122547200 bytes

255 heads, 63 sectors/track, 39162 cylinders, total 629145600 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

Disk /dev/disks/naa.6090a05860f2721837ac54020000909b: 322.1 GB, 322122547200 bytes

255 heads, 63 sectors/track, 39162 cylinders, total 629145600 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

Disk /dev/disks/naa.6090a05860f212182bac240200003088: 676.4 GB, 676457349120 bytes

255 heads, 63 sectors/track, 82241 cylinders, total 1321205760 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

Disk /dev/disks/naa.6090a05860f2824a29acf4010000304b: 2199.0 GB, 2199021158400 bytes

255 heads, 63 sectors/track, 267349 cylinders, total 4294963200 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

Disk /dev/disks/mpx.vmhba2:C0:T1:L0: 146.6 GB, 146685296640 bytes

255 heads, 63 sectors/track, 17833 cylinders, total 286494720 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

/dev/disks/mpx.vmhba2:C0:T1:L0p1 128 286487144 143243508+ fb VMFS

Disk /dev/disks/mpx.vmhba2:C0:T0:L0: 146.6 GB, 146685296640 bytes

64 heads, 32 sectors/track, 139890 cylinders, total 286494720 sectors

Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System

/dev/disks/mpx.vmhba2:C0:T0:L0p1 8192 1843199 917504 5 Extended

/dev/disks/mpx.vmhba2:C0:T0:L0p2 1843200 10229759 4193280 6 FAT16

/dev/disks/mpx.vmhba2:C0:T0:L0p3 10229760 286494719 138132480 fb VMFS

/dev/disks/mpx.vmhba2:C0:T0:L0p4 * 32 8191 4080 4 FAT16 <32M

/dev/disks/mpx.vmhba2:C0:T0:L0p5 8224 520191 255984 6 FAT16

/dev/disks/mpx.vmhba2:C0:T0:L0p6 520224 1032191 255984 6 FAT16

/dev/disks/mpx.vmhba2:C0:T0:L0p7 1032224 1257471 112624 fc VMKcore

/dev/disks/mpx.vmhba2:C0:T0:L0p8 1257504 1843199 292848 6 FAT16

Partition table entries are not in disk order

[manfriday]

OK.

It looks like I got things working again thanks to this lovely blog post:

http://www.yellow-bricks.com/2008/02/11/vcb-i-forgot-all-about-automount-disable-what-now/

Mr. Epping, you are the wind beneath my wings, brother..