Windows VMs lost disks?

[stevebnz]

So, I have an odd one:

I'm setting up a small business virtual environment.  Right now, I only have 2 servers involved:

Disk: 2012 R2 physical server that has a Dell SCSI array attached to it.  It shared out those disks over iSCSI.  I also have vSphere installed on there (not optimal, but it will work for a while)

Host: ESX 5.5 host with small business licensing on it.  I have 2 VMs spun up, both 2012 R2, one a DC and the other a terminal server.

The Host machine sees the Disk's arrays via a cross over cable.  Not great - but it works.

It was all setup and working fine, with the host looking at the iscsi disks on the Disk server, and the VMs running on the host.  Then, I'm guessing we had a power outage at some point?

The disk server came back up fine.  It sees the SCSI disks and has no errors.

The host came back up, but the VMs did not.  They didn't have auto boot turned on.  I booted them by hand.

My problem: both my VMs came up saying that the operating system can't be found, like the disk is missing?

- The disk is there an attached.  I verified in vsphere and the config file

- The iscsi disks are mountable and I can browse and view all the files just fine

- I removed the VMs from inventory and readded, same problem.

- I removed the hard disks and readded them, same thing.

Its almost like both VMs lost their boot partitions or something?  I boot the VMs into their BIOS, and they don't see the disks, even though they are attached.

Any thoughts on how to fix this?  I just started, and could start over, but I"m concerned I have a bigger issue.  I can't have this happen once these are in production.

Thanks!!!

[CoolRam]

For window Vm the controller is "LSI Logic parallel/LSI Logic SAS" . Can you change the controller and boot the VM again. This Might fix the problem.

[garish22]

If its 2012 R2, then its LSI Logic SAS only.

[stevebnz]

Yeah, I tried that too - it was set for SAS, I changed it to parallel just to try, and then changed it back.  Neither way worked, still, both VMs come up and say No Operating System Found.

I'm trying to see if I can figure out what the piece is that maps the VMDK to the VM.  Maybe even past that - where do the boot sectors of the VMDK live?  Did they get lost or corrupted somehow?

[continuum]

Boot the VM with your favorite LiveCD.
Check if the partition is flagged as bootable. For Windows 7 and later that usually is the 100 MB systempartition.
If the partition is bootable next check with "testdisk" wether the NTFS partitionbootsectors are ok ?

Also check the VM configuration - the disk either has to use SCSI0:0 or SCSI1:0 ....
If it uses something like SCSI0:1 then boot sequence in BIOS has to be edited.
By the way - SCSI-disks do not appear in BIOS like IDE-disks.
You only can check the SCSI-disks in BIOS under the boot-sequence tab.

[stevebnz]

10-4, let me work on it.  Thanks for the reply. 

The disks are setup right now to be 0:0 by the way.  I changed to 1:0 just to try, and no fixie.

Thanks!

[stevebnz]

So I tried with Ubuntu and with WIndows, and yes, it sees the disks as there, but sees them as empty. 

When I go through the windows installer, and I choose custom, it sees a 60 GB disk, which is correct, but sees it as being 100% free, which is not.  Ubuntu sees things similarly.

What would make my VMDKs go completely blank?  Are they blank?  Is there a pointer that got broken somewhere?

I won't lie - I'm really concerned this happened.  I can't have this system die in production....

[stevebnz]

So, I'm doing some digging, and it looks my VMs only have 1 .vmdk file in their locations.  On the 1 VM, I have the following:

server.vmdk

server.nvram

server.vmx

server.vmxf

server.vmsd

server.vmx~

server.vswp

server.vmx.lck

server.vswp (there are 2 of these, one is above)

and a bunch of log files

Is this everything it needs to run?

[stevebnz]

One last spamming of something I tried - I copied the VDMK and built a new VM with it set as the disk - same issue.

My VMDK's broke somehow.  Any ideas on how to fix them?

[continuum]

Next troubleshooting step is to find out wether you have corrupted vmdks or blank new ones.
So far we only know that they are not bootable any more which in itself may be completely harmless.

Please use a linux LiveCD and have a look at the partitions of the disk.

Another possible cause for this may be the fact that the original - working - vmdks had snapshots and that you are now trying to use the vmdks without them.
A vmware.log from back in the days where everything was fine would help.

[stevebnz]

So, I'm no Linux expert - but I actually don't see any partitions.  When I browse to /dev, I see no SDA1 files or anything.  Can you tell me what I'm looking for in detail?

I attached the oldest log I had.  I think it might be after the catastrophe, but I don't see an older one.

[continuum]

Did you test the attached file ? - seems to be empty.

The result of fdisk -lu or a screenshot of gparted would help.

[stevebnz]

Wow, you are right, thats was big stupids.  My bad.

fdisk replied with nothing, so I rand gparted.  It tells me that

Partition - Unallocated

File System - Unallocated

Size - 60.00 GB

Used - '-'

Unused - '-'

flagged - '-'

[continuum]

Ok - so we dont have to assume that you lost a snapshot.
Next I would check with testdisk if there has been a partitiontable on that disk before.

If fdisk "replies with nothing" you may need to use it as root - depending on the LiveCD you use can either switch to the root account or use sudo.

sudo fdisk -lu
sudo testdisk

[stevebnz]

OOOO!  GPARTED finally finished.  I thought it hung:

(I have to type this - its being stupid)

(gpartedbin:5497): Glib-CRITICAL **: Source ID 7 was not found when attempting to remove it

     The same repeats for 6, 10, 9, 13, 12, 16, 15, 19, 18, 38, 37, 48, 47, 53, 52, 57, 56, 60, and 59

/dev/sda: unrecognized disk label

I'm trying to install testdisk - but I can't find it and it will take me a while to dig through my scary linux skills to do it.  Give me a bit.

Thanks for the patience!!!!

[continuum]

http://sanbarrow.com/iscsiworkshop/moa64dvd.iso

Thats a UbuntuLiveCD that I created myself for this kind of work. Has everything you need

When you boot that and run this command
dd if=/dev/sda bs=1M count=200 | hexdump | less
and this displays only zeroes then you have a blank vmdk

If it displays lots of cryptic looking stuff  then you have a corrupted vmdk

[stevebnz]

OK - I got fdisk to work:

Disk /dev/sda: 64.4 GB, 64424509440 bytes

2565 heads, 63 sectors/track, 7832 cylinders, total 125829120 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O Size (minimum/optimal) : 512 bytes / 512 bytes

Disk identifier: 0xaa3caa3c

Disk /dev/sda doesn't contain a valid partition table

[stevebnz]

Sorry, cutting and pasting isn't working so well:

0000000 0000 0000 0000 0000 0000 0000 0000 0000

*

00001b0 0000 0000 0000 0000 aa3c aa3c 0000 0000

00001c0 0000 0000 0000 0000 0000 0000 0000 0000

*

c800000

200+0 records in

200+0 records out

209715200 bytest (210 MB) copied, 1.40556 s, 149 MB/s

[continuum]

Hi Steve
quite unexpected results !
That looks almosr blank
There is something really strange going on here - if you are interested lets have a closer look via Teamviewer.
Call me via skype "sanbarrow" to arrange something.
Right now I do not know what to check next - but hopefully that will easier sitting in front of the problem ...

Ulli

[kuenzir]

I have the same problem guys. Did you find a solution? With Diskpart Windows can't see the disks anymore.