JoeBlogs2011101
Contributor
Contributor

Error on rebooting VM after image deployment- black screen with solid cursor

Hello,

I have a really frustrating issue that I have tried everything I can think of to fix, but I haven't gotten anywhere.

Some details:

-I use VMWare Workstation 7.1 installed on Windows 7 Professional 64bit

-The client image I am deploying to my virtual machine (VM) is Windows XP Professional 32bit

-To deploy images we use an open source system called FOG

I maintain a standard operating environment SOE) for our organisation, which at present is Windows XP based. Up until recently, the process I have used to work-on/make changes to our SOE is as follows:

-Deloy the current SOE image to my virtual machine

-Make whatever changes are required

-Image the VM back to the imaging system

Up until recently this has worked perfectly.

A few weeks ago I recieved a brand new office machine, with a fresh installation of Windows 7. I reinstalled all my normal applications including VMWare Workstation 7.1. I then created a VM to work on my SOE, and imaged the VM with the SOE image (as I have always done).

The imaging process went fine, and the normal sysprep / minisetup stuff happened post imaging. Note that as part of this process the VM reboots several times, and each time it booted up again with no issue. After all post imaging tasks were finished, I was presented with the windows desktop and began making changes as normal.

The problem became apparent on the first reboot of the VM. Instead of booting into windows, the screen is black with a single, solid (non blinking) cursor in the top left corner of the screen).

At first, I assuemd that it must have been a windows update or some other change I had made to the VM. I simply reimaged the virtual machine with the same image as before. This time however, when I was presented with the Windows desktop post imaging I manually rebooted the VM to confirm all was working. This worked fine, and the VM rebooted normally. Thinking that I had arrived at a 'known good point' I created a snapshot of the VM that I could revert back to in the event of further problems.

I then proceeded to install some windows updates on the VM, which required a reboot. When the VM tried to boot into Windows the same issue appeared- solid cursor on black background. Thinking that an update must have cuased the issue, I reverted back to the known good snapshot (that as mentioned, I was able to mannually reboot without issue).

Once the snapshot had been reverted, I decided to manually reboot the VM just to be sure. Imagine my surprise and annoyance when it had the same problem- solid cursor on black screen and no booting into Windows!

I ahve already done some searching and have come across a few links related to this issue, and I have tried what they suggest such as using the fixmbr and fixboot commands in the windows recovery console. Since some people suggest the issue is related to the way that some imaging system don't record/ignore mbr information, I have also tried manually creating a mbr on the VM disk before imaging.

Can anyone suggest anything else I can try? and can anyone explain why my 'known good' snapshot had the same error after I reverted to it? Or why my VM is able to properly reboot several times during the sysprep/minisetup process 100% of the time, yet I get errors when rebooting manually and/or as part of software / windows update installation.

Many thanks for any help.

0 Kudos
21 Replies
WoodyZ
Immortal
Immortal

What CPU did the old physical computer have and what does the new on have?

I'm going to ask this just so I'm clear about what you've said... If you drop the SOE Image on a new VM and let it go through normal process to finish the sysprep install and once back at the Desktop and without making any other changes if you reboot again you never see anything other then a black screen and solid cursor in the upper left hand corner?  If yes does this happen whether or not you do a shutdown and then start the VM again or just a normal reboot?

0 Kudos
JoeBlogs2011101
Contributor
Contributor

Hi and thanks for the reply,

What CPU did the old physical computer have and what does the new on have?

If by 'physical computer' you mean the computer that VMWare Workstation is running on, my old PC had a Intel i5 750, and my new PC has an AMD

1055T CPU (I think I see where you are going with this- I didn't consider the fact that we have changed CPU brands. I will look more into this).

I'm going to ask this just so I'm clear about what you've said... If you drop the SOE Image on a new VM and let it go through normal process to finish the sysprep install and once back at the Desktop and without making any other changes if you reboot again you never see anything other then a black screen and solid cursor in the upper left hand corner? Sometimes- and this is the annoying thing. It isn't consistent. One time, as mentioned, I had rebooted from desktop without making any other changes and it rebooted fine. Other times, I have done exactly the same thing and it hasn't rebooted.  If yes does this happen whether or not you do a shutdown and then start the VM again or just a normal reboot? Yes, it doesn't matter if I've chosen to do a reboot, or shutdown and start the VM.

Another point to note- the very same image that I'm having trouble with on VMs works perfectly on all our physical machines (we have a single image that is deployed to 6 or so different model computers, with appropriate driver sets for each model). One thing they all have in common- all are Intel based.

Many thanks for your reply

0 Kudos
JoeBlogs2011101
Contributor
Contributor

Some additional information:

  • Yesterday, I checked the bios settings on my host machine that is running VMWare workstation. I noticed that the processor virtualisation setting was off, so I turned it on.
  • I then created a new VM and imaged it with the same image that I have been trying to use the last few weeks
  • The imaging process went fine, and the first thing I did when it completed was to manually reboot the VM. It rebooted fine
  • I then started to make changes to the VM- installing windows updates, installing new applications etc. As part of this I would have rebooted the VM at least 5 times throughout the day- each time it rebooted perfectly.
  • My host machine remained turned on overnight- it doesn't use suspend or hibernation or anything like that. The VM was also left 'on' as I was in the middle of working on it when I went home from work.
  • Today, I started using the VM again. I manually rebooted it to check something...and the problem is back! Rather than rebooting and loading windows, it displays the same solid cursor on a black screen striaght after the vmware splash screen
  • I have autoprotect turned on for this VM, so I tried a few different snapshots and all have the same issue!

Anyone have any ideas? This is really frustrating- yesterday it looked like the problem was fixed.

0 Kudos
continuum
Immortal
Immortal

you have autoprotect enabled and left it runnig over night ?

ok - that makes more sense now - probably you have a very large amount of snapshots ???

please attach latest vmware.log - files and a directory listing

Do you need support with a recovery problem ? - send a message via skype "sanbarrow"
0 Kudos
JoeBlogs2011101
Contributor
Contributor

Hi,

Yes, I didn't realise that autoprotect could cause problems? I had it set to take up to 10 snapshots every half hour, in case of power outage (which happens here sometimes). I don't really need them at the moment though so I've turned off autoprotect and have removed the snapshots.

Directory listing is as follows:

08/03/2011  12:05 PM    <DIR>          .
08/03/2011  12:05 PM    <DIR>          ..
08/03/2011  12:05 PM             8,684 SOE.nvram
08/03/2011  12:05 PM                 0 SOE.vmsd
08/03/2011  12:05 PM             2,985 SOE.vmx
08/03/2011  11:53 AM               258 SOE.vmxf
08/03/2011  12:04 PM            77,144 vmware-0.log
08/03/2011  12:01 PM           100,182 vmware-1.log
08/03/2011  11:58 AM            77,118 vmware-2.log
08/03/2011  12:05 PM            77,046 vmware.log
08/03/2011  12:05 PM            15,388 vprintproxy.log

And I store the Virtual disks on a secondary hard drive in my machine as I use a smaller SSD for my system drive:

E:\VMware disks>dir
Volume in drive E is New Volume
Volume Serial Number is 5607-875A

Directory of E:\VMware disks

08/03/2011  12:06 PM    <DIR>          .
08/03/2011  12:06 PM    <DIR>          ..
08/03/2011  11:58 AM    42,949,672,960 SOE-flat.vmdk
08/03/2011  11:58 AM               478 SOE.vmdk
               2 File(s) 42,949,673,438 bytes
               2 Dir(s)  916,998,041,600 bytes free

Attached is my vmware.log file.

Many thanks for your help.

0 Kudos
WoodyZ
Immortal
Immortal

in case of power outage (which happens here sometimes)

That is what a UPS is for! Smiley Wink

I don't really need them at the moment though so I've turned off autoprotect and have removed the snapshots.

From a diagnostic/troubleshooting perspective it would have been better to have a file listing and vmware.log from before you removing especially if you have been using the self destructive AutpProtect feature all along! Smiley Wink

Also on a totally separate issue, why are you using a monolithicFlat .vmdk, is it that you have no choice because of FOG?  (Which is why I use Symantec Norton Ghost.)

0 Kudos
JoeBlogs2011101
Contributor
Contributor

We have plenty of UPS here for the servers and network core, but none for the machine in my office :smileycry:

Yes, sorry about that. I was trying a few thing myself and as part of it I deleted the autoprotect stuff.

Noe sure what you mean about the flat.vmdk. I just use the recommended options when creating a new Windows XP VM and that is what VMware choses for the disk. Nothing to do with FOG- FOG is actually really good and able to cope with everything we need (and I say this as an ex-ghost user).

Thanks again for the help everyone.

0 Kudos
WoodyZ
Immortal
Immortal

We have plenty of UPS here for the servers and network core, but none for the machine in my office :smileycry:

I know at work you may not have any say in the matter however. ...

I've seen cases in severe surges companies loosing considerable numbers of computers and related devices because of not using UPS and Surge Protectors and IMO using a Computer without using a UPS and Surge Protector whether built-in or separate surge protector in the case of a Notebook which of course already has a battery is at a minimum plain ignorance or beyond that just outright stupidity considering how inexpensive they are vs. the cost of replacing hardware from an unprotected surge or lost time due to data lose or having to restore software, etc over the life of the system and or UPS and Surge Protector.  Granted that really large companies playing the odds just protecting the servers, network core and data and rolling the dice with User's Desktop systems is probably more cost effective for the normal users however considering what you're doing they should be providing you with one. Smiley Wink

Yes, sorry about that. I was trying a few thing myself and as part of it I deleted the autoprotect stuff.

I guess the question should be, are you having this issue without using AutoProtect?

Noe sure what you mean about the flat.vmdk. I just use the recommended options when creating a new Windows XP VM and that is what VMware choses for the disk. Nothing to do with FOG- FOG is actually really good and able to cope with everything we need (and I say this as an ex-ghost user).

How much disk space is used on the virtual C: Drive, assuming the .vmdk just has one partition?

I do have and use pre-allocated (monolithicFlat) .vmdk's however that typically is only on my primary production VM's used under VMware Workstation and VMware Fusion however an image being made and used for the express purpose to image physical machines from then I wouldn't be using a pre-allocated disk! Smiley Wink

0 Kudos
JoeBlogs2011101
Contributor
Contributor

Yes, UPS for my office would be nice. Maybe next budget... Smiley Wink

I thought you might ask if the problem exists without autoprotect. The answer is yes. With autoprotect turned off, I created a brand new VM and imaged it and started to make changes. I rebooted the VM a few times for various things with issue. I then powered off the VM as a test. When I powered the VM back on, the issue was there again.

The VM disk I use is a pre-allocated 40GB in a single file.

I mainly use pre-allocated disks for two reasons:

-From what I've seen, performance is better in some circumstances when using a pre-allocated virtual disk rather than a dynamically growing one. I espeically notice this when I pull down an image from our image server to a VM.

-Hard drive space is cheap these days :smileylaugh: I've got 1TB for VM usage, so one or two 40GB VM disks hardly make a dent.

I'm going to try VirtualBox to see if the issue exists there too.

0 Kudos
WoodyZ
Immortal
Immortal

You didn't answer my question... How much disk space is used on the virtual C: Drive, assuming the .vmdk just has one partition?

0 Kudos
JoeBlogs2011101
Contributor
Contributor

Sorry, I misread your post.

The virtual disk is just one partition of 40GB, and of that around 23GB is used (the image contains many different software packages that we have site licenses for).

0 Kudos
WoodyZ
Immortal
Immortal

Thanks, I was really just curious and I don't think that is the cause of the issue.  BTW How big is the FOG Image that you drop on the physical machine?

0 Kudos
WoodyZ
Immortal
Immortal

I have been thinking about this issue in the back of my head and I just don't have anything concrete at the moment other then to ask/say. ... I assume that you did not have this issues on your old computer at all, correct?

If you didn't then here is what I'd do before doing anything else regardless of how new the computer is and that is first, at a minimum, boot the physical machine with an appropriate diagnostic disk and perform software level diagnostics on the RAM and preferable actually do it in a hardware memory tester if available and if not be sure to switch memory around in its banks if two or more are being used.  I would also then use the manufactured supplied diagnostics on the physical HDD's and additionally if the manufacture like Dell's have run full diagnostics on the hardware.

Once this is done and if all checks out then I'd run MD5 or SHA1 checksums one given source test file of size equivalent to the files being transferred between physical disks and over the Network and validate the MD5 or SHA1 checksums with each transfer from point to point regardless of the fact you're not getting any error messages since you haven't mentioned getting any during these types of transactions.  I would probably write a script to automate this MD5 or SHA1 checksums testing process so it would run a series of tests between all disks concerned and at a minimum several times and this could run after work and or during the night so as not to interfere with normal network traffic.  All of this would at least rule out these areas that can cause issues even when not directly giving hard errors under normal use and because of using a new system and I'm assuming didn't have these problems with the old computer.

That's all I can think of at the moment.

JoeBlogs2011101
Contributor
Contributor

No worries. The FOG image when stored on the FOG server is around 12GB. As mentioned, its a pretty impressive system Smiley Happy

Yes, fair call on checking the ram and hard drive(s). Before asking for help on here I have done a reasonable amount of testing myself and in the back of my mind I was starting to wonder if it could be that sort of issue. And yes, no issues on the old system.

One other possibility that was hinted at in a previous post was the fact that my previous system was an intel based machine, and my new system is an amd based one. I haven't had much luck in finding out if that could be the cause of the issue though.

Once again, thanks for all the replies/suggestions.

0 Kudos
WoodyZ
Immortal
Immortal

One other possibility that was hinted at in a previous post was the fact that my previous system was an intel based machine, and my new system is an amd based one. I haven't had much luck in finding out if that could be the cause of the issue though.

Yes I asked about that and I've never had anything but Intel Processors since using virtualization so can't do any testing however I believe this is really only an issue with Linux VM's not Windows XP however I like knowing what was different between the old physical system and new one so that is why I asked originally.

Thanks for the FOG related info and I've started am looking at it, since you mentioned it in this thread, as a replacement for Ghost and GhostCast Server for my Clients that can't afford the Symantec Solution.  Although I've use Ghost for so long and it is so reliable that it might not be cost effective for the Clients in the long run.  Will have to implement a FOG Server in my Office and put it through its paces to see what the real cost differential will be with my time that the Client is paying for.

0 Kudos
JoeBlogs2011101
Contributor
Contributor

Yes, my own reading led me to some Vmware knowledgebase articles mentioning the issue with Red Hat Linux VMs, due to the way the kernal is compiled on installation to suit a specific architecture.

I've used most of the well known open source imaging systems (such as PING, Clonezilla and of course FOG) and FOG is by far the most feature rich and polished. Plus, you don't need expensive hardware either. Our current FOG server is actually an intel atom based system with 1GB ram and consumer grade 500GB sata drive running Ubuntu 10.04 Server. With those specs I can reimage 30-40 machines at once in around 8 minutes (via multicast).

Unless Ghost has changed a lot in recent years, FOG actually offers many features that Ghost doesn't. Anyhow, I'm obviously a fan :smileylaugh: but check out the website for further info:

http://www.fogproject.org/

0 Kudos
WoodyZ
Immortal
Immortal

Ghost is not the only imaging product I've used although it's been the primary one under Wintel and even on my MacBook Pro Boot Camp partition however I've been using Ghost since its creation by Binary Research years before Symantec purchased it from them so I have a well established track use and mythology behind its implementation however I'm always willing to look at and test new and different solutions. Smiley Happy

0 Kudos
JoeBlogs2011101
Contributor
Contributor

Okay, a further update to this:

Yesterday, I ran memtest on my host machine (it took a while to get through 12GB of ram!). No errors though.

I've completely uninstalled VMWare workstation and reisntalled it, and updated to the latest version (7.1.3 build-324285).

I've also done all possible Windows updates for my machine, including Service Pack 1.

I created a new VM, and this time I used the dynamic disk rather than pre-allocated. Speed wise it is much better than it was in old version of workstation (around v4 or v5 was last I tried).

Anyhow, I imaged the VM and started making changes, and sure enough after a few restarts it gave me the same old error. I decided to try something new and reimage the VM straight away after the error appeared. The image deployed to the VM fine, but as soon as the VM restarted itself to start the sysprep minisetup process the error with the non-flashing cursor was their straight away (whereas EVERY time I reimage a new VM it has no problems with any of the sysprep stuff- it is only a few reboots post-sysprep that the problem comes up).

Sigh Smiley Sad I'll keep trying more things and post back when I work out the issue.

0 Kudos
WoodyZ
Immortal
Immortal

Did you do any testing to validate MD5 or SHA1 checksums on large file network transfers?

Have you tried creating a totally new VM and installing Windows XP from CD or ISO Image and not the SOE Image and do any testing on it to see if you end up with the same results you're getting with the SOE Image?

0 Kudos