VM startup slow, long delay before guest OS starts booting

[rn701]

Evaluating VMWare to set up VM for legacy software maintenance. Set up a virtual machine, installed Windows 7 SP1 32-bit, everything works fine. Once started, performance is excellent.

The symptom is that when starting the VM, either from fresh boot or suspend, there is a significant delay before the VMware power on/startup screen appears. Once it appears, the OS starts booting (or the suspend state is resumed) quickly and performance is normal after that. During the startup delay, the entire system is unresponsive, both host and guest OS.

See the the log excerpt below at time 09:54:28.619 where the delay occurs followed by the entry for time 09:55:11.625 when startup resumes, a 43 second delay during which nothing seems to happen and the system is unresponsive. After that, everything is fast.

Any ideas or suggestions?

More info...

VMware Player version 6.0.1 build-1379776

Host PC: Dell Optiplex, Intel Core2 Quad PU Q9650 3.00GHz

Installed memory: 4.00GB (3.25 GB usable)

Boot disk C: Intel SSD, 180GB (where VMware is installed)

Data disk F: Seagate 1TB (where the VM resides)

Host OS: Windows 7 Professional SP1 32-bit

Guest OS: Windows 7 Professional SP1 32-bit

vmware.log excerpt:

2014-02-19T09:54:28.619-05:00| vmx| I120: VMMEM: Initial Reservation: 1442MB (MainMem=1024MB)

2014-02-19T09:54:28.619-05:00| vmx| I120: MemSched_PowerOn: balloon minGuestSize 104857 (80% of min required size 131072)

2014-02-19T09:54:28.619-05:00| vmx| I120: MemSched: reserved mem (in MB) min 128 max 2493 recommended 2493

2014-02-19T09:54:28.619-05:00| vmx| I120: PShare: enabled 1 adaptive 1 scanRate [16, 400]

2014-02-19T09:54:28.619-05:00| vmx| I120: MemSched: pg 590258 np 35890 anon 5159 mem 262144

2014-02-19T09:55:11.625-05:00| vmx| I120: MemSched: numvm 1 locked pages: num 0 max 605661

2014-02-19T09:55:11.626-05:00| vmx| I120: MemSched: locked Page Limit: host 638514 config 638208 dynam 613853

2014-02-19T09:55:11.626-05:00| vmx| I120: MemSched: minmempct 50 minalloc 0 admitted 1

2014-02-19T09:55:11.626-05:00| vmx| W110: NUMA: automatic VM sizing request ignored

vm config (.vmx) file:

.encoding = "windows-1252"

config.version = "8"

virtualHW.version = "10"

scsi0.present = "TRUE"

scsi0.virtualDev = "lsisas1068"

sata0.present = "TRUE"

memsize = "1024"

mem.hotadd = "TRUE"

scsi0:0.present = "TRUE"

scsi0:0.fileName = "BCDEVVM.vmdk"

sata0:1.present = "TRUE"

sata0:1.fileName = "E:"

sata0:1.deviceType = "cdrom-raw"

ethernet0.present = "TRUE"

ethernet0.connectionType = "bridged"

ethernet0.virtualDev = "e1000"

ethernet0.wakeOnPcktRcv = "FALSE"

ethernet0.addressType = "generated"

usb.present = "TRUE"

ehci.present = "TRUE"

ehci.pciSlotNumber = "35"

sound.present = "TRUE"

sound.virtualDev = "hdaudio"

sound.fileName = "Speakers (SoundMAX Integrated D"

sound.autodetect = "FALSE"

mks.enable3d = "TRUE"

serial0.present = "TRUE"

serial0.fileType = "thinprint"

pciBridge0.present = "TRUE"

pciBridge4.present = "TRUE"

pciBridge4.virtualDev = "pcieRootPort"

pciBridge4.functions = "8"

pciBridge5.present = "TRUE"

pciBridge5.virtualDev = "pcieRootPort"

pciBridge5.functions = "8"

pciBridge6.present = "TRUE"

pciBridge6.virtualDev = "pcieRootPort"

pciBridge6.functions = "8"

pciBridge7.present = "TRUE"

pciBridge7.virtualDev = "pcieRootPort"

pciBridge7.functions = "8"

vmci0.present = "TRUE"

hpet0.present = "TRUE"

usb.vbluetooth.startConnected = "FALSE"

displayName = "BCDEVVM"

guestOS = "windows7"

nvram = "BCDEVVM.nvram"

virtualHW.productCompatibility = "hosted"

gui.exitOnCLIHLT = "FALSE"

powerType.powerOff = "soft"

powerType.powerOn = "soft"

powerType.suspend = "soft"

powerType.reset = "soft"

extendedConfigFile = "BCDEVVM.vmxf"

gui.lastPoweredViewMode = "fullscreen"

uuid.bios = "56 4d d2 32 bb c9 b7 bd-f2 3c d2 46 59 e6 38 e3"

uuid.location = "56 4d d2 32 bb c9 b7 bd-f2 3c d2 46 59 e6 38 e3"

replay.supported = "FALSE"

replay.filename = ""

scsi0:0.redo = ""

pciBridge0.pciSlotNumber = "17"

pciBridge4.pciSlotNumber = "21"

pciBridge5.pciSlotNumber = "22"

pciBridge6.pciSlotNumber = "23"

pciBridge7.pciSlotNumber = "24"

scsi0.pciSlotNumber = "160"

usb.pciSlotNumber = "32"

ethernet0.pciSlotNumber = "33"

sound.pciSlotNumber = "34"

vmci0.pciSlotNumber = "36"

sata0.pciSlotNumber = "37"

scsi0.sasWWID = "50 05 05 62 bb c9 b7 b0"

ethernet0.generatedAddress = "00:0C:29:E6:38:E3"

ethernet0.generatedAddressOffset = "0"

vmci0.id = "1508260067"

vmotion.checkpointFBSize = "134217728"

cleanShutdown = "TRUE"

softPowerOff = "TRUE"

usb:1.speed = "2"

usb:1.present = "TRUE"

usb:1.deviceType = "hub"

usb:1.port = "1"

usb:1.parent = "-1"

sata0:1.autodetect = "FALSE"

tools.syncTime = "FALSE"

unity.wasCapable = "FALSE"

tools.remindInstall = "FALSE"

toolsInstallManager.updateCounter = "1"

checkpoint.vmState = ""

monitor.virtual_mmu = "automatic"

monitor.virtual_exec = "automatic"

usb:0.present = "TRUE"

usb:0.deviceType = "hid"

usb:0.port = "0"

usb:0.parent = "-1"

usb.generic.autoconnect = "FALSE"

serial1.present = "TRUE"

serial1.fileName = "COM1"

floppy0.present = "FALSE"

[wila]

Hi,

One of the things that VMware does for your virtual machine on booting is to write out a memory backed file to disk. In other words, during that 43 second delay I suspect that your host OS is busy with writing that file out to disk. As the guest then needs access to about 1350Mb of memory (1024Mb guest OS+256Mb 3D memory+overhead) it will also swap out any memory on the host OS that is currently using the 1350 memory the guest OS needs.

So if you want more speed on boot, you'll either have to:

- be able to have more memory in your host OS (tricky with a 32 bit system and almost at 4GB)

- free as much memory in use as you can before booting

- move the VM backed memory file to a faster disk, for example by moving the whole VM to the SSD

hope this helps,

--

Wil

[rn701]

Thanks. Getting a new machine for this project and some other stuff with Windows 64-bit and 24GB. Hopefully that will help, will see what happens.

[wila]

Hi,

That should make a huge difference.

Being memory constrained like you are now, is one of the worst bottlenecks you can have causing the VM to slow down and causes unexpected moments with everything slowing down while the host is swapping for more memory.

--

Wil

[AD77]

In my original post - Re: VMWare Workstation 10 - slow load before "boot" starts

I forgot mention that I use dedicated new 64GB SSD disk to save Virtual Disk + Memory file/swap. So I don't think there is a problem.

I try to look deeper into problem with Sysinternals tools ( Windows Sysinternals: Documentation, downloads and additional resources )  and found up, it has something with interrupts (heavy % CPU load).

Or, can be something wrong set-up in BIOS of my computer? Any ideas?

[wila]

Hello,

Sysinternals has many tools, I think you are referring to the Process Explorer?

Writing to disk triggers interrupts, without any details on what interrupts you are seeing it is difficult to comment on that. Also heavy CPU load doesn't tell me much besides the comment, busy doing what? Which process is running at a high percentage?

Please also beware that while it is a pretty decent CPU, the Q9650 is from 2008 which isn't helping for virtualisation either.

If I'm reading your SSD remark correctly you now moved the VM to a dedicated new 64GB SSD and are still experiencing the same delay?

re. BIOS maybe your VT-x support is disabled? If so please be aware that upon changing it to enabled you will have to completely remove power before the motherboard will actually reward the change. IIRC though, VMware would popup a warning if it needs the setting and while slower if having to use binary translation I would not expect to see your specific problem.

Anyways, it never hurts to verify/check, more info on that is here: Trouble-shooting Intel VT-x Issues

--

Wil

[wmarusiak]

Hi,

I had the same issue with my vCenter VM. There was ISO attached to VM and my Kaspersky Antivirus was apparently scanning it during every boot. After disconnecting CD I don't have issues with slow boot of VM.

[rn701]

That was a different person, I think. But sounds like we are having a similar problem.

[rn701]

Thanks for the suggestion. I checked to make sure there was no ISO file attached. I have the CD mapped to the host physical CD and there's no CD mounted.

I also excluded the VM folders and subfolders from Norton virus scanner.

I also turned off the virus scanner and firewall.

None of that helped.

I purchased a license, same result.

I called support, and they said this is expected behavior and there is no problem.

When running a VM under Player Plus (version 6), I get a black screen for about 42 to 43 seconds, the entire system is slow and unresponsive, then the VMware "power on" screen appears briefly and the OS starts booting. After that it is very fast.

Running a similar VM using Windows 7 Virtual PC takes about six seconds to begin loading the OS.

Is the VMware startup delay indeed normal? I am curious how long it takes for others to start a VM, from the time you start the VM until the OS starts to load?

[wila]

rn701 wrote:

That was a different person, I think. But sounds like we are having a similar problem.

Oh yes, sorry about that, didn't notice that it wasn't you, so mixed things up in my reply.

But in my defense you did also post a question for help in the thread AD77 had posted about the similar issue

--

Wil

[rn701]

Yes, sorry about the cross posting. Thought I searched...

[wila · Accepted solution]

Hi,

I know what you mean as I have seen it occasionally as well. But mostly on older hardware or if there are problems with regards to particular resources. For example I saw this on an ESX host (yes I know different product) where there was a problem with storage not supplying enough IOPS.

I have also seen this on older versions of Workstation under linux (IIRC version 7) where the host had an atom based CPU, there I updated to more RAM and a newer version of Workstation and the problem disappeared.

Then I have also seen it on Fusion 4/5 when my host OS was running out of memory and I was trying to boot yet another guest OS anyways.

No I do not consider it normal behavior for the boot to be slow and unresponsive, it is an indication that the hypervisor is trying to get some resources but is not getting them quick enough.

The reasons for that can be numerous. Indeed things like an antivirus might be biting you if it tries to scan all files (including virtual disks) upon opening them.

FWIW, I also understand the answer from support as it might be expected if your host is a little thin on resources (eg. trying to boot a 1GB VM on a Win7 host with only 3.5 GB RAM) and if the VM runs OK after the initial boot delay, it is mostly just an annoyance.

--

Wil

[wila]

no worries about it, I understand.

[rn701]

OK, never mind. My apologies for wasting everyone's time.

I installed the VMware Player on a notebook with a Core I-5 3320M @ 2.60 GHz, 8GB RAM (7.7 GB usable) and and SSD.

I copied the VM from the desktop to the notebook.

The VM launches and starts booting the guest OS almost instantaneously, and the OS boots in about 15 seconds to the login prompt.

I am now a happy camper, and will be a much happier camper when I get my new desktop for this project.

Thanks to everyone for your suggestions and help.

[AD77]

So, finally I have understood what's going on more deeply.

On my computer I observed this problem -- increasing "before" load time -- from two previous versions of VMware Workstation. On v8 is was quite short, at v9 longer and with latest v10 much more longer.

So for me this is the final answer.

// my computer HW is also from ~ 2008-2009

Thanks to all.

[bonnie201110141]

I can reproduce it internally. Now we have a bug to track it. Thanks for your post!

[wila]

Glad to hear you got it resolved and for letting us know what fixed it for you.

A problem was there, so you did not waste anybodies time in my opinion.

--

Wil

[wila]

bonnie201110141 that's interesting, thanks for testing and tracking the bug. I would be interested to hear what causes this.

AD77 from the sounds of bonnie's reply it isn't final yet. The pattern you are describing that it takes longer to boot on each higher version of  Workstation is peculiar. Can't say that I have seen that.

One thing that changed over time is the virtual hardware and what -if your time permits- might be worthwhile in that case is trying to create a machine with older virtual hardware. Eg. if you can reproduce this with any type of guest I would personally probably test this with a linux CD as it is very quick to setup. Set it up with virtual hardware only compatible with workstation 8 and test if it boots faster.

--

Wil

[_ZAB_]

Any news here? The bug start to appear on 10.0.7 and it is the last version for 2003 server host. It worked just fine on previous versions.