VM_a1
Contributor
Contributor

"Operating System not found" errors on multiple VMs

I'm currently running Workstation Pro 16.2.3 but was getting the same problem when using 16.2.1 and possibly early versions (it's been going on for 4 months now and I tried several versions during testing).

Essentially lots of my VMs, regardless of windows version installed (they're all Windows based VMs), have been failing to boot. When they haven't found the boot disk they've attempted a network boot before failing completely saying "operating system not found" or something similar depending on the windows version installed.

This is now affecting multiple VMs and I am unable to work. I thought it was a Windows update and have even tried a completely separate laptop. I've even taken clean copies of the VMs and had the same thing happen, so it seems to be Windows/ VMware issue but cannot pinpoint where.

I'm at a loss as to what else I can do and am desperate for any assistance anyone can offer.

Tags (1)
0 Kudos
40 Replies
bluefirestorm
Champion
Champion

I looked at just one of the vmware.log for this VM E:\SWAdmin71036-PCS7v81SP1\SWAdmin71036-PCS7v81SP1\SWAdmin71036.vmx

So it looks like the VM s stored on a WD 5400rpm external disk. Are all the other VMs that fails to boot on the same external drive?

What is the storage format of the external drive (NTFS, exFAT, FAT32)? Does it have NTFS compression? Best is to have NTFS without compression.

Assuming there is enough space on the laptops's internal storage, try copying one of the VMs there and see if it can boot up.

There are some other weird entries (weird meaning first I time I am seeing this)

2022-04-05T15:46:07.332Z Wa(03) vcpu-0 SMBIOS: Cannot find guest System Information structure.
2022-04-05T15:46:07.332Z Wa(03) vcpu-0 SMBIOS: Can't find guest System Enclosure structure.
2022-04-05T15:46:07.332Z Wa(03) vcpu-0 SMBIOS: Can't find guest Base Board Information structure.
2022-04-05T15:46:07.333Z Wa(03) vcpu-0 SMBIOS: Can't find OEM Strings structure.

These lines would suggest that the nvram file is corrupt.

So possible scenarios:
1 - External drive is slowly dying
2 - domain policy recently introduced not allowing to write to external USB storage. But this seems unlikely as there should be other errors such as failing to create and/or write files to the drive E:\ as the VM is powering up.

0 Kudos
SchaalPatrick
Enthusiast
Enthusiast

Tags (1)
0 Kudos
VM_a1
Contributor
Contributor

hi @SchaalPatrick and @bluefirestorm 

Just to clarify my laptop hardware/ storage. My original laptop has an M2 SDD C:\ drive. D:\ drive is an 2.1/2" HDD located instead of a CD. E:\ drive is an external USB3 drive

When the issue first started happening all my VMs were located on D:\. I then moved them to my external drive E:\ so I could move them to my laptop. But due to space limitations I couldn't copy them onto the internal storage so have been running them from E:\ drive.

I've extracted clean copies of the VMs to various locations but I cannot find a situation where they work.

So I've run the same VMs on 2 separate machines on 2 separate disks and been getting the same results. So this to me seems to rule out the hardware issues. Also free space has not been a problem on the drives in question.

I will do the chkdsk overnight as it will take a long time I'm sure. I've attached dir info requested of the VM folder. You will note there's 2 .old files in there (vmx file and nvram file). I renamed the nvram file just to see what would happen and it created new. The vmx file I copied from the clean VM to try. As you can see I'm getting deperate now....

0 Kudos
SchaalPatrick
Enthusiast
Enthusiast

0 Kudos
SchaalPatrick
Enthusiast
Enthusiast

0 Kudos
VM_a1
Contributor
Contributor

@SchaalPatrick I am not creating a new VM, I am restoring a zipped up backup of the original VM from when it was created. This was made maybe 3-4 years ago and every time I have used this back up previously it has worked fine. It's only more recently that it corrupts, reporting it cannot find the operating system.

As it wont boot I can't boot in Safe Mode or do a repair, unless you are talking about using a Windows Install disk/ iso to try and repair the VM. I did actually do this successfully with another VM and was able to get what I needed off before we deleted that VM.

Regarding the upgrade, no it was not upgraded but it was working fine until around December without upgrade. Then it and various other VMs have stopped as I can't see how the upgrade affects it. My IT don't know for sure what the upgrade does but they think it chnages a value in the vmx file but what the does within they don't know.

Coincidentally I got given a completely new laptop yesterday which had Workstation Pro 16.2.2 so I removed this and installed 16.1.1 instead. I then downloaded another clean copy of the same VM I've been using for testing and it too failed to boot.

As a further test I have just taken a copy of a colleagues WinXP based VM which he tested before and it was working fine. When I ran it it initially started booting fine showing the Windows splash screen before it BSOD'd. I've attached the BSOD screen shot, the vmware.log and also the vmx file (txt file...)


This is just getting stupid! I can't explain any of this anymore.....

0 Kudos
SchaalPatrick
Enthusiast
Enthusiast

Tags (1)
0 Kudos
VM_a1
Contributor
Contributor

Thanks @SchaalPatrick I'll try again, but I think it will just break again even if I was able to repair it...

0 Kudos
bluefirestorm
Champion
Champion

Looks like you now used Workstation Pro 15.5.7 with your colleague's working XP Pro VM and it BSOD'd. It was running on the local D drive storage.
So with that one test means it isn't a Workstation Pro/Player 16.2.x problem (your previous Win7 showed using Player 16.2.2). Even if it was the instability/problems from 16.2.x most of the reported with this version are hangs/graphics issues. But your situation is quite different and inconsistent. The Win7 VM BSOD (which progressed further beyong VM POST) and then ended up with what appears to the NVRAM problems and then unable to boot.

Problems occur regardless if it is 15.5.7 or 16.2.2.

The problem occurs on D local storage or E external USB storage. So it isn't an issue of domain policy restrictions on removable storage.

Which really leaves it might be a specific laptop problem (which seems rather unlikely as you should be seeing other weird problems probably with just using regular applications such as Office, email, or a browser).
Other possibilities would be if your domain account and/or laptop has some restrictions that your colleagues account/laptop don't.

The vmware.log files suggest you don't have VBS enabled through domain policy as it is still using the Intel native VT-x hypervisor.

By any chance are your drives D and E using compression? I have come across posts here where VMware Workstation/Player generally won't be happy with NTFS Compressed files/folders.

How about Bitlocker? Is it enabled? I am not aware of any possible conflict between Bitlocker and VMware Workstation Pro/Player but maybe this is one of those differences between your colleague's machine and your laptop.

0 Kudos
SchaalPatrick
Enthusiast
Enthusiast

0 Kudos
RDPetruska
Leadership
Leadership

Make sure you have a backup before you make those changes, as that will change the virtual hardware...  if your VM doesn't have the device drivers for those disk controller and network card devices, Windows will complain and potentially not boot.

0 Kudos
SchaalPatrick
Enthusiast
Enthusiast

Deleted by wila

Tags (1)
0 Kudos
bluefirestorm
Champion
Champion

SchaalPatrick,

To be honest, you are out of your depth here.

Virtual hardware version 10 is fine with his colleague, So it is pointless to change virtual hardware version.

The test VM is XP so changing the VM controller from buslogic to lsisas1068 is not appropriate.

You asked for vmx file when that can be seen in the vmware.log file. Or are you just wanting to give us some authentic offshore outsourced helpdesk experience that asks for things/items/information that have been provided? <Sarcasm>

 

continuum
Immortal
Immortal

> Or are you just wanting to give us some authentic offshore outsourced helpdesk experience that asks for things/items/information that have been provided? <Sarcasm>

Coud not have said that better 😎
Ulli


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

0 Kudos
VM_a1
Contributor
Contributor

Thanks @bluefirestorm 

You're right in that I downgraded to 15.5.6 to try and rule out 15.X to 16.X upgrade issues. This then updated to 15.5.7 but I denied the 16.x update.

I struggle to understand how it could be a specific laptop issue as I have been having similar problems across three separate machines.

Regarding domain restrictions I did discuss that with our IT chaps earlier today. We were talking whether my profile could be to blame, but as he pointed out, on the new laptops they have no profile so would only be downloading the domain part (I think that's what he said...).

I don't know what you mean by VBS enabled...

The drives are not compressed but they are bitlockered. But the've always been bitlockered even before the problems started.

0 Kudos
bluefirestorm
Champion
Champion

It has been a long time since I have had to mess around AD/GPO. I think a new laptop/PC won't necessarily have a full user profile stored locally. You can force a domain policy update by issuing gpupdate rather than waiting for it occur in the background (but I think gpupdate only works for machine policy). I don't know if there is some equivalent for user policy settings. I think some policy settings exists for both machine and user level and one can override the other (not 100% sure of this last statement). Anyway, your AD/GPO people should know better than me on this topic.

On the surface, the root of the problem does not appear to be VMware Workstation Pro/Player 15.5.x/16.2.x itself. Which really leaves to either settings on the host machine and/or your account profile causing all this grief.

VBS is Virtualisation Based Security which could turn on memory integrity checks. But this should not cause VMs crashing though; even if it somehow could, it should have been more consistent (not BSOD now and unable to boot later).

I also doubt if it is a specific hardware problem otherwise ordinary things like office, browser would also have problems, too. Chances are the browser and Office applications would not be touching the drive D or drive E. Which is what led to me think of compression and Bitlocker. There could be other things like such as ASLR or Ransomware protection folder access control, or antivirus drive/folder exclusions to look at and possibly compare these with your colleague's machine settings.

I guess the other thing to try that is if you and your colleague are physically located in the same office (somehow there is >50% chance you are not) is to have your colleague login to any of your 3 problematic machines and try with their working VMs. If they don't have any BSOD/boot problem, it could then be pointing something in your domain user profile that is the root cause.

0 Kudos
VM_a1
Contributor
Contributor

Sorry @bluefirestorm for the delayed reply. I lost the will to live last week so had a weekend off thinking about VMs considering it's work....

I've been finding more and more users in our company having similar issues so its clearly a wider problem. Interestingly on my 3rd laptop I got a colleague to log in and they were able to run a clean copy of a VM that failed repeatedly with me. So this does seem to point to a user based problem, i.e. profiles, rather than a machine/ Windows/ VMware compatibility issue.

I need to do some more testing but I can see we may need wipe my profile and start again. But this is not a 5 minute thing....

Do you know how we can contact VMWare directly to see if they can support me?

0 Kudos
bluefirestorm
Champion
Champion

I don't work for VMware so I don't really have a direct contact person from VMware to refer you to.

A possible alternative to wiping out your profile is create a second domain account to start off with a clean slate. I realise that this might require special management or even CISO/CTO approval/dispensation as it might be a IT security policy that an employee have only one account and exceptions will likely be IT admins. But this is probably a better alternative than wiping out the profile as in essence it is like wiping out the cause/evidence. A second account will also let your IT folks experiment with whatever settings that is needed. That should also let you keep using your current domain account to do other work stuff that is not development VM related. I think your manager or your manager's manager and further up the chain should back you up on this to get a second account considering that this problem has brought your productivity down to near zero; and potentially other colleagues as well.

0 Kudos
Technogeezer
Champion
Champion

<redacted after I read the rest of the thread and steps taken to date>

0 Kudos
VM_a1
Contributor
Contributor

@bluefirestorm Sorry for the delay in replying and just an update. I'm still no further in solving this but have manged to get to our IT who manage our licensing agreements, as without details of our accoutn I'm not able to raise a support ticket with VMware. Anyway, I hope this is now in and I should get some contact from VMware tech support at some stage.

If I get any solution I'll update the discussion but thank you in the meantime for your support.

I had hoped some VMware employees would be more forthcoming with their support in this forum but I appreciate this community is not a formal support route to VMware. So we'll see what happens via the official route.

Have a great weekend everyone.

0 Kudos