brianberliner
Contributor
Contributor

VirtualBox Trashes VMware Fusion VMDK Files [+Workaround]

I just installed the security update that was released today from Apple.

It said to "Restart", so before pushing that button, I went to my VMware box (running Windows XP) to suspend it first.

VMware gave me an error I had never seen before, saying that it couldn't find my VMDK file. VMware Fusion then shut down.

I restarted and the security update (it also installed Safari 3.1) finished it's installation.

But, my VMware Fusion virtual machine would not start.

Nor would ANY of my VMware Fusion virtual machines.

They all fail with "File not found" messages, yet the files are there, and when I select them, it just continues to tell me that the file is not there.

Is this just me?

There is lots of nastyness in the attached vmware.log file.

Help!

-Brian

0 Kudos
22 Replies
admin
Immortal
Immortal

I don't see a vmware.log file, try again?

What version of OS X is this? What version of Fusion?

When you say the files are there, do you mean the .vmdk files?

0 Kudos
brianberliner
Contributor
Contributor

One more time, hopefully with the vmware.log file.

This is on Mac OS X Server 10.5 (Leopard Server).

All latest patches applied from Apple Software Update.

Thanks!

-Brian

0 Kudos
admin
Immortal
Immortal

I still don't see a log file.

I just tried installing the security update (and Safari update) on my MacBook Pro running 10.5.2, and I'm still able to run virtual machines. Possible differences are that Fusion wasn't running when I ran the update, I'm running Leopard Client instead of Server, and I'm running a developer build of Fusion (i.e. different from yours). Something to try would be reinstalling Fusion - this won't affect your virtual machines.

0 Kudos
brianberliner
Contributor
Contributor

Thanks for doing a quick test. It may just be me, it sounds like.

I will re-install Fusion 1.1.1 and see if that helps.

-Brian

0 Kudos
brianberliner
Contributor
Contributor

No change for me after re-installing.

Off on holiday for a bit, then I'll try again.

-Brian

0 Kudos
admin
Immortal
Immortal

What type of Mac is this? Anything unusual about it? Copy/paste in vmware.log?

0 Kudos
brianberliner
Contributor
Contributor

I'm still on Holiday, so I don't have access to the vmware.log file right now, but here's some info about the machine & software:

- iMac 24" with 4GB memory (2.8Ghz Core 2 Duo, I believe)

- VMware Fusion 1.1.1 (latest)

- Mac OS X Server 10.5 (Leopard Server)

- Guest: Windows XP with 20GB virtual disk

- Guest was moved over with VMware Converter

- Guest has worked flawlessly for months, since this problem.

I do run VMware under Leopard Spaces, and put Fusion and the Guest in Space #2.

I also did recently (a week ago or so) install VirtualBox as part of testing something else. I have since deleted it, by dragging it to the Trash. Could VirtualBox have messed something up between the two?

I did attach the vmware.log, but it doesn't seem to be here. Will get it for you next week.

Thanks again for all your help. Looks like it's only my box having his problem.

-Brian

0 Kudos
ktula
Contributor
Contributor

Which vmdk file was Fusion complaining of being missing when you tried to start the VM?

0 Kudos
greenberry
Contributor
Contributor

I am also having the same issue. The upgrade to VM1.1.1 was easy and painless but the minute I tried to access the web I ran into problems. Now I just realized that I had implemented my airport upgrade yesterday and it worked with the previous version of VMware (1.1). So it really is not an issue with the my airport but with VMware. I am able to ping my gateway on XP...

0 Kudos
brianberliner
Contributor
Contributor

I dug into this a bit more today.

Looks like something bad has happened to the VMDK files for THREE of my Virtual Machines (the ones with large disks) on this system, but the other TWO VM's that I have run just fine (the ones with small disks, as it turns out).

Here's a snippet of the log file for my straight-up Ubuntu VM:

Mar 24 08:39:53.381: vmfusion| DISKLIB-DSCPTR: Failed to parse embedded descriptor file in normal mode: Wrong line format.

Mar 24 08:39:53.382: vmfusion| DISKLIB-LINK : "/Users/brianberliner/Documents/Virtual Machines/Ubuntu 6.06/Ubuntu.vmdk" : failed to open (The file specified is not a virtual disk).

Mar 24 08:39:53.382: vmfusion| DISKLIB-CHAIN : "/Users/brianberliner/Documents/Virtual Machines/Ubuntu 6.06/Ubuntu.vmdk" : failed to open (The file specified is not a virtual disk).

Mar 24 08:39:53.382: vmfusion| DISKLIB-LIB : Failed to open '/Users/brianberliner/Documents/Virtual Machines/Ubuntu 6.06/Ubuntu.vmdk' with flags 0x15 (The file specified is not a virtual disk).

Mar 24 08:39:53.382: vmfusion| DISKLIB-LIB : Failed to enum '/Users/brianberliner/Documents/Virtual Machines/Ubuntu 6.06/Ubuntu.vmdk' : 16

Mar 24 08:39:53.382: vmfusion| SNAPSHOT: Unable to find all files for 'Ubuntu.vmdk'

Mar 24 08:39:53.382: vmfusion| SNAPSHOT: SnapshotConfigInfoExpandDisks: Error 7

Mar 24 08:39:53.382: vmfusion| Missing virtual disk Ubuntu.vmdk of type disk

Mar 24 08:39:53.382: vmfusion| Missing VM file: Ubuntu.vmdk

The files are there, but Fusion thinks that the VMDK files are corrupted in some way.

As I mentioned earlier, I had briefly installed VirtualBox, but I don't get why the three large disk files are corrupted, yet the small ones appear to be fine.

Any help with the above errors on recovering my VM's?

Thanks!

-Brian

0 Kudos
aetm
Hot Shot
Hot Shot

Hi Brian,

Do you have a bakcup or copy of these files before they were played with Virtualbox? I would like to compare the checksums with the vmdk files on your hard drive and the ones one backup, to see if we find a difference there.

Are these vms in your internal hard drive or other hd or storage? How is the volume they are stored in formatted? What is the size of these affected VMs?

SnapshotConfigInfoExpandDisks: Error 7

Looks interesting. Did you have a snapshot before? It looks like it can't be found or even that is corrupted.

0 Kudos
brianberliner
Contributor
Contributor

Hi aetm,

Problem resolved (at least for the 1 VM that I use daily)!

It was, indeed, trashing of the VMDK file done by VirtualBox.

Unfortunately, the error codes and messages out of VMware Fusion did not really give me much guidance in figuring out where the problem might be, so I had to slog through it and try a bunch of things.

Here's the very complex steps that I took to recover the VMDK file, after trying numerous things and referring to the VirtualBox source code for hints.

Starting at byte offset 512 in my VMDK file, you will find a "Disk DescriptorFile" and a "The Disk Data Base" section of text that appears to be padded by NULL characters.

In the corrupted VMDK that was mucked with by VirtualBox, mine looked like the following:

  1. Disk DescriptorFile

version=1

CID=4c06b51e

parentCID=f6d5af3d

createType="monolithicSparse"

parentFileNameHint="AmyOffice.vmdk"

  1. Extent description

RW 41963828 SPARSE "AmyOffice-000001.vmdk"

  1. The Disk Data Base

#DDB

ddb.toolsVersion = "7362"

ddb.virtualHWVersion = "6"

ddb.uuid.image="309b24ab-4acf-4b11-cbbe-f6935882d848"

ddb.uuid.modification="12ff42d5-f691-4ba3-7abb-6686efa6d59f"

ddb.uuid.parent="00000000-0000-0000-0000-000000000000"

ddb.geometry.cylinders"0"

ddb.geometry.heads"16"

ddb.geometry.sectors"63"

ddb.geometry.cylinders="0"

ddb.geometry.heads="16"

ddb.geometry.sectors="63"

And, it was padded by NULL characters. I extracted it from the VMDK using "dd":

  1. dd if=AmyOffice-000001.vmdk bs=512 skip=1 count=2 > out

I didn't like the look of the entries: Note that there is not an equals sign between the cyclinders/heads/sectors section at the bottom and the values.

That did not sit well with me at all. Nor did I like the value of the cylinders was 0 and the heads/sectors did not match the binary values in the firs data structure after the MAGIC KMDV in the VMDK file.

Badness.

Comparing this text header info with a very old backup of my VMDK file, it appeared that I could drastically simplify this section.

So, I did.

I edited the "out" file, and made the following changes:

- Changed the CID to 613506eb

- Deleted everything after this line (kept it): ddb.toolsVersion = "7362"

- Extended the NULL padding at the end to get the file back up to 1024 bytes exactly.

Then, I patched the VMDK with the following command:

  1. dd conv=notrunc if=out of=AmyOffice-000001.vmdk bs=512 oseek=1 count=2

Don't forget the "conv=notrunc", or your VMDK file will be truncated (which I did, and had to copy over from backup again. Ugh).

And, guess what, the Virtual Machine booted with absolutely no problem.

I immediately grabbed the data that I wanted out of the VM, then created a fresh Snapshot.

All is well with my silly Windows VM now. I promise to implement a bettwer Windows VM backup process (it would be great if I could just let Time Machine handle this, but the VM is 30GB)...

Anyway, hope this helps someone else that gets bit by VirtualBox, as I did.

-Brian

0 Kudos
brianberliner
Contributor
Contributor

Marked as answered.

Please add any other VirtualBox coexistence issues here, please.

-Brian

0 Kudos
brianberliner
Contributor
Contributor

Attempting to change the Subject line to reflect the actual issue (VirtualBox)

0 Kudos
kleinfelter
Contributor
Contributor

There is another version of this problem. If you're using a vmdk with a separate data file and descriptor file (e.g. foo.vmdk plus foo-flat.vmdk), VirtualBox adds some lines to the descriptor file that break vmware-mount. If you delete the offending lines (that all begin with "ddb.geometry.bios", you can again vmware-mount the disk. Full details are at

0 Kudos
chriswolf
Contributor
Contributor

The problem with the "dd" commands offered by "[~225873]", (above) is that you only get the disk-descriptor section in the resulting

"cleaned" VMDK file, in other words, the file is truncated and unusable.

Alternatively, you can try these steps, which I put into a script. I probably would not have thought

of this without "[~225873]" "dd" editing ideas.

I tested the following shell script on a monolithic, growable (type 0) VMDK file which got trashed by VirtualBox:

#!/bin/sh
# Strip out VirtualBox settings from VMDK file
# Usage: $0 <vmdkfile>
#
dd if=$1 bs=512 count=1 of=vm.hdr && \
dd if=$1 bs=512 skip=1 count=2 of=vm.ddesc && \
cat vm.ddesc | grep -a -v ddb.uuid.image \
| grep -a -v ddb.uuid.modification \
| grep -a -v ddb.uuid.parent \
| grep -a -v ddb.uuid.parentmodification \
| grep -a -v ddb.geometry.biosCylinders \
| grep -a -v ddb.geometry.biosHeads \
| grep -a -v ddb.geometry.biosSectors > vm.novb.ddesc && \
dd if=vm.novb.ddesc conv=sync,notrunc of=vm.new.ddesc bs=512 count=2 && \
dd if=$1 bs=512 skip=3 of=vm.nohdr && \
cat vm.hdr vm.new.ddesc vm.nohdr > $1.fixed

Note, this script has minimal error checking and validation. The output file is named by using

the input vmdk file name appended with ".fixed", which you will need to rename before

using.

It would be a lot easier if VMware could just ignore the VirtualBox ddb settings. I am running

VMware Fusion 2.0.6 on MacOSX 10.5.8.

This workaround should work for any guest OS contained in a type=0 VMDK (monolithicSparse)

0 Kudos
chriswolf
Contributor
Contributor

Since performing a cleanup to remove VirtualBox-specific settings from the VMDK file can take

a long time (due to moving dozens of gigabytes of data around on the harddrive), the solution

of changing the VMDK file is not practical if you still want to be able to quickly switch back

and forth to/from VirtualBox and VMware (e.g. during evaluation).

In that case, I have another workaround which leaves the VMDK file with the extra VirtualBox

ddb settings. Instead, you can reconfigure grub to search for it's boot device by the

device name rather then the UUID of the device. This way, you can boot the same, unchanged

VMDK file with either VMware or VirtualBox. The only caveat is that the device name

gets "baked in" to grub's config when you run "update-grub", such that if you are booting from

an external USB drive, you need to make sure it always gets assigned the same device

name - usaully by insuring you plug in the same USB devices in the same order from boot

to boot of the host OS. (this is one advantage of searching for boot devices by UUID, this search

method is immune to device name changes due to adding devices in different sequence)

This solution only works with Linux VMDK files as guest OSs.

When the Linux guest is up and running either on VMware or VirtualBox, perform

these steps:

become root

edit "/etc/default/grub"

uncomment: GRUB_DISABLE_LINUX_UUID=true

You're supposed to be able to just run "update-grub" and be done, but

there appears to be a bug in grub (at least the version with Ubuntu-9.1,

1.97_beta4) such that update-grub still adds UUID-related searching to

the grub menu items.

To workaround this problem, as root, go to this directory:

cd /etc/grub.d

Make a backup of the script module, "10_linux":

cp 10_linux bak.10_linux

chmod 444 bak.10_linux

(Note the backup file name MUST NOT have a number/digit prefix)

Edit "10_lunux" and comment out the line which begins with:

prepare_grub_to_access_device

Now run "update-grub".

The result is that the grub config file:

/boot/grub/grub.cfg

Will have changes to the menu selections (which you don't even see unless

you hit escape, then 'e' on a given item)

...from UUID-type device search:

search --no-floppy --fs-uuid --set 472bb5e7-c394-405f-9d5e-f7639ecf143a

linux /boot/vmlinuz-2.6.31-14-generic root=UUID=472bb5e7-c394-405f-9d5e-f7639ecf143a ro single

...to older style device-name search:

linux /boot/vmlinuz-2.6.31-14-generic root=/dev/sda1 ro single

Now this VirtualBox VMDK can boot either from VMware or VirtualBox.

BTW, if you're nervous about updating the grub configuration, you can

reconfigure on-the-fly by hitting escape as soon as grub starts to load,

then you'll be presented with a menu of OSs to boot - typically the

normal Linux and a recovery-mode Linux, you just highlight the one

you want with the up/down keys, then press 'e' (to edit the menu item)

and change the search style from UUID to device (see lines, above),

then hit Control-X to boot.

0 Kudos
chriswolf
Contributor
Contributor

This problem might possibly be due to ambiguous vmdk spec. I am looking a a document called,

Virtual Disk Format 1.1, filename, vmdk_specs.pdf. There is a section called.

The Disk Database, but all it says is:

Additional information about the virtual disk is stored in the disk database section of the

descriptor. Each line corresponds to one entry. Each entry is formatted as follows:

ddb.<nameOfEntry> = "<value of entry>"

There's no word on what values of are valid, so I suppose there was an assumption

by the VirtualBox people that any value could be used. However, if I sift through the strings of the

vmware-2.0.6 executable, I see:

strings vmware | grep ddb

ddb.

ddb.%s = "%s"

bora/lib/disklib/ddb.c

DISKLIB-DSCPTR: Unrecognized ddb entry. ID='%s' Val='%s'

ddb.toolsVersion

ddb.virtualHWVersion

ddb.adapterType

ddb.geometry.cylinders

ddb.geometry.heads

ddb.geometry.sectors

Which seems to support the idea that only certain values of ddb.<nameOfEntry> will be accepted

by VMware, although I fail to find any string, "Unrecognized ddb entry" in any log file, unless I need

to change the log level, but I have no idea how to do that.

0 Kudos
ChipMcK
Hot Shot
Hot Shot

I frequently use a vmdk file under VMware Fusion and VirtualBox with no problems, even booting up the virtual machine.

SnapShots and AutoProtect are not used with such virtual machines (used under both products) as those features/facilities modify the vmdk files.

For good measure, I get a back-up of the files before 'flipping' products

0 Kudos