VMware Communities
grahamg
Contributor
Contributor

Workstation 9 VMDK file corruption in Windows 8 Storage Spaces

There is a related thread on this topic that seems to have died out so I am starting a new one Smiley Wink

I experiencing extremely  frequent .vmdk disk file issues when running VMWare Workstation 9 VMs  from Storage Spaces under Windows 8 Pro (x64).  The attached image is an  example.

Storage  Spaces is configured with one disk redundancy (3 x 3 TB storage pool  with 1 disk redundancy, partitioned into 2 x 2.04 TB storage spaces).  I  have no issues when running VMs from non-storage space drives but  almost always, after shutting down a VM on a storage space, I will get a  .vmdk internal consistency warning on the next boot.  I have configured  VMWare Workstation to "Fit all virtual machine memory into reserved  host RAM" with no change in behaviour.  The issue persists with version  9.02.

I  have not observed file corruption with any other applications that  write to the storage spaces, including write intensive applications such  as media ripping.

Corrupted vmdk files can generally be repaired with "vmware-vdiskmanager.exe -R".

Has anyone else come across this issue?

Reply
0 Kudos
12 Replies
WoodyZ
Immortal
Immortal

I don't have an answer for you, sorry, although a comment... It has always been a best practice to use identical disks in a RAID and also to favor Hardware RAID over Software Raid.  AFAIC Windows 8 Storage Spaces conforms to neither best practice and I'll never personally use the Storage Spaces technology and will stick to matched hardware in a hardware RAID especially when running VM's, if not from an individual device.

grahamg
Contributor
Contributor

WoodyZ, although I appreciate and agree with your sentiments, I really want to figure out what is going on in this situation.  Storage Spaces has been pretty thoroughly tested and, in the guise of Server 2012, is supposed to be ready for enterprise storage loads.  The fact that file corruption seems to be specific to VMWare Workstation suggests to me the issue is not at the OS layer.

With regard to your comments about RAID best practices, remember this is not RAID in the usual sense.  That said, the three drives I am using are identical models.

Graham

Reply
0 Kudos
WoodyZ
Immortal
Immortal

grahamg wrote: The fact that file corruption seems to be specific to VMWare Workstation suggests to me the issue is not at the OS layer.

IIRC Over the years I've read posts that when using software RAID that it was not unusual to have .vmdk corruption and I do not necessarily disagree with where the issue may be at fault.  From what you've said it sure looks more like a VMware issue then an OS issue and although I'd like to, unfortunately I'm not in a position at the moment to do any testing.

With regard to your comments about RAID best practices, remember this is not RAID in the usual sense.

If one selects Two-way mirror or Parity then IMO that's tantamount to a software RAID.

Anyway, hopefully one of the Developers will catch this thread and offer something more useful. Smiley Happy

Reply
0 Kudos
continuum
Immortal
Immortal


I experiencing extremely  frequent .vmdk disk file issues when running VMWare Workstation 9 VMs ...

Corrupted vmdk files can generally be repaired with "vmware-vdiskmanager.exe -R".

Has anyone else come across this issue?


Yes and I would not  blame your storage so early.

I used to repair a lots of these errors for other folks but never get one myself.
But no longer with WS 9 - now I get this errors quite often myself.

With 9.0 I got disk corruption after enabling sharing mode - did not even need to start tzhe VM once.
Sometimes this errors go away on their own which never happened with earlier versions.

With version nine I have not seen a single  successful run of vmware-vdiskmanager -R.
It often will say so but if you look at the vdiskmanager.log and the next vmware.log you will probably dont get see a difference

to to avoid this problems ... this helps here ...
dont use very large growing vmdks
avoid snapshots
never use autoprotect
never suspend two VMs at the same time
do not overcommit RAM

make sure antivirus leaves vmdks, vmem, vmx alone ...


I regard the -R function as broken or deffect since 8.0
As replacement I use vdiskmanager from 6.5 and the one from the VDDK 5.0 - not the latest !!!

Still one of the best vmdk reading tools is vdk.exe 32bit exectuted on a XP or 2003 host or VM.
Nowadays you first have to rewrite the descriptor to make it old enough for vdk.exe to read it.
So if you have a vmdk that -R can not repair there are few other tools that still may be able to read the vmdk.

It is a pity that you are very likely screwed if you have graintable errors in the vmware.logs.
In the last few years I noticed only one or two successful repairs here in the forum.

...  sorry - your question is still unanswered

Ulli


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
grahamg
Contributor
Contributor

Thanks, Ulli.

Disturbing news regarding failures of vmware-vdiskmanager -R. Given that I am seeing problems only on VMs residing with a Storage Space volume I still lean toward my issues being specific to Workstation and Storage Spaces.  I moved my most critical VMs (ones that I snapshot a lot) off to other volumes and these all seem stable.

Unfortunately the purchase price for VMWare Workstation does not include support and I am not willing to pony up for something that appears to be a flaw in the product.  Maybe it is time to look more closely at Hyper-V, seeing as it is included in Win 8.

Graham

Reply
0 Kudos
random3355
Contributor
Contributor

I also have VMDK file corruption using workstation 9 and Windows 8 Storage Spaces.

I'm using a mirrored storage space (two-way mirror) with identical disks.  No other software apart from VMware has any corruption problems and I have individually tested each disk to be OK.

This has been ongoing since the release of WS9 and I've updated it to latest version but no fix.  Moving the vm's onto non-storage pool stops new corruption from appearing.  However I do continue to run test machines from the storage pool as that's where most of my space is.

I agree that this must be some bug in vmware workstation 9. 

I did contact VMware and lodge a support request. They were very helpful but unable to resolve other than recommend to not keep my vm's on the storage space.  They couldn't reproduce from their end so at least its reassuring to see other users with the same issue.

Reply
0 Kudos
mfelker
Expert
Expert

To me it is not self-evident taht this is a bug in WS 9. Windows 8 Storage Places is not yet a mature product. Do a Google search on Storage Spaces vs Dynamic disks. Storage Space is more likely to corrupt data. I am in the process of creating mirrored dynamic disk in Windows (or will be shortly). My brief time with Storage Spaces left me with a bad taste.

Reply
0 Kudos
mfelker
Expert
Expert

Ini fact I did put VM's on Storage Spaces and I had to rebuild my entire system as trying to run the VM's on the Spaces seemed mess stuff up (can't be more specific right now).  I had software RAID 2 and would have stuck with that except my backup software would not recognize the RAID.  Switching to Spaces , which it did recognize it was a bad mistake.

Reply
0 Kudos
grahamg
Contributor
Contributor

Perhaps a silly question but why would one use dynamic disks on top of Storage Spaces?  Shouldn't Storage Spaces provide all the functionality of dynamic disks and then some? In any event, my understanding is that Microsoft is deprecating dynamic disks.

With regard to where the bug lies, although it is possible that VMWare has tickled something obscure in Storage Spaces, I think it is equally possible that the VMWare hypervisor has a bug that is only revealed by the new storage subsystem.  Over the course of approximately 2 months of constant use in multiple use cases the only data corruption I encountered was with VMWare Workstation 9 and that was with very high repeatability.  At the very least this points to something pretty unique about Workstation 9.

Reply
0 Kudos
random3355
Contributor
Contributor

In my case, I am not using dynamic disks at all.  I have a two-way mirror storage space pre-allocated using all available space on each physical drive. 

The only app I have that has problems with storage on the storage space is WS9 (the vmdk corruption).  This did not occur for me when I was running WS8 on the same storage space.  So that's why my guess it is a WS9 bug.

As I'm aware of the issue, I only run test vms on the space and keep important ones on other drives.  I do trust the storage space to keep non-vmware backups etc.  I haven't otherwise encounted any issues with storage spaces.

Cheers

Reply
0 Kudos
nickaclark
Contributor
Contributor

I am having the same problem but with Workstation 10.  My VMDK's get corrupt all the time plus erratic mouse issue and I am now to the point to where I cannot even create a new VM.  It just crashes during OS installation at random points.  I am going to switch to the built in Microsoft Windows 8 Hyper-V solution since support for storage spaces is so bad.  Unfortunate that I spent $300 for a workstation license and now have to use hyper-v (which is free).

Plus the whole "Use hardware RAID, software RAID sucks" is not an answer.  Let's move into the future people, RAID technologies are changing, storage spaces is a far superior technology than hardware RAID for consumers so lets break out of that box please!

Reply
0 Kudos
natenjo
Contributor
Contributor

I can confirm this Problem.  It still exists in Workstation 10.0.3.

I ran into this while wanting to install Proxmox as a VM, after the installation I always ran into a grub error.

When I installed it again on a "normal disk"  everything worked fine.

So this is a reproducible error of vmware!

See my posting on the Proxmox forum

edit:// A hack just came to my mind: "What if i create a vhd on the storage space and try to install the vm into it". This sounds a bit crazy but it actually works and boots!.. But i wouldn't consider it as recommended..

Reply
0 Kudos