VMware Communities
xilex
Contributor
Contributor

Help troubleshooting a recurring vmdk disk error

Hi everyone, I have been having this issue that was intermittent but now occurring with increasing frequency. I have the virtual machine on a second SSD, and when this error occurs, the VM cannot continue and must be terminated. I cannot restart the VM unless I restart the host machine.

If I move the VM to same disk where host OS is, then the entire system is non-functional after this error occurs. But if the VM is on the second disk, I can continue to use host OS, but must restart the host to run the VMs again. This does not happen only on this VM, but on newly created VMs, too. I ran the basic check disk from Windows but it did not note any serious errors. A SMART check did not reveal anything significant from what I can tell, either.

Do you know if this is a symptom of corrupt vmdk(s), underlying issue with the host hardware, or something else? I had lower suspicion of host hardware, because the host OS is running fine without interruption. I have attached some excerpts/logs below. Thank you. 

The error dialog that appears in VM Workstation is this:

 

 

The operation on file "E:\virtual machines\ubuntu-dev\Ubuntu 19.04 (64bit)-cl1.vmdk" failed.

If the file resides on a remote file system, make sure that the network connection and the server where this disk resides are functioning properly. If the file resides on removable media, reattach the media.

Select Retry to attempt the operation again.

Select Cancel to power off the virtual machine.

Select Continue to forward the error to the guest operating system.

 

 

In VM logs, I have several error messages:

 

 

2022-05-11T21:54:04.072-07:00| vmx| I125: VMXAIOMGR: Retry on read "E:\virtual machines\ubuntu-dev\Ubuntu 19.04 (64bit)-cl1.vmdk" : The request could not be performed because of an I/O device error.
2022-05-11T21:54:04.072-07:00| vmx| I125: VMXAIOMGR: system : err=45d0002 errCode=1117 freeSpace=97490825216
2022-05-11T21:54:04.072-07:00| vmx| I125: VMXAIOMGR: "E:\virtual machines\ubuntu-dev\Ubuntu 19.04 (64bit)-cl1.vmdk" : read s=52956434432 n=61440 ne=1, fai=0
2022-05-11T21:54:04.072-07:00| vmx| I125: VMXAIOMGR:             v[0]=8B53000:61440
2022-05-11T21:54:04.073-07:00| vmx| I125: VMXAIOMGR: Retry on read "E:\virtual machines\ubuntu-dev\Ubuntu 19.04 (64bit)-cl1.vmdk" : The request could not be performed because of an I/O device error.
2022-05-11T21:54:04.073-07:00| vmx| I125: VMXAIOMGR: system : err=45d0002 errCode=1117 freeSpace=97490825216
2022-05-11T21:54:04.073-07:00| vmx| I125: VMXAIOMGR: "E:\virtual machines\ubuntu-dev\Ubuntu 19.04 (64bit)-cl1.vmdk" : read s=52956434432 n=61440 ne=1, fai=0
2022-05-11T21:54:04.074-07:00| vmx| I125: VMXAIOMGR:             v[0]=8B53000:61440
2022-05-11T21:54:04.082-07:00| vmx| I125: Msg_Question:
2022-05-11T21:54:04.082-07:00| vmx| I125: [msg.vmxaiomgr.retrycontabort.rudeunplug] The operation on file "E:\virtual machines\ubuntu-dev\Ubuntu 19.04 (64bit)-cl1.vmdk" failed.
2022-05-11T21:54:04.082-07:00| vmx| I125+ If the file resides on a remote file system, make sure that the network connection and the server where this disk resides are functioning properly. If the file resides on removable media, reattach the media.
2022-05-11T21:54:04.082-07:00| vmx| I125+ Select _Retry to attempt the operation again.
2022-05-11T21:54:04.082-07:00| vmx| I125+ Select Cancel to power off the virtual machine.
2022-05-11T21:54:04.082-07:00| vmx| I125+ Select _Continue to forward the error to the guest operating system.
2022-05-11T21:54:04.082-07:00| vmx| I125: ----------------------------------------
2022-05-12T08:30:39.405-07:00| vmx| I125: MsgQuestion: msg.vmxaiomgr.retrycontabort.rudeunplug reply=1
2022-05-12T08:30:39.405-07:00| vmx| I125: Msg_Post: Warning
2022-05-12T08:30:39.405-07:00| vmx| I125: [msg.vmxaiomgr.corrupted] VMware Workstation cannot synchronize with the disk before canceling. Disk E:\virtual machines\ubuntu-dev\Ubuntu 19.04 (64bit)-cl1.vmdk may be inconsistent.
2022-05-12T08:30:39.405-07:00| vmx| I125: ----------------------------------------
2022-05-12T08:30:39.461-07:00| vmx| E105: PANIC: User requested abort: Exiting because of failed disk operation.

 

 

In Event Viewer (Windows 7), I have many entries of the following:

 

 

Log Name:      System
Source:        Disk
Date:          5/12/2022 8:31:27 AM
Event ID:      11
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      *********
Description:
The driver detected a controller error on \Device\Harddisk1\DR1.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Disk" />
    <EventID Qualifiers="49156">11</EventID>
    <Level>2</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2022-05-12T15:31:27.733700000Z" />
    <EventRecordID>28943222</EventRecordID>
    <Channel>System</Channel>
    <Computer>*********</Computer>
    <Security />
  </System>
  <EventData>
    <Data>\Device\Harddisk1\DR1</Data>
    <Binary>0407800001000000000000000B0004C00101000000000000000000000000000000403BFAC7000000AF15950000000000FFFFFFFF030000005800008802000000F3200A1282052040000002003C0000000000040000000000E00ED83680FAFFFF000000000000000050E3FE3180FAFFFF0000000000000000A01DFD63000000002A0863FD1DA000010000000000000000700004000000000A00000000000000000000000000000000</Binary>
  </EventData>
</Event>

 

 

xilex_0-1652377887459.png

 

 

0 Kudos
3 Replies
wila
Immortal
Immortal

Hi,

This message here:

VMXAIOMGR: Retry on read "E:\virtual machines\ubuntu-dev\Ubuntu 19.04 (64bit)-cl1.vmdk" : The request could not be performed because of an I/O device error.

 

is a pretty clear one I'd say.
Something is wrong with your disk and it gradually failing more often certainly is in line with a hardware error.

Note that it can be:
- the SSD itself (most likely)
- a bad cable or loose/dirty connector
- a malfunctioning controller

To me it does not sound like a software error. But if you insist on going down that route of suspicion then you should make sure that your virtual disk(s) are excluded by your antivirus software.

--
Wil

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva
0 Kudos
bluefirestorm
Champion
Champion

You didn't mention the version of Workstation/Player. Keep in mind that with version 16.x, Windows 7 is no longer supported as a host OS. The last version that support Windows 7 as a host OS was 15.5.x.

From one of the screenshots, the model of the SSD looked familiar to me. It looks like you have an Intel 660p 1TB. I got an Intel 660p 2TB. I did an update of the firmware from 002C to version 004C back in 2020 just shortly after I got the NVMe SSD. You may want to try updating the firmware. Do backup the data before the firmware update.

https://ark.intel.com/content/www/us/en/ark/products/149407/intel-ssd-660p-series-1-0tb-m-2-80mm-pci...

0 Kudos
xilex
Contributor
Contributor

Thanks, @wila and @bluefirestorm . I am running Workstation v15.0 right now. One drive is a M.2 NVMe (the Intel), the second drive is a SATA SSD. And I think the issue occurs on both drives when I moved the VM to either. And there is no antivirus on the host OS, including disabled Windows Defender.

During online searching on my error(s), I found this old thread, and after deleting these .lck files/directories, the VM(s) have been running stably. 🤔

0 Kudos