VMware Cloud Community
mdina
Contributor
Contributor

Error - WARNING: VSCSI: 3410: WRITE10 past end of virtual device

Ok vmware gurus, I've hit a road block and need some help. I just recently started getting an error in my vmkernel and vmkwarning logs which states: WARNING: VSCSI: 3410: WARNING: VSCSI: 3410: WRITE10 past end of virtual device with 20971520 numBlocks, offset 21252119, length 128. Further, my vmkwarning log is filling up with this log information such that the vmkwarning file has grown to ~1.4GB or more. The vmkernel file seems to roll off and start anew ever hour or so. This is a problem because the /var/log (which is on its own partition) will become completely full, and the result is the host becomes 'disconnected' within virtualcenter. I say 'disconnected' because it does not have the option to reconnect the host, and all of the VMs on that host appear disconnected. The VMs are still up running, as is the host. Also, the local storage on the ESX host appears offline, but the iscsi storage is still visible. It is not until I delete some of the old logs in the /var/log directory to free up some space that the host eventually will reconnect within virtualcenter. However the vmkwarning file is still huge and cannot be deleted b/c it is still in use. Rebooting the ESX host will roll off the huge vmwarning to vmkwarning.1 and then I can free it up.

My environment is as follows: HP DL385G1, 2 socket dual core 1.8GHz, 13GB RAM, using both local and iscsi (netapp FAS 3020) storage. I'm running ESX 3.01 build 32039 and my VC is 2.01. I’ve got plenty of free space on my vmfs’s.

So I guess I have two questions. First is, does anyone know what is causing the WRTIE10 error? My query of the forums has yielded me very little at this point. Second, does anyone know how to safely roll the vmkwarning files so that I do not have to reboot the host?

I have opened up an SR on the issue and the tech seems to think that there is an offending VM somewhere. I recently read another article on this forum that suggested that one of the .vmdk files was shrunk, but the OS did not recognize the shrink, but I have checked all of the vmdk files and all appear to match what the OS states.

Who wants first stab at this one??? Smiley Wink

Reply
0 Kudos
4 Replies
mdina
Contributor
Contributor

Update: I have found the offending VM at this point. It is a Win2K3 OS and has numerous 'write disk' errors. Going to reboot and see what happens.

Reply
0 Kudos
BUGCHK
Commander
Commander

does anyone know what is causing the WRTIE10 error?

WRITE10 is not an error. It means that a 10-byte SCSI WRITE command has been issued.

I think the tech's guess is right - a VM is trying to write outside a virtual disk's range.

Reply
0 Kudos
mdina
Contributor
Contributor

Well it looks like the reboot did not work. I also cloned the VM and the clone experienced the same issues, which led me to believe that the issue on the OS side was for some reason causing an issue on the ESX side. I simply asked the server owner to rebuild thier app on a new VM that had a fresh OS install and I am going to delete the offending VM. Seems like the easiest path to resolution.

Reply
0 Kudos
dmondal
Contributor
Contributor

Symptoms

Frequent WRITE10 past end of virtual device error logged in /var/log/vmkernal log

Event ID 51 may appear in Windows Event Log (System) for the problematic drive/s

Problem

The WRITE10 error logged into the vmkernel is being caused due to I/O attempts outside the boundaries of the virtual disk (VMDK) which indicates a disk geometry mismatch between the Guest OS and the VM (VM disk configuration).

Resolution

Try to identify the problematic VM first by browsing through the WRITE10 error messages and comparing the WID (World ID) of the VM. Another easy but time consuming approach can be comparing the VM Disk Sizes (VM Configuration) with the disk size inside the Guest OS (ex: Windows 2000 Server) to check for discrepancies.

Extending the VMDK size just for the required amount may fix this but the safer approach would be to backup the existing disk content, detach the old disk, create a new disk and then restore the data into the new disk using ntbackup or equivalent utility. That way you don't need to bother about the partition table and disk descriptor parameter anomalies.

Another option can be attaching a correctly sized newly created disk and the problematic disk with a helper VM and then do a diskcopy of the old drive to the new disk. The new disk once copied needs to be attached to the original VM after removing the old system drive.

Debashis

Reply
0 Kudos