4 Replies Latest reply on Apr 5, 2017 1:33 PM by a.p.

    ESX 6 The filesystem where disk disk.vmdk resides is full

    erickpty Lurker

      A recently converted virtual machine from Redhat virtualization was moved to ESX 6 host.

      the command to convert the RAW RHEV virtual machine was:

      qemu-img convert rhevvmdisk -O vmdk disk-id0.vmdk

       

      The same command was used for 8 other machines. The other 8 are just fine.

       

      then I just created a VM (they all are windows 2008 R2 64 bit, 8GB ram with thick disks) and attached the converted disk. The VM booted fine, I even had the chance to install the vm tools and booted about 10 times over a 24 hour period before the error appeared.

      Someone recommended to run vmkfstools -i DiskImage.vmdk -d thin /MyServer/DiskImage.vmdk but since the machine booted just fine, I did not do it to this or any of the others.

       

      The datastore is an NFS mount located in a synology 416 (4x3TB sata3 hdd, dual gbit nic in bond). This is just to run 8 simple VMs that generated very low network usage. async is disabled on the NFS server.

      VAAI primitives are loaded for the synology units.

       

      An error appeared that I have no clue how to troubleshoot, please see the attached image. I have also attached the corresponding VM vmware.log file.

      Message:

      Virtual Machine Message
      The operation on the file "/vmfs/devices/deltadisks/19fd63f4-worryfree-disk0-s001.vmdk" failed (14(Bad address)). 
      The file system where disk "/vmfs/devices/deltadisks/19fd63f4-WORRYFREE-disk0-s001.vmdk" resides is full. 
      Select button.retry to attempt the operation again. Select button.abort to end the session.
      

       

      The datastore in question has 5.12TB free so hard disk space is not a real issue here. And the other 8 VMs are humming just fine. Strange entries begin at around line 9725 in the vmware.log file

      2017-04-05T16:59:35.843Z| vmx| I120: ide0:0: Command WRITE(10) took 1.234 seconds (ok)

      2017-04-05T16:59:48.343Z| vmx| I120: ide0:0: Command WRITE(10) took 1.311 seconds (ok)

      2017-04-05T17:00:42.091Z| vmx| I120: ide0:0: Command WRITE(10) took 1.108 seconds (ok)

      2017-04-05T17:01:56.185Z| vmx| I120: DISKLIB-LIB   : numIOs = 1300000 numMergedIOs = 0 numSplitIOs = 0

      2017-04-05T17:08:40.021Z| Worker#1| I120: FileIOErrno2Result: Unexpected errno=14, Bad address

      2017-04-05T17:08:40.022Z| vmx| I120: VMXAIOMGR: Retry on write "/vmfs/devices/deltadisks/19fd63f4-WORRYFREE-disk0-s001.vmdk" : Bad address.

      2017-04-05T17:08:40.022Z| vmx| I120: VMXAIOMGR: system : err=e0002 errCode=14 freeSpace=0

      2017-04-05T17:08:40.022Z| vmx| I120: VMXAIOMGR: "/vmfs/devices/deltadisks/19fd63f4-WORRYFREE-disk0-s001.vmdk" : write s=7766048768 n=131072 ne=1, fai=0

      2017-04-05T17:08:40.022Z| vmx| I120: VMXAIOMGR:             v[0]=6800202E000:131072

      2017-04-05T17:08:40.022Z| Worker#0| I120: FileIOErrno2Result: Unexpected errno=14, Bad address

      2017-04-05T17:08:40.022Z| vmx| I120: VMXAIOMGR: Retry on write "/vmfs/devices/deltadisks/19fd63f4-WORRYFREE-disk0-s001.vmdk" : Bad address.

      2017-04-05T17:08:40.022Z| vmx| I120: VMXAIOMGR: system : err=e0002 errCode=14 freeSpace=0

      2017-04-05T17:08:40.022Z| vmx| I120: VMXAIOMGR: "/vmfs/devices/deltadisks/19fd63f4-WORRYFREE-disk0-s001.vmdk" : write s=7766048768 n=131072 ne=1, fai=0

      2017-04-05T17:08:40.022Z| vmx| I120: VMXAIOMGR:             v[0]=6800202E000:131072

      2017-04-05T17:08:40.022Z| Worker#1| I120: FileIOErrno2Result: Unexpected errno=14, Bad address

      2017

       

      Please advice on places to look for clues.