0 Replies Latest reply on Jun 30, 2016 2:57 PM by ymaxwell

    A general system error occurred: A fatal internal error occurred. See the virtual machine's log for more details. Failed to copy one or more disks

    ymaxwell Lurker

      We are attempting to storage vMotion a drive between datastores and it keeps failing with the below event details.

       

       

       

      ----begin event details----

      event 6/30/2016 12:36:40 PM, Cannot migrate chilhqdt02 (P1)(New Paperless) from host8., ch1nim01-t1-07 to host8.,ch1nim01-t1-07 in CH1

       

      Description:

       

      Failed to migrate the virtual machine for reasons described in the event message

       

      Possible causes:

       

      Cause: The virtual machine did not migrate. This condition can occur if vMotion IPs are not configured, the source and destination hosts are not accessible, and so on.

       

      Action: Check the reason in the event message to find the cause of the failure. Ensure that the vMotion IPs are configured on source and destination hosts, the hosts are accessible, and so on.

      --end event details---

       

       

      So far we've tried these actions to no avail:

       

      1. Upgraded the source datastore from VMFS3 to VMFS5

      2. Attempted the storage vMotion several times including overnight when I/O would be very low for that VM and source /destination datastores.

      3. Ensured vMotion is configured properly on the host

      4. Ensured we have no issues with our network traffic

      5. We've been able to successfully storage vMotion several other drives connected to same VM to the same destination datastore

      6. vMotioned the VM to another host and tried storage vMotion again.

       

      I also tried to complete the storage vMotion again while watching progress of the hostd.log and the result of greps for "fail" "storage" "VMotion" "vMotion" yielded the below. I wasn't able to find concrete root causes for this but I did find a line pointing to checking the virtual machine logs but that fails when attempting to download the file from thick client as well as web client. Anyone run across this scenario willing to share your solutions? I realize several things could be the true root cause.

       

      2016-06-30T14:48:03.317Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] DeleteFeatureCompat failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.317Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.317Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.318Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.346Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.349Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.353Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.356Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.357Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.357Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.359Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.367Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.369Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.370Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.375Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.385Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] Is disk present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.386Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-c4-eb-28 user=vpxuser] NIC: is present failed: vim.fault.GenericVmConfigFault

      2016-06-30T14:48:03.584Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:03.585Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:03.586Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:03.587Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:03.589Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:03.590Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:03.591Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:03.592Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:03.594Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:03.595Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:03.596Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:03.597Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:03.599Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:03.600Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).21B-c4-eb-28 user=vpxuser] PopulateCache failed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:12.684Z [6A180B70 info 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-12 user=vtlog state changed from failure to none

      2016-06-30T14:48:13.157Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.158Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.160Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.161Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.162Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.163Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.164Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.165Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.166Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.167Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.169Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.170Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.171Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.172Z [6A180B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:13.188Z [6F8C1B70 info 'Hbrsvc'] Replicator: ReconfigListener failed to look up VM (id=28)

      2016-06-30T14:48:19.508Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:19.509Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:19.510Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:19.511Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:19.512Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:19.514Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:19.515Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:19.516Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:19.517Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:19.518Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:19.519Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:19.521Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:19.522Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:19.523Z [6A380B70 warning 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).ed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.374Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.375Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.376Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.378Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.379Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.380Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.381Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.382Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.383Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.384Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.386Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.387Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.388Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.389Z [FFF875B0 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.455Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.456Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.458Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.459Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.460Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.461Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.462Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.463Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.465Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.466Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.467Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.468Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.469Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T14:48:33.471Z [6F680B70 warning 'Vmsvc.vm:/vmfs/volumes/4eec6ef5-6501dba0-7d60-0025b500003d/chilhqdt02/chilhqdt02.vmx' opID=A2B5CBBC-0000021B-c4-eb-b userfailed: _diskAccess : false, _storageAccessible : true

      2016-06-30T15:10:53.914Z [66DB8B70 verbose 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).b:: hostlog state changed from emigrating to failure

      2016-06-30T15:54:21.440Z [694C6B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-43-11-12 user=vpxuser] VMotionInit: hostlog state changed from failure to none

      2016-06-30T16:14:21.388Z [6F280B70 verbose 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).b:: hostlog state changed from emigrating to failure

      2016-06-30T17:14:32.240Z [66DB8B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-2c-5-64 user=vpxuser] VMotionInit: hostlog state changed from failure to none

      2016-06-30T17:36:39.276Z [66F40B70 verbose 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).b:: hostlog state changed from emigrating to failure

      2016-06-30T20:29:23.714Z [66DB8B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-9f-37-28 user=vpxuser] VMotionInit: hostlog state changed from failure to none

      2016-06-30T20:53:00.389Z [696C1B70 verbose 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).b:: hostlog state changed from emigrating to failure

      2016-06-30T21:08:16.511Z [66DB8B70 info 'Vmsvc.vm:/vmfs/volumes/576f5f18-eda38b00-fb57-0025b52a0cef/chilhqdt02 (P1)(New Paperless)/chilhqdt02 (P1)(New Paperless).vmx-78-b7-f1 user=vpxuser] VMotionInit: hostlog state changed from failure to none

      /var/log #