marioffm
Contributor
Contributor

Error after deploy VM from Template "file not found" result with locked open files on ESX Host

Problem:

After deploy a VM from a template the Process Canceld with the Error:

"file not found". Few day in front of this error the deployment with

the same file run without any problems. How could this happen, that the

Template perhaps crash?

Could i scan my san storage for defect segments from the esx site (like chkdsk)

Also at this moment some files on the ESX Host are locked. How could this happen?

Replacing the Template and reboot the Host resolve the Problem. But it is not the final Resolution!

Attached are the vmkernel Log File:

Jun 2 11:19:15 vmName vmkernel: 0:00:54:09.146 cpu1:1035)FS3: 1973:

Checking if lock holders are live for lock [type 10c00001 offset

38354944 v 198, hb offset 3941888

Jun 2 11:19:15 vmName vmkernel: gen 13, mode 1, owner 4a24e1d0-c25a7708-66e1-0022649ae96c mtime 518]

Jun 2 13:16:41 vmName vmkernel: 0:02:51:35.206 cpu3:1036)World: vm 1090: 901: Starting world vmware-vmx with flags 4

Jun 2 13:16:42 vmName vmkernel: 0:02:51:36.217 cpu3:1090)World: vm 1091: 901: Starting world vmm0:denu1v635 with flags 8

Jun 2 13:16:42 vmName vmkernel: 0:02:51:36.217 cpu3:1090)Sched: vm

1091: 5333: adding 'vmm0:denu1v635': group 'host/user': cpu: shares=-3

min=0 minLimit=-1 max=-1

Jun 2 13:16:42 vmName vmkernel: 0:02:51:36.217 cpu3:1090)Sched: vm 1091: 5352: renamed group 18 to vm.1090

Jun 2 13:16:42 vmName vmkernel: 0:02:51:36.217 cpu3:1090)Sched: vm 1091: 5366: moved group 18 to be under group 4

Jun 2 13:16:42 vmName vmkernel: 0:02:51:36.221 cpu3:1090)Swap: vm 1091: 2169: extending swap to 524288 KB

Jun 2 13:16:45 vmName vmkernel: 0:02:51:39.028 cpu3:1090)VSCSI: 4060:

Creating Virtual Device for world 1091 vscsi0:0 (handle 8196)

Jun 2 13:16:46 vmName vmkernel: 0:02:51:39.300 cpu1:1090)World: vm 1092: 901: Starting world vmware-vmx with flags 44

Jun 2 13:16:46 vmName vmkernel: 0:02:51:39.300 cpu2:1092)World: vm 1093: 901: Starting world vmware-vmx with flags 44

Jun 2 13:16:46 vmName vmkernel: 0:02:51:39.301 cpu2:1092)World: vm 1094: 901: Starting world vmware-vmx with flags 44

Jun 2 13:16:46 vmName vmkernel: 0:02:51:39.301 cpu0:1091)Init: 1057: Received INIT from world 1091

Jun 2 13:17:25 vmName vmkernel: 0:02:52:19.088 cpu2:1091)Net: 4259: unicastAddr 00:50:56:b9:66:57;

Jun 2 13:17:30 vmName vmkernel: 0:02:52:23.381 cpu3:1091)Net: 4259: unicastAddr 00:50:56:b9:66:57;

Jun 2 13:31:09 vmName vmkernel: 0:03:06:02.584 cpu1:1037)World: vm 1095: 901: Starting world vmware-vmx with flags 4

Jun 2 13:31:09 vmName vmkernel: 0:03:06:03.138 cpu0:1095)World: vm 1096: 901: Starting world vmm0:denu1v634 with flags 8

Jun 2 13:31:09 vmName vmkernel: 0:03:06:03.138 cpu0:1095)Sched: vm

1096: 5333: adding 'vmm0:denu1v634': group 'host/user': cpu: shares=-3

min=0 minLimit=-1 max=-1

Jun 2 13:31:09 vmName vmkernel: 0:03:06:03.138 cpu0:1095)Sched: vm 1096: 5352: renamed group 19 to vm.1095

Jun 2 13:31:09 vmName vmkernel: 0:03:06:03.138 cpu0:1095)Sched: vm 1096: 5366: moved group 19 to be under group 4

Jun 2 13:31:09 vmName vmkernel: 0:03:06:03.148 cpu0:1095)Swap: vm 1096: 2169: extending swap to 2097152 KB

Jun 2 13:31:11 vmName vmkernel: 0:03:06:04.755 cpu3:1095)VSCSI: 4060:

Creating Virtual Device for world 1096 vscsi0:0 (handle 8197)

Jun 2 13:31:11 vmName vmkernel: 0:03:06:04.831 cpu0:1095)World: vm 1097: 901: Starting world vmware-vmx with flags 44

Jun 2 13:31:11 vmName vmkernel: 0:03:06:04.832 cpu1:1097)World: vm 1098: 901: Starting world vmware-vmx with flags 44

Jun 2 13:31:11 vmName vmkernel: 0:03:06:04.832 cpu1:1097)World: vm 1099: 901: Starting world vmware-vmx with flags 44

Jun 2 13:31:11 vmName vmkernel: 0:03:06:04.832 cpu1:1096)Init: 1057: Received INIT from world 1096

Jun 2 13:32:16 vmName vmkernel: 0:03:07:10.113 cpu2:1096)Net: 4259: unicastAddr 00:50:56:b9:00:21;

Jun 2 13:32:25 vmName vmkernel: 0:03:07:18.372 cpu1:1096)Net: 4259: unicastAddr 00:50:56:b9:00:21;

Jun 2 14:41:44 vmName vmkernel: 0:04:16:37.612 cpu0:1037)DevFS: 2300:

Unable to find device: 102b-t - w2k3s32r2sp2en-000001-delta.vmdk

Jun 2 14:41:44 vmName vmkernel: 0:04:16:37.614 cpu0:1024)VMNIX: VMKFS:

1129: t - w2k3s32r2sp2en-000001-delta.vmdk (4139), status -9

Jun 2 14:41:44 vmName vmkernel: 0:04:16:37.614 cpu0:1024)VMNIX: VMKFS: 1129: t - w2k3s32r2sp2en-flat.vmdk (24618), status -9

Jun 2 14:42:12 vmName vmkernel: 0:04:17:05.297 cpu2:1037)FS3: 1973:

Checking if lock holders are live for lock [type 10c00001 offset

66834432 v 586, hb offset 3941888

Jun 2 14:42:12 vmName vmkernel: gen 13, mode 1, owner 4a24e1d0-c25a7708-66e1-0022649ae96c mtime 15451]

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.870 cpu3:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.871 cpu3:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.871 cpu3:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.872 cpu3:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.872 cpu3:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.874 cpu3:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.874 cpu3:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.874 cpu3:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.874 cpu3:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.875 cpu3:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.875 cpu3:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.875 cpu3:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.876 cpu0:1035)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.895 cpu3:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.895 cpu0:1024)VMNIX: VMKFS: 1890: status = -2

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.895 cpu3:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.895 cpu0:1024)VMNIX: VMKFS: 1890: status = -2

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.896 cpu3:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.896 cpu0:1024)VMNIX: VMKFS: 1890: status = -2

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.896 cpu3:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.896 cpu0:1024)VMNIX: VMKFS: 1890: status = -2

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.897 cpu3:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 14:56:51 vmName vmkernel: 0:04:31:44.897 cpu0:1024)VMNIX: VMKFS: 1890: status = -2

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.558 cpu1:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.559 cpu2:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.559 cpu1:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.559 cpu1:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.560 cpu2:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.561 cpu2:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.561 cpu1:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.561 cpu2:1035)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.562 cpu1:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName last message repeated 3 times

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.563 cpu1:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.563 cpu1:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.564 cpu1:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.564 cpu1:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.583 cpu1:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.583 cpu0:1024)VMNIX: VMKFS: 1890: status = -2

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.584 cpu1:1035)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.584 cpu0:1024)VMNIX: VMKFS: 1890: status = -2

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.584 cpu1:1035)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.584 cpu0:1024)VMNIX: VMKFS: 1890: status = -2

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.585 cpu1:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.585 cpu0:1024)VMNIX: VMKFS: 1890: status = -2

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.585 cpu1:1035)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:11 vmName vmkernel: 0:05:01:04.585 cpu0:1024)VMNIX: VMKFS: 1890: status = -2

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.755 cpu1:1035)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.755 cpu1:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 denu1s121 vmkernel: 0:05:01:37.755 cpu1:1035)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.755 cpu1:1035)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.756 cpu1:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.756 cpu1:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.756 cpu1:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.756 cpu1:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.757 cpu1:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.757 cpu1:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.757 cpu1:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.757 cpu1:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.757 cpu1:1035)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.758 cpu1:1035)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.758 cpu1:1037)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.758 cpu1:1036)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

Jun 2 15:26:44 vmName vmkernel: 0:05:01:37.777 cpu0:1035)FSS: 390:

Failed with status Not found for f530 28 2 494a23d2 7687dc50 220057a3

367ba164 9802384 212 0 0 0 0 0

0 Kudos
1 Reply
marioffm
Contributor
Contributor

more error and failure messages in the hostd log file:

Failed to validate VM IP address: unknown

Hw info file: /etc/vmware/hostd/hwInfo.xml

Config target info loaded

Failed to validate VM IP address: unknown

Failed to validate VM IP address: unknown

Failed to validate VM IP address: unknown

--- more -


DISKLIB-LINK : "/vmfs/volumes/494a23d2-7687dc50-57a3-002264a17b36/t - w2k3s32r2sp2en/t - w2k3s32r2sp2en.vmdk" : failed to open (The system cannot find the file specified).

DISKLIB-CHAIN : "/vmfs/volumes/494a23d2-7687dc50-57a3-002264a17b36/t - w2k3s32r2sp2en/t - w2k3s32r2sp2en.vmdk" : failed to open (The system cannot find the file specified).

DISKLIB-LIB : Failed to open '/vmfs/volumes/494a23d2-7687dc50-57a3-002264a17b36/t - w2k3s32r2sp2en/t - w2k3s32r2sp2en.vmdk' with flags 0x11 (The system cannot find the file specified).

DISKLIB-LINK : "/vmfs/volumes/494a23d2-7687dc50-57a3-002264a17b36/t - w2k3s32r2sp2en/t - w2k3s32r2sp2en.vmdk" : failed to open (The system cannot find the file specified).

DISKLIB-CHAIN : "/vmfs/volumes/494a23d2-7687dc50-57a3-002264a17b36/t - w2k3s32r2sp2en/t - w2k3s32r2sp2en.vmdk" : failed to open (The system cannot find the file specified).

DISKLIB-LIB : Failed to open '/vmfs/volumes/494a23d2-7687dc50-57a3-002264a17b36/t - w2k3s32r2sp2en/t - w2k3s32r2sp2en.vmdk' with flags 0x17 (The system cannot find the file specified).

Failed to VixHost_OpenObject: Error: A file was not found

0 Kudos