Hello
I have ran into a rather scary situation. For some reasons my VM have become non responsive. So I decided to restart it by issuing the "reset" command. Unfortunately this resulted in the VM not being able to start up. I tried rebooting the ESXi server and am still given the following message when attempting to start the VM.
Failed to start the virtual machine.
Module Disk power on failed.
Cannot open the disk '/vmfs/volumes/557dc2e2-2c11eba8-1494-00259066ac58/<VM Name>/<VM Name>.vmdk' or one of the snapshot disks it depends on.
22 (Invalid argument)
The screen shot looks like the image attached named "PowerOnError.jpg".
Even more puzzling is the ESXi shell isn't able to find the .vmx file. Note I tried to list content of the directory it attempts to include the .vmx file but isn't able to. In fact before the reboot it was included in the output of this command, in addition to showing something unable to stat file or something like that. And if I tried to copy it it'd say Invalid Argument. If I do something like 'strace cp <VM Name>.vmx <dest>' it'd show the error happened during the Linux system call 'open' operation with returned value EINVAL -1.
[root@esxi6prim:/vmfs/volumes/557dc2e2-2c11eba8-1494-00259066ac58/<VM Name>] ls -ltr
ls: ./<VM Name>.vmx: No such file or directory
total 52833280
-rw-r--r-- 1 root root 0 Oct 4 2015 <VM Name>.vmsd
-rw-r--r-- 1 root root 179092 Oct 9 2015 vmware-1.log
-rw-r--r-- 1 root root 177929 Oct 14 2015 vmware-2.log
-rw-r--r-- 1 root root 409599604 Nov 24 07:26 vmware-3.log
-rw------- 1 root root 524 Nov 24 09:55 <VM Name>.vmdk
-rw------- 1 root root 53687091200 Feb 17 12:52 <VM Name>-flat.vmdk
-rw------- 1 root root 8684 Mar 28 04:42 <VM Name>.nvram
-rw-r--r-- 1 root root 270087 Mar 28 04:42 vmware.log
If I try to copy the directory to another location it'd say the following. Note it says the <VM Name>-flat.vmdk can't be opened due to an Invalid argument despite it's shown as having a size of 53687091200 on the above 'ls' output? 53687091200
[root@esxi6prim:/vmfs/volumes/557dc2e2-2c11eba8-1494-00259066ac58] cp -rp '<VM Name>' '<VM Name>.orig.copi
ed.after.reboot'
cp: can't stat '<VM Name>/<VM Name>.vmx': No such file or directory
cp: can't open '<VM Name>/<VM Name>-flat.vmdk': Invalid argument
I'm seriously out of ideas here. Any suggestions on how to fix this?
Thanks.
This looks like an issue with the VMFS locks. Although there are several posts with hints on the Internet, you may try to reach continuum via Skype. He's an expert when it comes to such issues.
André
ok the vmx is missing in this directory - but if you are able to start the vm (but fails) there must be a vmx in another place.
no lockfile is visible (*,lck)
the vmsd should not be 0 bytes
is the vm spreaded on different datastores ?
edit the vm and have a look on "vm-options->general options" there you find the entry for "vm config file"
also see the actual vmware.log for errors
Sometimes it could help to completly migrate the vm to another host and datastore.
Or as your error suggest: or one of the snapshot disks it depends on.
If the vm has multiple snapshots remove or consolidate them
If the vms isn't responsive then and it's running you could kill it by using esxicli vm process kill
A vm could be shown as powered off, and still be a process on the hypervisor, checking this is never a loss.
Hope some of the solutions will work for you
> Note it says the <VM Name>-flat.vmdk can't be opened due to an Invalid argument despite it's shown as having a size of 53687091200 on the above 'ls' output?
Can you open the flat.vmdk with hexdump -C name-flat.vmdk | less ?
I assume that the flat.vmdk has I/O errors ...
Hi just tried but am seeing errors like this.
[root@esxi6prim:/vmfs/volumes/557dc2e2-2c11eba8-1494-00259066ac58/<VM name>] ls -ltr
ls: ./<VM name>.vmx: No such file or directory
total 52833280
-rw-r--r-- 1 root root 0 Oct 4 2015 <VM name>.vmsd
-rw-r--r-- 1 root root 179092 Oct 9 2015 vmware-1.log
-rw-r--r-- 1 root root 177929 Oct 14 2015 vmware-2.log
-rw-r--r-- 1 root root 409599604 Nov 24 07:26 vmware-3.log
-rw------- 1 root root 524 Nov 24 09:55 <VM name>.vmdk
-rw------- 1 root root 53687091200 Feb 17 12:52 <VM name>-flat.vmdk
-rw------- 1 root root 8684 Mar 28 04:42 <VM name>.nvram
-rw-r--r-- 1 root root 270087 Mar 28 04:42 vmware.log
[root@esxi6prim:/vmfs/volumes/557dc2e2-2c11eba8-1494-00259066ac58/<VM name>]
[root@esxi6prim:/vmfs/volumes/557dc2e2-2c11eba8-1494-00259066ac58/<VM name>] hexdump -C '<VM name>-flat.vmdk' | less
[root@esxi6prim:/vmfs/volumes/557dc2e2-2c11eba8-1494-00259066ac58/<VM name>] hexdump -C '<VM name>-flat.vmdk'
hexdump: <VM name>-flat.vmdk: Invalid argument
[root@esxi6prim:/vmfs/volumes/557dc2e2-2c11eba8-1494-00259066ac58/<VM name>] which hexdump
/bin/hexdump
[root@esxi6prim:/vmfs/volumes/557dc2e2-2c11eba8-1494-00259066ac58/<VM name>] ls -ltr '<VM name>-flat.vmdk'
-rw------- 1 root root 53687091200 Feb 17 12:52 <VM name>-flat.vmdk
[root@esxi6prim:/vmfs/volumes/557dc2e2-2c11eba8-1494-00259066ac58/<VM name>] cp '<VM name>-flat.vmdk' ../.
cp: can't open '<VM name>-flat.vmdk': Invalid argument
I'm somewhat lost atm. I have taken a backup by copying out the directory and noticed now the backup itself is missing this <VM name>-flat.vmdk file in the backed up directory, despite it is showing a sizable 53687091200 bytes in the original directory. I suspect during the copy it was having the same issue with "Invalid argument".
A strace shows a copy operation failed during the system call 'open' with a returned error EINVAL which is Invalid Argument.
[root@esxi6prim:/vmfs/volumes/557dc2e2-2c11eba8-1494-00259066ac58/<VM name>] strace cp '<VM name>-flat.vmdk' ../.
execve("/bin/cp", ["cp", "<VM name>-fl"..., "../."], [/* 18 vars */]) = 0
[ Process PID=202994 runs in 32 bit mode. ]
brk(0) = 0x1f000000
uname({sys="VMkernel", node="esxi6prim", ...}) = 0
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib/tls/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/lib/tls", 0xffb11728) = -1 ENOENT (No such file or directory)
open("/lib/libm.so.6", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\2004\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0555, st_size=169969, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xffffffffffb12000
mmap2(NULL, 147584, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x259c000
mmap2(0x25bf000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x22) = 0x25bf000
close(3) = 0
open("/lib/libpam.so.0", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\27\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0555, st_size=43568, ...}) = 0
mmap2(NULL, 46380, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x25c1000
mmap2(0x25cc000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xa) = 0x25cc000
close(3) = 0
open("/lib/libpam_misc.so.0", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\n\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0555, st_size=8596, ...}) = 0
mmap2(NULL, 11376, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x25cd000
mmap2(0x25cf000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0x25cf000
close(3) = 0
open("/lib/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\214\1\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0555, st_size=1638750, ...}) = 0
mmap2(NULL, 1452696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x25d0000
mmap2(0x272d000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15c) = 0x272d000
mmap2(0x2730000, 10904, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x2730000
close(3) = 0
open("/lib/libdl.so.2", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\n\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0555, st_size=13168, ...}) = 0
mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x2733000
mmap2(0x2735000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0x2735000
close(3) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xffffffffffb13000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xffffffffffb14000
set_thread_area(0xffb11b94) = 0
mprotect(0x2735000, 4096, PROT_READ) = 0
mprotect(0x272d000, 8192, PROT_READ) = 0
mprotect(0x25bf000, 4096, PROT_READ) = 0
mprotect(0x259a000, 4096, PROT_READ) = 0
getuid32() = 0
stat64("<VM name>-flat.vmdk", {st_mode=S_IFREG|0600, st_size=53687091200, ...}) = 0
stat64("../.", {st_mode=S_IFDIR|S_ISVTX|0755, st_size=4620, ...}) = 0
brk(0) = 0x1f000000
brk(0x1f021000) = 0x1f021000
stat64("<VM name>-flat.vmdk", {st_mode=S_IFREG|0600, st_size=53687091200, ...}) = 0
lstat64(".././<VM name>-flat.vmdk", 0xffb11a90) = -1 ENOENT (No such file or directory)
open("<VM name>-flat.vmdk", O_RDONLY|O_LARGEFILE) = -1 EINVAL (Invalid argument)
write(2, "cp: can't open '<VM name>"..., 75cp: can't open '<VM name>-flat.vmdk': Invalid argument
) = 75
exit_group(1) = ?
[root@esxi6prim:/vmfs/volumes/557dc2e2-2c11eba8-1494-00259066ac58/<VM name>]
Would it be possible to do a screen share over the weekend?
I have other hosts but none of them are ESXi and the process isn't running anymore as I have rebooted the machine.
The problem is I am unable to start this VM. And in fact the original VM shown on the ESXi thick client isn't there anymore.
I was searching online and came across a suggestion to remove the VM, in order to try adding it back from the datastore with "Add to Inventory".
The problem was either the .vmx file wasn't there or it was there but it didn't work.
The VM isn't spreaded on different datastores.
I can't do the edit VM operation and check "vm-options->general options" anymore as the VM was removed from inventory.
If you are sure the VM is not registered;
Rebuild the vmx file you should have enough data in the vmware.log files in that folder to save yourself:
Try register the VM manually via the CLI: VMware Knowledge Base
vim-cmd solo/registervm /vmfs/volumes/datastore_name/VM_directory/VM_name.vmx
Then try power if on from either the gui or cli:
vim-cmd vmsvc/getallvms |grep <vm name>
vim-cmd vmsvc/power.on <vmid>
Thanks. I just did the first step but a few observations and am stuck.
http://www.aionsolution.com/it-support/vmware/recover-vmx-from-log-file/
#!/usr/bin/vmware
[root@esxi6prim]./Corp PROD - Atlassian PRIM.sh
-sh: ./Corp: not found
[root@esxi6prim]./Corp\ PROD\ -\ Atlassian\ PRIM.sh
vmware: Invalid arguments
vmware -v, --version Get version information for vmkernel
-l, --level Get release level information
-h, --help Print this menu
Questions:
[root@esxi6prim]ls -ltr 'Corp PROD - Atlassian PRIM-flat.vmdk'
-rw------- 1 root root 53687091200 Feb 17 12:52 Corp PROD - Atlassian PRIM-flat.vmdk
[root@esxi6prim]
[root@esxi6prim]cp -p 'Corp PROD - Atlassian PRIM-flat.vmdk' ../.
cp: can't open 'Corp PROD - Atlassian PRIM-flat.vmdk': Invalid argument
[root@esxi6prim]
[root@esxi6prim]pwd
/vmfs/volumes/datastore1/Corp PROD - Atlassian PRIM
[root@esxi6prim]
[root@esxi6prim]cp -p 'Corp PROD - Atlassian PRIM-flat.vmdk' /vmfs/volumes/datastore1/.
cp: can't open 'Corp PROD - Atlassian PRIM-flat.vmdk': Invalid argument
[root@esxi6prim]
If I tried to create the vmx file something strange happens as if the vmx file is still there and can't be overwritten despite it isn't shown by 'ls'.
[root@esxi6prim]cp 'Corp PROD - Atlassian PRIM.sh' 'Corp PROD - Atlassian PRIM.vmx'
cp: can't create 'Corp PROD - Atlassian PRIM.vmx': File exists
[root@esxi6prim]ls -lt
ls: ./Corp PROD - Atlassian PRIM.vmx: No such file or directory
rtotal 52833288
-rwxr-xr-x 1 root root 6781 Apr 4 04:34 Corp PROD - Atlassian PRIM.sh
-rw-r--r-- 1 root root 270087 Mar 28 04:42 vmware.log
-rw------- 1 root root 8684 Mar 28 04:42 Corp PROD - Atlassian PRIM.nvram
-rw------- 1 root root 53687091200 Feb 17 12:52 Corp PROD - Atlassian PRIM-flat.vmdk
-rw------- 1 root root 524 Nov 24 09:55 Corp PROD - Atlassian PRIM.vmdk
-rw-r--r-- 1 root root 409599604 Nov 24 07:26 vmware-3.log
-rw-r--r-- 1 root root 177929 Oct 14 2015 vmware-2.log
-rw-r--r-- 1 root root 179092 Oct 9 2015 vmware-1.log
-rw-r--r-- 1 root root 0 Oct 4 2015 Corp PROD - Atlassian PRIM.vmsd
'ls' uses the syscall 'lstat64' and is able to read the stats from the file. Anything like 'cp' or 'less' require the syscall 'open' and that's when it'd return an error. The actual error shown when I 'strace' it is the following:
open("Corp PROD - Atlassian PRIM-flat.vmdk", O_RDONLY|O_LARGEFILE) = -1 EINVAL (Invalid argument)
Looks like there are corruption on the VMFS volume. Yikes!
[root@esxi6prim:/var/log] grep -i corrupt /scratch/log/vmkernel.log
2020-04-04T04:28:08.827Z cpu6:346397)WARNING: DLX: 921: Volume 557dc2e2-2c11eba8-1494-00259066ac58 ("datastore1") might be damaged on the disk. Corrupt lock detected at offset 82253824
2020-04-04T04:28:49.702Z cpu3:346432)WARNING: DLX: 921: Volume 557dc2e2-2c11eba8-1494-00259066ac58 ("datastore1") might be damaged on the disk. Corrupt lock detected at offset 82253824
2020-04-04T04:29:47.035Z cpu9:346463)WARNING: DLX: 921: Volume 557dc2e2-2c11eba8-1494-00259066ac58 ("datastore1") might be damaged on the disk. Corrupt lock detected at offset 82253824
2020-04-04T05:02:45.627Z cpu11:347827)WARNING: DLX: 921: Volume 557dc2e2-2c11eba8-1494-00259066ac58 ("datastore1") might be damaged on the disk. Corrupt lock detected at offset 82253824
2020-04-04T05:03:21.536Z cpu2:347830)WARNING: DLX: 921: Volume 557dc2e2-2c11eba8-1494-00259066ac58 ("datastore1") might be damaged on the disk. Corrupt lock detected at offset 82253824
2020-04-04T05:05:29.728Z cpu9:347938)WARNING: DLX: 921: Volume 557dc2e2-2c11eba8-1494-00259066ac58 ("datastore1") might be damaged on the disk. Corrupt lock detected at offset 82253824
2020-04-04T05:05:45.786Z cpu12:347966)WARNING: DLX: 921: Volume 557dc2e2-2c11eba8-1494-00259066ac58 ("datastore1") might be damaged on the disk. Corrupt lock detected at offset 82253824
2020-04-04T05:16:08.766Z cpu5:348326)WARNING: DLX: 921: Volume 557dc2e2-2c11eba8-1494-00259066ac58 ("datastore1") might be damaged on the disk. Corrupt lock detected at offset 82253824
[root@esxi6prim:/var/log] less vmkwarning.log
2020-03-23T14:38:38.731Z cpu11:5371123)WARNING: [lockAddr 82253824] Invalid lock addr 4728680755771736379
2020-03-23T14:38:38.731Z cpu11:5371123)WARNING: [lockAddr 4728680755771736379] Invalid state: Owner 000a6b57-00000000-0000-000000000000 mode 0 numHolders 0 gblNumHolders 1501603542
2020-03-23T14:38:38.731Z cpu11:5371123)WARNING: DLX: 921: Volume 557dc2e2-2c11eba8-1494-00259066ac58 ("datastore1") might be damaged on the disk. Corrupt lock detected at offset 82253824
2020-03-23T14:38:38.731Z cpu11:5371123)WARNING: [type 81a4 offset 4728680755771736379 v 16, hb offset 4728680757079534805
2020-03-23T14:38:38.737Z cpu13:5371124)WARNING: Invalid lock address 7308814929491424288
2020-03-23T14:38:38.737Z cpu13:5371124)WARNING: [lockAddr 7308814929491424288] Invalid lock type 0x6e692065
2020-03-23T14:38:38.737Z cpu13:5371124)WARNING: [lockAddr 81836032] Invalid lock addr 7308814929491424288
2020-03-23T14:38:38.737Z cpu13:5371124)WARNING: [lockAddr 7308814929491424288] Invalid lock numHolders 1629513843
2020-03-23T14:38:38.737Z cpu13:5371124)WARNING: [lockAddr 7308814929491424288] Invalid lock mode 2036425843
2020-03-23T14:38:38.737Z cpu13:5371124)WARNING: DLX: 921: Volume 557dc2e2-2c11eba8-1494-00259066ac58 ("datastore1") might be damaged on the disk. Corrupt lock detected at offset 81836032
2020-03-23T14:38:38.737Z cpu13:5371124)WARNING: [type 6e692065 offset 7308814929491424288 v 7310842101425793127, hb offset 7526207048542349161
2020-03-23T14:38:39.131Z cpu19:5756607)WARNING: Invalid lock address 4728680755771736379
2020-03-23T14:38:39.131Z cpu19:5756607)WARNING: [lockAddr 4728680755771736379] Invalid lock type 0x81a4
2020-03-23T14:38:39.131Z cpu19:5756607)WARNING: [lockAddr 82253824] Invalid lock addr 4728680755771736379
2020-03-23T14:38:39.131Z cpu19:5756607)WARNING: [lockAddr 4728680755771736379] Invalid state: Owner 000a6b57-00000000-0000-000000000000 mode 0 numHolders 0 gblNumHolders 1501603542
2020-03-23T14:38:39.131Z cpu19:5756607)WARNING: DLX: 921: Volume 557dc2e2-2c11eba8-1494-00259066ac58 ("datastore1") might be damaged on the disk. Corrupt lock detected at offset 82253824
2020-03-23T14:38:39.131Z cpu19:5756607)WARNING: [type 81a4 offset 4728680755771736379 v 16, hb offset 4728680757079534805
2020-03-23T14:38:39.137Z cpu10:5756609)WARNING: Invalid lock address 7308814929491424288
2020-03-23T14:38:39.137Z cpu10:5756609)WARNING: [lockAddr 7308814929491424288] Invalid lock type 0x6e692065
2020-03-23T14:38:39.137Z cpu10:5756609)WARNING: [lockAddr 81836032] Invalid lock addr 7308814929491424288
2020-03-23T14:38:39.137Z cpu10:5756609)WARNING: [lockAddr 7308814929491424288] Invalid lock numHolders 1629513843
2020-03-23T14:38:39.137Z cpu10:5756609)WARNING: [lockAddr 7308814929491424288] Invalid lock mode 2036425843
2020-03-23T14:38:39.137Z cpu10:5756609)WARNING: DLX: 921: Volume 557dc2e2-2c11eba8-1494-00259066ac58 ("datastore1") might be damaged on the disk. Corrupt lock detected at offset 81836032
2020-03-23T14:38:39.137Z cpu10:5756609)WARNING: [type 6e692065 offset 7308814929491424288 v 7310842101425793127, hb offset 7526207048542349161
2020-03-23T14:38:39.531Z cpu5:5756602)WARNING: Invalid lock address 4728680755771736379
2020-03-23T14:38:39.531Z cpu5:5756602)WARNING: [lockAddr 4728680755771736379] Invalid lock type 0x81a4
2020-03-23T14:38:39.531Z cpu5:5756602)WARNING: [lockAddr 82253824] Invalid lock addr 4728680755771736379
2020-03-23T14:38:39.531Z cpu5:5756602)WARNING: [lockAddr 4728680755771736379] Invalid state: Owner 000a6b57-00000000-0000-000000000000 mode 0 numHolders 0 gblNumHolders 1501603542
2020-03-23T14:38:39.531Z cpu5:5756602)WARNING: DLX: 921: Volume 557dc2e2-2c11eba8-1494-00259066ac58 ("datastore1") might be damaged on the disk. Corrupt lock detected at offset 82253824
2020-03-23T14:38:39.531Z cpu5:5756602)WARNING: [type 81a4 offset 4728680755771736379 v 16, hb offset 4728680757079534805
2020-03-23T14:38:39.537Z cpu23:4729105)WARNING: Invalid lock address 7308814929491424288
2020-03-23T14:38:39.537Z cpu23:4729105)WARNING: [lockAddr 7308814929491424288] Invalid lock type 0x6e692065
2020-03-23T14:38:39.537Z cpu23:4729105)WARNING: [lockAddr 81836032] Invalid lock addr 7308814929491424288
2020-03-23T14:38:39.537Z cpu23:4729105)WARNING: [lockAddr 7308814929491424288] Invalid lock numHolders 1629513843
2020-03-23T14:38:39.537Z cpu23:4729105)WARNING: [lockAddr 7308814929491424288] Invalid lock mode 2036425843
Depending on the ESXi version, Checking Metadata Consistency with VOMA may help.
André
Ok ...
make sure lsof does not list "Corp PROD - Atlassian PRIM-flat.vmdk|
Try vmkfstools -p 0 "Corp PROD - Atlassian PRIM-flat.vmdk" > /tmp/flat.map
If you get /tmp/flat,map - send the file to me.
If you dont get /tmp/flat.map
we need to do more.
Download http://sanbarrow.com/livecds/moa64-nogui/MOA64-nogui-incl-src-111014-efi.iso
upload it to an unaffected datastore.
Create Ubuntu64 VM with 5gb ram and 2 vcpus.
Make sure it gets a nic that allows to talk to your ESXi.
I expect that you have putty, winscp and teamviewer/anydesk on your admin host.
We will very likely need to clone via Linux .....