Can't start VM or Copy [Invalid argument]

[akha666]

I have got an issue when start my VM in ESXi 5.5 connected to OpenFiler storage after power failure , and I when I create new_VM and assign existing VM.vmdk I got error msg

"cannot open the disk '/vmfs/volumes/xxxxx-xxxxxx/VM/VM.vmdk' or one of the snapshot disks it depends on"

and I can't copy any file of this VM Folder I got error [Invalid argument]

[akha666]

here is my voma log

Checking if device is actively used by other hosts

Running VMFS Checker version 1.0 in check mode

Initializing LVM metadata, Basic Checks will be done

Phase 1: Checking VMFS header and resource files

   Detected VMFS file system (labeled:'4TB') with UUID:54450bfc-89706a36-abeb-001517aff7f2, Version 5:60

Phase 2: Checking VMFS heartbeat region

ON-DISK ERROR: Invalid HB address <10758557897232416816>

Phase 3: Checking all file descriptors.

   <FD c264 r1> : Wrong Epoch Block Count 354 should be 41156

   Found stale lock [type 10c00001 offset 132714496 v 39, hb offset 3772416

  gen 143, mode 1, owner 545247ff-d487db1b-40e5-0015c5f9ca62 mtime 11873

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 132804608 v 125, hb offset 3772416

  gen 197, mode 1, owner 54ce8dae-81d0bc43-e5b8-0015c5f9ca62 mtime 1485

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 132806656 v 126, hb offset 3772416

  gen 197, mode 1, owner 54ce8dae-81d0bc43-e5b8-0015c5f9ca62 mtime 1500

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 132808704 v 127, hb offset 3772416

  gen 197, mode 1, owner 54ce8dae-81d0bc43-e5b8-0015c5f9ca62 mtime 1511

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 132810752 v 128, hb offset 3772416

  gen 197, mode 1, owner 54ce8dae-81d0bc43-e5b8-0015c5f9ca62 mtime 1535

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 132812800 v 129, hb offset 3772416

  gen 197, mode 1, owner 54ce8dae-81d0bc43-e5b8-0015c5f9ca62 mtime 1562

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 215189504 v 197, hb offset 3772416

  gen 197, mode 1, owner 54ce8dae-81d0bc43-e5b8-0015c5f9ca62 mtime 1502

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 215207936 v 51, hb offset 3772416

  gen 197, mode 1, owner 54ce8dae-81d0bc43-e5b8-0015c5f9ca62 mtime 1726

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 215212032 v 54, hb offset 3772416

  gen 197, mode 1, owner 54ce8dae-81d0bc43-e5b8-0015c5f9ca62 mtime 1724

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 215216128 v 51, hb offset 3772416

  gen 197, mode 1, owner 54ce8dae-81d0bc43-e5b8-0015c5f9ca62 mtime 1730

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 234899456 v 43, hb offset 4009984

  gen 29, mode 1, owner 548faa26-00567a32-4b23-001517aff7f2 mtime 21414

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 234950656 v 132, hb offset 4009984

  gen 29, mode 1, owner 548faa26-00567a32-4b23-001517aff7f2 mtime 21283

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 234960896 v 60, hb offset 4009984

  gen 15, mode 1, owner 5419d88f-17cd53a3-b39a-001517aff7f2 mtime 64933

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 234971136 v 105, hb offset 4009984

  gen 29, mode 1, owner 548faa26-00567a32-4b23-001517aff7f2 mtime 21411

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 234975232 v 98, hb offset 4009984

  gen 29, mode 1, owner 548faa26-00567a32-4b23-001517aff7f2 mtime 21417

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 235026432 v 191, hb offset 4009984

  gen 15, mode 1, owner 5419d88f-17cd53a3-b39a-001517aff7f2 mtime 116846

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 235032576 v 198, hb offset 4009984

  gen 29, mode 1, owner 548faa26-00567a32-4b23-001517aff7f2 mtime 21269

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 235034624 v 199, hb offset 4009984

  gen 29, mode 1, owner 548faa26-00567a32-4b23-001517aff7f2 mtime 21281

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 235036672 v 200, hb offset 4009984

  gen 29, mode 1, owner 548faa26-00567a32-4b23-001517aff7f2 mtime 21292

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 235038720 v 201, hb offset 4009984

  gen 29, mode 1, owner 548faa26-00567a32-4b23-001517aff7f2 mtime 21316

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 235040768 v 202, hb offset 4009984

  gen 29, mode 1, owner 548faa26-00567a32-4b23-001517aff7f2 mtime 21331

  num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 235044864 v 576, hb offset 4010496

  gen 239, mode 1, owner 54ce5756-aa396d57-3e1e-001517aff7f2 mtime 40249

  num 0 gblnum 0 gblgen 0 gblbrk 0]

Phase 4: Checking pathname and connectivity.

Phase 5: Checking resource reference counts.

ON-DISK ERROR: FB inconsistency found: (12257,0) allocated in bitmap, but never used

Total Errors Found:           2

how can I fix the 2 error ?????

[continuum]

Try to read that volume with Linux. The vmfs-fuse driver seems to ignore heartbeat-errors.
Sometimes it also helps to resignature the volume.

[akha666]

I got this Idea and I boot from Ubuntu live USB and install vmfs-tools

but I face and trouble that I have to mount RAID in Ubuntu , because my Openfiler Storage on RAID 10

I'll try and feed back

thanks for replay

[continuum]

I see.
In such a case I connect the Linux-LiveCD to the ESXi that sees the storage via sshfs.
Then you can use vmfs-fuse without having to use iscsi or fibrechannel from Linux.

mkdir /esxi
mkdir /vmfs
sshfs root@esxihostname:/ /esxi
cd /esxi/dev/disks
vmfs-fuse naa.*:1 /vmfs

[akha666]

Thank you continuum

Great help , you are really point me to the right way.

Finally I got myVM copied out the storage, Your sshfs method working for me

But I got another workaround for fast copy from storage, I would like to share

1- Install Ubuntu 12.04 LTS or whatever LIVE USB

2- Install ISCSI Initiator

     iSCSI Initiator

3- Connect Ubuntu to My Storage [OpenFiler]

4- Mount Storage contain myVM

5- Compile vmfs-tools from here

     VMFS: Unsupported version 5 - How to mount VMFS5 on Ubuntu |My Technical Blog

6- Mount VMFS in Ubuntu @ /mnt

7- I can read all VMs Files and copy them to another location or my local HD

8- Create NewVM in VMWare Workstation and attached existing VMDK

Thank you again my issues is solved