VMware Cloud Community
kaikaisd
Contributor
Contributor

Trouble in VMFS-unknown version on ESXi 6.5 but it still can be accessed and reached out is vmfs6 by using recovery software

After i reinstall the ESXi , and my SSD can't load the filesystem to vmfs6 ,then i have made a lot of research to repair the filesystem to vmfs6 , but there are need to pay a huge money for a key.Also, i have already tested and passed by voma with 0 error , So , does anyone can help me to repair the filesystem to give me to access the files because i have a lot of important data on a single VM.

REPORT 1:

[root@server:~] df -h

          Filesystem   Size   Used Available Use% Mounted on

VMFS-unknown version   0.0B   0.0B      0.0B   0% /vmfs/volumes/SSD 128G

vfat                 249.7M   8.0K    249.7M   0% /vmfs/volumes/238e93e3-dfb17682-5562-ef490f6ff7d6

vfat                 249.7M 143.6M    106.1M  58% /vmfs/volumes/bd4b17e7-3f022a3f-d65c-938d21b31e31

vfat                 285.8M 205.8M     80.0M  72% /vmfs/volumes/495c2082-1ded4d5a-d9f4-74d4356b4eb2

REPORT 2(VOMA):

[root@server:/vmfs/volumes] voma -m vmfs -d /vmfs/devices/disks/naa.5707c181005a45cc

Checking if device is actively used by other hosts       

Running VMFS Checker version 2.1 in default mode         

Initializing LVM metadata, Basic Checks will be done     

Phase 1: Checking VMFS header and resource files         

   Detected VMFS-6 file system (labeled:'SSD 128G') with UUID:5aa2a8e4-6d0f985e-f0ef-74d4356b4eb2, Version 6:81

   Found stale lock [type 10c00002 offset 9756672 v 58, hb offset 3637248

gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 524426

num 0 gblnum 0 gblgen 0 gblbrk 0]

Phase 2: Checking VMFS heartbeat region                  

Marking Journal addr (5, 0) in use                       

Phase 3: Checking all file descriptors.                  

   Found stale lock [type 10c00001 offset 34799616 v 172, hb offset 3637248

gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2719

num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 34807808 v 42, hb offset 3637248

gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2777

num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 34873344 v 38, hb offset 3637248

gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2788

num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 34996224 v 48, hb offset 3637248

gen 111, mode 1, owner 5aa54177-0ea4b366-0f31-74d4356b4eb2 mtime 295

num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 35086336 v 67, hb offset 3637248

gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2706

num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 35094528 v 68, hb offset 3637248

gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2717

num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 35102720 v 69, hb offset 3637248

gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2729

num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 35110912 v 70, hb offset 3637248

gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2754

num 0 gblnum 0 gblgen 0 gblbrk 0]

   Found stale lock [type 10c00001 offset 35119104 v 71, hb offset 3637248

gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2764

num 0 gblnum 0 gblgen 0 gblbrk 0]

Phase 4: Checking pathname and connectivity.             

Phase 5: Checking resource reference counts.             

                                                         

Total Errors Found:           0

Tags (4)
Reply
0 Kudos
6 Replies
a_p_
Leadership
Leadership

Welcome to the Community,

what's the output of the following commands:

esxcli storage filesystem list

esxcli storage vmfs snapshot list

André

Reply
0 Kudos
kaikaisd
Contributor
Contributor

Hello , a.p.

Thank for your reply first

here is the result

[root@server:~] esxcli storage filesystem list

Mount Point                                        Volume Name  UUID                                 Mounted  Type                       Size       Free

-------------------------------------------------  -----------  -----------------------------------  -------  --------------------  ---------  ---------

                                                            SSD 128G     5aa2a8e4-6d0f985e-f0ef-74d4356b4eb2    false  VMFS-unknown version          0          0

/vmfs/volumes/bd4b17e7-3f022a3f-d65c-938d21b31e31               bd4b17e7-3f022a3f-d65c-938d21b31e31     true  vfat                  261853184  111271936

/vmfs/volumes/495c2082-1ded4d5a-d9f4-74d4356b4eb2               495c2082-1ded4d5a-d9f4-74d4356b4eb2     true  vfat                  299712512   83927040

/vmfs/volumes/238e93e3-dfb17682-5562-ef490f6ff7d6               238e93e3-dfb17682-5562-ef490f6ff7d6     true  vfat                  261853184  261844992

[root@server:~]

[root@server:~] esxcli storage vmfs snapshot list

[root@server:~]

Reply
0 Kudos
a_p_
Leadership
Leadership

According to the output ithe datastore is just not mounted, so we need to find out why.

Do you receive an error message (please provide the output) when you try to mount the SSD manually

esxcli storage filesystem mount --volume-label "SSD 128G"

and/or

esxcli storage filesystem mount --volume-uuid 5aa2a8e4-6d0f985e-f0ef-74d4356b4eb2

In case mounting doesn't work, run

esxcli storage core adapter rescan --all

and then check whether the vmkernel.log contains helpful information, i.e. a reason why ESXi won't mount the datastore.

André

Reply
0 Kudos
kaikaisd
Contributor
Contributor

Here is the result:

[root@server:~] esxcli storage filesystem mount --volume-label "SSD 128G"

Unable to complete Sysinfo operation.  Please see the VMkernel log file for more details.: Sysinfo error: TimeoutSee VMkernel log for details.

[root@server:~]

[root@server:~] esxcli storage core adapter rescan --all

[root@server:~]

[root@server:~] cat /var/log/vmkernel.log

...(is the same output of the line at below)

2018-04-10T19:43:57.273Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

2018-04-10T19:43:57.313Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

2018-04-10T19:43:57.318Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

2018-04-10T19:43:57.358Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

2018-04-10T19:43:57.363Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

2018-04-10T19:43:57.404Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

2018-04-10T19:43:57.408Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

2018-04-10T19:43:57.449Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

2018-04-10T19:43:57.454Z cpu1:65562)ScsiDeviceIO: 2962: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x3 D:0x0 P:0x0 Invalid sense data: 0x0 0x0 0x0.

2018-04-10T19:43:57.454Z cpu4:67909 opID=76c37a09)LVM: 6491: Forcing APD unregistration of devID 5aa2a8e4-6a577140-a6c9-74d4356b4eb2 in state 1.

2018-04-10T19:43:57.454Z cpu4:67909 opID=76c37a09)LVM: 15060: Failed to open device naa.5707c181005a45cc:1 : Timeout

2018-04-10T19:44:00.775Z cpu7:67909 opID=c47e0e68)World: 12230: VC opID esxcli-09-8436 maps to vmkernel opID c47e0e68

2018-04-10T19:44:00.775Z cpu7:67909 opID=c47e0e68)WARNING: xpt_scsi_adapter_discover:1224: unable to find target 0: No connection

2018-04-10T19:44:00.775Z cpu7:67909 opID=c47e0e68)WARNING: xpt_scsi_adapter_discover:1224: unable to find target 1: No connection

2018-04-10T19:44:00.775Z cpu7:67909 opID=c47e0e68)WARNING: xpt_scsi_adapter_discover:1224: unable to find target 0: No connection

2018-04-10T19:44:00.775Z cpu7:67909 opID=c47e0e68)WARNING: xpt_scsi_adapter_discover:1224: unable to find target 1: No connection

2018-04-10T19:44:00.775Z cpu7:67909 opID=c47e0e68)WARNING: xpt_scsi_adapter_discover:1224: unable to find target 0: No connection

2018-04-10T19:44:00.776Z cpu7:67909 opID=c47e0e68)WARNING: xpt_scsi_adapter_discover:1224: unable to find target 1: No connection

2018-04-10T19:44:00.779Z cpu6:65751)ScsiScan: 446: Path 'vmhba32:C0:T0:L0': Failed to read VPD Serial id page: Not supported

2018-04-10T19:44:10.084Z cpu0:65981)NMP: nmp_ResetDeviceLogThrottling:3348: last error status from device naa.5707c181005a45cc repeated 6760 times

Reply
0 Kudos
SrAdamski
Contributor
Contributor

Hello Kaika,

I have pretty similar problem here, how you fixed this problem?

Reply
0 Kudos
continuum
Immortal
Immortal

Do not assume that being able to display the content of a suspicious VMFS-volume with thirdparty VMFS-readers also means that it should be possible to mount the VMFS-volume again with ESXi.
Anyway - if you provide a VMFS header dump I can analyse the volume for you.
See Create a VMFS-Header-dump using an ESXi-Host in production | VM-Sickbay
Ulli
Be careful: using commands you found in a forum or in a Knowledgebase article maybe fine in the case for which those commands were created.
Do not use any specific commands you found for a "similar case" for your problem. There are no "similar" VMFS-problems .... treat each case as a different beast.


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos