6 Replies Latest reply on Jul 18, 2018 6:07 AM by continuum

    Trouble in VMFS-unknown version on ESXi 6.5 but it still can be accessed and reached out is vmfs6 by using recovery software

    kaikaisd Lurker

      After i reinstall the ESXi , and my SSD can't load the filesystem to vmfs6 ,then i have made a lot of research to repair the filesystem to vmfs6 , but there are need to pay a huge money for a key.Also, i have already tested and passed by voma with 0 error , So , does anyone can help me to repair the filesystem to give me to access the files because i have a lot of important data on a single VM.

       

      REPORT 1:

      [root@server:~] df -h

                Filesystem   Size   Used Available Use% Mounted on

      VMFS-unknown version   0.0B   0.0B      0.0B   0% /vmfs/volumes/SSD 128G

      vfat                 249.7M   8.0K    249.7M   0% /vmfs/volumes/238e93e3-dfb17682-5562-ef490f6ff7d6

      vfat                 249.7M 143.6M    106.1M  58% /vmfs/volumes/bd4b17e7-3f022a3f-d65c-938d21b31e31

      vfat                 285.8M 205.8M     80.0M  72% /vmfs/volumes/495c2082-1ded4d5a-d9f4-74d4356b4eb2

       

      REPORT 2(VOMA):

      [root@server:/vmfs/volumes] voma -m vmfs -d /vmfs/devices/disks/naa.5707c181005a45cc

      Checking if device is actively used by other hosts       

      Running VMFS Checker version 2.1 in default mode         

      Initializing LVM metadata, Basic Checks will be done     

      Phase 1: Checking VMFS header and resource files         

         Detected VMFS-6 file system (labeled:'SSD 128G') with UUID:5aa2a8e4-6d0f985e-f0ef-74d4356b4eb2, Version 6:81

         Found stale lock [type 10c00002 offset 9756672 v 58, hb offset 3637248

      gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 524426

      num 0 gblnum 0 gblgen 0 gblbrk 0]

      Phase 2: Checking VMFS heartbeat region                  

      Marking Journal addr (5, 0) in use                       

      Phase 3: Checking all file descriptors.                  

         Found stale lock [type 10c00001 offset 34799616 v 172, hb offset 3637248

      gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2719

      num 0 gblnum 0 gblgen 0 gblbrk 0]

         Found stale lock [type 10c00001 offset 34807808 v 42, hb offset 3637248

      gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2777

      num 0 gblnum 0 gblgen 0 gblbrk 0]

         Found stale lock [type 10c00001 offset 34873344 v 38, hb offset 3637248

      gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2788

      num 0 gblnum 0 gblgen 0 gblbrk 0]

         Found stale lock [type 10c00001 offset 34996224 v 48, hb offset 3637248

      gen 111, mode 1, owner 5aa54177-0ea4b366-0f31-74d4356b4eb2 mtime 295

      num 0 gblnum 0 gblgen 0 gblbrk 0]

         Found stale lock [type 10c00001 offset 35086336 v 67, hb offset 3637248

      gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2706

      num 0 gblnum 0 gblgen 0 gblbrk 0]

         Found stale lock [type 10c00001 offset 35094528 v 68, hb offset 3637248

      gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2717

      num 0 gblnum 0 gblgen 0 gblbrk 0]

         Found stale lock [type 10c00001 offset 35102720 v 69, hb offset 3637248

      gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2729

      num 0 gblnum 0 gblgen 0 gblbrk 0]

         Found stale lock [type 10c00001 offset 35110912 v 70, hb offset 3637248

      gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2754

      num 0 gblnum 0 gblgen 0 gblbrk 0]

         Found stale lock [type 10c00001 offset 35119104 v 71, hb offset 3637248

      gen 127, mode 1, owner 5aa5469e-265dc7e2-b24d-74d4356b4eb2 mtime 2764

      num 0 gblnum 0 gblgen 0 gblbrk 0]

      Phase 4: Checking pathname and connectivity.             

      Phase 5: Checking resource reference counts.             

                                                               

      Total Errors Found:           0

        • 1. Re: Trouble in VMFS-unknown version on ESXi 6.5 but it still can be accessed and reached out is vmfs6 by using recovery software
          a.p. Guru
          vExpertCommunity WarriorsUser Moderators

          Welcome to the Community,

           

          what's the output of the following commands:

          esxcli storage filesystem list

          esxcli storage vmfs snapshot list

           

          André

          • 2. Re: Trouble in VMFS-unknown version on ESXi 6.5 but it still can be accessed and reached out is vmfs6 by using recovery software
            kaikaisd Lurker

            Hello , a.p.

            Thank for your reply first

            here is the result

             

            [root@server:~] esxcli storage filesystem list

             

            Mount Point                                        Volume Name  UUID                                 Mounted  Type                       Size       Free

            -------------------------------------------------  -----------  -----------------------------------  -------  --------------------  ---------  ---------

                                                                        SSD 128G     5aa2a8e4-6d0f985e-f0ef-74d4356b4eb2    false  VMFS-unknown version          0          0

            /vmfs/volumes/bd4b17e7-3f022a3f-d65c-938d21b31e31               bd4b17e7-3f022a3f-d65c-938d21b31e31     true  vfat                  261853184  111271936

            /vmfs/volumes/495c2082-1ded4d5a-d9f4-74d4356b4eb2               495c2082-1ded4d5a-d9f4-74d4356b4eb2     true  vfat                  299712512   83927040

            /vmfs/volumes/238e93e3-dfb17682-5562-ef490f6ff7d6               238e93e3-dfb17682-5562-ef490f6ff7d6     true  vfat                  261853184  261844992

            [root@server:~]

            [root@server:~] esxcli storage vmfs snapshot list

            [root@server:~]

            • 3. Re: Trouble in VMFS-unknown version on ESXi 6.5 but it still can be accessed and reached out is vmfs6 by using recovery software
              a.p. Guru
              vExpertUser ModeratorsCommunity Warriors

              According to the output ithe datastore is just not mounted, so we need to find out why.

               

              Do you receive an error message (please provide the output) when you try to mount the SSD manually

              esxcli storage filesystem mount --volume-label "SSD 128G"

              and/or

              esxcli storage filesystem mount --volume-uuid 5aa2a8e4-6d0f985e-f0ef-74d4356b4eb2

               

              In case mounting doesn't work, run

              esxcli storage core adapter rescan --all

              and then check whether the vmkernel.log contains helpful information, i.e. a reason why ESXi won't mount the datastore.

               

              André

              • 4. Re: Trouble in VMFS-unknown version on ESXi 6.5 but it still can be accessed and reached out is vmfs6 by using recovery software
                kaikaisd Lurker

                Here is the result:

                 

                 

                [root@server:~] esxcli storage filesystem mount --volume-label "SSD 128G"

                Unable to complete Sysinfo operation.  Please see the VMkernel log file for more details.: Sysinfo error: TimeoutSee VMkernel log for details.

                [root@server:~]

                [root@server:~] esxcli storage core adapter rescan --all

                [root@server:~]

                [root@server:~] cat /var/log/vmkernel.log

                ...(is the same output of the line at below)

                2018-04-10T19:43:57.273Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

                2018-04-10T19:43:57.313Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

                2018-04-10T19:43:57.318Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

                2018-04-10T19:43:57.358Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

                2018-04-10T19:43:57.363Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

                2018-04-10T19:43:57.404Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

                2018-04-10T19:43:57.408Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

                2018-04-10T19:43:57.449Z cpu1:65562)ScsiDeviceIO: 2927: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0xb 0x0 0x0.

                2018-04-10T19:43:57.454Z cpu1:65562)ScsiDeviceIO: 2962: Cmd(0x439500978f00) 0x2a, CmdSN 0x2 from world 66753 to dev "naa.5707c181005a45cc" failed H:0x3 D:0x0 P:0x0 Invalid sense data: 0x0 0x0 0x0.

                2018-04-10T19:43:57.454Z cpu4:67909 opID=76c37a09)LVM: 6491: Forcing APD unregistration of devID 5aa2a8e4-6a577140-a6c9-74d4356b4eb2 in state 1.

                2018-04-10T19:43:57.454Z cpu4:67909 opID=76c37a09)LVM: 15060: Failed to open device naa.5707c181005a45cc:1 : Timeout

                2018-04-10T19:44:00.775Z cpu7:67909 opID=c47e0e68)World: 12230: VC opID esxcli-09-8436 maps to vmkernel opID c47e0e68

                2018-04-10T19:44:00.775Z cpu7:67909 opID=c47e0e68)WARNING: xpt_scsi_adapter_discover:1224: unable to find target 0: No connection

                2018-04-10T19:44:00.775Z cpu7:67909 opID=c47e0e68)WARNING: xpt_scsi_adapter_discover:1224: unable to find target 1: No connection

                2018-04-10T19:44:00.775Z cpu7:67909 opID=c47e0e68)WARNING: xpt_scsi_adapter_discover:1224: unable to find target 0: No connection

                2018-04-10T19:44:00.775Z cpu7:67909 opID=c47e0e68)WARNING: xpt_scsi_adapter_discover:1224: unable to find target 1: No connection

                2018-04-10T19:44:00.775Z cpu7:67909 opID=c47e0e68)WARNING: xpt_scsi_adapter_discover:1224: unable to find target 0: No connection

                2018-04-10T19:44:00.776Z cpu7:67909 opID=c47e0e68)WARNING: xpt_scsi_adapter_discover:1224: unable to find target 1: No connection

                2018-04-10T19:44:00.779Z cpu6:65751)ScsiScan: 446: Path 'vmhba32:C0:T0:L0': Failed to read VPD Serial id page: Not supported

                2018-04-10T19:44:10.084Z cpu0:65981)NMP: nmp_ResetDeviceLogThrottling:3348: last error status from device naa.5707c181005a45cc repeated 6760 times

                • 5. Re: Trouble in VMFS-unknown version on ESXi 6.5 but it still can be accessed and reached out is vmfs6 by using recovery software
                  SrAdamski Lurker

                  Hello Kaika,

                   

                  I have pretty similar problem here, how you fixed this problem?

                  • 6. Re: Trouble in VMFS-unknown version on ESXi 6.5 but it still can be accessed and reached out is vmfs6 by using recovery software
                    continuum Guru
                    Community WarriorsUser ModeratorsvExpert

                    Do not assume that being able to display the content of a suspicious VMFS-volume with thirdparty VMFS-readers also means that it should be possible to mount the VMFS-volume again with ESXi.
                    Anyway - if you provide a VMFS header dump I can analyse the volume for you.
                    See Create a VMFS-Header-dump using an ESXi-Host in production | VM-Sickbay
                    Ulli
                    Be careful: using commands you found in a forum or in a Knowledgebase article maybe fine in the case for which those commands were created.
                    Do not use any specific commands you found for a "similar case" for your problem. There are no "similar" VMFS-problems .... treat each case as a different beast.