9 Replies Latest reply on Nov 5, 2019 7:07 AM by MaxRayRus

    Who has seen a voma module named advfix in action and is willing to talk about it ?

    continuum Guru
    User ModeratorsCommunity WarriorsvExpert

      So far unconfirmed rumours claim that ESXi 6.7 u 2 comes with a voma module called "advfix".

      According to the rumours that module can actually really clean up stale locks in a VMFS 6 heartbeat section.

      Has anybody seen that module in action ? ... and is willing to talk about it ?

      I am aware that this may be regarded as a delicate and intimate question so feel free to contact me via PMs or email if you do not want to talk about this in public.

       

      Ulli

        • 1. Re: Who has seen a voma module named advcfg in action and is willing to talk about it ?
          Tayfun DEGER Hot Shot
          vExpert

          Hi,

           

          With vSphere 6.7, voma version 0.8 has been released and VMFS 6 supports it.

           

          voma -m vmfs -f advfix -d

           

          There is a command called advfix but I haven't encountered advcfg before.

           

          Thanks.

          --
          Blog: https://www.tayfundeger.com
          Twitter: https://www.twitter.com/tayfundeger

          vBlogger, vExpert, Cisco Champions

          Please, if this solution helped your problem, "Helpful" if it solves your problem "Correct Answer" to mark.
          1 person found this helpful
          • 2. Re: Who has seen a voma module named advcfg in action and is willing to talk about it ?
            continuum Guru
            User ModeratorsCommunity WarriorsvExpert

            Oops - sorry - my bad.

            The option I was looking for is indeed called advfix - not advcfg.

            1 person found this helpful
            • 3. Re: Who has seen a voma module named advfix in action and is willing to talk about it ?
              Tayfun DEGER Hot Shot
              vExpert

              Hi continuum

               

              No problem, you can see the command example and output below.

               

              [root@esxi01:~] voma -m vmfs -f check -d /vmfs/devices/disks/vml.0000000000766d686261303a303a30

              Checking if device is actively used by other hosts

              Running VMFS Checker version 2.1 in check mode

              Initializing LVM metadata, Basic Checks will be done

              Initializing LVM metadata..-

              LVM magic not found at expected Offset,

              It might take long time to search in rest of the disk.

               

               

              VMware ESX Question:

              Do you want to continue (Y/N)?

               

               

              0) _Yes

              1) _No

               

               

              Select a number from 0-1: 0

               

               

              Phase 1: Checking VMFS header and resource files

                 Detected VMFS-6 file system (labeled:'datastore1') with UUID:5ba6e058-63704ba3-f4fa-0050568cb081, Version 6:82

              Phase 2: Checking VMFS heartbeat region

              Phase 3: Checking all file descriptors.

              Phase 4: Checking pathname and connectivity.

              Phase 5: Checking resource reference counts.

               

               

              Total Errors Found:           0

               

              [root@esxi01:~] voma -m vmfs -f advfix -d /vmfs/devices/disks/vml.0000000000766d686261303a303a30 -p /vmfs/volumes/datastore1/

               

               

              ########################################################################

              #   Warning !!!                                                        #

              #                                                                      #

              #   You are about to execute VOMA in Advanced Fix mode. This mode can  #

              #   potentially end up in deleting some files/vmdks from datastore     #

              #   if it is heavily corrupted.                                        #

              #                                                                      #

              #   This mode is suppored only under VMware Support supervision.       #

              ########################################################################

               

               

              VMware ESX Question:

              Do you want to continue (Y/N)?

               

               

              0) _Yes

              1) _No

               

               

              Select a number from 0-1: 0

               

               

                 Successfully created patch file /vmfs/volumes/datastore1/mpx.vmhba0:C0:T0:L0_Wed_Oct__2_09:57:21_2019

              Checking if device is actively used by other hosts

              Running VMFS Checker version 2.1 in advfix mode

              Initializing LVM metadata, Basic Checks will be done

              Initializing LVM metadata..-

              LVM magic not found at expected Offset,

              It might take long time to search in rest of the disk.

               

               

              VMware ESX Question:

              Do you want to continue (Y/N)?

               

               

              0) _Yes

              1) _No

               

               

              Select a number from 0-1: 0

               

               

                       ERROR: Fix Mode is not yet supported for VMFS-6

                 VOMA failed to check device : Not Supported

               

               

              Total Errors Found:           0

              Total Errors Fixed:           0

              Total Partially Fixed errors: 0

                 Kindly Consult VMware Support for further assistance

              [root@esxi01:~]

              --
              Blog: https://www.tayfundeger.com
              Twitter: https://www.twitter.com/tayfundeger

              vBlogger, vExpert, Cisco Champions

              Please, if this solution helped your problem, "Helpful" if it solves your problem "Correct Answer" to mark.
              1 person found this helpful
              • 4. Re: Who has seen a voma module named advfix in action and is willing to talk about it ?
                continuum Guru
                User ModeratorsvExpertCommunity Warriors

                Wow - I have to admit that I  was not prepared at all to get an answer like this.

                Last questions I posted rather had a subtext like "the official documentation about topic XY really sucks- fix it - it is anoying that we dont have a good documentation about this topic"

                And now I get an answer much faster than expected from a new user that I had not even noticed before  - and as the icing on the cake that guy goes by the name Tayfun.

                What a nice surprise -welcome to the madhouse  - hope to read more from you in future.

                 

                Let me explain why I asked this question ...

                I first heard about the advfix option from a user-comment in my vm-sickbay blog

                Locked files with VMFS 6 – updated for ESXi 6.5 and higher | VM-Sickbay

                The advfix option really looked like a possible solution - at least thats how I understood that comment.

                The deeper I looked into the problem the more improbable it appeared to me that we would get a tool like that from VMware itself.

                Then some weeks went by until I got a new support request on a vmfs-6 locked files problem last week.

                In that case the result of a few bad entries in the heartbeat section have so drastic effects that it is really ridiculous to see that VMware is no help at all with issues like that.

                For me it is obviously more fun to claim that Voma is a bluff - especially if I am able to prove my point and at the same time can offer help with positive feedback.

                On the other hand this special case I am dealing with right now is so ugly that I would actually really welcome a working advfix voma option.

                 

                Now your reply answers all qiestions I had:

                > ERROR: Fix Mode is not yet supported for VMFS-6

                No further questions ...

                Assumption: Voma is a bluff: confirmed.
                You just saved me from half a week of experiments ....

                Thanks


                Let me give you a warning about VMTN  ...

                It can happen very easily that you hang around here a lot, enjoy yourself and feel like you are at home among friends ...

                You may even start to think that you are acchieving something good here  ...

                Well thats how it started in my case ...( VMTN-addict since 2004 )

                 

                Tayfun - it was a pleasure to meet you - welcome

                 

                Ulli

                1 person found this helpful
                • 5. Re: Who has seen a voma module named advfix in action and is willing to talk about it ?
                  Tayfun DEGER Hot Shot
                  vExpert

                  Thank you for your good wishes

                  --
                  Blog: https://www.tayfundeger.com
                  Twitter: https://www.twitter.com/tayfundeger

                  vBlogger, vExpert, Cisco Champions

                  Please, if this solution helped your problem, "Helpful" if it solves your problem "Correct Answer" to mark.
                  • 6. Re: Who has seen a voma module named advfix in action and is willing to talk about it ?
                    MaxRayRus Lurker

                    Thank you for the hint on undocumented options. If we look, then neither adfix, nor -p are described in the help on voma.

                    Can you tell me, if there is a way to start the advfix on the partition on which the ESXi are directly installed. For all my attempts, the system refuses under the pretext that this partition is used by the host (in fact, this is the localhost, in my case 130.21).

                    How do i repair the VMFS on such installations? Can i unmount or other way? I tried to choose shift+R/O even when loading, there is no way to openl the console for cli ((

                     

                    [root@ESXi-U1-DL8:~] esxcli storage vmfs extent list
                    Volume Name  VMFS UUID                            Extent Number  Device Name                           Partition
                    -----------  -----------------------------------  -------------  ------------------------------------  ---------
                    datastore1   5d4468de-4daee3be-0f47-509a4c9492de              0  naa.64cd98f01bea570024d71af30c1c246a          3
                    [root@ESXi-U1-DL8:~] voma -m vmfs -f advfix -d /vmfs/devices/disks/naa.64cd98f01bea570024d71af30c1c246a:3 -p /vmfs/volumes/datastore1/
                    
                       Successfully created patch file /vmfs/volumes/datastore1/naa.64cd98f01bea570024d71af30c1c246a:3_Sun_Nov__3_08:56:58_2019
                    Checking if device is actively used by other hosts
                    Scanning for VMFS-6 host activity (4096 bytes/HB, 1024 HBs).
                    Found 1 actively heartbeating hosts on device '/vmfs/devices/disks/naa.64cd98f01bea570024d71af30c1c246a:3'
                    1): MAC address 50:9a:4c:94:92:de, IP 192.168.130.21
                    [root@ESXi-U1-DL8:~] voma -v
                    voma version 0.8
                    
                    
                    
                    
                    • 7. Re: Who has seen a voma module named advfix in action and is willing to talk about it ?
                      continuum Guru
                      User ModeratorsCommunity WarriorsvExpert

                      Hi

                      in my recovery work I have no use for voma - so I am not uptodate with the latest versions.

                      When I saw that the advfix option is still readonly for vmfs 6 I was quite disapointed ...

                      Anyway - what do you want to acchieve ?

                      Do you have trouble with locked files too ?

                      Then I may have workarounds ...

                      Ulli

                      • 8. Re: Who has seen a voma module named advfix in action and is willing to talk about it ?
                        MaxRayRus Lurker

                        I had a power outage on this host. After starting the voma in chk mode, many errors were found. Accordingly, now I'm trying to understand, how to fix them.

                         

                        Apparently voma, this is the only one stock tool for this.

                         

                        ...end of voma chk-log:

                         

                        ON-DISK ERROR: LFB inconsistency found: (454,10) allocated in bitmap, but never used

                        ON-DISK ERROR: LFB inconsistency found: (454,11) allocated in bitmap, but never used

                        ON-DISK ERROR: LFB inconsistency found: (454,12) allocated in bitmap, but never used

                        ON-DISK ERROR: LFB inconsistency found: (454,13) allocated in bitmap, but never used

                        ON-DISK ERROR: LFB inconsistency found: (454,14) allocated in bitmap, but never used

                        ON-DISK ERROR: LFB inconsistency found: (454,15) allocated in bitmap, but never used

                         

                        Total Errors Found:           4732

                         

                        • 9. Re: Who has seen a voma module named advfix in action and is willing to talk about it ?
                          MaxRayRus Lurker

                          If, however, we will consider third-party file system repair tools (standart linux tools), then I tried to use your assembly based on ubuntu: http://sanbarrow.com/livecds/moa64-nogui/MOA64-nogui-incl-src-111014-efi.iso

                          As we can see, version 6 of the vmfs is not supported (on screenshot) ((

                          MOA_ubuntu14_recovery_info_from_ESXihost.PNG

                          ver_miss.PNG