8 Replies Latest reply on Jun 14, 2019 7:52 PM by insearchof

    Failed to Add VSAN to VMKernal

    insearchof Enthusiast

      VMWare 6.5

      I have two vCenters both running 6.5

      I have 9 ESXI hosts all running 6.5 and up to date will all patches.

       

      Trying to implement VSAN on both ESXI Hosts.

       

      1. Created a Datacenter on both vCenters.

      2. Add VSAN to my VMkernal on each ESXI host.

       

      I had a VMkernal already defined the same name on all 9 hosts.

       

      I started to add VSAN to each one and on several it added but a few it did not.

       

      After a while I tried again and they added the VSAN

       

      I have one ESXI 6.5 host that will not add it.

       

      A General System error occurred     any ideas?

       

      vmwarevmkernal.PNG

       

      Thank you

        • 1. Re: Failed to Add VSAN to VMKernal
          TheBobkin Virtuoso
          vExpertVMware Employees

          From what you have said it seems like a vCenter/client/account issue - are you positive all hosts have the same build version installed and the vCenter is on equivalent or higher build? (e.g. be more specific - build 8294253 not '6.5 U2').

          Any of the above potential causes can be ruled out via setting this via CLI e.g.

          # esxcli vsan network ip add -i vmkX

          # esxcli vsan network list

           

          Bob

          • 2. Re: Failed to Add VSAN to VMKernal
            insearchof Enthusiast

            Thebobkin

             

            First thank you for responding.

             

            The command failed and all hosts are at same version as you can see below.  Any other command to try?

             

            [root@TGCSESXI-7:~] esxcli vsan network ip add -i list

            Unable to complete Sysinfo operation.  Please see the VMkernel log file for more details.: Sysinfo error: Not foundSee VMkernel log for details.

             

            [root@TGCSESXI-7:~] esxcli vsan network list

            [root@TGCSESXI-7:~] esxcli system version get

               Product: VMware ESXi

               Version: 6.5.0

               Build: Releasebuild-13635690

               Update: 2

               Patch: 88

             

            [root@TGCSESXI-5:~] esxcli system version get

               Product: VMware ESXi

               Version: 6.5.0

               Build: Releasebuild-13635690

               Update: 2

               Patch: 88

             

            [root@TGCSESXI-6:~] esxcli system version get

               Product: VMware ESXi

               Version: 6.5.0

               Build: Releasebuild-13635690

               Update: 2

               Patch: 88

             

            [root@TGCSESXI-8:~] esxcli system version get

               Product: VMware ESXi

               Version: 6.5.0

               Build: Releasebuild-13635690

               Update: 2

               Patch: 88

             

             

            Thank you

             

            Tom

             

             

            Update   I tried this     VMK3 is the interface that I am trying to add VSAN to

             

            [root@TGCSESXI-7:~] esxcli network ip interface tag get -i vmk0

               Tags: Management

            [root@TGCSESXI-7:~] esxcli network ip interface tag get -i vmk1

               Tags:

            [root@TGCSESXI-7:~] esxcli network ip interface tag get -i vmk2

               Tags: VMotion

            [root@TGCSESXI-7:~] esxcli network ip interface tag get -i vmk3

               Tags:

             

            So then I ran this

             

            [root@TGCSESXI-7:~] esxcli network ip interface tag add -i vmk3 -t VSAN

            [root@TGCSESXI-7:~] esxcli network ip interface tag get -i vmk3

               Tags: VSAN

            [root@TGCSESXI-7:~]

             

            Now it has the Tag but when I look on vcenter the vmkernal still does not show the VSAN

             

             

            Thoughts

            • 3. Re: Failed to Add VSAN to VMKernal
              TheBobkin Virtuoso
              VMware EmployeesvExpert

              Hello Tom,

               

              You ran the commands wrong and thus the fail message - please look at these carefully - if ever in doubt about syntax simply type most of the command and -h , it will show you the next available options and syntax.

              Please also inform of both vCenter build versions.

               

              Bob

              • 4. Re: Failed to Add VSAN to VMKernal
                insearchof Enthusiast

                Hi this time I copied it from your posting

                 

                [root@TGCSESXI-7:~] esxcli vsan network ip add -i vmk3

                Unable to load module /usr/lib/vmware/vmkmod/cmmds: Out of memory

                 

                 

                 

                vCenter  6.5.0

                 

                Build 7119157

                 

                OUT OF Memory??????????

                • 5. Re: Failed to Add VSAN to VMKernal
                  TheBobkin Virtuoso
                  vExpertVMware Employees

                  Unlikely your problem here but don't manage hosts with a vCenter of lower version - this needs to be on the same or higher release version (e.g. 6.5 U2 here):

                  VMware Product Interoperability Matrices

                   

                  So, the network configuration can't be configured because one of the underlying vSAN services (CMMDS) is having issues - are these nested hosts or physical servers and with what specs?

                   

                  Bob

                  • 6. Re: Failed to Add VSAN to VMKernal
                    insearchof Enthusiast

                    Bob

                     

                    They are all Physical hosts

                     

                    On this vCenter/Cluster

                     

                    Dell PowerEdge R210     Host 1   VSAN working

                     

                    Dell PowerEdge R300     Host 2  VSAN working

                     

                     

                    Dell PowerEdge R210     Host 3 VSAN NOT WORKING

                     

                     

                    Dell PowerEdge R310     Host 4 VSAN Working

                     

                     

                     

                    I tried running this   ./sbin/services.sh restart   but I did not receive any responses on the console.

                    I thought about restarting the services on the host but that did not work

                     

                    Any ideas?

                     

                    • 7. Re: Failed to Add VSAN to VMKernal
                      TheBobkin Virtuoso
                      vExpertVMware Employees

                      None of those servers are supported to run anything past ESXi 5.5 so random instability such as this comes with the territory - I hope this is a homelab .

                      You *might* fare better with ESXi 6.0 on these.

                       

                      If there are zombie processes that can't be stopped and/or something hogging inits (already limited) memory then restarting services likely isn't going to help (as the services don't cleanly go down). Try rebooting the host. If you have any 3rd-party plug-ins, monitoring tools or utilities that eat resources (e.g. RecoverPoint) then consider temporarily disabling these.

                       

                      Bob

                      • 8. Re: Failed to Add VSAN to VMKernal
                        insearchof Enthusiast

                        restarting the host took care of this

                         

                        Thanks everyone