1 2 Previous Next 26 Replies Latest reply on Jun 21, 2019 8:05 PM by insearchof

    VSAN Datastore ZERO

    insearchof Enthusiast

      VMware 6.5

       

      Just install SSD drives on my ESXI Hosts and configured VSAN.

       

      After claiming the drive on 4 hosts

       

      The VSANDatastore was created on all the hosts but has Zero space available

       

      Any reason why ??

       

      Thank you

       

      Tom

        • 1. Re: VSAN Datastore ZERO
          TheBobkin Virtuoso
          VMware EmployeesvExpert

          Hello Tom,

           

          By claimed SSDs do you mean that you have configured vSAN Disk-Groups?

          If these have been created then other causes can be hosts in Maintenance Mode or the cluster being network-partitioned - vSAN Health check should inform further:

          Cluster > Monitor > vSAN > Health

           

          Bob

          • 2. Re: VSAN Datastore ZERO
            insearchof Enthusiast

            vsanhealth.PNG

            Hi

             

            I did not create a disk group although I see GOUP 1 on the display

             

            The error above not helpful what is cluster error?

             

            vsancap.PNG

            • 3. Re: VSAN Datastore ZERO
              depping Champion
              User ModeratorsVMware Employees

              Check the physical disks section to see if any disks were claimed, cause it seems something has failed. I would also recommend using the H5 interface if you have vSphere 6.7

              • 4. Re: VSAN Datastore ZERO
                insearchof Enthusiast

                Hello

                 

                I am running VMWARE 6.5 not 6.7

                 

                I believe all my claimed disks are H5  see images

                 

                vsandisk.PNG

                 

                vsandiskhd.PNG

                 

                vsandcerror.PNG

                 

                The error on the Data Center reflects that I do not have any capacity

                 

                The Disk Format version column shows 5 that's H5 you mentioned correct?

                 

                IF the disks are claimed which I did manually I do not understand why I do not have a capacity

                 

                Any other way to check command line ??

                 

                Thank you.

                 

                Tom

                • 5. Re: VSAN Datastore ZERO
                  matze007 Lurker

                  Did you check the Firmware Version?

                   

                  You can look here in the compatibility list:

                  VMware Compatibility Guide - vsan

                  • 6. Re: VSAN Datastore ZERO
                    TheBobkin Virtuoso
                    VMware EmployeesvExpert

                    Check whether the nodes have a non-zero vSAN decom state:

                    # cmmds-tool find -t NODE_DECOM_STATE

                     

                    Per your other Communities post (assuming it is the same cluster and igf so, no idea why you started another thread) you are losing packets which strongly indicates you are having network issues and/or it is is misconfigured - if the cluster is partitioned or flapping then this would explain 0 bytes storage.

                    As an aside "vmkping -I VMK1 10.10.10.10 -s 9000 -c 100" as MJMSRI  advised is not a very good test as a) you have not indicated whether you are using jumbo frames (and thus would need to even check large MTU), b) 9000 byte test pings WILL actually pass through 1500 MTU network unless you set 'don't fragment' flag (-d) and c) in a  correctly configured VMware environment using jumbo frames 9000 won't actually pass as we have our own additional packet headers (28 bytes). The correct way to check it (if you DO have Jumbo frames enabled) would be:

                    # vmkping -I vmkX -s 8972 -d <DestIP>

                     

                    Bob

                    • 7. Re: VSAN Datastore ZERO
                      MJMSRI Novice

                      Not a very good test? But then you advised to use the same test and simply changed the command slightly from 9000 to 8972 and removed -c which is for count so the command runs for a set amount.

                      • 8. Re: VSAN Datastore ZERO
                        TheBobkin Virtuoso
                        VMware EmployeesvExpert

                        "simply changed the command slightly from 9000 to 8972"

                        Yes because -s 9000 -d won't pass on a functional Jumbo-frame enabled network (unless it has been raised to something higher than necessary on the swtich) - test it if you do not believe me.

                        And without -d it doesn't validate whether full Jumbo-frame can pass (which is what I assume you were advising to test as otherwise I don't know why you would specify a size value at all).

                         

                        Bob

                        • 9. Re: VSAN Datastore ZERO
                          insearchof Enthusiast

                          To All

                           

                          This Host has this error warning and wondering if this is the cause

                           

                          Also do I need to open the VSAN rule sets on the Hosts firewall settings?

                           

                          I have another issue opened with this error  but thought it might be relevant

                          vsanissue.PNG

                           

                          I ran this on one host that seems ok and the one in the picture above

                           

                          [root@TGCSESXI-9:~]
                          cmmds-tool find -t NODE_DECOM_STATE

                           

                          owner=598cfc1b-c9e1-d832-d6a2-f04da207add3(Health:
                          Healthy) uuid=598cfc1b-c9e1-d                                                                                      
                          832-d6a2-f04da207add3 type=NODE_DECOM_STATE rev=2 minHostVer=0  [content = (i0 i                                                                                      
                          0 UUID_NULL i0 [ ] i0 i0 i0)], errorStr=(null)

                           

                          owner=5c69a839-08c6-a636-c148-782bcb054c38(Health:
                          Healthy) uuid=5c69a839-08c6-a                                                                                      
                          636-c148-782bcb054c38 type=NODE_DECOM_STATE rev=2 minHostVer=0  [content = (i0 i                                                                                      
                          0 UUID_NULL i0 [ ] i0 i0 i0)], errorStr=(null)

                           

                          owner=5a598968-4aa6-8b4a-0925-782bcb1399f7(Health:
                          Healthy) uuid=5a598968-4aa6-8                                                                                      
                          b4a-0925-782bcb1399f7 type=NODE_DECOM_STATE rev=5 minHostVer=0  [content = (i0 i                                                                                      
                          0 UUID_NULL i0 [ ] i0 i0 i0)], errorStr=(null)

                           

                          owner=5710250b-cfcf-67fc-9dc0-a4badb376d3f(Health:
                          Healthy) uuid=5710250b-cfcf-6                                                                                      
                          7fc-9dc0-a4badb376d3f type=NODE_DECOM_STATE rev=2 minHostVer=0  [content = (i0 i                                                                                      
                          0 UUID_NULL i0 [ ] i0 i0 i0)], errorStr=(null)

                           

                          Host with Issue

                           

                          [root@TGCSESXI-4:~]
                          cmmds-tool find -t NODE_DECOM_STATE

                           

                           

                          [root@TGCSESXI-4:~]

                           

                          Thank you

                          • 10. Re: VSAN Datastore ZERO
                            insearchof Enthusiast

                            [root@TGCSESXI-4:~] vmkping -I vmk2 -s 8972 -d 10.2.8.62
                            PING 10.2.8.62 (10.2.8.62): 8972 data bytes
                            sendto() failed (Message too long)
                            sendto() failed (Message too long)
                            sendto() failed (Message too long)

                            --- 10.2.8.62 ping statistics ---
                            3 packets transmitted, 0 packets received, 100% packet loss
                            [root@TGCSESXI-4:~]

                             

                             

                            ???

                             

                            I tried on another host with same results

                             

                             

                            [root@TGCSESXI-9:~] vmkping -I vmk2 -s 8972 -d 10.2.8.62
                            PING 10.2.8.62 (10.2.8.62): 8972 data bytes
                            sendto() failed (Message too long)
                            sendto() failed (Message too long)
                            sendto() failed (Message too long)

                            --- 10.2.8.62 ping statistics ---
                            3 packets transmitted, 0 packets received, 100% packet loss
                            [root@TGCSESXI-9:~]

                            • 11. Re: VSAN Datastore ZERO
                              daphnissov Guru
                              vExpertCommunity Warriors

                              These messages indicate you don't have jumbo frames configured properly on your external switching infrastructure.

                              • 12. Re: VSAN Datastore ZERO
                                insearchof Enthusiast

                                How do I check that?

                                 

                                 

                                Where do I do that also?

                                 

                                On the Network adapters on the ESXI host?

                                • 13. Re: VSAN Datastore ZERO
                                  daphnissov Guru
                                  vExpertCommunity Warriors

                                  You need to check everything end-to-end including the vswitch where these vmkernel ports are connected and your physical switches.

                                  • 14. Re: VSAN Datastore ZERO
                                    insearchof Enthusiast

                                    Hello

                                     

                                    I changed the MTU to 9000 on all Hosts vswitch and vmkernal adapters.

                                    I saw Update vSAN configuration in recent tasks

                                     

                                    And then the vsandatastore had 6.2 GB

                                     

                                    Then I thought wow it is working

                                     

                                    After I updated the last host then the Vsandatastore went back to ZERO

                                     

                                    What else can we check

                                     

                                    Something is wrong here?

                                    1 2 Previous Next