11 Replies Latest reply on May 10, 2019 6:35 AM by TheBobkin

    vSAN Datastore Capacity Report... again

    pirx666 Novice

      Hi,

       

      I'm scratching my head and can't find the right solution. I've a littel powercli script that reports a summary of our clusters (because vROPs too complicated for most people here). Thus I need some data for our vSAN datastores. Sounds easy right?

       

      What I get from Get-VsanSpaceUsage:

       

      CapacityGB           : 41079,41796875

      FreeSpaceGB          : 23406,8515624544

      VirtualDiskUsageGB   : 9794,29296875

      VMHomeUsageGB        : 961,78125

      FileSystemOverheadGB : 601,609375045635

      ChecksumOverheadGB   : 50,53125

      PrimaryVMDataGB      : 8494,50390625

      VsanOverheadGB       : 9178,06250004564

       

       

      This is a 7 node Hybrid vSAN cluster (7x900GB capacity tier). We use FTT=1 and 25% buffer for reblancing.

       

       

      So I added this to my report:

       

       

                      vSAN Capacity (RAW):                                               40.12 TB

                      vSAN Free Capacity (RAW):                                       22.90 TB

                      vSAN Max. Usable Capacity:                                      15.04 TB   (40.12 / 2 * 0.75)

                      vSAN Used Capacity (vDisks, Swap, Overhead):       17.22 TB

       

      But this doesn't look right to me. My max usable capacity is 15TB and the used capacity is already 17TB. I guess it's because FTT=1 is already there in the numbers.

       

      My goal is pretty simple, I want to add a number to which capacity we can deploy new VM's.

       

       

       

        • 1. Re: vSAN Datastore Capacity Report... again
          TheBobkin Virtuoso
          VMware EmployeesvExpert

          Hello pirx666,

           

           

          "But this doesn't look right to me. My max usable capacity is 15TB and the used capacity is already 17TB. I guess it's because FTT=1 is already there in the numbers."

          Yes, it includes the replica data that's what the 'VsanOverheadGB' usage is in addition to the 'PrimaryVMDataGB'.

           

          "My goal is pretty simple, I want to add a number to which capacity we can deploy new VM's."

          That would be 'vSAN Capacity (RAW)' 40.12TB * 0.75 e.g. 'vSAN Used Capacity' of ~30TB

           

          You have a fair chunk of 'VM overreserved' there (5.6TB) - is this intentional?

          e.g. some VMs with Object Space Reservation = 100 or vswp set to (pre-6.7) default of Thick?

          The latter can of course be manually changed:

          Set Sparse Swap files for vSAN 6.2 or Greater - Samples - VMware {code}

           

           

          Bob

          • 2. Re: vSAN Datastore Capacity Report... again
            pirx666 Novice

            "That would be 'vSAN Capacity (RAW)' 40.12TB * 0.75 e.g. 'vSAN Used Capacity' of ~30TB"

             

            Okay, maybe I was a bit unprecise, I need to know how much storage we can assign to VM's. This 30TB will result in 15TB of vmdk's with FTT=1. So we can deploy VM's with 15TB of vmdk's/vDisks? So my "vSAN Max. Usable Capacity:  15.04 TB   (40.12 / 2 * 0.75)" was correct - but maybe not named correctly.

             

            Is there place where the output of Get-VsanSpaceUsage command (VMHomeUsageGB etc) is described? Online help did not show much. I've to check the other topic you have raised.

            • 3. Re: vSAN Datastore Capacity Report... again
              TheBobkin Virtuoso
              VMware EmployeesvExpert

              Hello pirx666

               

               

              VMHomeUsageGB refers to the amount of space used by namespace folders of registered VMs - 'VM Home Objects' in the capacity break-down, the rest are all fairly explanatory other than 'Other' which is data stored in namespaces folders not associated with a VM e.g. if you created a folder on vsanDatastore for storing ISOs.

              Namespaces (that I have seen) for VMs typically use ~900MB per VM as base and then whatever else MB/low-GB of usage depending on the caveats for your OS and usage varying e.g. Citrix XenDesktop places small -flat.vmdk in some places, older vSAN versions use file not Object-based for .vmem data etc. .

              If you want to see how this looks directly from a host and what uses what then use:

              # ls -lahS /vmfs/volumes/vsanDatastoreName/*

               

              Jasemccarty  might be able to answer more on this, what with being author of one of the only public documentation of this cmdlet usage (the code.vmware cmdlet doesn't expound on the specifics of any output types that I can find):

              https://storagehub.vmware.com/section-assets/powercli-cookbook-for-vsan

               

               

              Bob

              • 4. Re: vSAN Datastore Capacity Report... again
                Jasemccarty Champion
                VMware EmployeesvExpert

                This is covered in v1.4 of the PowerCLI Cookbook for vSAN on page 87 (as updates are published, this page may change).

                The section is: Useable vSAN Capacity based on Storage Policy

                 

                PowerCLI 11.2 Added the VsanWhatIfCapacity property was added to the Get-VsanSpaceUsage cmdlet.

                 

                First you have to grab the policy that you'll use, and then

                 

                Here's the code from the example in the cookbook:

                PS /> $DefaultPolicy = Get-SpbmStoragePolicy -Name "vSAN Default Storage Policy" 
                PS /> $RAID5Policy = Get-SpbmStoragePolicy -Name "RAID5"
                PS /> (Get-VsanSpaceUsage -StoragePolicy $DefaultPolicy).VsanWhatIfCapacity
                StoragePolicy                TotalWhatIfCapacityGB FreeWhatIfCapacityGB
                -------------                --------------------- --------------------
                vSAN Default Storage Policy             4471.21875     4201.66796873882

                PS /> (Get-VsanSpaceUsage -StoragePolicy $RAID5Policy).VsanWhatIfCapacity
                StoragePolicy TotalWhatIfCapacityGB FreeWhatIfCapacityGB
                ------------- --------------------- --------------------
                RAID5               6706.8298017066     6302.50352873374

                Remember that different policies will give you different usable space.

                 

                The FreeWhatIfCapacityGB is what I think you are looking for.

                • 5. Re: vSAN Datastore Capacity Report... again
                  pirx666 Novice

                  Thanks, this looks good but we are still on 6.5U2 and "VMware PowerCLI 6.5 Release 1 build 4624819". Even with powercli 11.2+ it will not work, because of 6.5U2?

                  • 6. Re: vSAN Datastore Capacity Report... again
                    Jasemccarty Champion
                    VMware EmployeesvExpert

                    pirx666 I'll have to spin up a 6.5U2 cluster and see if there's an issue.

                     

                    The API has been pretty standard since vSAN 6.2, so I'd imagine it would work.

                    I'm assuming you're using vC/VCSA 6.5U2 and ESXi 6.5U2?

                     

                    Happy to check for you.


                    Update: Deployed a test VCSA 6.5U2c and vSphere 6.5U2 environment

                     

                    It looks this is exposed in vCenter 6.7. Here's what I found:

                     

                    PS /Users/jase/PowerCLI> $Global:DefaultVIServer | Select Name, Version, Build

                     

                    Name         Version Build

                    ----         ------- -----

                    10.160.23.90 6.5.0   10964411

                     

                    PS /Users/jase/PowerCLI> (Get-VsanSpaceUsage -StoragePolicy (Get-SpbmStoragePolicy -Name "vSAN Default Storage Policy")) |fl

                     

                    Cluster              : vSAN-Cluster

                    CapacityGB           : 719.9296875

                    FreeSpaceGB          : 710.332031241618

                    VirtualDiskUsageGB   :

                    VMHomeUsageGB        :

                    FileSystemOverheadGB : 8.6484375083819

                    ChecksumOverheadGB   : 0.94921875

                    PrimaryVMDataGB      : 0

                    VsanOverheadGB       : 9.5976562583819

                    IscsiTargetUsageGB   :

                    IscsiLunUsedGB       :

                    VsanWhatIfCapacity   : {}

                    So it doesn't appear it is exposed in vCenter 6.5x


                    Jase

                    • 7. Re: vSAN Datastore Capacity Report... again
                      pirx666 Novice

                      Thanks for checking, I had to fight with powercli 11.2 update, but from the description in the cookbook I already assumed it's only for 6.7.

                      • 8. Re: vSAN Datastore Capacity Report... again
                        Jasemccarty Champion
                        VMware EmployeesvExpert

                        Some stuff in the cookbook still works on 6.5/6.6... Hit and miss.

                         

                        Given 6.7 was the latest, that's where I started when I wrote it.
                        Might be able to get the same from a Get-VsanView, but I'd have to play with it.

                        • 9. Re: vSAN Datastore Capacity Report... again
                          pirx666 Novice

                          I can't even use Get-SpbmStoragePolicy cmdlet.

                           

                          PS D:\scripts> Get-SpbmStoragePolicy -Name "vSAN Default Storage Policy"

                          Get-SpbmStoragePolicy : 09.05.2019 13:10:39     Get-SpbmStoragePolicy           There is an error in the XML document.

                           

                           

                          At line:1 char:1

                          + Get-SpbmStoragePolicy -Name "vSAN Default Storage Policy"

                          + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                              + CategoryInfo          : NotSpecified: (:) [Get-SpbmStoragePolicy], VimException

                              + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.Storage.Commands.Cmdlets.Spbm.GetStora

                             geProfile

                           

                           

                          Get-SpbmStoragePolicy : 09.05.2019 13:10:39     Get-SpbmStoragePolicy           SpbmStoragePolicy with name 'vSAN Defaul

                          t Storage

                          Policy' was not found using the specified filter(s).

                          At line:1 char:1

                          + Get-SpbmStoragePolicy -Name "vSAN Default Storage Policy"

                          + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                              + CategoryInfo          : ObjectNotFound: (:) [Get-SpbmStoragePolicy], VimException

                              + FullyQualifiedErrorId : Core_OutputHelper_WriteNotFoundError,VMware.VimAutomation.Storage.Commands.Cmdlets.Spbm.

                             GetStorageProfile

                          • 10. Re: vSAN Datastore Capacity Report... again
                            pirx666 Novice

                            I've to bring this up again, because I'm still struggling. I've compared the webclient view with powercli output. Seems that webclient shows more values than powercli output.

                             

                            - If 'Used - Physicially written' is 11,16 TB, how can 'Free' be only 22,78 TB (I'd expect 40,12 - 11,16 = 29 TB free)

                            - where is Swap Objects in Get-VsanSpaceUsage included?

                            - There are VirtualDiskUsageGB, VsanOverheadGB, PrimaryVMDataGB

                             

                            > Yes, it includes the replica data that's what the 'VsanOverheadGB' usage is in addition to the 'PrimaryVMDataGB'.

                             

                            So, VsanOverheadGB + PrimaryVMDataGB is the total used data (really everything incl FTT=1) that is stored on the datastore?

                             

                            Web Client vSAN Capacity Overview Get-VsanSpaceUsage
                            Capacity40,12 TBCapacityGB41079 GB
                            Used - Physicially written11,16 TB
                            Used - VM overreserved5,54 TB
                            Used - Total16,70 TB
                            vSAN system overhead653,02 GBFileSystemOverheadGB + ChecksumOverheadGB652
                            Free22,78 TBFreeSpaceGB23323 GB
                            Virtual Disks9,70 TBVirtualDiskUsageGB9935
                            VM Home Objects964,66 GBVMHomeUsageGB964
                            Swap Objects5,68 TB
                            Performance management objects103,86
                            File system overhead602,49FileSystemOverheadGB602
                            Checksum overhead50,53 GBChecksumOverheadGB50
                            Other286,12 GB
                            PrimaryVMDataGB8535
                            VsanOverheadGB9219

                             

                             

                            CapacityGB           : 41079,41796875

                            FreeSpaceGB          : 23323,9453124544

                            VirtualDiskUsageGB   : 9935,5078125

                            VMHomeUsageGB        : 964,6640625

                            FileSystemOverheadGB : 602,468750045635

                            ChecksumOverheadGB   : 50,53125

                            PrimaryVMDataGB      : 8535,48046875

                            VsanOverheadGB       : 9219,99218754564

                            IscsiTargetUsageGB   :

                            IscsiLunUsedGB       :

                            VsanWhatIfCapacity   : {}

                            • 11. Re: vSAN Datastore Capacity Report... again
                              TheBobkin Virtuoso
                              VMware EmployeesvExpert

                              Hello pirx666,

                               

                               

                              "- If 'Used - Physicially written' is 11,16 TB, how can 'Free' be only 22,78 TB (I'd expect 40,12 - 11,16 = 29 TB free)"

                              No, because you have Thick vswps and while the space for these is not 'Physically written' it is reserved and thus not usable/free.

                               

                              "So, VsanOverheadGB + PrimaryVMDataGB is the total used data (really everything incl FTT=1) that is stored on the datastore?"

                              Yes including vswps - just not including namespace and FS-overhead usage.

                               

                               

                              Bob