8 Replies Latest reply on Apr 25, 2020 7:36 AM by TheBobkin

    VCSA Thick to Thin Provision on VxRail

    BarryUWSEFS Enthusiast

      I have the nagging

      "vSAN online health alarm 'Thick-provisioned VMs on vSAN' " and I finally decided to investigate further now that I have time. The VM in question is one of my two VCSA. According to the VCSA summary page it is taking up a whopping 1.73 TB. My first question is, Isn't that really huge for VCSA??

      According to  VMware Knowledge Base 66758 there are several options for converting this to thin. Because of the size of it, and because I have never performed this before, I could use some advice on the best option to convert this to thin. I do have two vCenters that are linked, each controls a different VxRail cluster. So I can perform operations on the thick VCSA from the other VCSA.

       

      It appears that the most popular option is to vStoragemotion to another SAN and back. In my case that would mean I would move it from one VxRail cluster to the other, and back. Because of the size I am concerned it would take forever. This VCSA runs our Horizon desktops, and I fear that having it turned off for a long period of time is going to disrupt VM provisoning etc. of our desktop pools.

       

      Any advice on how to handle this is much appreciated!

       

       

       

       

        • 2. Re: VCSA Thick to Thin Provision on VxRail
          depping Champion
          VMware EmployeesUser Moderators

          You may want to verify with Dell/EMC what their support stance is on this. I also wonder if it is "thick" provisioned or if they have a policy with 100% space reservation. I haven't looked at VxRail in a while, but I believe it was 100% reserved.

          • 3. Re: VCSA Thick to Thin Provision on VxRail
            TheBobkin Virtuoso
            vExpertVMware Employees

            Hello Barry,

             

            Can you clarify whether the vmdks are set as thick at the primitive level (e.g. they have provisioned type of Eager-Zero Thick or Thick) and/or whether the 'Thick' is applied via a Storage Policy (SP) with Object Space Reservation = 100?

            If it is the former then unfortunately you will need to SvMotion/Clone the VM to essentially create new Thin Objects, if it is just set at the SP level then you can just apply a SP with Object Space Reservation = 0 (or one without this rule at all which will default to it being 0) - provided you don't apply a SP with any other rule changes that would change the structure of the Objects (e.g. different Stripe-width, FTM or FTT) this shouldn't do a complete rebuild of the Objects and just thin them in-place, though to be cautious of space-usage etc. you could just apply the new SP to one vmdk at a time and validate whether you have resultant resyncs or not.

             

            Regarding SvMotion to another cluster - you don't have to power off the vCenter to do this and also it may not take as long if there isn't actually 1.7TB written to disk (and yes that is relatively huge) - can you SSH to the vCenter and run 'df -h' to see what the current usage is there?

             

            Bob

            • 4. Re: VCSA Thick to Thin Provision on VxRail
              BarryUWSEFS Enthusiast

              depping, we are using the default vSAN storage policy, and I am not sure how to find out if it is 100% reserved, but the object space reservation is thin provisioning. I also believe this VM is thick because it was deployed from an OVA and never previously changed to thin VMware Knowledge Base

              I think you are right though, should see what Dell/EMC has to say.

              • 5. Re: VCSA Thick to Thin Provision on VxRail
                BarryUWSEFS Enthusiast

                Hi Bob,

                 

                We only have and use the vSAN default storage policy which is thin provision. I believe it is thick because it was deployed from an OVA VMware Knowledge Base or actually, I had to restore it from the administration backup a while back, which is an OVA deployment. If this powercli is correct then it is only using 69.1GB!

                PS C:\WINDOWS\system32> Get-VM vcenter* -datastore vxrail* | Select Name,

                >> @{N="HD Used (GB)";E={[math]::Round(($_.Guest.Disks | %{

                >>       $_.CapacityGB - $_.FreeSpaceGB} | Measure-Object -Sum |

                >>       Select -ExpandProperty Sum),1)}}

                 

                Name            HD Used (GB)

                ----            ------------

                vCenter-WOPR              80

                vCenterMadrona1         69.1

                • 6. Re: VCSA Thick to Thin Provision on VxRail
                  TheBobkin Virtuoso
                  VMware EmployeesvExpert

                  Hello Barry,

                   

                  Might you be able to share (or PM me) the output of the following that will show all the information relating to this VMs vmdk storage usage:

                  # esxcli vsan debug object list --vm-name=<VMNameHere>

                  or

                  # esxcli vsan debug object list -g <UUID of the VMs namespace folder>

                   

                  Bob

                  • 7. Re: VCSA Thick to Thin Provision on VxRail
                    BarryUWSEFS Enthusiast

                    Hi Bob, the output was huge from those commands! I PM'd so as not to take up miles of space here.

                    Thanks

                     

                    I also did open a ticket with Dell/EMC. The first option he recommended was vStorageMotion to our other vSAN, but it required a vCenter to vCenter vMotion and apparently our "standard" vSphere license does not permit cross vCenter vMotion. Then the option to clone the vCenter. However I ran into a very strange issue with the clone wizard, which prevented that for now. That will be the subject of a different post. But as of for now I have not done anything to solve this.

                    • 8. Re: VCSA Thick to Thin Provision on VxRail
                      TheBobkin Virtuoso
                      vExpertVMware Employees

                      Hello Barry,

                       

                      "the output was huge from those commands! I PM'd so as not to take up miles of space here."

                      Yes, because it is the information for the policy, DOM and LSOM layout of 41 Objects (13 base-disks, 26 snapshots, vswp and namespace).

                       

                      First off, that vCenter looks to be using ~250GB on disk - the base vmdks are not thin, they have "proportionalCapacity: 100" meaning they are Thick-provisioned at the SPVM level.

                      Start with consolidating the snapshots as re-applying Storage Policies (SP) now will only apply it to the last disks in the chain.

                      Check what your actual 'vSAN Default Storage Policy' has in it's rules - my 'vSAN Default Storage Policy' doesn't necessarily equal your 'vSAN Default Storage Policy' as this is basically just a label/template and the rules of what it defines can be manually changed.

                      If it doesn't have 'Object Space Reservation = 100' (same thing as 'proportionalCapacity: 100') then this *should* be 'Object Space Reservation = 0' by default.

                      If without snapshots and OSR=0 (or rule is not present) in the SP and the VM is showing as compliant with it's SP when checked, depending on how/when this vCenter was initially created/migrated to this cluster these may be unintentionally Thick - please if you could ask DellEMC to open a B2B case with VMware vSAN team and ask for Paul Twomey (me) so that I can take a further look, I have what is likely a simple fix for such issues.

                       

                      Bob