Trouble converting thick to thin on vsan 6.7u1

[RainerJM]

Hi

I have found a number of VMs on our vSAN clusters that have thick-provisioned disks, and no matter what I try I can't get them to convert to thin.

The default storage policy has OSR=0, everything shows as compliant with this, but some VMs are definitely thick provisioned: they show as thick in the web client, in powershell and rvc (proportionalCapacity = 100).

I've tried creating another SP with the same settings as the default policy and adding a rule for 'IOPS Limit=0' - when I applied this to a VM it did say it would result in reclaimed space, but nothing happened except that proportionalCapacity is now 0, no change in vmdk size and the disks still show as thick in the client and powershell.

I found one article suggesting that space is only reclaimed if you do this via powershell, but that seems to make no difference either.

Any suggestions? These are the discussions I've found, none of which seem to work for me:

https://communities.vmware.com/message/2728535#2728535

https://communities.vmware.com/thread/592237

https://communities.vmware.com/thread/588664

https://www.virtuallyghetto.com/2016/06/heads-up-ovfova-always-deployed-as-thick-on-vsan-when-using-...

I only have vsan datastores available to me, so I can't svmotion it off and back and change it that way

Thanks in advance

[TheBobkin]

Hello RainerJM​

Welcome to Communities.

Thanks for doing the due diligence and doing some research and trying some stuff before asking - we ned more like yourself around here :smileygrin: .

If the proportionalCapacity=0 after applying the workaround I mentioned in the other posts then the usage of the VM is likely organic - e.g. a 100GB vmdk with 100GB used inside the Guest-OS (or max EVER used was 100GB) will be 200GB on datastore (with FTT=1, FTM=RAID1) regardless of whether it is thin or thick.

If you are using significantly less within the Guest-OS ( e.g. were using 100GB/100GB but deleted some data and are now for instance 10GB/100GB used) you will have to enable UNMAP/TRIM features in order to reclaim this space without using swing-storage:

UNMAP/TRIM Space Reclamation on vSAN

VSAN 6.7 U1 UNMAP issue

Note that even after this if you are experiencing the issue related to this Health check VMware Knowledge Base  you would likely need to apply the workaround in order to clear the warning (note however that this is distinctly different from actual/used space issues).

Bob

[RainerJM]

Thanks Bob

vmware support have come back to me and say that when they tested in their lab they didn't see the previously thick disks reported as thin until after they vmotioned to another vsan cluster and back again, even after changing the SP, so I'll try that. They didn't say anything about disk space reclaims after this, so I'll have to try it and see, I have plenty of thick VMs to test this on.

One thing that has come out of this: do you know how the provisionedspaceGB attribute of a VM on vsan is calculated? The values I get seem to follow no logic that I can see. vmware tell me it's the allocated space, but that doesn't look right.

e.g. I have a VM with hard disks summing to 120GB, 12GB of ram, no snapshots, so say roughly 132GB in size. The usedspaceGB is 145GB, which roughly matches the used space as seen by the OS, doubled for FTT1, but the provisonedspaceGB is 180GB. I have plenty of others with sizing stats that just don't look right, so I presume I'm making some wrong assumptions as to how they are calculated

Scratching my head on that one...

Rainer

[TheBobkin]

Hello Rainer,

"One thing that has come out of this: do you know how the provisionedspaceGB attribute of a VM on vsan is calculated? The values I get seem to follow no logic that I can see. vmware tell me it's the allocated space, but that doesn't look right.

e.g. I have a VM with hard disks summing to 120GB, 12GB of ram, no snapshots, so say roughly 132GB in size. The usedspaceGB is 145GB, which roughly matches the used space as seen by the OS, doubled for FTT1, but the provisonedspaceGB is 180GB. I have plenty of others with sizing stats that just don't look right, so I presume I'm making some wrong assumptions as to how they are calculated"

Are you referring to the 'Storage Usage' in the VM summary tab or something else? This refers to the total 'Capacity Used' of all Objects associated with the VM including the used for FTT - note that .vswp have FTT also (but these should be thin by default in 6.7 U1)- one other thing I have noted from troubleshooting such calculations is dependencies or links to other vmdks e.g. with Citrix MCS even 'Full Clones' have some form of parent vmdk referenced in their vmdk descriptor which gets added to the summary total used of the VMs. Other things to note are any data files stored in the namespace (e.g. relatively small -flat.vmdk for some Citrix and other types of deployments)

Assuming you are speaking of the total VM usage (or something pulling data from the same metrics) this will depend largely on the max ever used space of the VM without UNMAP/TRIM being applied.

If you look at the VM Objects in RVC (vsan.vm_object_info <path_to_vm>) or using #esxcli vsan debug object list (helps to > output to a txt file) it generally makes more sense.

https://www.virten.net/2017/06/vsan-6-6-rvc-guide-part-3-object-management/#vsan-vm_object_info

Using Esxcli Commands with vSAN

e.g. an empty 5GB vmdk with OSR=100/proportionalCapacity=100/primitive Thick will look like so:

Capacity Used: 5.11GB    Physical Capacity Used 0.00GB   Address Space 5.00GB

While a Thin/OSR=0  5GB vmdk will have these attributes:

Capacity Used: 0.01GB    Physical Capacity Used 0.00GB   Address Space 5.00GB

Bob

[RainerJM]

Hi Bob

I'm pulling those stats from powercli, e.g.

Get-VM |

Select Name,ProvisionedSpaceGB,UsedSpaceGB,

    @{N='FileTotal';E={($_.ExtensionData.LayoutEx.File | Measure-Object -Property Size -Sum).Sum/1GB}}

For the server I was referring to this gives:

ProvisionedSpaceGB     180.75404063705354928970336914

UsedSpaceGB                145.44935313705354928970336914

FileTotal                          146.41067596525

and 'Capacity Used' on the VM summary tab shows 145.45, as you say and as expected.

But given that most of the allocated space for a VM will be its storage footprint and (get-vm | Get-HardDisk | Measure-Object -sum capacitygb).sum gives 120GB, I just don't see where that 180GB figure comes from.

I have a webex session later today with vmware support so we'll see what they say.

Rainer

[RainerJM]

Hi

Just an update from discussions with vmware support:

There doesn't seem to be a way to convert think to thin on vsan unless you have a non-vsan datastore available to svmotion a VM off to and then back to vsan again. However, vmware's opinion is that it's really a labeling thing: applying a different SP sets proportionalcapacity to 0 and that's the important bit.

On the UsedSpaceGB and ProvisionedSpaceGB VM attributes - the definition my support engineer as given by one of the vsan architects didn't agree with the observed figures in rvc or anywhere else, even UsedSpaceGB miscalculated the space taken up by the vswp file, so it's being passed back up the chain for more investigation at vmware. Short answer from them was not to rely on those attributes at all in a vsan environment.