NFS datastore sizing mystery

[HendersonD]

I have carved out a 1,000GB (1TB) volume on my Netapp filer and

exported that via NFS. I have added this data store to all three of me

ESX 3.5 hosts and migrated 18 VMs to this datastore. If I add up the

disk sizes of all 18 VMs, they add to 470GB so I would expect about

530GB of free space on this datastore. When I look at the datastore in

the VI Client as well as my Netapp it shows that I have 770GB used and

230GB of free space. Where did the 300GB go to?

The snapshots that have been taken of this volume on my Netapp chew

up about 16GB so this does not account for all of this space. I have

restarted my VirtualCenter server thinking that would do some type of

database refresh but that did nothing.

[kooltechies · Accepted solution]

Already replied on another thread , puting it here also..

Apart from disks and snapshots the space is also consumed by the VM's

swap file. Count all the swap file size add that to disk and snapshots

you will be near to amount shown by VI. VM's swap file has an extension

.vswp.

Thanks,

Samir

P.S : If you think that the answer is helpful please consider rewarding points.

[HendersonD]

Samir,

Sorry about the double post, I posted it in ESXi accidently. Where would I find the swap files? In other words, how can I figure out how much space they are chewing up?

Dave

[kooltechies]

Hi,

You can look for them in the same folder where the disk files are residing they have .vswp extension, you can check the size with ll command in the directory. However swap files will not get created every time as it depends on memory reservation settings if you have memory reservation equal to assigned memory then you will not get swap file.

Thanks,

Samir

P.S : If you think that the answer is helpful please consider rewarding points.

[HendersonD]

How do I check the memory reservation versus assigned memory? When I edit a VMs settings, under memory, is that the reservation or the assigned?

[kooltechies]

You should be able to check that under edit settings > options > Memory tab. I will give you an example of the swap size.

Assigned Memmory to VM : 1000 Mb

Memory reservation : 500 Mb

Swap file size : Assigned Memory - Reserved Memory

Swap size : 1000Mb -500Mb = 500Mb

Thanks,

Samir

P.S : If you think that the answer is helpful please consider rewarding points.

[HendersonD]

I see this now. What is best practice here for memory reservation? It appears that I have all of my VMs set to 0 for reservations and the unlimited checkbox checked. Should these settings be changed? If I do change them and the swap file gets smaller or is eliminated, is this dynamic? In other words, would I immediately see a change in storage free/used space?

[kooltechies]

Hi,

Swap file will get created when the VM is power on and it gets deleted when the VM is off , the sizing should take effect without reboot but to be honest I don't remember it at this moment. You can do this on a test VM.

Thanks,

Samir

[HendersonD]

Great, thanks for your help on this, I will try on a test VM. Should I just set my memory reservations equal to my allocated memory on all of VMs?

[kooltechies]

Hi,

You can do that if you have enough physical memory on the machine , lets say you have 15 VM's with 2 GB memory allocation then you will need 30 GB physical memory in order to set memory reservation.

Thanks,

Samir

[Erik_Zandboer]

Hi,

Do not confuse allocated memory to a VM with memory reservation. As soon as you start a VM, the memory allocated to the VM is used to create a swapfile of that size. Memory reservation has nothing to do with that.

You should check all the files that have been created on the NFS and add them up, if you "miss" 300GB I do not think all those GBs went into the sum of allocated VM memory...(unless you put 16GB in each VM)

Visit my blog at http://erikzandboer.wordpress.com

[HendersonD]

You must have been reading my mind. I took another long look at this and still have no ready explanation where I am losing space. I did go back and look at the volume I created on my Netapp and I had space reservations set to 20% when Netapp recommends 0%. I changed this and resized the volume to 1TB and now I have a 1TB volume presented to VMWare. All told, the hard drive space for my 18 VMs add up to 470GB so I should have 530GB of free space showing minus whatever the swap files chew up.

My 18 VMs are allocated a total of 23GB of RAM. Since I have no memory reservation set on any of my VMs the swap files should chew up 23GB. This still should leave me with about 500GB of free space on this volume. When I look at the Netapp or at the datastore through the VI client, I only have 240GB of free space. Any ideas where the other 260GB are?

[HendersonD]

Unfortunately the VI Client does not display in one location the disk allocation for each VM. I just downloaded a nice utility MCS Storageview and it works great. Launch, login to the VC server and its display information about disk usage for each VM. I have my Windows print server in a VM that has a 10GB boot volume and a 20GB volume for the spooled print jobs. This is just a second disk I added to this VM out of the shared storage so when I came up with the 470GB figure for the total space my VMs use, I included 30GB for this print server. This utility displays exactly what I have allocated for disk space for each VM. I wanted to make sure that I did not miscount so the total allocated is still 470GB.

My Exchange 2007 mailbox server is also virtualized with an 80GB boot volume that sits on this same NFS datastore. This server has presented to it two luns from my Netapp filer through iSCSI. Is there a possibility that somehow the VI client is adding this in as consuming space on my NFS datastore even though they are luns? Is it a possibility that the VI client just has a hard time figuring out used and free space on an NFS datastore used to house VMs.

I am still trying to come up with some explanation of the discrepancy between how much space that my VMs should be chewing up on my datastore and what the VI Client and my Netapp are reporting is actually chewed up

[HendersonD]

Mystery solved. I had fat fingers on my spreadsheet I was using to calculate the amount of storage space in total based on how much each VM got. Now that I have fixed this calculaition and understand how much space is chewed up by swap files, all of it is accounted for. Thanks for everyone's help