Simple answer to why you VMs are balooning is that you have overcommited.
When ESXi is short on physical memory it will start to reclaim "inactive" memory on VMs and release it to other VMs when needed.
There's only 2 server 08 r2 sp1 virtual machines and there hardly using any memory. How do you get it to release the memory? After awhile the second machine balloons up for no apparent reason.
Server has 8gb of ram
3.60GB Consumed of 6116MB
Balloon is 2219936kb and not releasing
1.88 Consumed of 5852Mb
Ballon was 3894816kb rebooted and it released it sitting at 0
How much RAM does your host have ?
Its has 8gb I am going to get another 8gb