If (as you wrote) your ESXi-box has 8GB physical memory, it does not mean it has 8GB RAM available for VMs. You have to substract 1 or 2GB for hypervisor itself. So if you created two VMs, each with 4GB and both are running, you are over-commiting memory. If both VMs really want to use 4GB, only part of that 2x4GB total is mapped to ESXi-host physical memory. Rest is handled by swapping (or memory compression, tps, etc).
So first of all, try to apply Rule No.1: do not overcommit ESXi-host memory, unless you really have to. In your case, substract memory used by hypervisor itself (iirc, it is about 1.5GB), and use only the rest for VMs (i.e. assign only 2x 3GB for VMs). Then check VMs for some time if they can live with this memory...
_____________________________________________
If you found my answer useful please do *not* mark it as "correct" or "helpful". It is hard to pretend being noob with all those points!