I don't think there is really any "rule of thumb" as to what your environment should be at in terms of CPU and memory usage. It varies so much depending on each environment and the workloads that you're running. Rather than getting hung up on a set of numbers for CPU and memory, I would just make sure your environment is designed for the following:
1) High Availability - Can you sustain the loss of one (or more) host without overloading the remaining hosts? You really need to determine what your SLA is in terms of how many hosts you can afford to lose and if you need all VMs to run in the event of a host loss. Making sure you have enough resources available to sustain the loss of a host is a really important point and can easily get overlooked as it is so easy to continue to deploy new virtual machines. Striving to get CPU and memory utilization in the 60-70% range can limit your ability to lose hosts without incurring serious performance degradation.
2) Growth - do you have the capacity to grow your environment as the needs of your business change? Can you add new virtual machines if there is a company merger, a new application is brought in house, or if you need to ramp up dev/test, etc? Growth is often overlooked but planning for it ahead of time can save you money down the road.
3) Performance - How are the guests performing in your environment, and how well do you want them to perform? As you start to add more virtual machines and resource utilization increases, you may start to experience performance problems in the guests. Same point as above - it is so easy to deploy new VMs that you can quickly get to the point where you have too many VMs per host and no easy way to balance out the load with vMotion. If you are still meeting the performance needs of the applications, and the above two items have been satisfied based on your organization's needs, then you're doing something right!
I know you're probably looking for an answer that more closely resembles something like "All hosts should be at 50% CPU and 60% memory usage" but there really isn't an easy way to give that answer. I would try to ensure that the above 3 criteria are met and worry less about what the actual numbers look like.
Matt
My blog:
Matt | http://www.thelowercasew.com | @mattliebowitz
