Rolling out ESX for the first time

[offlinetn]

We are converting our data center with 300 plus servers and I am trying to collect some information from other people who have deployed ESX...

Basically we need to figure out a starting point for reserve resource allocation etc. IE a best practices or what other people are doing kind of thing. I'm sure these will change once we start using it, but I'd like to make a best guess attempt.

IE for memory if we a Windows server should we by default set it up to reserve 25% of it's memory when we create a new virtual machine. So for 1GB we would reserve 256MB etc.

Same thing for CPU, should we reserve say 10% of the cpu. So if we had a single proc VM we would reserve 10% of 1 cpu, if we had dual proc we would reserve 10% of 2 cpu's etc.

Should we make the SAN LUNs RAID 10, or are most people doing RAID 5 etc.

Any other best/recommended practices.

I know that most of these answers will not apply to every situation, but I'm just looking to get defaults.

Thanks for any help,

Chris

[Odurasler2]

Chris, Have you purchased the HW for your ESX environment? If so, what are they?

the reason i asked is because it helps to know what you have to be able to offer better advice. for example, if you have a clariion cx for storage, i would recommend a metalun for your data instead of R5. so it helps to know a little bit about your environment.

[weinstein5]

My recommendations is to not start with reservations - because if the vmkernel can not gaurantee the reservation the vm will not start - also in terms of multi virtual CPUs - start out with one vCPU you can always add -

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful

[Rodos]

Chris, welcome to the forums.

Thats a good size deployment. Do you have en experienced VMware architect involved to assist? A few days in their head is going to get you a long way fast. Thats my first recommendation.

Second, why do you want reservations. A CPU reservation on the VM gives the resource to the VM if it needs it or not (opposed to pool reservations). So you are going to waste a LOT of CPU that could be used by machines that do want it. (Someone will correct me if I am wrong here, its to avoid any ramp up latency). Sure you may have some specific VMs that need a reservation but that should be an exception and a very rare one and not a rule. You want to be thinking about shares and resource pools more.

Have you done your sizing analysis yet? Worked out your VM per core rations, RAM per VM ratios (in normal model and in maintenance mode, aka down a host). Have you done your cluster sizing yet? Are you scaling up or scaling out. If you don't get the balance of RAM vs CPU right you are going to waste a bit of money. Has VMware or a partner run a Consolidation Estimate yet? It has good data to feed this analysis.

Keep us posted of your progress and good luck.

Rodos

Consider the use of the helpful or correct buttons to award points. Blog: http://rodos.haywood.org/

[offlinetn]

Thanks for the help guys! I just attend the VMWare infastructure class and have played with our lab equipment. We will be using Dell 1950's and 2950's depending on wether they need HBA cards for the EMC SAN or not. We do not have an experienced VMWare tech on had, your looking at it.

What is the best way to do this...

Have you done your

sizing analysis yet? Worked out your VM per core rations, RAM per VM

ratios (in normal model and in maintenance mode, aka down a host). Have

you done your cluster sizing yet? Are you scaling up or scaling out. If

you don't get the balance of RAM vs CPU right you are going to waste a

bit of money. Has VMware or a partner run a Consolidation Estimate yet?

It has good data to feed this analysis.

Thanks Again!

Chris Edwards

[offlinetn]

Sorry, we have two EMC separate EMC's that we will be using to connect the 1950's and 2950's to, The new one is a CX4-240 with iSCSI and FC the older is a CX3-40.

[JasonVmware · Accepted solution]

Hello,

Everything Rodos said should be done or looked into. There is many ways to spec out what your Vmware systems are needed. One way is to base line your systems and find out what there low, average, and highend needs are on a daily basis, then you will be able to better spec out the resources on your resource pools / shares better. Also it is important to only use reservations on systems that really need to have that resouce garenteed at any time, and to use resource pool reservations / shares like mentioned earlyer.

One tool that may help you spec out your current system before the flip is the Vmware Capacity Planne. Wha this tool does is installs a thing agent on selected servers and then it monitors them for an extended period of time. Essentially the agent is like a perfmon in windows with a TON of conters tracking everything and dumping it into reports. So typically what people will do is install it and let it collect data for 1-2 weeks which will give you plenty of data / reports that show you your low, average, and highend requirements on your servers. Its very important to know this information as you could have a VM that really only needs 1GB of memory most of the day except it may need 3GB for just 1 hour a day durning a high load time. Knowing this will give you the ability to setup your shares better.

Another thing to take into consideration is your Virutal Switching and Port Groups. A good article on this topic is here:

8 nics - http://www.networkworld.com/community/node/36691

6 nics - http://www.networkworld.com/community/node/36359

4 nics - http://www.networkworld.com/community/node/35854

3 nics - http://www.networkworld.com/community/node/35640

2 nics - http://www.networkworld.com/community/node/35388

Also if you have those psyhical network card modules that have 2 or 4 nics on 1 psyhical pci card or module make sure to pair up opposite psyhical nics incase of a nic / module failure. For instace say you have 4 nics on your server.

pNic0 - Onboard nic

pNic1 - Onboard nic

pNic2 - Expansion module card

pNic3 - Expansion module card

In this expample your Expansion module card has 2 nics on the same module which is very common. In your virutal switching you would want to put pNic0 and pNic2 together in vSwitch0. Then put pNic1 and pNic3 together in vSwitch1, that way if your expansion module card dies your virutal switching will still function, where if you put all your vm's on the 2 nics that where on the module card you would be dead in the water.

Hope this helps, if you have any questions feel free to ask.

[Rodos]

Chris, you have 300 servers and 2 SANs, here is my recommended course of action.

1. Go and buy the book, VMware Infrastructure 3: Advanced Technical Design Guide and Advanced Operations Guide by Scott Herold, Ron Oglesby, Mike Laverick Review

2. Capacity and work load analysis. Find someone who can run a VMware Capacity Assessment for you against your 300 servers and ask for a spreadsheet dump of the data.

3. Create a candidate server worklist with transformed resource requirements. That is how many vCPU, how much RAM and new disk sizes and connected VLANs. Use the data from the CA to determine these based on load.

4. Based on your information work out your total number of VMs, total number of vCPUs and RAM.

5. Architect one or more VMware cluster based on your data and business requirements. Business requirements are things like existing hardware, vendor relationships, division of different workload types (prod, DMZ, dev, UAT), risk. There are many more inputs into architectural decisions ... but you get the drift. This should tell you how many hosts you need and how much resource each needs to deliver (number of cores and RAM).

6. Based on your number of hosts and needed resources pick a hardware platform, blade or server, required number of network ports, SAN connectivity considerations.

7. Based on your data perform a storage design, based on your data for sizes, performance.

8. Create a migration plan and test

9. Build and test your VMware environment, configure your monitoring

10. Migrate

11. Enjoy

There is probably more but I just did a brain dump.

Rodos

Consider the use of the helpful or correct buttons to award points. Blog: http://rodos.haywood.org/

[offlinetn]

Thanks for all of the advice! I will buy the book and give VMWare a call to se see if they can recommend someone to help with our capacity planning.

Chris