Skip navigation
henrikmj2 Novice

ESXi 4.1 Best Practices

Posted by henrikmj2 Nov 15, 2010

Here's a second attempt on listing best practices for VMware ESXi 4.1

 

Why

Basically I have yet to stumble upon a comprehensive Best Practices list for ESXi 4.1/vCenter that is relevant for beginners.

 

So here we go - I hope you find this of use and contribute with more Best Practices.

 

Before you install

It is never too late to educate yourselves. Here are a few good places to read up

 

The VMware installation best practices is a good read:

http://kb.vmware.com/kb/1022101

 

For a more detailed introduction, the Setup Guide is very good

http://www.vmware.com/pdf/vsphere4/r41/vsp_41_esxi_i_vc_setup_guide.pdf

 

Performance Best Practices from VMware

http://www.vmware.com/pdf/Perf_Best_Practices_vSphere4.1.pdf

 

Hardware

Ensure that your hardware is supported, but also ensure that you support your hardware. Check for drivers and monitoring tools that may improve your performance or monitoring capabilities. Enable Intel VT or other virtualization in BIOS.

 

Redundant systems

Carefully consider how you can rearrange your setup should your hardware fail. If you only have one host/storage box/switch, then you're risking a lot.

 

ESXi and hardware

Strangely, a good tip to install ESXi is to consider to install from other sources than VMware.

Dell and other might provide better out of the box support, but installing ESXi on HP hardware is a bad deal unless you insist on going through a lot of command line jogging to get hardware CIM (that's hardware errors) presented in vCenter console.

Pick it up from here

http://www.software.hp.com/portal/swdepot/displayProductInfo.do?productNumber=HPVM06

 

If you're already set and installed with HP hardware, the provider can be downloaded from here

http://h18000.www1.hp.com/products/servers/software/vmware-esx3i/offline_bundle.html

 

(Ugly direct link here: )

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?swItem=MTX-b98795300e7947d88f19ab56d6&lang=en&cc=us&idx=1&mode=4&

 

Monitoring

Apart from picking up hardware errors, ESXi may complain about disk latency and other bad things. These error messages shold not just be lurking in a console - pull them out and mail them to the administrator.

 

Nagios is a good pick for this task.

 

Networks

Face it: you probably bought your server with too few NIC's. Here

http://livingonthecloud.blogspot.com/2009/09/esxi-networking-best-practices.html

you can see that with redundant switches you need 8-12 ethernet ports in a midrange server these days.

 

  • 2 x NICs for VMkernel Administration Network


  • 2 x NICs for VMkernel Vmotion


  • 2 x NICs for VMkernel NFS/iSCSI (Storage)


  • 2 x NICs for VMkernel FT


and at least

  • 2 x NICs for Virtual Machine Network


 

In addition remember that you will want a monitoring adaptor like HP's ILO (remember that they require a separate license to have any value) It's got it's own NIC, but it adds to the switch requirements.

 

I'm curious on the best practices for network security. I'm guessing that Administration network may live in the same subnet as ILO's and Storage. Let me know what you do here.

 

Network Naming

VMware has this brilliant naming scheme: VMNet0, VMNet1, VMNet... brilliant if you're a computer but not good for real people. A best practice is to start naming VMware networks before it's too late.

  • Administration net


  • Storage net


  • DMZ


  • Office LAN


  • Internet


 

If you're already stuck with the default naming, be careful whan you change the nets - you probably need to go into each individual server and reconnect to the new network name.

 

Storage naming

As with networks, VMware is not creative in naming datastores. Names should make it easy to identify what you're working with

  • Direct attach servername location


  • SAN manufacturer location


i.e.

  • Direct attach server1 Copenhagen


  • SAN IBM Oslo


 

Tools

VMware PowerCLI is a good free product

The vSphere Management Assistant (vMA) should be mandatory on all hosts.

VMwares own description of vMA is good

VMwares own description of vMA is good

 

The vSphere Management Assistant (vMA) allows administrators and developers to run scripts and agents to manage ESX/ESXi and vCenter Server systems. vMA is a virtual machine that includes prepackaged software, a logging component, and an authentication component that supports non-interactive login.

 

You can download a vMA from the VMware appstore - but it's an old 4.0 version, the real thing is here

http://www.vmware.com/support/developer/vima/

 

For more tools checkout http://kendrickcoleman.com

 

vCenter

Be very careful when you configure vCenter. Without Database cleanup, you have a problem waiting to happen.

Limit your stats, here's how:

http://www.vcritical.com/2009/04/vmware-vcenter-server-performance-stats-levels/

 

Host setup

We have realized that some servers react pretty unexpected to a power failure: When power is back they just sit there and wait for someone to push the power button - on HP servers you can alter this setting through the ILO adapter.

 

Server setup

Every time you add a new server, be aware that the server will not automatically start with the host. Select the host - then the configuration tab -  Virtual Machine startup/shutdown.

Items that needs to be covered in future blog updates

 

Server setup

If you run into disk latency issues with your storage, windows servers would generally be unhappy. You can modify the registry to handle this.

 

Aligning partitions

 

Backup