VMware Communities > Blogs > Manual Automation > 2009

Blog Posts

Manual Automation : May 2009

Previous Next
3

Bye, Bye ESXi

Posted by Virtual_JTW May 20, 2009

What a long, frustrating trip it's been! Don't get me wrong, I really like the idea of ESXi: thin, fast install, small foot-print, BIOS-like host configuration, no Console OS (COS) to patch or support, can run from embedded USB key, etc, etc. But, my experience in supporting and managing an ESXi-based VI production environment tells a different story.

I've decided to convert all of my hosts from ESXi to ESX "Classic". There are three primary reasons:

  1. Support
  2. Reliability
  3. Compatibility

Support

Without the COS it's difficult to execute commands and view logs files "real-time". I've had more than one VMware support engineer complain about this during a trouble-shooting session (so it must be true!). There are alternatives: using the unsupported trick to get to the command line from the host's console, hacking SSH to open it up (which is also unsupported), capturing logs/diagnostic bundles via vCenter Server, RCLI, VIMA, etc. But none of the alternatives are as fast/clean/easy as SSH'ing right into the COS and working from there.

Reliability

I purchased eleven Hewlett-Packard USB keys w/unlicensed ESXi to embed in my ProLiant DL380 G5s. When I upgraded them via VMware Update Manger (VUM), the entire installation on the key became corrupted. I was not even able to revert back to the previous ESXi image on some keys. HP has since issued a customer advisory and I have replaced all of the keys: http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=c01605187

Unfortunately, this experience still leaves me with a less-than-fuzzy feeling for running ESXi on said embedded USB keys in critical production environments.

Compatibility

ESXi seems to lag ESX Classic in updates - specifically when it comes to compatibility. This implies that ESX Classic is developed, tested and certified against first. I manage two VI environments located in different datacenters. I use SRM at the primary for DR/fail-over to the secondary (see previous articles). It took around a month for VMware to release a patch for ESXi Update 3 that made it compatible with SRM 1.0 Update 1. Read your compatibility guides! More on this in a future article.

Many third-party tools and scripts require the COS. There are many examples of this: Snap Hunter, Vizioncore vOptimizer Pro, etc.

Unlike ESX Classic, HA in ESXi requires a ScratchConfig folder created on separate VMFS datastores for each host. This may not be a big deal for smaller clusters, but for a cluster with many servers, many datastores will be required.

Finally, for those of us that run HP servers to host ESXi, we have a specific firmware/ISO that contains the HP management providers. Unfortunately, even with the built-in providers you still can't monitor disk status - which is, of course, the one hardware component that fails the most often(!). As of this writing, here's what I've been ale to determine:

Licensed ESXi

  1. HP only supports ESXi with the proper "management providers" on Update 2 and Update 3 (as evidenced by VMware's downloads section of their web site). ESXi 3.5 Update 4 is not yet supported.
  2. Using VMware Update Manger to upgrade ESXi instances to Update 4 effectively breaks SIM manageability.

Free ESXi

  1. A new installable image is available for Update 4 with the management providers.
  2. Upgrading exiting hosts via the VMware Infrastructure Update tool effectively breaks SIM manageability. These hosts will have to be reinstalled from the ISO.

Once you're sure you have the right ESXi firmware image installed, it's time to add the host to HP Systems Insight Manager (SIM) for hardware monitoring. I was able to add only 3 out of about 20 of my ESXi hosts successfully. HP support wasn't able to help me out. The main suggestions I got were to reinstall(?) and to call VMware. With ESX Classic you install the Insight agent in the COS, add the host to SIM, and you're done. It just works.

Conclusion

Like I mentioned previously, I still really like the idea of ESXi. Once I saw that Hitachi was embedding a virtualization solution in their servers I knew it was only a matter of time before VMware came out with something similar.

I have many free ESXi installable instances. This is a great solution in cases where the budget it tight or non-existent. Utilizing the free ESXi still gives you many of the benefits of virtualization making it a better way to go than bare metal OS installation in most cases.

I think embedded/thin is the future. I hope vSphere 4 embedded improves on the issues described above.


5-20-2009 UPDATE:
Not being one to spread any FUD, I would to add to my comment "Using VMware Update Manger to upgrade ESXi instances to Update 4 effectively breaks SIM manageability". According to this article on the Yellow Bricks blog (http://www.yellow-bricks.com/2009/05/14/updating-an-esxi-server-with-vendor-agents/), VUM has the intelligence to download and apply the correct ESXi firmware image (i.e. the one with whatever OEMs management providers are pre-installed). And I believe it because Ducan is "the man". Note that I used the Virtual Infrastructure Update Tool, not VUM, so that may have made a difference. Regardless, I do know that after updating the image HP Insight Manager was no longer seeing all of the hardware components and in some cases failed to communicate to the server completely. This hasn't to me happened on my ESX "classic" hosts.

3 Comments Permalink
0

By the time I was twelve I knew I wanted to be in the IT business. I loved my Commodore 64, my 1541 disk drive and 300 baud modem! I loved dialing into bulletin board systems and typing in games with all of those peeks and pokes. I didn't know it at the time, but I was really learning a lot about computer hardware, storage media, telecommunications and software programming.

Responsibilities at my first corporate job included managing and maintaining LANtastic on a 10base2 network. Okay, I was mostly maintaining it - not very reliable as it ran on top of MS-DOS and the transceivers failed all of the time but hey, this was before the whole dot com boom era so you could expect much back then. I also had a Novell NetWare 3.x NOS running for one department (remember NetWare Loadable Modules, NLMs?). Once NetWare 4.11 came out I convinced management to dump LANtastic and 10baseT and run upgrade to NetWare 4.11 and Ethernet (remember NetWare Directory Services, NDS - can you say "AD"?). I didn't realize it at the time, but I was really learning a lot about network operating systems (NOS), networking hardware and standards.

It was around this time that I started to realize that I wanted to work with multiple technologies, especially newer technologies that were interesting because they solved a complex problem or saved businesses money. I didn't want to become a walking product manual for one piece of software or hardware component and focus on that for the rest of my life (or until I retired, which-ever came first)!

My next opportunity involved leading a team in the design, implementation and management of MS Systems Management Server (SMS) 2.0 to potentially hundreds of locations around the US. While the tie-in may not be oblivious, working with SMS allowed me to learn more about MS SQL Server 2000, Windows management technologies (CIM, WBEM, etc.) and data synchronization across slow links (anyone heard of Starburst?). Not-to-mention it was a great opportunity to work for a 3 billion dollar publicly traded company - a different culture than my previous employer to say the least.

Unfortunately, management made a short-sided, mostly political decision and shut down the SMS implementation after it was successfully deployed to the first remote site. However, this turned-out to be good news: I discovered virtualization around this same time - 2001/2002 timeframe via VMware Workstation. Shortly there-after I setup a production GSX server hosting three VMs that shared the same base image with each VM having a unique redo log (remember that VMware whitepaper?). The users couldn't tell the difference and were a bit surprised when I finally let them in on the secrest! And as they say: the rest is history.

To expand on a theme, virtualization has allowed me to work with and learn more about operating systems, enterprise-level server hardware (such as how CPUs work), enterprise-level storage and all of its related technologies (SAN hardware, iSCSI, fiber switches, etc).

I don't pretend to have the most unique career path in the world - I know there are many other Systems Administrators and Engineers that have lived similar experiences. And for that reason I'm identifying a new breed of IT professional. Systems Analyst, Systems Administrator and Systems Engineer titles are all less meaningful in this context. I've had all of these titles at some point in my career.

Regardless of title, we're IT professionals that stand apart based on our past experiences and constant passion to always be working with current and newer technologies that ultimately allow businesses operate faster, smarter and more efficiently. Virtualization, especially VMware Virtual Infrastructure and the coming vSphere products, has allowed us to take businesses to that next level.

And we won't stop there. We're constantly keeping an eye on cloud-based technologies, standards and initiatives. We'll be beta testing these products - and not just download, install, use for five minutes and throw-away. We're excited about the product and want to see it succeed so we'll provide feedback. We'll keep an eye on the bleeding edge and maybe sometimes participate in a limited way - such as a product install in a lab for evaluation, but we won't bet the company's business on it (at least those of us that have been around for awhile and have made that mistake before).

To bring us full circle I have to ask, what is the next NetWare? The next SMS? The next VMware VI? Is it more VMware? Quite possibly. Or maybe it's something we haven't thought of yet. Maybe its mainframe 2.0 with pervasive high-speed wireless connectivity brought about by a technology such as WiMAX? It's hard, if not impossible to predict. But one thing is certain, it will be "cool" and we'll be among the first of the light bulbs popping on throughout the IT industry and the businesses we work for will thanks us for it.

0 Comments Permalink
Click to view Virtual_JTW's profile

Virtual_JTW

Member since: Nov 1, 2004

I am a senior IT professional that has designed, implemented and managed the operations of several VI environments. This blog will detail design rationale, testing results and technical tips with a heavy focus on VI/vSphere, storage and cloud computing.

View Virtual_JTW's profile