VMware Developer Center Blog - Developer Spotlight +March 4, 2008
Tom Milner Developer nworks.com
I had an opportunity to catch up with Tom Milner from nworks to get his perspective on using the VMware SDKs. Tom also had some good feedback on how VMware can improve the SDK developer experience.
Pablo: Tell us about your programming background and work experience.
Tom: I guess my gray hairs speak to my experience. I've been a software developer for over 30 years and have worked with large companies like Huges Aircraft and Hewlett-Packard, and I've worked for startups and smaller companies (the smaller companies are more fun). I've got a lot of background with C and Java doing network management, device management, and performance management products. For the last two years I've been with a small software firm called nworks, which makes plug-ins and connectors for enterprise management software from Hewlett-Packard and Microsoft.
Pablo: Tell us about what your application does, and maybe describe your approach.
Tom: VMware Management products fill a niche that develops when virtualization is brought into a data center. Enterprise-class companies use products like HP's OpenView or Microsoft Operations Manager to monitor infrastructure like their phone systems, heating and air systems, manufacturing, networks, and data centers. These applications
are large rule based engines that look for abnormal situations before they become an issue and either fix the situation on their own or notify someone to take action. To monitor the servers in the data center, software agents are deployed on these servers to return hardware inventory, status, and operational performance data back to the enterprise software. This method worked fine for years until VMware came along. As many engineers are aware, the wall-clock in a virtual machine ("VM") is no longer steady over time. When VMs lose the physical processor their clock stops running. And when they regain the processor, one of the first maintenance steps must be to move the clock forward to the current time before proceeding. In other words, the VM's clock "jumps". So all of the tools that used to rely on the system clock to tell them how long they've been working no longer work correctly. Think about it, how do you figure out CPU utilization, for example, if you no longer have a clock? The VMware designers of the SDK were aware of this problem and added the performance interface to the SDK to fill this void.
Our product uses the SDK to pull in key performance data, plus the event stream from the Virtual Center (which is also missing from these applications), and integrates it into the enterprise application. This data is coupled with about 200 rules that we provide that are also installed so that the IT administrators can manage the virtual world the same
way they manage the physical one. In addition to performance and event data, we also use the SDK to provide management action capabilities like shutting down or suspending a VM, or placing a host into maintenance mode.This is very powerful stuff! Imagine a scenario where HP OpenView or Microsoft's Operation Manager receives an event that a fan has gone out in the middle of the night on one of its DRS servers in some data center hundreds of miles away from central IT
administration. The rules for this scenario can be customized so that a technician is notified automatically and the ESX Server is set into maintenance mode (through the nworks technology and the SDK). Because its a DRS server, all the VMs automatically begin migrating to other ESX Servers. When the technician arrives with the new fan, the
ESX Server is ready to be repaired, no downtime, no processor failure, and no human intervention required by IT Administration. The IT administration can watch the ESX Server status from the same enterprise console
that they use to monitor all of their other sytems.When the system with the bad fan is repaired, they can take the ESX Server out of maintenance mode directly from that console.
Pablo: This sounds like a very exciting approach. Can you tell us about your initial experience coming when you first started using our SDK ?
Tom: Well, I'd like to say it was easy, but it wasn't. I started before the first release of the
SDK, so documentation and samples were scarce, and so were people who knew how to do things with it. And you have to understand, the SDK supports an industrial strength API, no doubt about it. When I started with nworks, we were using the older COM API, which was nice for small systems and supported simple scripts. But it couldn't scale past a single system. The VMware designers were looking for something that could support hundreds of ESX Servers with thousands of VMs. They wanted an API that was so robust that it would satisfy all requirements, even for VMware's own VI Client. And that's what they came up with. The API is a SOAP based API, which means that the client-side is described in XML rather than shipping a client side library. When you get the SDK, you build the client-side libraries yourself by using the VMware-supplied XSL scripts to convert the XML definitions into C, C#, or Java source code. This conversion makes it easy to port the client-side of the SDK to almost any platform, but it also makes it difficult to document the client-side,and VMware hasn't done their best efforts in that regard. (Note: The VMware SDK ships with pre-compiled API stubs for Java) The SDK also returns different objects by using the same calls which requires a lot of casting for type-safe languages like C# and Java. Because the client side documentation is so weak, my first development cycle was: Develop code using the next API call,use the debugger to determine the exact object being returned, repeatIt wasn't until I had hit most of the API calls that I could really plow ahead without a debugger to decode what type of object was being returned.
Pablo: Sounds like you have some ideas about what things we could be doing to improve the developer experience?
Tom: It's easy to be a Monday-morning quarterback, and I know that the problem of supporting such a robust and critical API is an immense one, but my two recommendations for VMware are:
1. Code up many more stand-alone code samples (a good job for interns).
2. Put these samples on the web with a API cross-reference.
The VMware documentation for each API call is sufficient to explain what it does, but it's only documented from the server's side. The code samples will show what object is being returned on the client side and will show how the API calls interact.
Pablo: Are there any tips or hints you can provide to our readers, lessons learned of a different approach you would consider next time?
Tom: Despite some of the getting started pain, using the SDK has been very good for nworks. Other companies have tried to put agents in the console O/S and are now scrambling to figure out what to do with ESX 3i. And console agents means additional overhead for the customer's ESX Server which is NOT where you want to be putting it. So
for us, having an agent less architecture that scales well has been great. I love it when a customer with a dozen or more ESX Servers and 200 VMs asks us "What agents do I need to install on my virtual machines?" None. "Well, what about my ESX Servers?" None. That's usually when they smile.
If someone wants to pick up the SDK, the first thing I'd ask is "What do you want to do with it?" If you're looking for simple/quick scripts to dump out a simple list, perhaps you should look at the VI Perl Toolkit. If its a larger application in terms of data or functionality, then the SDK is the way to go. One of the tips I would give people is if you're going to use the SDK API, become familiar with the Managed Object Browser ("MOB"). The MOB is a great interface provided by VMware that gives a (relatively) raw view into all of the runtime properties of an ESX or VC Server. And if the MOB can see it, then so can the API. Secondly, the VI Client is implemented using the API, and according to VMware, there are no back doors... so if the VI Client can see it or do it, then so can the API.
There is some ramp up time to using the SDK, but once you understand it the sky is the limit.
