Monitoring Issues

On Sunday night into Monday morning, we had unexpected downtime on VMware Communities. As soon as we realized, we immediately rebooted VMware Communities and performance returned to reasonable levels, but that was after many community members in time zones other than Pacific experienced several hours of poor performance. We had been checking the site manually all weekend, but from Sunday night to Monday morning Pacific, the site was unusably slow. We are sorry for that, especially since we are aware how many of you rely on VMware Communities professionally and personally. We’ve now shortened the acceptable response time for our monitoring system to 20 seconds – if a page loads slower than 20 seconds, we get an alert and will examine system statistics and respond appropriately.

New Build Environment

On Thursday, we recreated our build environment by rolling in a series of performance patches we had made since going live. Recreating our build environment is a necessary step before we can roll out the additional usability and performance fixes that we have been (and continue to be) working on. The build appears to have caused some issues with the order of posts on the various community pages (although the timestamps are still correct). Jive Software is working on identifying a fix, and I will give an update here tomorrow.

Today's Fixes

The advantage of having a new build environment is that we are able today to roll out a set of usability and performance fixes:

Internet Explorer Performance: We replaced the JavaScript-based top menus with CSS-based ones, which makes them faster on Internet Explorer, both IE6 and IE7. Since people’s experiences have varied depending on their hardware, software, and location, please let us know if these menus now work for you.

Private Message notifications: The “My Content” in your menu bar now turns orange when you have a private message waiting for you.

Blog formatting: Some blog pages didn’t format correctly, and these are now fixed.

Meanwhile, we are continuing to work on performance, including the issues that caused the outage Sunday night / Monday morning. This week we are also working on the top features that were lost in the move from VMTN Forums to VMware Communities, as well as some other usability improvements. I expect some good news on those two areas as well as performance over the course of the week.

I promise to keep you all informed more frequently of progress.