Blog Posts

Nothing new to report. We're still keeping an eye on performance, which appears to be good. If, however, you are still experiencing problems, please report it to us with specifics of which page(s) are consistently slow, and if there is any pattern (e.g. hours of the day).

Thanks.

Badsah

NOTICE: Due to the U.S. Thanksgiving holiday, the VMware's U.S. offices will be closed Thursday and Friday, November 22 and 23, 2007.

Well, we are cautiously optimistic that we've reached a period of stability on the site. We have enjoyed four plus days of continuous uptime, and the performance has been consistently reasonable. We are still targeting end of next week (last week of November) for adding the second server to the cluster, to enhance the performance.

If you still encounter sporadic error messages, reload/refresh the page and you will usually see the error disappear. If the problem persists, please bring it to our attention. Details such as timing, page information, and specifics of what you were doing at the time are helpful, as always.

Thank you.

Badsah Mukherji, Sr. Community Manager, VMware

Last night we did an upgrade on the kernel to update the system clock to a newer version. This change is required before we add a second node to the cluster. The result of the update is that we have seen system performance improve significantly in the 18 hours since it was implemented. Users have reported performance improvements as well. If you have observations to share, please do: Performance?.

In addition to (and after) the system clock update, we also added in a second node to the cluster, but rolled it back because of some configuration issues. We are working actively to resolve those configuration issues so that we can get the second node running in the cluster.

Some users have reported errors in read tracking and private message notifications, and we believe this to be related to some work on the second node. We are monitoring it closely, and believe these errors to be in the presentation of the data, not in the data itself.

Performance and stability continue to be much improved.

Meanwhile, we are investigating 3 areas:

• 1. some compatibility errors we're seeing related to the JDK version and Jive's caching engine;
• 2. moving to a more accurate system clock to avoid some bottlenecks waiting on a new time;
• 3. we are accelerating our move to a clustered configuration.

I'll post again as we define the precise action plan around these.

Today we increased the heap size of the application server and then increased cache sizes in the application to increase their effectiveness. We also rolled out a patch to the AuthFactory that fixes a locking problem in the status level calculator and removes the root cause of one class of system slowdowns.

The result of these fixes, along with the switch to JDK 1.5 last Friday, is that we are seeing improved stability and overall lower cpu usage. However, we are still seeing periodic performance slowdowns that we continue to monitor closely.

We are actively working to implement a clustered solution for the application server, which will allow for larger caches and therefore fewer hits against the backend database. Meanwhile, we have decided to hold off for a few days on the feature enhancements we discussed last week, so that we can focus fully on the app server cluster.

Stability

Tonight we rolled out the following stability / performance changes:

• Database Connections: We increased the number of database connections by 50% because peak traffic occasionally maxed the number of database connections.

• APIC Setting: We disabled a power management setting that was causing timer-related problems.

• AuthFactory fix: We implemented a fix that allows us to avoid locking in the status level calculator. This removes the root cause of one relatively infrequent class of system slowdowns.

While these fixes help, we are actively working on some additional stability / performance fixes that we will roll out next week.

Features

We rolled out the feature fixes described in yesterday's blog post:

• Filter searches by user: There's an additional field in Advanced Search where you can specify an individual user so that search returns matching results from that user only.

• Search returns individual messages, not threads: Search returns a result for each thread (instead of for each message) matching the search term.

• Email notification of private messages: There is a new preferences that allows you to specify whether you get sent an email when you receive a private message. This works if your email is displayed or hidden.

• Email monitor improved: The email monitor now catches (and discards) a wider range of out-of-office replies.

• Edit Email in Profile: You can now edit your email address in your profile.

And also a couple of additional fixes:

• Status Level Legend: A status level legend showing the points for each status level now displays on the people page.

• Documents-only Communities: They now show the documents tab, e.g. Community Sample Code -- before they were erroneously showing a discussions tab even though discussions were disabled.