Blog Posts

Since applying the performance patch this morning (see previous post in this blog), we've had consistently good performance all day, and no one has reported performance issues. To tune performance further, we are akamaizing (i.e. moving to Akamai edge servers) some static files like .gifs. And while we continue to monitor the systems closely, it seems it's time to turn our attention to usability.

And with that in mind, John updated the VMware Communities Known Issues today, which we are using to prioritize and drive our development tasks. Be sure to read through it and let us know if you perceive things differently. We plan to provide more updates next week on when you can expect to start seeing these on the site -- we are working to get the top items as quickly as possible.

Performance: We implemented performance fixes today, which have decreased load times of pages and removed the intermittent delays many users were experiencing.

• Configuration changes on the proxy
• Increased cache size on the application server
• Removal of Javascript from the homepage dropdowns and the right column

We are working to further reduce the amount of Javascript in the next few days, especially on the homepage. And we are continuing to monitor performance and work with Jive Software to resolve issues.

We are now turning our focus to the many usability fixes and improvements you all have been reporting.

Lost in Transition: To start off, we are working on the following Forums features missing in Clearspace:

1. Threaded view
2. See all posts from user -- now you can only see the new threads started by a user, not all posts and replies
3. View communities with unread threads at a glance, so users can scan the list of communities and see which ones have new posts since they last visited
4. Visible indicator of watches/favorites so users can track threads they were researching or working on answering.
5. Set # of posts per page as a per-user configurable preference
6. Search returns per-Message results, i.e. if a search term shows up in several messages of a thread, it should return one result instead of one result per message
7. Turn off the editor and use a plain text editor
8. User filter in search, i.e. ability to search by user
9. Code markup missing, e.g. for [code], pre , and [url]{noformat}
10. Search in this community (currently only search in all communities)
11. Advanced search so users don't have to do a blank search to get to the advanced search functionality
12. No PM notification if private email

These are roughly in priority order, although we may increase the priority of the editor fix, since that prevents some users from posting. We will update here and in the Missing Features thread.

Other Usability Fixes: We are also looking at other usability fixes, and will have a fully-prioritized list tomorrow. Among others, we are looking at the following:

• Making Discussions the default tab (not All Content), which will make read tracking of threads more visible
• Fixing OR search
• Turn off default email notification

Low-Hanging Fruit: While we investigated the more important issues, we slipped in some easy fixes:

• Fixing of Reply button so it doesn't fall off the page -- we enclosed those pages in a <table>
• We will add tables to a few other pages with similar problems
• Put "Up to Discussions" link at the bottom of threads as well -- they were only at the top
• Put points in Top Members boxes
• Ensure all listings have Previous and Next buttons -- listings on All Contents tab did not

I know many of you continue to be frustrated, but I ask you to bear with us while we fix these issues. With a little patience, I still firmly believe the benefits of the new features will become more and more apparent. One is already apparent to us behind the scenes -- this platform is much easier to customize and bug-fix than Forums. We are working to push out fixes as fast as we can.

We think we have some of the major problems fixed, after spending many long days, it seems we have solved most of the major issues with the site. Performance seem to be very good now, stability seems to be where we would like it to be, and the functionality is behaving as expected.

Thanks for eveyone's help, I know we called several of you on the phone asking specific question about what you were seeing at your location. Thanks for everyone hanging in there and giving us the load we needed to solve the problems.

Hopefully we are now back to where we had hoped to be Saturday. We have new software, with new features. We know there are things missing, and we look forward to all the communities input and help. This platform has a great SDK, and we have a lot of new interesting features we can add going forward.

I hope we see a collection of good documents show up, and hope eveyone enjoys using the the new features.

Now.. I think th entire team is going to sleep for 3 days straight... see you on Friday. =)

Eric N.

VMware

Communities Team

I thought it was time to start a blog to describe what we have been doing here at VMware with regard to upgrading from Jive Forums, to Jive Clearspace. I suspect everyone out on the VMTN forums might find regular posts interesting, or at least a little calming.

The VMware web team has been working pretty much round the clock since we put VMTN in read-only mode last Thursday evening. The goal was to convert the existing forum records (well over 500,000) to the new clearspace format, and in the process not lose points, any significant data, and real useability. It has been a tall order, but we all respect and really enjoy the activity and relationships with the community members. So it was important to us to preserve as much as we could, while moving to a platform that would allow everyone to create/upload and share documents.

Possibly we will do some additional posts to give you a sence of the timeline over the long weekend, but I thought I would start at where we are now, and later post a few other enteries of where we have been.

Where we are:

o We have managed to debug some serious issues around points calculations, which was bringing the system to it's knees on Sunday. It turns out, as the big users
logged into the system, we discovered that the points calculation system was not able to handle so many big point folks on all at once. We eventually got a fix and
solved this issue.

o We next ran into a database connection issue, where over time, database connections were not being dropped, this too was fixed.

o We next found a point system bug that was still causing excessive database lookups on points. It was Sunday evening when we got this but fix in, and when we got
it in, performance improved dramatically. We went from 7-20 second page load times, down to 2 second page load times. However, after being in production for over
2 hours, things began to gradually get slower, until the system finally crashed.

o Today, we have been working all day on a memory leak, that we believed was causing the problem. We put a performance script in place that would restart Clearspace when
page loads exceeded 30 seconds. This resulted in a apache restart approximately every 2 hours.

o Tonight (Monday Night) at 8:00pm, we deployed the memory leak fix, which seem to improve performance, and memory remained constant. During this deployment, we were
encouraged due to the average page load hovering around 1 second and as fast as .5 seconds. But, after 58 minutes in production, with an average of 50 users online, and the
team pounding on the system, it crashed. Crash dumps indicate a garbage collection issue, possibly due to an imporper collection model.

o We are done for tonight, we have put an apache restart script in place on the hour, this should keep the site pretty fast (under 4 second page loads consistently) it also will
give people an idea when to save before we restart. At 5 minutes to the hour, save your work, and be ready for the site to reset on the hour, the reset takes 1-2 minutes.

Tomorrow we will be back, in the morning, looking at stack traces, the vendors top engineers, and the founder of the company (Jive Software) is working 14 hours a day to solve these issues. Each day we make progress, tonight the site was very fast for a while, we will keep working on it, and hopefully, will solve the last of our problems this week. If we
fall back to the old forums, we will have to restart the entire process (including the 4 days read only) in a few months when the next deployment window opens. We are hoping to solve these problems this week, so we can all upload documents and do collaboratoin together.

Thanks for all the feedback, and encouragement.. we have gotten some good layout ideas, and have a nice list of upgrades to work on once we solve our pressing issues of system stability and performance. More status tomorrow.

The VMware communities team.