Perf Charts service experienced an internal error

[germoles]

Hi all,

we are in the process of setting up a new vSphere 4.0 Update 1 farm.

Trying to access the Performance Chart Overview page from our vCenter server all we get is the following error:

Perf Charts service experienced an internal error.

Message: Report application initialization is not completed successfully. Retry in 60 seconds.

Please note that our vCenter server connects to a local SQL Server 2005 Standard Edition RDBMS (we do not use trusted connection, here). According to our security policies, the SQL2005 instance is configured to listen to a non-standard port (i.e. different than the 1433, the default one).

I've found several posts in VMware Communities, but unfortunately none of the suggested steps have allowed us to resolve our issue.

I've also found the following document in VMware's KB:

http://kb.vmware.com/kb/1013877

and have tried to build up a proper Tomcat's connection string as follows (in the context.xml file under C:\Program Files\VMware\Infrastructure\tomcat\webapps\statsreport\META-INF) but again with no success:

jdbc:sqlserver://VCSRV01:3341;DbDriver=com.microsoft.sqlserver.jdbc.SQLServerDriver;integratedSecurity=false;DbUser=vcsrv

It looks like the settings in the context.xml file are not read at all by Tomcat.

We have by the way tried to temporarely reconfigure the SQL2005 instance to listen to its default port (1433), and by doing so the Performance Chart Overview page works as expected.

Unfortunately our security policies do not allow us to maintain such a configuration, and so we can't really leave the RDBMS set up to listen to the default port.

Is there any way we can override such a constraint in order to successfully configure the Performance Chart Overview page to work as due in a non standard SQL2005 configuration?

I've read from another source that there may be a way to configure all the needed settings in vCenter server's Registry, in the following key, but I'm not sure of what and how to specify there:

HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI

Thanks a lot in advance for your kind help.

Best regards,

Salvatore

SKY Italia s.r.l.

[NTShad0w]

Germoles,

i dont have an answer (yet) but I have similar problem,

its quite normal enviroment on default ports, i just upgrade VI3 to vS4 (U1) and... observe same 1 issue, on any performance tab ("Overwiew", "Advanced" working fine) I have same error:

Perf Charts service experienced an internal error

Message: Report application initialization is not completed succesfully. Retry in 60 seconds.

I'm just investigating the issue, vCenter Server is on MS Windows 2003 R2 x32 Standard EN and working perfectly good before upgrade, right now working almost ideally with only this one issue.

One more I remember form this instalaltion that VM admin here dont install any windows patches here, I decide to first upgrade VI3 to vS4 and then when observe the issue I install all patches but MS IE 8... here is an MS IE 6 installed, Im thinking to upgrade it but I dont think it may be an issue... but if You have quite same config, gimme an info.

If I fond something interesting I'll give an info here,

kind regards

Dawid Fusek

IT Security Consultant

Virtual Infrastructure Architect

COMP SA

[NTShad0w]

ok I have something that fix my problem...

Hope it will help U2

1. Close all sessions of vSphere Client.

2. Go Start > Administrative Tools > Services

3. Find VMware VirtualCenter Management Webservices and Right Click Stop

4. Don’t close Services.mmc but find a service SQL Server Browser

a. Right Click > Properties

b. Change Startup Type: Automatic

c. OK

d. Right Click and Start the Service (SQL Server Browser)

5. Start VMware VirtualCenter Management Webservices

6. wait 1min...

7. Login into vCenter and Look at Performance > Overview you should see Nice Graphs of CPU, Memory, Disk. You have options to pick between History Graphs and Real-time Graphs.

this solution is even found in remotexpert.net

kind regards

Dawid Fusek

IT Security Consultant

Virtual Infrastructure Architect

COMP SA

[brystro]

I have the same problem as my SQL server is on a non-standard port. My vCenter server (which is seperate from SQL server), does not have an SQL Browser service. Any other ideas?

[IOWDave · Accepted solution]

Good Afternoon,

We have had exactly the same problem. VC on one server, SQL DB on another server, and a non-standard SQL port. The only way that I can get this to work is to look at the ODBC settings in the registry (they may be under Wow6432Node if you are using 64 bit 2008) and add a value called Database with the name of the VC database and then change the Server value from<SERVER>,<PORT> to <SERVER>:<PORT>. If I restart the VC Web Management service the overview chart works. The bad news is that VC won't start unless I change the Server value to use a comma instead of a colon. This means that the Server value has to be changed to use a colon, the VC Web Management service restarted, and then the Server value changed to use a comma after every reboot of the VC server.

It's a pain, but it does appear to work.

Perhaps someone at VMware will be able to tell us why it behaves in this way.

Regards,

Dave.

[bggb29]

I just went through this issue with support this week. We also use a non standard port.

This will be fixed in a future release not the next release which should be 2 be update 3.

I think the se also said for vcenter 2.5 u6 or u5 was supposed to have the fix.

There is a jodbc call which apparently does not fully support the dsn calls via the system dsn

[germoles]

Hi Dave,

first of all, my apologies for the long delay in giving you my feedback.

I've been able to resolve the proposed issue by following your suggested steps.

I't just a little annoying that we'll need to remind ourselves to redo the same registry tweak every time we'll restart the vCenter server, but it works!

Hope that VMware will soon release a fix for such an issue, since it's now quite common to secure the RDBMS servers by also changing the default ports of the relevant services to reduce the possible attack surface.

Thank you very much for sharing with us your solution,

Best regards,

Salvatore

[virtualizeme215]

Germoles,

I have the same exact issue you have. Going through SQL DB hardening i have also lost use of the Perf Charts.

Can you tell me where you added that registry key exactly. I have done a search for "Wow6432Node" and it appears in multiple locations.

Thanks in advance

[germoles]

Hi,

please note that my vCenter Server is a WS2003 Standard Edition SP2 32bit system, not a 64bit one.

Anyway, let's say by example that:

- the vCenter Server's hostname is 'VCENTERSRV';

- the System DSN pointing to the (remote or local) SQL Server instance to be used by vCenter is named 'VMware VirtualCenter';

- the TCP port on which that same SQL Server instance is 'securely listening for incoming connections is the '2433'.

Said this, in my case I had to tweak the following Registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\ODBC\ODBC.INI\VMware VirtualCenter

while in case of a 64bit WS2003 OS the Registry key should be the following, instead (this means that the vCenter suite is actually a 32bit application even if installed on a 64bit WS2003 server):

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ODBC\ODBC.INI\VMware VirtualCenter

If you cannot see the 'Perf Charts Overview' graphs (and this will happen at each server reboot and also at each restart of the 'VMware VirtualCenter Management Webservices' service) then:

a) set the 'Server' Registry value from 'VCENTERSRV' to:

VCENTERSRV:2433

b) restart the 'VMware VirtualCenter Management Webservices'' service

c) reset the 'Server' Registry value back to its original content:

VCENTERSRV

And that should be all you need to do in order to make the 'Perf Charts Overview' feature work on your vCenter Server!

Best regards,

Salvatore

[Falch]

We manage to solve our problem by doing this for Windows 2008 64Bit with SQL: Taken from the

Perform the following steps to start the Performance chart application:

1. Open the Performance chart database configuration file. For vSphere 4.0, it is context.xml located at \tomcat\webapps\statsreport\META-INF by default. For vSphere 4.0 Update 1, it is statsreport.xml located at \tomcat\conf\Catalina\localhost.

2. Edit the file as follows for your database server:

3. Restart VMware VirtualCenter Management Webservices.

We changed the file statsreport.xml located at C:\Program Files (x86)\VMware\Infrastructure\tomcat\conf\Catalina\localhost

As you see we added 2 lines below with bold, and used our input at the red text. Then Restart VMware VirtualCenter Management Webservices.

Restart the Client too, to see the changes in the client.

<Resource auth="Container"

name="jdbc/StatsDS"

type="javax.sql.DataSource"

factory="org.apache.tomcat.dbcp.dbcp.BasicDataSourceFactory"

driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"

url="jdbc:sqlserver://Myserver:1433;instanceName=MyInstancename;databaseName=MyDatabasename;integratedSecurity=true;"

initialSize="3"

maxActive="10"

maxIdle="3"

maxWait="10000"

defaultReadOnly="true"

defaultTransactionIsolation="READ_COMMITTED"

removeAbandoned="true"

removeAbandonedTimeout="60"

/>

Maybe this will help some of you ?

[germoles]

Hi Falch,

thank you very much for your suggestion: I've just tested it on my 4.01 vCenter server, and it works fine.

I've been able to resolve once for all the issue, with no further need to modify the registry each time we'll restart the 'VMware VirtualCenter Management Webservices' service.

Thank you again and best regards,

Salvatore

[Falch]

Nice to know that the solution works for other too.

[androidtopp]

That solution is not working for me - I've tried a number of different connection strings, such as:

url="jdbc:sqlserver://:1433;database=vCenter;integratedSecurity=true;"

And after restarting my management web service, as well as everything else I can think of, I'm still seeing the same error. Does that connection string look correct? My "driverclassname" line is:

driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"

It was like this already - I only had to make changes to the "url" line.

Thanks,

Andrew

[germoles]

Hi Andrew,

here's the "url" string in the C:\Program Files\VMware\Infrastructure\tomcat\conf\Catalina\localhost\statsreport.xml file since we have a vSphere 4 Update 1 server (I've only masked the sensitive info):

url="jdbc:sqlserver://SQLESX:2434;instanceName=SQLESX;databaseName=VSPHERE;integratedSecurity=true"

From your post I can see that both the instanceName and the databaseName information are missing (also, the databaseName option is named as database, and I'm not sure if that's OK anyway or not):

url="jdbc:sqlserver://:1433;database=vCenter;integratedSecurity=true;"

Finally, the port number in your example is the default 1433, and if I'm not wrong that by itself should let the Perf Charts work with no needs for any tweaks (in our case, in fact, the Perf Charts didn't work after changing the SQL Server default port to a different one).

Best regards,

Salvatore

[androidtopp]

Salvatore,

That's my typo - I am using "databaseName" in the actual connection string.

Turns out it was the presence of the port number - as soon as I removed the ":1433" from the string, everything started working. For some reason, even though this is the default port, I still had to follow these steps to get things working properly. I've seen this on multiple vCenter installs, all using remote SQL on default ports, so maybe I'm just an idiot - either way, I've got a fix now...

Thanks,

Andrew

[germoles]

Hi Andrew,

I'm glad you made it work!

I don't think it's been your fault: I rather think it's a badly documented feature!

Best regards,

Salvatore

[WilderB]

Falch,

Thanks for the precise and helpfull post.

I tried to edit statsrepost.xml to connect to a SQL 2008 using sa user (not integrated security). The file looks like this:

<Resource auth="Container"

name="jdbc/StatsDS"

type="javax.sql.DataSource"

factory="org.apache.tomcat.dbcp.dbcp.BasicDataSourceFactory"

driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"

url="jdbc:sqlserver://deneb:2510;instanceName=SQLPRD10;databaseName=VCDB;integratedSecurity=false;user=sa;password=P@ssw0rd;"

initialSize="3"

maxActive="10"

maxIdle="3"

maxWait="10000"

defaultReadOnly="true"

defaultTransactionIsolation="READ_COMMITTED"

removeAbandoned="true"

removeAbandonedTimeout="60"

/>

When I open vSphere Client and select Performance I receive the following error:

HTTP Status 404 - /statsreport/vicr.do

-

---

type Status report

message /statsreport/vicr.do

description The requested resource (/statsreport/vicr.do) is not available.

-

---

Apache Tomcat/6.0.20

Any ideas??

[kgrnathan]

Editing the Statsreport.xml *and restarting the Webservice worked fine for me, but after many attempts with different formats .

Thanks a lot for the solution

[germoles]

Hi, I'm happy to know you too managed to resolve your issue.

To be honest, it's Falch the one who deserves your thanks, not me.

Best regards,

Salvatore

[stratolynne]

In reference to:

a) set the 'Server' Registry value from 'VCENTERSRV' to:

VCENTERSRV:2433

When I look in the 'Server' registry entry is: localhost\SQLEXP_VIM

So where do I put the port?

Thanks!