Update Storage Views fails with popup error

[AndyShinn]

This used to work and now it does not. I am not sure what has changed. But when I go to the Storage Views tab, then click the Update link, it comes back with a popup error:

An Unknown error has occured

vCenter logs for this time period have a lot of data. But nothing seems be relevant to storage.

Where should I begin looking for debug information?

[RParker]

An Unknown error has occured

Since its unknown it's most likely not trapped, therefore not logged. So you may be out of luck. Did you check Windows logs? It could simply be a VI Client error / Application issue, and not vCenter. Those are hard to diagnose.

[AndyShinn]

OK, I think I have a little more information to go by. Been debugging this all day and here is what I found:

I am guessing 'Storage Views' and storage information comes from the Storage Management Service (sms running in Tomcat). In my vCenter Service Status I have an error:

Storage Management Service: Service initialization failed

So I finally found where this is located and run from ( C:\Program Files\VMware\Infrastructure\tomcat\webapps\sms on my system ) and the logs fromTomcat show this error:

Jul 9, 2009 7:54:05 PM org.apache.catalina.loader.WebappClassLoader validateJarFile

INFO: validateJarFile(C:\Program Files\VMware\Infrastructure\tomcat\webapps\sms\WEB-INF\lib\servlet-api.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class

Since this error is from sms I am guessing the servlet is not starting up properly. But from here I am not sure where to go.

Anyone have any ideas?

[Michelle_Laveri]

I have the exact same error. Any resolution/progress...?

Regards

Mike Laverick

RTFM Education

http://www.rtfm-ed.co.uk

Author of the SRM Book: http://www.lulu.com/content/4343147

[AndyShinn]

Nope, not yet. Can I get your OS and SQL version you are running for vCenter? I am on Windows 2008 and SQL Standard 2005. Am wondering if it is related to Windows 2008.

[Michelle_Laveri]

I'm on W2K3.

Are you running VC and SQL2005 on the SAME Windows Instance....

If you are - I might have something for you...

Regards

Mike Laverick

RTFM Education

http://www.rtfm-ed.co.uk

Author of the SRM Book: http://www.lulu.com/content/4343147

[AndyShinn]

Yes, I am running SQL 2005 and vCenter on the same Windows 2008 machine. It is a physical machine, not a VM.

The machine actually has a hardware problem so I am replacing it with a new vCenter / W2K8 / SQL 2005 setup soon. I will see if a freshinstall changes anything... This combination seemed to be very difficult to get running properly...

[Michelle_Laveri]

Ah ha!

On the vmware training courses they run VC+SQL on the same Windows instance.

They say to do this:

Choose Start > Programs > Microsoft SQL Server 2005 > Configuration Tools > SQL Server Configuration Manager.

In the SQL Server Configuration Manager, do the following:

Expand SQL Server 2005 Network Configuration and select Protocols for SQLEXP_VIM.

In the right-hand window, right-click TCP/IP and select Enable.

Click OK.

Restart the services named SQL Server (SQLEXP_VIM) and SQL Server Browser

I tried on my VC but it didn't help as SQL runs as separate instance. In fact the Server Browser was not started and was set to be disabled.

I would have thought you might have to restart VC and web-service for this to have any affect...

Regards

Mike Laverick

RTFM Education

http://www.rtfm-ed.co.uk

Author of the SRM Book: http://www.lulu.com/content/4343147

[AndyShinn]

Oh, I have alreadyd done this long ago. I don't think any of the issues I am having have been SQL related so far. The vCenter <-> SQL server seems to be pretty solid. It's the plugin components / Tomcat web service stuff that seems to be tricky...

[iamsiju]

Hi,

Thanks for your cooperation!

In windows 2008, could you please verify "UAC" is ON in case if logged on user is not local administrator.

Also could you upload tomcat, vpxd and vSphere client logs for more information?

Thanks,

[AndyShinn]

I just reinstalled vCenter. This time I am using Windows 2008 and SQL 2008. I have UAC disabled.

The same problem has happened, Storage manager works fine first startup. After rebooting server it no longer works and the Service status says:

Storage Management Service: Service initialization failed

Attached is the catalina logs.

[Michelle_Laveri]

Don't know if this helps you.

But it seems clear that my problems are caused by the PowerPath for VMware PSP installed to my ESX hosts...

Waiting for feedback from EMC about how to proceed... In the meantime I've turned off PowerPath with the escli command...

Regards

Mike Laverick

RTFM Education

http://www.rtfm-ed.co.uk

Author of the SRM Book: http://www.lulu.com/content/4343147

[Joonie]

Hi,

I was wondering if you can share the escli commands you are running to turn off Powerpath. Thakns

Joonie

[Michelle_Laveri]

*Proceed with caution...*

The EMC PowerPath (EMC® PowerPath®/VE Version 5.4 for VMware vSphere - P/N 300-008-703) guide talks of using

esxcli corestorage claiming unclaim

Unclaims a path or set of paths, and disassociates those paths

from an MPP plugin. You can only unclaim active paths with no I/O.

The second part is the scary bit... I would use maintanace mode to achieve this...

I've raised the issue with Chad Sakac of EMC.

He gave me this work around... which I'm just about to test. If you hang for second I can tell if it works

1. Goto the SMS folder for vSphere 4.0. So, if vSphere 4.0 was installed under "c:\Program Files\VMware\Infrastructure", run the following command:

C:\> cd "c:\Program Files\VMware\Infrastructure\tomcat\webapps\sms\WEB-INF\services\smService\com\vmware\vim\sms\data\xml"

2. Rename the VcDbQueryMsSqlMap.xml, VcDbQueryDb2SqlMap.xml, VcDbQueryOracleSqlMap.xml

- move VcDbQueryMsSqlMap.xml VcDbQueryMsSqlMap.xml.org

- move VcDbQueryDb2SqlMap.xml VcDbQueryDb2SqlMap.xml.org

- move VcDbQueryOracleSqlMap.xml VcDbQueryOracleSqlMap.xml.org

3. Copy the three xml files, attached with this email, to "c:\Program Files\VMware\Infrastructure\tomcat\webapps\sms\WEB-INF\services\smService\com\vmware\vim\sms\data\xml" (assuming that vSpehere 4.0 is installed under "c:\Program Files\VMware\Infrastructure").

4. Restart "VMware VirtualCenter Management Webservices".

Regards

Mike Laverick

RTFM Education

http://www.rtfm-ed.co.uk

Author of the SRM Book: http://www.lulu.com/content/4343147

[Michelle_Laveri]

BTW. he forgot to attach the .xml files - so I've email him to request them.

I'm not sure if these .xml files will be publically released or indeed who is precisely responsible. Is this EMC or VMW problem...

I will try to get back to you today (GMT:3.35PM) with some kind of update....

Regards

Mike Laverick

RTFM Education

http://www.rtfm-ed.co.uk

Author of the SRM Book: http://www.lulu.com/content/4343147

[AndyShinn]

Hi Mike,

When you are able to attach the files I will also try these steps even though I am not using any EMC storage. If this fixes the problem for me then I would guess it is a VMware issue and not EMC.

[Michelle_Laveri]

OK. Chad's asked me to hang fire while a formal patch plan is put together - it is a VMware Fix.

Whether it fixes others peoples problems is anyone's guess. I'm ass-um-ing that my error message is PowerPath related...

It does seem like a "known issue" if there's a patch on its way...

Regards

Mike Laverick

RTFM Education

http://www.rtfm-ed.co.uk

Author of the SRM Book: http://www.lulu.com/content/4343147

[nbhms]

Did this ever get resolved? I'm having the same issue, however, there is no EMC in the environment.

VirtualCenter 4 running on a Windows 2003 server standard guest on an ESXi 3 host. SQL database is SQL 2005 standard running on another virtual Windows 2003 box on the same ESXi Host.

Storage Views was working fine. I rebooted the SQL server and the Virtual Center server, (no changes, just a reboot), and ever since I'm getting the "Storage Management Service" "Service Initialization Failed" error on the vCenter Service Status tab and Storage Views no longer works.

The rest of Virtual Center is working fine, and the tomcat page is up and working for downloading of the Infrastructure client.

[Michelle_Laveri]

Nope is not been resolved.

That said, its been a while since I did a VMware Update on my esx hosts...

Regards

Mike Laverick

RTFM Education

http://www.rtfm-ed.co.uk

Author of the SRM Book: http://www.lulu.com/content/4343147

[AndyShinn]

Nope, I never got it resolved. At this point, I am waiting the next update to vCenter to see if it changes anything. I think VMware is aware of the problem.

-Andy