VMware Cloud Community
OphirZahavi
Contributor
Contributor

Failed insertion of statistics - vSphere 4.1.0

Hello,

I have a problem, when viewing the "events" shown in my vCenter I get the following error "Stats insertion failed for entity XXXX due to ODBC error".

I get this error for a bunch of my ESX's.

I  looked up the problem and found a bunch of solutions that talked about the SQL jobs and Programmability - Stored Procedures, I did all of the checks and everything is as it should be.

Is there anyother thing that can be checked regrading this error?

Thanks!

0 Kudos
29 Replies
OsburnM
Hot Shot
Hot Shot

Ugh...  so they gave you not solution?  I'm kicking around the idea of going back to level 1 stats if there's no fix.

0 Kudos
squebel
Contributor
Contributor

Nope, not yet. BTW, my colleague found this KB this morning: VMware KB: Historical performance data in vCenter Server does not get saved

I've asked Support if this could help w/ our issue and we've asked our DBA to check on the problem as well. We're waiting to hear back.

0 Kudos
OsburnM
Hot Shot
Hot Shot

Nice find.  I'm currently running 5.0 Update 2 so one would think it shouldn't apply to me.  However, our VC is an upgrade of an upgrade of an upgrade so who knows what stored procedures are still good.

0 Kudos
squebel
Contributor
Contributor

So, Support has provided little to no help at all and we are still seeing the insertion errors. We've tried pretty much everything we could think of and there's be no help. This issue has been around for far too long and I can't believe there aren't more customers seeing the same problem that Engineering hasn't been forced to come up with a solution. Support only suggests subscribing to the KB so that we know when there are updates but I am not expecting much.

0 Kudos
OsburnM
Hot Shot
Hot Shot

squebel wrote:

So, Support has provided little to no help at all and we are still seeing the insertion errors. We've tried pretty much everything we could think of and there's be no help. This issue has been around for far too long and I can't believe there aren't more customers seeing the same problem that Engineering hasn't been forced to come up with a solution. Support only suggests subscribing to the KB so that we know when there are updates but I am not expecting much.

Just wanted to update that I have sense upgraded my vSphere environment to 5.1-Update 1.  The SQL DB was totally changed and it went from a few HIST STAT tables to a LOT.  Doing so allowed me to put my stat levels back up to 2 w/o any disruption and I no-longer see any breaks in my performance logs and/or errors in events logs.

0 Kudos
squebel
Contributor
Contributor

Interesting. Support has said that this issue is also affecting 5.1 instances but they didn't provide any more info than that. It sure seems like this problem is hitting some people but not others and if you do get nailed with it, there's apparently no way to fix it.

Did you say that you had your 5-minute stats set up to Level 2 because of 3rd party app that access the database through the webservice SDK? If so, would you mind telling me which app that was?

0 Kudos
VMCHIS
Enthusiast
Enthusiast

Has there been any solutions to this issue?  I am also experiencing the same problem and recreated the SQL jobs and that did not work.  I too have my 5 Minutes stats interval set to Save For 1 Days and Stats Level set too 2 for use with our Foglight software.  I would really like to get rid of these pesky errors.

0 Kudos
squebel
Contributor
Contributor

VMCHIS wrote:

Has there been any solutions to this issue?  I am also experiencing the same problem and recreated the SQL jobs and that did not work.  I too have my 5 Minutes stats interval set to Save For 1 Days and Stats Level set too 2 for use with our Foglight software.  I would really like to get rid of these pesky errors.

Yes, I received a "solution" from engineering... sorry for never updating and finishing this thread. Engineer suggest that I try the following:

The following are two parameters that are designed for user to configure. We can increase this parameter values and see if it can help improve the situation. Edit the VC configuration file (vpxd.cfg) by adding the following parameters under the <vpxd> tag and restart VC service for them to take effect.

  1. e.g.

<vpxd>

  <stats>

<maxProcessingThreads>3</maxProcessingThreads>

  </stats>

</vpxd>

------------------------

- vpxd.stats.maxProcessingThreads -- Number of concurrent temp

  table processing stored procedures can be invoked. Default 1.

To do: Increase the default value from 1 to 3 and see if it helps.

- vpxd.stats.maxTableAcqTimeSec -- How long should a thread

  wait to get a table before giving up on persisting a set of

  stats to the DB. This limit should only be hit is all the

  temp tables are pending for processing indicating an overly

  loaded DB. Default 30

To do: Increase this value to 600

I asked Engineering why they don't just supply this information in a KB article and they said that they look at this problem on a case by case scenario. They look at the specific logs and then try to suggest a solution. So, apparently for us, this was what they suggested for or situation and it DID seem to work. We were able to run our 5-min logging level back up to "2" and the insertion errors disappeared. Not saying this will fix your problem but it's pretty easy change to implement and then check. Hopefully this helps!

0 Kudos
VMCHIS
Enthusiast
Enthusiast

Where do i put the following?


- vpxd.stats.maxProcessingThreads -- Number of concurrent temp

  table processing stored procedures can be invoked. Default 1.

To do: Increase the default value from 1 to 3 and see if it helps.

- vpxd.stats.maxTableAcqTimeSec -- How long should a thread

  wait to get a table before giving up on persisting a set of

  stats to the DB. This limit should only be hit is all the

  temp tables are pending for processing indicating an overly

  loaded DB. Default 30

To do: Increase this value to 600

0 Kudos
squebel
Contributor
Contributor

I believe there should be a <vpxd> </vpxd> section in your vpxd.cfg file. I think (I remember) there being one in mine and it may have only had one thing in it. Mine looks like this:

  <vpxd>

       <stats>

            <maxProcessingThreads>3</maxProcessingThreads>

            <maxTableAcqTimeSec>600</maxTableAcqTimeSec>

       </stats>

  <filterOverheadLimitIssues>true</filterOverheadLimitIssues>

  </vpxd>

0 Kudos