Graph data retention for VCenter 5 appliance

[technobuddha]

Hi all,

I notice about a month ago, that our graph retention went down to 1 month of collected data.

I used to have all the months of graphs since the install.

I read somewhere that you could go to:

vCenter Server Settings -> Database Retention Policy.

it was checked at 356 days. I unchecked it to see if that worked, and rebooted.

Nope.

Something changed, since the install, and I can't figure out what.

I did the latest update, hoping that would help but it didn't.

Does anyone know where I could troubleshoot this?

thanks!

[admin]

Did you upgrade your vCenter appliance from an older version to 5? What RDBMS do you use?

[technobuddha]

Thanks for asking.

This is a fresh install of VSphere 5.0,

with a fresh downloaded VCenter 5.0 appliance.

I noticed one day, that the graphing history changed and was only giving me 1 months worth of data.

I had installed Vsphere and Vcenter appliance in Feb. and in april one day most of it was gone but 1 months worth.

I havent chagned anything, and now i'm going around trying to find what log files I need to look at.

so when you ask me what database it is, I don't know, as it came with the vcenter 5.0 appliance.

Perhaps someone can help me point to the exact log files that would hold errors?

there's like a billion of them all over the place! LOL

thanks!

[admin]

Just to clarify, are you looking at Performance charts?

I'm asking because from your first post I get the impression that you're looking at perf charts but then you try to configure a different retention policy.

[technobuddha]

yes I am looking at the Performance charts.

with regards to "retention policy", I assumed (my bad?) that was with regards to the graph data.

I'm trying to think of everything as to why the appliance would just simply only provide 1 month of data for the performance graph.

[admin]

For performance charts configuration you should look into: vCenter Server Settings -> Statistics

You have 4 different type of aggregations: daily, weekly, monthly and yearly. Each one has their specific statistics level and retention policy.

For more information about each aggregation please select it and press "Edit...".

Back to your original question:

> I notice about a month ago, that our graph retention went down to 1 month of collected data.

> I used to have all the months of graphs since the install.

This is because on monthly level the retention policy (aka "Keep Samples for") is configured at 1 month.

[technobuddha]

Thank you ivaivanov, but if you could tell me where I would find this setting "monthly level the retention policy".

I looked under vCenter Server Settings and don't see it.

Perhaps its somewhere else, or is a command line switch?

[technobuddha]

And just to clairfy, I found the area where I think your talking about (attached picture),

BUT, I can only change the "Keep samples for:" for "1 Day", and the "5 Minutes"....

I think you may mean the "2 Hours", which is set for "1 Month" but i can't change that variable for some reason.

[admin]

This is the screen I'm refering to. If you select help you can see that the retention could be changed only for daily (aka 5 minutes) and yearly (aka 1 day) levels.

[technobuddha]

Yep, that's correct.

SO, going back to square one.

what happened to my previous 3 months of data?

is there an internal switch?

I was able to see all the performance data that was collected since vsphere/vcenter was installed.

then POOF, after about 3 months, the collection stopped, and now all I get is a consistant 1 month of performance data collection.

have any more ideas?

[admin]

The montly level data retention policy is to keep up to 1 month stats data. This is why your old data had been purged. You can monitor old historical data by looking at year stats data.

> the collection stopped,

No new statistics data is being gathered? Do you have any warning/error messages in Home -> Administration -> vCenter Service Status ?

[technobuddha]

It seems i'm bad with explaining what's going on.

so i took a snapshot of the actual memory graph of one of my servers.

As you can see, I choose an "advanced" graph in the performance tab.

Normally, I would see a graph that would go all the way to Feb. 21st!

BUT, since the begining of  May, all of the graph data dissapeared from Feb. 21st to end of April.

And as the days go by, I only see 1 month of data now.

I want it back to its original state: accumulative from the beinging of the install.

does that make sense?

[admin]

Now that is strange. You need to see what is the status of your DB2 (VCVA embedded) database.

1. login as root to your VCVA

2. cat /etc/vmware-vpx/embedded.cfg and search for VCDB user password (e.g.<pass>)

3. su - db2inst1

4. db2 connect to VCDB user vc using <pass>

5. db2 "select min(sample_time) from vpx_sample_time1"

6. db2 "select min(sample_time) from vpx_sample_time2"

7. db2 "select min(sample_time) from vpx_sample_time3"

8. db2 "select min(sample_time) from vpx_sample_time4"

9. db2 "select interval_def_name, interval_val, interval_length, rollup_enabled_flg from VPX_STAT_INTERVAL_DEF"

Those queries should give more details about the issue you are facing.

[technobuddha]

Thank you XXX for your help!

I deeply appreciate your time and efforts!

I'll go and try and figure out what all this means!

Here's the results:

db2inst1@localhost:~> db2 connect to VCDB user vc using mqsArk2iYGw4Ix2L

   Database Connection Information

Database server        = DB2/LINUXX8664 9.7.2
SQL authorization ID   = VC
Local database alias   = VCDB

db2inst1@localhost:~> db2 "select min(sample_time) from vpx_sample_time1"

1                        
--------------------------
2012-07-06-09.45.00.000000

  1 record(s) selected.

db2inst1@localhost:~> db2 "select min(sample_time) from vpx_sample_time2"

1                        
--------------------------
2012-07-04-09.00.00.000000

  1 record(s) selected.

db2inst1@localhost:~> db2 "select min(sample_time) from vpx_sample_time3"

1                        
--------------------------
2012-06-11-11.00.00.000000

  1 record(s) selected.

db2inst1@localhost:~> db2 "select min(sample_time) from vpx_sample_time4"

1                        
--------------------------
2012-06-12-00.00.00.000000

  1 record(s) selected.

db2inst1@localhost:~> db2 "select interval_def_name, interval_val, interval_length, rollup_enabled_flg from VPX_STAT_INTERVAL_DEF"

INTERVAL_DEF_NAME                                            INTERVAL_VAL INTERVAL_LENGTH ROLLUP_ENABLED_FLG
------------------------------------------------------------ ------------ --------------- ------------------
history.PastDay                                                      300.         432000.                 1.
history.PastWeek                                                    1800.         604800.                 1.
history.PastMonth                                                   7200.        2592000.                 1.
history.PastYear                                                   86400.       31536000.                 1.

  4 record(s) selected.

[admin]

Since I'm going on vacation today and I'll be back next Monday.

Can you rerun

db2 "select min(sample_time) from vpx_sample_time3"

and

db2 "select min(sample_time) from vpx_sample_time4"

again on Monday?

From the data you've provided everything looks fine but there are some funny numbers in those 2 tables. In order to better understand what's going on, I'll need to track how their data changes.

[technobuddha]

I sure will! Thanks for your efforts!

[technobuddha]

I hope this helps?

db2inst1@localhost:~> db2 "select min(sample_time) from vpx_sample_time3"

1                        
--------------------------
2012-06-16-01.00.00.000000

  1 record(s) selected.

db2inst1@localhost:~> db2 "select min(sample_time) from vpx_sample_time4"

1                        
--------------------------
2012-06-16-00.00.00.000000

  1 record(s) selected.

thanks!

[admin · Accepted solution]

Now this is a problem...

The best workaround for your missing perf stats is to issue the following update:

db2 "update VPX_STAT_INTERVAL_DEF set INTERVAL_LENGTH = 31536000 where INTERVAL_SEQ_NUM = 3"

Unfortunatelly this won't restore your perf data but you'll no longer have the same issue with missing stats data.

[technobuddha]

Ok, let me see how that goes and i'll let you know tomorrow.!

thank you for your time!

[technobuddha]

YES! that solved the problem!

Thank you so much for your time.

I wouldn't have figured it out without you!