vCenter High disk I/O after 7.0.2 update

[sbd27]

After updating vCenter from 7.0.1 to 7.0.2 build 17958471,  Disk I/O on the appliance went from about 1k avg to now around 75k. The jump was so severe my Storage Team had to bring it to my attention as total I/O on our Pure SAN jumped almost 10%!

I have also noticed CPU and Memory resource utilization have drastically increased above their normal average. Look at the chart below and you can see when the update occurred. Rebooting the VCA did not help. Anybody else seeing anything like this?

[depping]

just looking at my environment, and not seeing a similar trend. Not sure what could be causing this, you could of course always ask support to inspect this, as it is an extremely high number of IOPS.

[sbd27]

I also have a separate vCenter that is much smaller then this one (only 4  cluster and about 10 hosts), and like you said it does not have the same issues. The vCenter that is seeing this issue has close to 60 hosts and 8 different clusters.

I plan to reach out to VMWare today.

[depping]

Do you have any of the newer services enabled? (Kubernetes)

[sbd27]

No, but

I have dug a little deeper. Llooks like something it reading the heck out of my SEAT db

sdh is the disk that holds the SEAT filesystem. I though maybe it was our monitoring software, but I disabled it and nothing. Does anyone know how to dive into postgres and get info on where these reads are coming from?

[depping · Accepted solution]

strange, this is where the events are stored... check this KB, it is on a different topic, but it could be useful to clean out the db:

https://kb.vmware.com/s/article/2119809

[sbd27]

@depping That was it! Sorry for the long turn around in my response, because I was working with VMWare support and not getting far. I finally started pruning truncating the event tables (vpx_event_* and vpx_event_args_*) and the I/O finally dropped down to more normal sub 1Mbps!

All other resource indicators dropped too.

[depping]

good to hear it worked, thanks for the update!