0 Replies Latest reply on Dec 5, 2017 5:17 AM by crichardson23

    Invalid performance metrics

    crichardson23 Novice

      We've just recently completed migrating our infrastructure to a new cluster of Dell R740 servers running ESXi 6.5 6765664 (Dell Customized). We use PowerShell to send vCenter performance metrics to Graphite and review the data using Grafana. Shortly after migrating servers we began to notice that invalid performance metrics were appearing in Graphite. In some cases our datastore read and write latency was greater than "1 day" and in some cases "1 year". I checked the time on the ESXi servers and everything appears to be in order. The issue is occurring on multiple ESXi hosts.

       

      There's no impact to production but it does completely skew our performance metrics when trying to compare our performance gains after migrating to the new servers. It's also triggered a couple bogus datastore latency alerts (but none in the past couple of days). I'm concerned it could be skewing all performance metrics.

       

      I've attached a couple screenshots from vCenter and Grafana - see for yourself.

       

      Previous 24 hours (vCenter):

      SkewedPerformanceMetrics1.png

       

      Previous 7 days (Grafana/Graphite)

      SkewedPerformanceMetrics2.png

       

      Any suggestions would be greatly appreciated.