How does Wavefront handle inactive sources?

[AbhishekSK]

As part of our Wavefront implementation, we have a metric called "Review.Gathered.count". Back in early February, we made some changes to the tagging associated with this metric, and now in late April we see that when reviewing the data in the metric from before 2/9, we have no sources reporting data. I understand that Wavefront will hide inactive sources automatically, and I suspect that this is what has happened to our older data associated with this metric. Is there a way to find that data and see the full history, both before and after the tag structure change?

[AbhishekSK · Accepted solution]

Great question nlaughlin! This is a case where having Include Obsolete Metrics turned on would help. If you dive into an individual chart, this option can be found under the General tab:

Metrics/Series that have not reported data in the last 4 weeks are not included in chart results. In this case, the tag change that was made created a new set of series. The series before the tag change was made in February are therefore considered to be more than 4 weeks old and not returned on the chart. Turning on Include Obsolete Metrics can slow down query execution, but it will display the data you are looking for. We suggest only having this feature turned on if you need to see data older than 4 weeks. The following link will show you how it works (and is only valid for users within your organization)...

Link: https://metrics.wavefront.com/u/6wkLN3vmw8

It does look like there is a gap present between 2/8 (~4p PDT) and 2/9 (~6p PDT). Is this expected?

[AbhishekSK]

Jason, great answer! That is exactly what I was looking for. The gap is a regression-triggered monitoring outage, and the response was an indication of how important our Wavefront metrics had become.