Tanzu Observability

Hi there!

I try to create a chart that should show percentage of several events. So I use such metrics:

sum(ts(my.*.metric.1.count, source=my-source)) / sum(ts(my.*.metric.*.count, source=my-source))*100

sum(ts(my.*.metric.2.count, source=my-source)) / sum(ts(my.*.metric.*.count, source=my-source))*100

sum(ts(my.*.metric.3.count, source=my-source)) / sum(ts(my.*.metric.*.count, source=my-source))*100

But I have the notification like this:

• The expression: "ts(my.*.metric.*.count, source=my-source)" has been pre-aligned, making it equivalent to "align(120s, mean, ts(my.*.metric.*.count, source=my-source))" in order to improve the performance of the sum() aggregation operation. You can wrap the expression with align() to explicitly state the periodicity and desired summarization strategy.

And my chart show some magic numbers.

Also I have similar chart with percentage and similar metrics:

sum(ts(my.metric.1.count, source=my-source)) / sum(ts(my.metric.*.count, source=my-source))*100

sum(ts(my.metric.2.count, source=my-source)) / sum(ts(my.metric.*.count, source=my-source))*100

sum(ts(my.metric.3.count, source=my-source)) / sum(ts(my.metric.*.count, source=my-source))*100

So this one looks great.

So the question is: What is wrong with chart using pre-aligned metrics? Does it try to summarize too many metrics? How should I fix it?

Hi Mikhail,

The metrics are automatically pre-aligned for performance reasons when more than 100 time series are used in an aggregate function (like sum() in your example). In most cases, for instance, where a metric reflects a parameter changing over time, there is absolutely nothing wrong with using pre-aligned metrics - in this case you can safely ignore that warning message and it's not necessarily something that needs to be fixed.

However, for some very specific use cases pre-alignment may cause undesirable side effects. Let's use a hypothetical example - you have a http.requests.count metric, that reports values once a minute and represents the total number of HTTP requests per minute, and you are trying to calculate the number of HTTP requests over the 1-hour sliding window (i.e. msum(1h, rawsum(ts(http.requests.count, source="web*") ). If your data set is pre-aligned like in your example to a 2-minute (120s) time window, using the default "Summarize by" strategy, which is, by default, "Average"; align(120s, mean) will average the values for every 2-minute window, effectively cutting the number of values available to msum() in half - which dramatically changes the result of the calculation. In this case, doing an explicit align with "SUM" as an aggregation method, or changing "Summarize By" to "SUM" for this chart would solve this. We always show this warning message if your data set was automatically pre-aligned so you're aware of the possibility of side effects.

Hope that was helpful - and if you would like us to take a look, please share the short URL to the chart you're working on with us. These URLs are secure and only users who have access to your instance will be able to see them; or you can always send that link to us over email at support@wavefront.com.

-Vasily

Is there a way to get the warning icon to go away? I'm using the pre-alignment, which is fine, but the little warning icon on the dashboard is annoying.