summarize by and align

[AbhishekSK]

Can someone explain the distinction between "Summarize By" and align(..., sum, ...) specifically, how they work together, do you need them both (it appears not), etc.

[AbhishekSK · Accepted solution]

'Summarize By' and align() do share some similarities, but it's the added benefits of align() that differentiates the two. Allow me to explain...

At a basic level both 'Summarize By' and align() group points into a bucket and allow you to specify how those points are grouped together (e.g. summed, counted, averaged). The align() function allows you to specify the desired bucket size whereas 'Summarize By' is completely based on the supported resolution. Supported resolution is based on two key factors: Chart window and dedicated screen pixels for the displayed chart. For example (assuming browser is full-screen), a 10m chart window on a 13" screen vs. a 1-week chart window on a 13" screen vs. a 1-week chart window on a 27" screen will all have different resolutions. The smaller the chart window and more dedicated pixels will result in higher resolution. For that previous example:

10m chart window + 13" screen = Best Probable Resolution (e.g. ~1 second buckets)

1w chart window + 27" screen = Second Best Probable Resolution (e.g. ~30 minute buckets)

1w chart window + 13" screen = Third Best Probably Resolution (e.g. ~60 minute buckets)

Take the '1w chart window + 27" screen' as an example -- 'Summarize By' will group any points reported within the given ~30 minute bucket and will summarize them based on your selection. align() on the other hand allows you to specify the size of the bucket ahead of time. If we kept with the same example, then align() would also give you the option of grouping the points in 45 minute buckets, 2 hour buckets, 1 day buckets, etc. Keep in mind that the supported resolution is the most granular view you can get. So with the previous example, specifying "align(15m,)" will not result in 15-minute buckets being displayed on the screen due to the ~30 minute buckets already associated with the chart. In this case, our system would align the values into the desired 15-minute buckets first, and then take the two aligned values and summarize those based on the 'Summarize By' selection in order to determine the proper value to display.

Now these two functionalities do not necessarily need to be used together. The 'Summarize By' functionality is always in play. Imagine you have a 10m chart with 1-second resolution. If your data is reported once a second, summarization is still occurring within the 1-second bucket, but it simply represents a single value and therefore the summarization should be the explicit value (excluding count). In the majority of cases, align() is only considered when explicitly added to the ts() query. However, "pre-alignment" will occur in circumstances where aggregate functions are being applied to more than 100+ unique series (more about this below). This pre-alignment is tied to a window of time more so than the actual unique series reporting data at any given moment. For example, if 100+ series are reporting data at the same time for 50% of your chart but less than 100 series for the other 50%, then the entire chart window will be pre-aligned if you attempt to aggregate those series into a single series. An additional note to mention when aggregating 100+ time series, the pre-alignment that occurs will include a time window equal to the resolution bucket and the summarize strategy for the pre-alignment will match the 'Summarize By' selected. So if the resolution bucket is ~1800 seconds and sum is selected as the 'Summarize By' strategy, then the pre-alignment that would occur in the case of aggregating 100+ time series would be 1800s buckets that are summed.

As I mentioned at the beginning, there are added benefits to align() that help to differentiate the two:

• ability to specify your buckets with align()
• ability to speed up queries to a degree due to less rendering

The speed at which a query gets executed/displayed is partially based on the amount of data values that need to be displayed on the chart. In a simple example, imagine a 60-minute chart with data values reported every 60-seconds (assume resolution is ~30s). The amount of time it takes to render and display 60 values should be longer than it would take to display 6 values that were aligned at 10-minute buckets. Now in reality displaying 60 data values vs. 6 data values probably does not have any noticeable difference in load time, but this would be the case if we started talking about 1-week or 1-month windows.

The final benefit for align() is tied to aggregate functions as well as the previous point I made about query speed. Imagine you have 10 series typically reported every 60-seconds that you want to average together over a 2-hour window. Now let's say that 1 of those 10 series is being reported intermittently and may only be reported every 120-seconds now and again. Non-Raw aggregate functions will apply an interpolated value if possible for that 1 intermittent series when at least one of the other nine series reported a value. So at 9:03a there were only 9 reported values and therefore avg() will apply an interpolated value for the 10th series when possible before aggregating. Interpolated values requires resources which can affect the speed at which the query is displayed. So imagine if this was the case for 1000+ series where they report every 5-seconds, but not always at the same 5-second interval. This could require a lot of interpolation and slow down the query speed. However, if you align these values in this example to a larger bucket (e.g. align(1m, mean, ts("my.metric")), then the amount of times an interpolated value can occur changes from 12 times a minute without align() (in the best case) to 1 time a minute with align().

I hope this provides you with a better understanding in regards to the similarities and differences of 'Summarize By' and align(). Feel free to ask additional questions in this thread and our team (or other customers) will do our best to provide answers. Thanks!