I have an application that pulls down data from Wavefront (using the v2 chart api) at a rate of several unique queries per minute (made in parallel, but mostly evenly spread out rather than done all at once), pulling down about 5 minutes worth of data with each query. Each query results in somewhere between 1-10 time series on average. Occasionally (probably a couple times a week), I find that Wavefront stops returning the expected json containing the timeseries results and/or it takes a long time to return from each query (up to 5 minutes or so, it seems). This usually occurs for several independent queries before things return to normal. This is despite the fact that I've changed nothing at all about the rate at which queries are made or what the content of those queries is. So my questions are: (1) Am I just catching rare times when Wavefront is unable to respond to queries in general? (2) If not, is this due to my usage patterns? (3) If it is due to my usage patterns, is there anything I can do to reduce the chances of this happening?
What cluster are you on? I believe we do throttle at some point on some clusters. Also include the name of your proxy, so we can watch it.