mission: "no longer require graphite-web"

[Dieterbe]

We've made good strides towards a more complete render api, and even when an instance only uses functions that MT has, and doesn't require any graphite proxying, we still pay some subtle costs.

Here's all the downsides to our current approach:

• resources needed (reserved) for graphite pods (which need to be reserved in case we need graphite api processing) (for most of our instances: graphite consumes 4~12% the memory the mt-read cluster does but note we currently have temporarily disabled built-in processing for most instances, we should re-enable that first)
• infrastructure/deployment complexity
• slower performance
• Graphite-web tends to return 5xx responses sometimes and it's harder / more time consuming to troubleshoot and fix than in the case where we only have MT and not graphite-web.
• means some features we want to push into MT, also need to be synchronized with a corresponding change in the graphite project, they are generally open to improvements, and having @DanCech as a co-maintainer helps, but it's still extra work
  (examples: jaeger tracing, indication of returned data is rollup data #1091, support for multiple rollups, meta tags)
• we have tech debt in the graphite-mt image and its deployment (e.g. we have no server-status monitoring, we may be affected by https://bz.apache.org/bugzilla/show_bug.cgi?id=53555 but not sure, graphite doesn't emit jaeger spans, ...)

I would like to transition from:

• "MT+graphite proxy enabled to cover for the lag between functions coming into graphite vs when they're introduced into MT"
  to
• "become 100% in-sync with where graphite is now, or was not too long ago, so we can get rid of graphite-web, and just work on keeping the lag of functions being introduced between graphite MT reasonable"

I suppose a downside of this approach is that it removes some of the incentive to contribute useful fixes and features to graphite upstream, but we don't really have much time for that currently (until we hire more) anyway.

[Dieterbe]

hitlist of next set of functions to implement:

aggregate
groupByNode
groupByNodes
aliasByMetric
aliasByTags
aliasQuery # this is probably a hard one. can leave for later
group
maximumAbove
removeEmptySeries
sumSeriesWithWildcards
minimumBelow
maximumBelow
smartSummarize
linearRegression
averageSeriesWithWildcards
averageAbove
percentileOfSeries
nPercentile
applyByNode
cumulative
integral
interpolate
absolute
currentAbove
fallbackSeries
mapSeries
reduceSeries

PS: timeShift(), offset() and moving* can't be done yet until #926 is fixed, i plan to fix that issue in the next couple of weeks

[shalstea]

As part of this effort maybe you can fix #1135

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[robert-milan]

re-opening issue

[robert-milan]

Will reevaluate in a few months.