support measure_time

[mikehale]

In the case of pulling metrics from cloudwatch AWS provides the measure time for metrics. To use l2met for publishing cloudwatch metrics we would need a way to provide the measure_time in the event instead of using the time at which the metric was logged or received by l2met.

[mikehale]

Allowing the time a sample was taken to be specified to l2met would not only help with accuracy for current measurements, but would also allow historical measurements to be piped through l2met to fill out the history. In the case of cloudwatch the last 7-14 days of data are typically available.

[ryandotsmith]

@mikehale Don't you feel like it is wrong that we have a timestamp in the header and the body of the syslog packet?

[mikehale]

@ryandotsmith hmm I forgot about the timestamp in the syslog header. Does l2met take that timestamp into consideration when determining the time to record the measurement? If so I think the intent behind this issue is satisfied and it can be closed.

[ryandotsmith]

L2met definitely uses the timestamp in the header to route the measurement to the correct bucket. However, l2met drops messages that are "late" to the receiver.

https://github.com/ryandotsmith/l2met/blob/master/receiver/receiver.go#L101-108

[mikehale]

Not being an expert on how the receiver stuff works... what are the effective limits on wanting to backfill sample data? Are buckets even required for sample data?

[ryandotsmith]

l2met doesn't handle backfilling data well at all. The period in which you can submit data is dependent on the resolution of your drain. For instance, if you are using a resolution of 60s you can submit syslog packets where the timestamps are within the current 60s of the server's time.

[nextmat]

FWIW, if you are using the Librato outlet and want CloudWatch metrics, the CloudWatch integration may provide a simpler path. I understand this doesn't help if you're using a different outlet though.