Consider allowing setting a timestamp for ConstMetrics.

[beorn7]

With a lot of caveats….

[beorn7]

Use-case: Monitored client cares about an exact timestamp, and the timestamp is close to the time of the scrape (e.g. it is between beginning and end of a scrape, which implies no problem with staleness).

[brian-brazil]

That's a highly advanced use case that should generally be discouraged, as having different timestamps for metrics from the same scrape will cause oddness and may prevent some features from working correctly.

[bits01]

Another use case case is for hardware devices that generate counters with their own time stamps. These are retrieved and pushed to some server that is then scraped by Prometheus. It helps to use the source timestamp to get an accurate rate.
Unlike the comment by @beorn7 above, this timestamp cannot be guaranteed to be between start and end of a scrape, it sounds like that may be a problem.

[ccassar]

@bits01 above is applicable to network devices too. Would I be right in saying, that, without this enhancement, a gateway implemented in go and terminating telemetry streams from network devices, and wishing to persist to prometheus while preserving original timestamps must push via pushgw to do so. It would be great if the telemetry gateway could be scraped directly and network device timestamps could be preserved.

[brian-brazil]

Those are the okay use cases (presuming you aren't talking about SNMP).

However offering any functionality like this before prometheus/prometheus#398 is complete would be reckless, as the vast majority of users looking to use timestamps it turned out that they should either have used normal pull or the pushgateway without timetamps. Had those users used timestamps, they would have run into a number of problems (as the few users that figured out how discovered).

[ccassar]

Indeed. The interesting case is streaming telemetry not SNMP. The context is a network pushing timestamped content. I would like to push the timestamped content into prometheus. We can accommodate collapsing multiple updates in the same series to the latest within a scrape period, but we would like to preserve the original timestamp. We are currently achieving something close using the pushgateway, but that has its own downsides.

[beorn7]

Coming from the discussion about the collectd_exporter, an exporter that does something equivalent to federation, i.e. it exposes regularly updated timestamped metrics from another monitoring system, would be a good example for a use case.

I'll mark this with the v0.10 milestone because the API to create const metrics will be reworked in v0.10, and it is the right time to think about how the timestamp would be passed in. At least I want to avoid a breaking change for this feature.

[brian-brazil]

My stance remains that this shouldn't be exposed before prometheus/prometheus#398 is resolved. We continue to see users abusing and misusing timestamps, which'd only get far worse if the official client libraries actually supported it.

[bits01]

Any progress being made on #398? It has been open for a long time.

[ptone]

I just wanted to voice an additional position on the relevance of this for device telemetry. Prometheus makes a great tool for analyzing IoT device telemetry. It is relatively easy to make a telemetry extractor that is scraped instead of using the push gateway - but the problem is time skew between event time (on the device) and processing time (on the extractor/target) can impact the interpretation of the resulting visualized data (in grafana in this case)

[brian-brazil]

With staleness handling complete and timestamp abuse mitigation in place, I believe it's now okay to expose timestamp support for the handful of advanced use cases that require it.

[beorn7]

Due to public demand, this was fixed for v0.9 after all.