Same url (with max time in future) got diffenent data from Prometheus and thanos query

[husoule]

What happened:
I want a query which endtime is a future time. If I play with Prometheus directly, I see the latest result being less then now + lookback-delta. This is as expected。 But via thanos query, it got the latest result data of future 5m.

via thanos query:

Prometheus directly:

What you expected to happen:
get the same data

Anything else we need to know:

Environment:

• OS (e.g. from /etc/os-release): centos7

• Kernel (e.g. uname -a): Linux VM_120_12_centos 3.10.107-1

• Others:
  prometheus version: v2.18.1
  thanos query version: thanos:v0.12.0
  thanos query args:

    - query
    - --log.level=debug
    - --query.auto-downsampling
    - --grpc-address=0.0.0.0:10901
    - --http-address=0.0.0.0:9090
    - --query.partial-response
    - --query.replica-label=prometheus_replica
    - --query.replica-label=rule_replica
  

[bwplotka]

Ack, might be the deduplication (: But well if ask for future - this is what deduplication interpolates ;p

[bwplotka]

How critical this is? Why it would cause issues?

[husoule]

How critical this is? Why it would cause issues?

I have a stress test service platform for many users . Use prometheus+thanos+grafana to collect and display graphs of stress test data. The stress test data is more accurate in terms of time, but the user actually saw the data in the chart that has not yet occurred, which is unacceptable. So I adjusted prometheus' --query.lookback-delta=5s to reduce future data. And prometheus is as expected, but there will still be data for the next 5m after thanos. So in fact, in my scenario, future data is not needed. I don't know how to not get the data of future time via thanos query.

[krasi-georgiev]

Thanos uses the same promql engine so @husoule have you tried to adjust the look-back in the Thanos code as well? If that works you can open a PR to make this configurable the same way as in Prometheus. Of course, if the Thanos maintainers would accept such a change.

btw in the original comment, you should also mention that the look-back in Prometheus is set to 5s instead of the 5m default. I was looking at the images and was quite confused before I realise that the Prometheus server was started with a 5s look-back.

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗