Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using point in time API instead of client side time conversion #88480

Closed
Tracked by #166068
Dosant opened this issue Jan 15, 2021 · 4 comments
Closed
Tracked by #166068

Using point in time API instead of client side time conversion #88480

Dosant opened this issue Jan 15, 2021 · 4 comments
Labels
Feature:Search Sessions Feature:Search Querying infrastructure in Kibana Icebox impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. loe:small Small Level of Effort Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.

Comments

@Dosant
Copy link
Contributor

Dosant commented Jan 15, 2021

part of #80406

Kibana converts relative time range to absolute on a client and search executed with absolute time range.
This is used for multiple things:

  • Sync time on a dashboard between different panels
  • Generating reports using forceNow query param to set original client's time now [Reporting] remove forceNow hack #110568
  • Caching in vega and + soon lens
  • Restoring search sessions

Currently not all places that communicate with the server convert to absolute time range.
For example,

  • aggs/buckets/date_range.ts is using relative time ranges.
  • We do not convert input in query and filter
  • Conversion isn't centralized, so not really reliable, there are probably other places we don't know about.

This now will likely soon come up in server-side expression.

Instead of addressing all places where we don't do the conversion, we could explore using point in time API: https://www.elastic.co/guide/en/elasticsearch/reference/master/point-in-time-api.html instead

cc @ppisljar

@Dosant Dosant added Feature:Search Querying infrastructure in Kibana Team:AppServices labels Jan 15, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app-services (Team:AppServices)

@exalate-issue-sync exalate-issue-sync bot added impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. loe:small Small Level of Effort labels Jun 2, 2021
@tsullivan
Copy link
Member

Generating reports using forceNow query param to set original client's time

cc @elastic/kibana-reporting-services

@lukasolson
Copy link
Member

Following up on our discussion this morning, I'm not sure PIT will solve this particular use case. For example, follow these steps:

  1. Index a couple of documents into an index (one before now & one a few minutes from now):
POST /my-index-000001/_doc
{
  "@timestamp": "2021-09-20T22:58:28.820Z
}
POST /my-index-000001/_doc
{
  "@timestamp": "2021-09-20T23:01:28.820Z"
}

Then, create a PIT:

POST /my-index-000001/_pit?keep_alive=10m

Search the PIT using a filter of < now:

POST /_search
{
  "query": {
    "range": {
      "@timestamp": {
        "lt": "now"
      }
    }
  },
  "pit": {
    "id": "your_pid_id"
  }
}

Notice that only one document shows up. Wait until the timestamp of the second document is before "now", then search the PIT again, and you'll see that both documents show up. This shows that now is not relative to when the PIT itself was created.

@exalate-issue-sync exalate-issue-sync bot added loe:medium Medium Level of Effort and removed loe:small Small Level of Effort labels Sep 28, 2021
@exalate-issue-sync exalate-issue-sync bot added loe:small Small Level of Effort and removed loe:medium Medium Level of Effort labels Jan 4, 2022
@petrklapka petrklapka added Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. and removed Team:AppServicesUx labels Nov 21, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Search Sessions Feature:Search Querying infrastructure in Kibana Icebox impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. loe:small Small Level of Effort Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.
Projects
None yet
Development

No branches or pull requests

6 participants