Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Schema]Update Observability Indices with Fields sorting #73

Open
YANG-DB opened this issue Aug 17, 2023 · 0 comments
Open

[Schema]Update Observability Indices with Fields sorting #73

YANG-DB opened this issue Aug 17, 2023 · 0 comments
Labels
schema schema related issue

Comments

@YANG-DB
Copy link
Member

YANG-DB commented Aug 17, 2023

Which domain protocol is relevant for this schema ?

Add support for index template predetermined sort fields:

When creating a new index in Opensearch it is possible to configure how the Segments inside each Shard will be sorted. By > default Lucene does not apply any sort. The index.sort.* settings define which fields should be used to sort the documents inside each Segment.

For example in the trace.mapping use the following sort fields:

  "settings": {
    "index": {
      "sort.field": [ "@timestamp", "traceId" ], 
      "sort.order": [ "asc", "asc" ]       
    }
  }

What is the schema resource ?

  • Traces
  • Metrics
  • Services
  • Logs

Describe the schema resource you are interested in update or create and attach a link to the schema resource

Source Schema - Add necessary repository


Do you have any additional context?

Index sorting can be useful in order to organize Lucene doc ids (not to be conflated with _id) in a way that makes conjunctions (a AND b AND …​) more efficient.
This only works with low-cardinality fields, you should sort first on fields that both have a low cardinality and are frequently used for filtering. The sort order actually doesn't matter as it only care about putting values that would match the same clauses close to each other.

@YANG-DB YANG-DB added untriaged schema schema related issue labels Aug 17, 2023
@YANG-DB YANG-DB removed the untriaged label Sep 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
schema schema related issue
Projects
Status: No status
Development

No branches or pull requests

1 participant