Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partially parse source documents to speed up source access #52591

Closed
stu-elastic opened this issue Feb 20, 2020 · 5 comments
Closed

Partially parse source documents to speed up source access #52591

stu-elastic opened this issue Feb 20, 2020 · 5 comments
Labels
:Core/Infra/Scripting Scripting abstractions, Painless, and Mustache >enhancement Team:Core/Infra Meta label for core/infra team

Comments

@stu-elastic
Copy link
Contributor

When source is accessed, it is fully parsed. If a script only uses a single field, most parsing and object construction is wasted.

We could implement a "skip parser" that does not extract values but rather keeps track of parsing context (inside string, inside map, etc) until we hit the desired field, and provide that field to the script.

@stu-elastic stu-elastic added >enhancement :Core/Infra/Scripting Scripting abstractions, Painless, and Mustache labels Feb 20, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (:Core/Infra/Scripting)

@stu-elastic
Copy link
Contributor Author

Here's the approach we're gonna try:

  • Given a list of fields, provide a view of those values (including the ES specific logic of folding multiple instances into an array)
  • Try to avoid as many allocations as possible, do this by using a facade to provide access to those objects, including nested objects

For first deliverable the list of fields are concrete, no wild cards.

@rjernst rjernst added the Team:Core/Infra Meta label for core/infra team label May 4, 2020
@jtibshirani
Copy link
Contributor

@stu-elastic how to you see this relating to #25168? Would it make sense to close one of the two issues?

@stu-elastic
Copy link
Contributor Author

I'll put it up for discussion in painless triage meeting today.

@stu-elastic
Copy link
Contributor Author

Yup, adding a label to that one and putting it on our project board.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Core/Infra/Scripting Scripting abstractions, Painless, and Mustache >enhancement Team:Core/Infra Meta label for core/infra team
Projects
None yet
Development

No branches or pull requests

4 participants