Multi level nested sorting incorrectly using bitset filter cache #31783
Assignees
Labels
Comments
martijnvg
added
>bug
discuss
:Search/Search
|
Pinging @elastic/es-search-aggs |
This was referenced Jul 19, 2018
jimczi
added a commit
that referenced
this issue
Jul 20, 2018
The parent filter for nested sort should always match **all** parents regardless of the child queries. It is used to find the boundaries of a single parent and we use the child query to match all the filters set in the nested tree so there is no need to repeat the nested filters. With this change we ensure that we build bitset filters only to find the root docs (or the docs at the level where the sort applies) that can be reused among queries. Closes #31554 Closes #32130 Closes #31783 Co-authored-by: Dominic Bevacqua <bev@treatwell.com>
jimczi
added a commit
that referenced
this issue
Jul 20, 2018
The parent filter for nested sort should always match **all** parents regardless of the child queries. It is used to find the boundaries of a single parent and we use the child query to match all the filters set in the nested tree so there is no need to repeat the nested filters. With this change we ensure that we build bitset filters only to find the root docs (or the docs at the level where the sort applies) that can be reused among queries. Closes #31554 Closes #32130 Closes #31783 Co-authored-by: Dominic Bevacqua <bev@treatwell.com>
jimczi
added a commit
that referenced
this issue
Jul 20, 2018
The parent filter for nested sort should always match **all** parents regardless of the child queries. It is used to find the boundaries of a single parent and we use the child query to match all the filters set in the nested tree so there is no need to repeat the nested filters. With this change we ensure that we build bitset filters only to find the root docs (or the docs at the level where the sort applies) that can be reused among queries. Closes #31554 Closes #32130 Closes #31783 Co-authored-by: Dominic Bevacqua <bev@treatwell.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
While looking into #31554, I noticed that in case multi level nested sorting is used and levels use a nested filter then the nested field type filter AND specified nested filter by the user are cached in the bitset filter cache.
In the case of #31554 this ended up being cached:
+_type:__nested1 #ToParentBlockJoinQuery (nested1.nested1_keyword:nested1_foo). I would only expect+_type:__nested1to be cached by the bitset filter cache.I think this is not intended and if many different nested sort filter where to be specified in separate search requests then this could cause the bitset filter cache to consume a lot of jvm heap space.
The problem is in
SortBuilderclass line 251 where this happens:context.bitsetFilter(parentQuery)and in the case of multi level sorting
parentQuerycould be set to the nested filter of the parent nested sort definition.I'm not sure how this can be best be fixed. I'm opening this issue to track the problem.
The text was updated successfully, but these errors were encountered: