Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ESQL: grouping by just a keyword ignores null values #100109

Closed
costin opened this issue Sep 30, 2023 · 4 comments · Fixed by #100117
Closed

ESQL: grouping by just a keyword ignores null values #100109

costin opened this issue Sep 30, 2023 · 4 comments · Fixed by #100117
Assignees
@dnhatn
Labels
:Analytics/ES|QL AKA ESQL >enhancement >non-issue Team:QL (Deprecated) Meta label for query languages team

Comments

@costin
Copy link
Member

costin commented Sep 30, 2023

Description

Not sure if it's a regression or not but when grouping by just one keyword field, the null group is not returned:

from employees | stats by gender | sort gender

should return [F, M, null] yet only [F, M] are.

It looks like the hashing function (whether ordinal or not) ends up dropping null along the way.
This doesn't happen with multiple keys as long as the keyword is not the first key:

from employees | stats by languages, gender | sort desc

null | null
null | F
null | M
etc...
@costin costin added the :Analytics/ES|QL AKA ESQL label Sep 30, 2023
@elasticsearchmachine elasticsearchmachine added the Team:QL (Deprecated) Meta label for query languages team label Sep 30, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-ql (Team:QL)

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/elasticsearch-esql (:Query Languages/ES|QL)

@dnhatn
Copy link
Member

dnhatn commented Sep 30, 2023

Thanks Costin. I can reproduce the issue, and I think OrdinalsGroupingOperator doesn't handle nulls properly.

This was referenced Sep 30, 2023
Closed
Merged
@dnhatn dnhatn mentioned this issue Oct 1, 2023
Merged
@dnhatn
Copy link
Member

dnhatn commented Oct 1, 2023

I've opened #100117.

dnhatn added a commit that referenced this issue Oct 1, 2023
@dnhatn
Handle nulls in OrdinalsGroupingOperator (#100117)
c6f4616 
This change introduces null handling in the OrdinalsGroupingOperator, 
replacing the current behavior which skips null keys. Ordinals are now
incremented by 1, with 0 being used to represent null ordinals.

Closes #100109
piergm pushed a commit to piergm/elasticsearch that referenced this issue Oct 2, 2023
@dnhatn @piergm
Handle nulls in OrdinalsGroupingOperator (elastic#100117)
290d989 
This change introduces null handling in the OrdinalsGroupingOperator, 
replacing the current behavior which skips null keys. Ordinals are now
incremented by 1, with 0 being used to represent null ordinals.

Closes elastic#100109
jakelandis pushed a commit to jakelandis/elasticsearch that referenced this issue Oct 2, 2023
@dnhatn @jakelandis
Handle nulls in OrdinalsGroupingOperator (elastic#100117)
a63e5c7 
This change introduces null handling in the OrdinalsGroupingOperator, 
replacing the current behavior which skips null keys. Ordinals are now
incremented by 1, with 0 being used to represent null ordinals.

Closes elastic#100109
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dnhatn dnhatn

Labels
:Analytics/ES|QL AKA ESQL >enhancement >non-issue Team:QL (Deprecated) Meta label for query languages team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Handle nulls in OrdinalsGroupingOperator dnhatn/elasticsearch
3 participants
@costin @dnhatn @elasticsearchmachine