-
Notifications
You must be signed in to change notification settings - Fork 25.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
QL: IndexResolver issue when dealing with dotted fields #100869
Labels
Comments
Pinging @elastic/es-ql (Team:QL) |
Pinging @elastic/elasticsearch-esql (:Query Languages/ES|QL) |
This was referenced Oct 14, 2023
costin
added a commit
to costin/elasticsearch
that referenced
this issue
Oct 15, 2023
In certain scenarios, a field can be mapped both as a primitive and object, causing it to be marked as unsupported, losing any potential subfields that might have been discovered before. This commit preserve them to avoid subfields from being incorrectly reported as missing. Fix elastic#100869
This should only be valid for ESQL. SQL hides these fields. |
costin
added a commit
that referenced
this issue
Oct 16, 2023
In certain scenarios, a field can be mapped both as a primitive and object, causing it to be marked as unsupported, losing any potential subfields that might have been discovered before. This commit preserve them to avoid subfields from being incorrectly reported as missing. Fix #100869
costin
added a commit
to costin/elasticsearch
that referenced
this issue
Oct 16, 2023
In certain scenarios, a field can be mapped both as a primitive and object, causing it to be marked as unsupported, losing any potential subfields that might have been discovered before. This commit preserve them to avoid subfields from being incorrectly reported as missing. Fix elastic#100869
elasticsearchmachine
pushed a commit
that referenced
this issue
Oct 16, 2023
In certain scenarios, a field can be mapped both as a primitive and object, causing it to be marked as unsupported, losing any potential subfields that might have been discovered before. This commit preserve them to avoid subfields from being incorrectly reported as missing. Fix #100869
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Description
IndexResolver
tries to create a tree structure for field names that contain dots (e.g. ECS) which can conflict with fields in other indices in which case all underlying fields end up being discarded as unsupported.Consider field "file.name" as keyword in "index-1" and "file" as keyword in "index-2". This results in the following mapping from field caps:
Notice that
file
is mapped both as anobject
inindex-1
andkeyword
inindex-2
. IndexResolver sees the conflict and thus discards as unsupported all the fields under it, includingfile.name
hence why the ESQL querywhere file.name is not null
fails.When a projection is added
where file.name is not null | keep file.name
, IndexResolver doesn't ask the mapping information forfile
which removes the conflict and thusfile.name
can be resolved.The main fix here is to NOT discard fields under an object leaf as unsupported if they have no conflict. In this example
file.name
is mapped as keyword in one index and unmapped in the other so across all indices it will have the same type.Furthermore listing the field as missing instead of say unsupported is incorrect.
The text was updated successfully, but these errors were encountered: