[BUG] Rename does not work with nested fields

[lezzago]

What is the bug?
Rename does not work with nested fields. When I tried to rename a nested field, the renamed field would just have empty values. This is an example query that has issues:

source=otel-metrics-2024.05| rename `attributes.resource.attributes.service@name` as ServiceName | dedup ServiceName | fields ServiceName

How can one reproduce the bug?
Make a similar query to the one below that renames a nested field

source=otel-metrics-2024.05| rename `attributes.resource.attributes.service@name` as ServiceName | fields ServiceName

What is the expected behavior?
The values from the renamed field should not be empty

What is your host/environment?

• OS: AWS OpenSearch 2.11

[LantaoJin]

Not sure if it relates to opensearch-project/OpenSearch#14355. @penghuo, how does the RenameOperator work in SQL engine? If the implementation relates to rename field processor, it might be the bug of OpenSearch Core.

[dblock]

[Catch All Triage - Attendees 1, 2, 3, 4, 5]