Add rename PPL function

[kt-eliatra]

Description

This PR implements the rename PPL function.

Issues Resolved

#509

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[kt-eliatra]

There is one problem, when the rename command is not followed by the fields command,
the results contain duplicate values - from both the original and the renamed column, e.g https://github.com/opensearch-project/opensearch-spark/pull/618/files#diff-cb628f2b7ed981862c709fe9b25149c53b5d7eb2fe87a8bb1bbfce4c702352b1R105-R108.
I'm not sure how to solve this. Any ideas? Or maybe this can be accepted as a known limitation?

[YANG-DB]

There is one problem, when the rename command is not followed by the fields command, the results contain duplicate values - from both the original and the renamed column, e.g https://github.com/opensearch-project/opensearch-spark/pull/618/files#diff-cb628f2b7ed981862c709fe9b25149c53b5d7eb2fe87a8bb1bbfce4c702352b1R105-R108. I'm not sure how to solve this. Any ideas? Or maybe this can be accepted as a known limitation?

please check out the next code that should help remove the double projected fields:

• Context projected fields list
• Usage of this list

[YANG-DB]

please also add README examples of this command usage

[kt-eliatra]

There is one problem, when the rename command is not followed by the fields command, the results contain duplicate values - from both the original and the renamed column, e.g https://github.com/opensearch-project/opensearch-spark/pull/618/files#diff-cb628f2b7ed981862c709fe9b25149c53b5d7eb2fe87a8bb1bbfce4c702352b1R105-R108. I'm not sure how to solve this. Any ideas? Or maybe this can be accepted as a known limitation?

please check out the next code that should help remove the double projected fields:

* [Context projected fields list](https://github.com/opensearch-project/opensearch-spark/blob/88ad15fd208b78c50b061226eb45e9d15f998ce3/ppl-spark-integration/src/main/java/org/opensearch/sql/ppl/CatalystPlanContext.java#L73)

* [Usage of this list](https://github.com/opensearch-project/opensearch-spark/blob/88ad15fd208b78c50b061226eb45e9d15f998ce3/ppl-spark-integration/src/main/java/org/opensearch/sql/ppl/utils/ParseStrategy.java#L43)

drop table if exists test;

create table test (
  name STRING, age INT, email STRING, street_address STRING
);

insert into test values 
("Alice", 30, "alice@example.com", "123 Main St, Seattle"),
("Bob", 55, "bob@test.org", "456 Els St, Portland"),
("Charlie", 65, "charlie@domain.net", "789 Pine St, San Francisco"),
("David", 19, "david@anotherdomain.com", "101 Maple St, New York");

source=test | parse email '.+@(?<email>.+)' | fields email;

returns

[AMBIGUOUS_REFERENCE] Reference `email` is ambiguous, could be: [`email`, `spark_catalog`.`default`.`test`.`email`].

Documentation says that parse command can override existing field, at least in the "main sql plugin" https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/cmd/parse.rst#example-2-override-an-existing-field

[YANG-DB]

@kt-eliatra thanks - this looks like an issue in the parse command ?
in addition can you plz add README examples of this command usage

[salyh]

@kt-eliatra thanks - this looks like an issue in the parse command ? in addition can you plz add README examples of this command usage

We opened this issue regarding the parse command: #650 (thx @kt-eliatra)

But our assumption is that replacing a existing column with new computed column with same name is impossible to do in spark.

[YANG-DB]

Some CI test are failing - plz review
Please add usage example in the readme.md doc

[kt-eliatra]

Some CI test are failing - plz review Please add usage example in the readme.md doc

it should be fine now

[YANG-DB]

Can u plz resolve the latest conflict ?
thanks

[kt-eliatra]

Can u plz resolve the latest conflict ? thanks

done

[YANG-DB]

@kt-eliatra sorry agains - plz resolve latest conflict...

[kt-eliatra]

@kt-eliatra sorry agains - plz resolve latest conflict...

np, done :)

[LantaoJin]

OH, there is a PR for rename, I opened a duplicated issue #697. I am starting to working on the implementation, but I am okey you have already started this. I will stop mine.
However, the rename is not just an alias of field, make sure the source field never be accessed again.
For example. source=t | rename a as b | fields a should throw exception.

Further, may not need to be done in this PR, the a field after renaming can't be accessed again in any commands:
e.g. source=t | rename a as b | stats sum(a).
And the identity a could be reused without conflicts:
e.g source=t | rename a as b | eval a = c + 1

[YANG-DB]

@kt-eliatra plz fix scalafmtCheck checkstyle

[salyh]

@kt-eliatra plz fix scalafmtCheck checkstyle

done

[LantaoJin]

Could you test the following case:

| source = $testTable | rename age as renamed_age | fields age

Ref #618 (comment)

[salyh]

@lukasz-soszynski-eliatra is on it ...

[lukasz-soszynski-eliatra]

So far, the query source = $testTable | rename age as renamed_age | fields age has worked "correctly"; that is, the field age was present in the query result. I introduced corrections and updated the tests. Currently, the execution of the previously mentioned query causes an error.