Native metrics do not always get mapped to the correct Spark nodes

[andygrove]

Describe the bug

CometNativeExec builds a tree of CometMetric nodes containing SQLMetrics for the Spark plan operators. This structure is passed into the Native.createPlan method.

During native execution, calls to update_metrics will recurse through the native ExecutionPlan tree and through the CometMetric tree simultaneously and copy native metrics to Spark metrics.

This approach assumes a 1:1 mapping between the Spark plan and the native plan. In some cases, this assumption is not correct and we therefore drop metrics and the Spark UI does not show metrics for some nodes.

One example is for aggregate queries, where the native planner sometimes inserts an additional projection.

Another example is that we insert CopyExec nodes around the inputs to aggregate, sort, and join operators.

Steps to reproduce

No response

Expected behavior

No response

Additional context

No response