Cherry-pick commit Support intermediate aggs in Orca plans (#13707)

[leborchuk]

Here I cherry-picked commit Jun 28, 2022 Support intermediate aggs in Orca plans (#13707) greenplum-db/gpdb-archive@5bf9fd5

This commit conflicts with 0a2bc5d Move per-agg and per-trans duplicate finding to the planner. - refactoring made by hlinnaka@

The interesting thing is that the commit is from November 24th, 2020 (which is more than a year older than the CBDB fork date) and is absent in the original gpdb repository. Also, I didn't find a PR where it was committed. It seems for me like a CBDB feature.

So I decided to create separate PR only for that commit

Fixes #ISSUE_Number

What does this PR do?

Type of Change

• Bug fix (non-breaking change)
• New feature (non-breaking change)
• Breaking change (fix or feature with breaking changes)
• Documentation update

Breaking Changes

Test Plan

• Unit tests added/updated
• Integration tests added/updated
• Passed make installcheck
• Passed make -C src/test installcheck-cbdb-parallel

Impact

Performance:

User-facing changes:

Dependencies:

Checklist

• Followed contribution guide
• Added/updated documentation
• Reviewed code for security implications
• Requested review from cloudberry committers

Additional Context

⚠️ To skip CI: Add [skip ci] to your PR title. Only use when necessary! ⚠️

---

[jiaqizho]

need waitting the ic-good-opt-on enable... current CI have not cover the ORCA in ICW tests..

[my-ship-it]

need waitting the ic-good-opt-on enable... current CI have not cover the ORCA in ICW tests..

@edespino Ed, appreciate your help on it.

[edespino]

need waitting the ic-good-opt-on enable... current CI have not cover the ORCA in ICW tests..

@edespino Ed, appreciate your help on it.

I have a branch with the changes. My repos's workflow is running: https://github.com/edespino/cloudberry/actions/runs/12115945979

[edespino]

@my-ship-it & @jiaqizho

My branch has a single commit based on this PR branch that is necessary to enable the ic-good-opt-on test configuration. Please note, it may have diffs that may or may not be related to this PR.

My branch: https://github.com/edespino/cloudberry/tree/refs/heads/CherryPickOrcaFeb2023_3-ic-good-opt-on
My workflow: https://github.com/edespino/cloudberry/actions/runs/12115945979

[leborchuk]

It seems to me that the ic-good-opt-off tests are flaky.

@edespino Ed, appreciate you help. ic-good-opt-on tests from Ed branch are green.

[my-ship-it]

It seems to me that the ic-good-opt-off tests are flaky.

@edespino Ed, appreciate you help. ic-good-opt-on tests from Ed branch are green.

@leborchuk @jiaqizho We need to investigate what flaky and fix them, thanks!

[edespino]

It seems to me that the ic-good-opt-off tests are flaky.

@edespino Ed, appreciate you help. ic-good-opt-on tests from Ed branch are green.

Thanks for flagging this. Since these flaky test failures involve the orca query optimizer behavior, we should first have an engineer familiar with the orca query optimizer review the test issues. Once they can determine whether this is a product issue or an environmental/infrastructure problem, I can assist with any infrastructure-related fixes needed. Please loop me back in after that assessment.

[leborchuk]

I see here only failed

test uao_compaction/stats         ... FAILED      333 ms (diff  123 ms)
test uao_compaction/index_stats   ... FAILED      575 ms (diff  139 ms)
test uao_compaction/index         ... ok          161 ms (diff   59 ms)
test uao_compaction/drop_column   ... FAILED      165 ms (diff  122 ms)

tests in Apache Cloudberry Build / ic-good-opt-off (pull_request) workflow run

I'm sure they will succeed if I run them again. (I cannot do it myself; I do not have the rights to launch the tests one more time.)

I opened issue to fix it - #789

[jiaqizho]

rebase pls. ic-good-opt-on already added in CI

[jiaqizho]

CREATE SEQUENCE complex_seq CACHE 1;
CREATE TABLE complex_ttbl (id INT4 DEFAULT NEXTVAL('complex_seq'), c COMPLEX) DISTRIBUTED BY (c);

INSERT INTO complex_ttbl(c) VALUES (COMPLEX(NEXTVAL('complex_seq'), NEXTVAL('complex_seq')));
INSERT INTO complex_ttbl(c) VALUES (COMPLEX(NEXTVAL('complex_seq'), NEXTVAL('complex_seq')));
INSERT INTO complex_ttbl(c) VALUES (COMPLEX(NEXTVAL('complex_seq'), NEXTVAL('complex_seq')));
INSERT INTO complex_ttbl(c) VALUES (COMPLEX(NEXTVAL('complex_seq'), NEXTVAL('complex_seq')));
INSERT INTO complex_ttbl(c) VALUES (COMPLEX(NEXTVAL('complex_seq'), NEXTVAL('complex_seq')));
INSERT INTO complex_ttbl(c) VALUES (COMPLEX(NEXTVAL('complex_seq'), NEXTVAL('complex_seq')));

INSERT INTO complex_ttbl(c) VALUES (COMPLEX(0, 0));
INSERT INTO complex_ttbl(c) VALUES (COMPLEX(0, -0));
INSERT INTO complex_ttbl(c) VALUES (COMPLEX(-0, 0));
INSERT INTO complex_ttbl(c) VALUES (COMPLEX(-0, -0));
SELECT COUNT(c) = 4 AS tr, COUNT(DISTINCT gp_segment_id) = 1 AS tr 
	FROM complex_ttbl 
	WHERE re(c) = 0 AND im(c) = 0;

hi @leborchuk, check this case pls, i think executor missing some implements..