Implement GroupsAccumulator for array_agg aggregation function

[lkt]

Is your feature request related to a problem or challenge?

Based on the article Aggregating Millions of Groups Fast in Apache Arrow DataFusion it's possible to optimise aggregation functions using GroupsAccumulator.

Describe the solution you'd like

Implement GroupsAccumulator for array_agg aggregation function

Describe alternatives you've considered

No response

Additional context

No response

[alamb]

Also related is the memory use in

• Fix array_agg memory over use #16346

[vegarsti]

take

[alamb]

take

I believe @duongcongtoai may be working on this too -- see #17446 (comment)

It would probably be a good idea to coordinate with him

[vegarsti]

take

I believe @duongcongtoai may be working on this too -- see #17446 (comment)

It would probably be a good idea to coordinate with him

Oh oops, I didn't realize that's what he was doing. Thanks!

[vegarsti]

take

I believe @duongcongtoai may be working on this too -- see #17446 (comment)

It would probably be a good idea to coordinate with him

I can't see a way to unassign myself. Sorry about that. Feel free to unassign.

Are there any others you can think of that are missing GroupsAccumulator? I couldn't find any on the issue tracker.

[vegarsti]

Indeed he is! #17446 (comment) Sorry!

[alamb]

Are there any others you can think of that are missing GroupsAccumulator? I couldn't find any on the issue tracker.

You can look for anything that implements AggregateUDFImpl that doesn't return GroupsAccumulator

https://github.com/search?q=repo%3Aapache%2Fdatafusion+%22impl+AggregateUDFImpl%22&type=code

There are a bunch of them, but given the complexity of implementing and testing GroupsAccumulator I think it would make sense to pick aggregate functions that we think are important to invest in (both code and review). For example, any aggregate functions that show up in benchmarks