Push grouping to the fdw level.

[cristianprice]

Hi there,
Thanks for this awesome project. Quick question: is it possible to push the group by clause at the level of the foreign data wrapper? There are situations when i want to push this as low as possible to avoid network traffic by bringing data to postgres .
Thanks,
C

[arturgspb]

Hi! Any updates?

[arturgspb]

@rdunklau ping, please

[rdunklau]

Hello !
In theory it is possible, however I have no idea how to map the API to do that cleanly.
We can keep this issue open in case someone has an idea or proposal, though.

[arturgspb]

Thx, a lot!

[frank-jiao-tre]

@rdunklau Do you have any plans to add this feature ? thanks a lot

[gruuya]

Hey there! First of all, thanks for making Multicorn - we at Splitgraph love it and have found it well suited for our own needs.

In fact, we went a step further and just implemented aggregation/grouping pushdown capability into our Multicorn fork. This capability is quite versatile, and basically allows for combinations of aggregation functions, GROUP BY clauses and qual (WHERE) clauses to be sent to the remote data source (if it supports those constructs of course). The relevant PRs are splitgraph#1 and splitgraph#2, while the practical FDW example that utilizes this feature (and only one so far) is our fork of Elastic search FDW. You can also take a look at our suite of end-to-end tests that exercises the full range of these capabilities.

Basically, our implementation closely follows the standard FDW approach for handling these things, best illustrated by postgres_fdw extension itself.

That said, our fork has diverged from the upstream significantly, such that contributing these changes back seems quite laborious. Nonetheless, we figured drawing attention to this can't hurt, as perhaps someone may find it useful. :)

Cheers!

[df7cb]

Hi @gruuya, were you able to fix the problems with the recent PG major versions in that fork?

[gruuya]

Hello there @df7cb! We are (for now) focused on PG 12 exclusively, so we haven't dealt with any problems in the later PG versions.