In batch mode, pos delete may trigger oom

[linfey90]

Feature Request / Improvement

We use a upsert table, then write data with flink batch mode. Of course there is no ck, the insertedRowMap
in BaseTaskWriter.java will be bigger and bigger,and the resource will not be released, so oom might happen.
method: BaseTaskWriter.write
PathOffset previous = insertedRowMap.put(copiedKey, pathOffset);
@openinx@rdblue
any advice can give me?
thanks!

Query engine

Flink

[singhpk234]

An interesting PR on similar lines (supporting spilling), addressing the issue by @openinx :

• Core: Add RocksDBStructLikeMap #2680

[linfey90]

An interesting PR on similar lines (supporting spilling), addressing the issue by @openinx :

• Core: Add RocksDBStructLikeMap #2680

thanks, I'll take a look.

[github-actions]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.