-
Notifications
You must be signed in to change notification settings - Fork 687
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
alter_distributed_table seems to OOM with many tables #6503
Comments
One potential problem could be that we keep def ConvertTable:
foreach_oid(partitionRelationId, partitionList)
attachPartitionCommands = lappend(attachPartitionCommands, attachPartitionCommand); and then keep This is NOT the only problem, but one problem. We should aggressively pfree (or better MemoryContextCreate/Delete) per table conversion |
For mitigation, we considered the following list of steps:
Edit: I got a transient error that vanished on a clean Citus build. I used to have an error message that is not possible to reproduce, so I deleted it. |
…tion and shards (#6726) 2 improvements to prevent memory leaks during altering or undistributing distributed tables with a lot of partitions and shards: 1. Free memory for each call to ConvertTable so that colocated and partition tables at `AlterDistributedTable`, `UndistributeTable`, or `AlterTableSetAccessMethod` will not cause an increase in memory usage, 2. Free memory while executing attach partition commands for each partition table at `AlterDistributedTable` to prevent an increase in memory usage. DESCRIPTION: Fixes memory leak issue during altering distributed table with a lot of partition and shards. Fixes #6503.
…tion and shards (#6726) 2 improvements to prevent memory leaks during altering or undistributing distributed tables with a lot of partitions and shards: 1. Free memory for each call to ConvertTable so that colocated and partition tables at `AlterDistributedTable`, `UndistributeTable`, or `AlterTableSetAccessMethod` will not cause an increase in memory usage, 2. Free memory while executing attach partition commands for each partition table at `AlterDistributedTable` to prevent an increase in memory usage. DESCRIPTION: Fixes memory leak issue during altering distributed table with a lot of partition and shards. Fixes #6503. (cherry picked from commit e2654de)
After some time, this gives OOM
The text was updated successfully, but these errors were encountered: