Skip to content

Commit

Permalink
fix(bulk): Fix memory held by b+ tree in reduce phase (#7161) (#7333)
Browse files Browse the repository at this point in the history
Fix memory held by b+ tree even in the reduce phase of bulk loader by bringing
back the change #4738.

(cherry picked from commit cc3dbfa)
  • Loading branch information
ahsanbarkati authored Jan 19, 2021
1 parent 260fc64 commit ff545be
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions xidmap/xidmap.go
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,12 @@ func (m *XidMap) AllocateUid() uint64 {

// Flush must be called if DB is provided to XidMap.
func (m *XidMap) Flush() error {
// While running bulk loader, this method is called at the completion of map phase. After this
// method returns xidmap of bulk loader is made nil. But xidmap still show up in memory profiles
// even during reduce phase. If bulk loader is running on large dataset, this occupies lot of
// memory and causing OOM sometimes. Making shards explicitly nil in this method fixes this.
// TODO: find why xidmap is not getting GCed without below line.
m.shards = nil
if m.writer == nil {
return nil
}
Expand Down

0 comments on commit ff545be

Please sign in to comment.