Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(lsBackup): Fix profiler conf in lsBackup #7729

Merged
merged 1 commit into from
Apr 15, 2021
Merged

fix(lsBackup): Fix profiler conf in lsBackup #7729

merged 1 commit into from
Apr 15, 2021

Conversation

ahsanbarkati
Copy link
Contributor

@ahsanbarkati ahsanbarkati commented Apr 15, 2021 •
edited by manishrjain
Loading

Before lsBackup was using wrong conf for starting profiling. This PR fixes it.

This change is Reviewable

@github-actions github-actions bot added the area/enterprise Related to proprietary features label Apr 15, 2021
@ahsanbarkati ahsanbarkati merged commit 3e03bf5 into master Apr 15, 2021
@ahsanbarkati ahsanbarkati deleted the ahsan/lsbackup branch April 15, 2021 15:30
ahsanbarkati added a commit that referenced this pull request Apr 23, 2021
@ahsanbarkati
fix(lsbackup): Fix profiler in lsBackup (#7729)
f79bcf0 
Make lsbackup use the right conf for profiler.

(cherry picked from commit 3e03bf5)
dshekhar95 pushed a commit that referenced this pull request Nov 14, 2022
@ahsanbarkati @dshekhar95
fix(lsbackup): Fix profiler in lsBackup (#7729)
5f1f4ce 
Make lsbackup use the right conf for profiler.
@joshua-goldstein joshua-goldstein mentioned this pull request Nov 29, 2022
Merged
dshekhar95 added a commit that referenced this pull request Dec 1, 2022
@dshekhar95 @ahsanbarkati
Fix(lsbackup): Fix profiler in lsBackup (#7729) (#8432)
5ac56a1 
cherry pick from #7729

## Problem
wrong config set for profiler (set to Restore instead of Backup)

## Solution
set profiler to Backup.conf


Co-authored-by: Ahsan Barkati <ahsanbarkati@gmail.com>
mangalaman93 pushed a commit that referenced this pull request Dec 13, 2022
@ahsanbarkati @mangalaman93
cherry-pick PR #7753
f0932d6 
fix(lsbackup): Fix profiler in lsBackup (#7729)
Bring back "perf(Backup): Improve backup performance (#7601)"
Opt(Backup): Make backups faster (#7680)
Fix s3 backup copy (#7669)
[BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
Perf(restore): Implement map-reduce based restore (#7664)
feat(backup): Merge backup refactoring
Revert "perf(Backup): Improve backup performance (#7601)"
@mangalaman93 mangalaman93 mentioned this pull request Dec 13, 2022
Merged
mangalaman93 pushed a commit that referenced this pull request Dec 14, 2022
@ahsanbarkati @mangalaman93
cherry-pick PR #7753
2a123f2 
fix(lsbackup): Fix profiler in lsBackup (#7729)
Bring back "perf(Backup): Improve backup performance (#7601)"
Opt(Backup): Make backups faster (#7680)
Fix s3 backup copy (#7669)
[BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
Perf(restore): Implement map-reduce based restore (#7664)
feat(backup): Merge backup refactoring
Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Dec 28, 2022
@ahsanbarkati @mangalaman93
cherry-pick PR #7753
9ad48cf 
This commit is a major rewrite of online restore code. It used
to use KVLoader in badger. Now it instead uses StreamWriter
that is much faster for writes in the case of restore.

following commits are cherry-picked (in reverse order):
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Dec 29, 2022
@ahsanbarkati @mangalaman93
cherry-pick PR #7753
543d4d9 
This commit is a major rewrite of online restore code. It used
to use KVLoader in badger. Now it instead uses StreamWriter
that is much faster for writes in the case of restore.

following commits are cherry-picked (in reverse order):
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 2, 2023
@ahsanbarkati @mangalaman93
cherry-pick PR #7753
1335343 
This commit is a major rewrite of online restore code. It used
to use KVLoader in badger. Now it instead uses StreamWriter
that is much faster for writes in the case of restore.

following commits are cherry-picked (in reverse order):
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix: fixing graphql schema update when the data is restored + skipping /probe/graphql from audit (#7925)
 * Don't ban namespace in export_backup
mangalaman93 pushed a commit that referenced this pull request Jan 2, 2023
@ahsanbarkati @mangalaman93
cherry-pick PR #7753
65d6273 
This commit is a major rewrite of online restore code. It used
to use KVLoader in badger. Now it instead uses StreamWriter
that is much faster for writes in the case of restore.

following commits are cherry-picked (in reverse order):
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix: fixing graphql schema update when the data is restored + skipping /probe/graphql from audit (#7925)
 * Don't ban namespace in export_backup
mangalaman93 pushed a commit that referenced this pull request Jan 2, 2023
@ahsanbarkati @mangalaman93
cherry-pick PR #7753
b8b51d2 
This commit is a major rewrite of online restore code. It used
to use KVLoader in badger. Now it instead uses StreamWriter
that is much faster for writes in the case of restore.

following commits are cherry-picked (in reverse order):
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix: fixing graphql schema update when the data is restored + skipping /probe/graphql from audit (#7925)
 * Don't ban namespace in export_backup
mangalaman93 pushed a commit that referenced this pull request Jan 3, 2023
@ahsanbarkati @mangalaman93
use StreamWriter to write to badger during backup/restore
af7ed17 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * Don't ban namespace in export_backup
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 3, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
27861d7 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * Don't ban namespace in export_backup
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 3, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
c51a385 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * Don't ban namespace in export_backup
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 4, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
0c75dae 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
 * fix(restore): return nil if there is error (#7899)
        skipping /probe/graphql from audit (#7925)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 4, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
545677f 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
 * fix(restore): return nil if there is error (#7899)
        skipping /probe/graphql from audit (#7925)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 6, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
d41203f 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): return nil if there is error (#7899)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 17, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
9c59b2b 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): return nil if there is error (#7899)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 17, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
2ac2673 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): return nil if there is error (#7899)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 17, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
3307aab 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): return nil if there is error (#7899)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
mangalaman93 pushed a commit that referenced this pull request Jan 18, 2023
@ahsanbarkati @mangalaman93
fix(backup): use StreamWriter to write to badger during backup/restore
dd950f7 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): return nil if there is error (#7899)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
all-seeing-code pushed a commit that referenced this pull request Jan 23, 2023
@ahsanbarkati @all-seeing-code
fix(backup): use StreamWriter to write to badger during backup/restore
9b20a70 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
 * fix(restore): return nil if there is error (#7899)
        skipping /probe/graphql from audit (#7925)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
all-seeing-code pushed a commit that referenced this pull request Jan 23, 2023
@ahsanbarkati @all-seeing-code
fix(backup): use StreamWriter to write to badger during backup/restore
1a6744a 
This commit is a major rewrite of backup and online restore
code. It used to use KVLoader in badger. Now it instead uses
StreamWriter that is much faster for writes.

cherry-pick PR #7753

following commits are cherry-picked (in reverse order):
 * opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984) (#7996)
 * fix(backup): Fix full backup request (#7932) (#7933)
 * fix: fixing graphql schema update when the data is restored +
        skipping /probe/graphql from audit (#7925)
 * fix(restore): return nil if there is error (#7899)
 * Don't ban namespace in export_backup
 * reset the kv.StreamId before sending to stream writer (#7833) (#7837)
 * fix(restore): Bump uid and namespace after restore (#7790) (#7800)
 * fix(ee): GetKeys should return an error (#7713) (#7797)
 * fix(backup): Free the UidPack after use (#7786)
 * fix(export-backup): Fix double free in export backup (#7780) (#7783)
 * fix(lsbackup): Fix profiler in lsBackup (#7729)
 * Bring back "perf(Backup): Improve backup performance (#7601)"
 * Opt(Backup): Make backups faster (#7680)
 * Fix s3 backup copy (#7669)
 * [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
 * Perf(restore): Implement map-reduce based restore (#7664)
 * feat(backup): Merge backup refactoring
 * Revert "perf(Backup): Improve backup performance (#7601)"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NamanJain8 NamanJain8 NamanJain8 approved these changes

@manishrjain manishrjain Awaiting requested review from manishrjain

@vvbalaji-dgraph vvbalaji-dgraph Awaiting requested review from vvbalaji-dgraph

Assignees
No one assigned
Labels
area/enterprise Related to proprietary features
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ahsanbarkati @NamanJain8