Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

br: error log optimization #29640

Merged
merged 41 commits into from
Dec 21, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
cc5c8d5
fix issue 27015
fengou1 Sep 27, 2021
65ccd96
Merge branch 'master' into issue_27015
fengou1 Sep 27, 2021
bd531c8
fix comments - add general function isRetryableError
fengou1 Sep 28, 2021
a3b0df0
Merge branch 'issue_27015' of https://github.com/fengou1/tidb into is…
fengou1 Sep 28, 2021
c1ddff9
Merge branch 'master' into issue_27015
fengou1 Sep 28, 2021
0fbd620
remote uncessary package errors
fengou1 Sep 28, 2021
0f1976c
Merge branch 'issue_27015' of https://github.com/fengou1/tidb into is…
fengou1 Sep 28, 2021
6dcdbdb
reused the retry code from lightning
fengou1 Sep 29, 2021
fe147b9
Merge branch 'master' into issue_27015
fengou1 Sep 29, 2021
8dcda02
Merge branch 'master' into issue_27015
fengou1 Oct 7, 2021
e97b450
refactoring retryable
fengou1 Oct 7, 2021
a825bea
Merge branch 'master' into issue_27015
fengou1 Oct 9, 2021
440cdf5
Merge branch 'master' into issue_27015
fengou1 Oct 12, 2021
c53ddc1
Merge branch 'master' into issue_27015
fengou1 Oct 12, 2021
d4004f3
backup failed summary should be quieter
fengou1 Oct 12, 2021
5a9ff74
Merge branch 'issue_27015' of https://github.com/fengou1/tidb into is…
fengou1 Oct 12, 2021
38e08e6
Merge branch 'master' into issue_27015
fengou1 Oct 12, 2021
afe3938
Merge branch 'master' into issue_27015
fengou1 Oct 12, 2021
ac36702
Merge branch 'master' into issue_27015
fengou1 Oct 12, 2021
07d1696
Merge branch 'master' into issue_27015
fengou1 Oct 13, 2021
b7ced92
Merge branch 'master' into issue_27015
fengou1 Oct 14, 2021
a1f292b
optimize error log
fengou1 Nov 10, 2021
b3f50cc
Merge branch 'master' into issue_27015
fengou1 Nov 10, 2021
fbdf057
fixed unit test for dumpling
fengou1 Nov 10, 2021
1a0001c
Merge branch 'master' into issue_27015
fengou1 Nov 15, 2021
c90fd96
Merge branch 'master' into issue_27015
fengou1 Nov 16, 2021
71d1d28
add a retry test
fengou1 Nov 22, 2021
001f048
Merge branch 'master' into issue_27015
fengou1 Nov 22, 2021
e8bcc59
Merge branch 'master' into issue_27015
fengou1 Nov 25, 2021
dbf2006
fix dumpling unit test error
fengou1 Nov 25, 2021
a09918d
Merge branch 'master' into issue_27015
fengou1 Nov 27, 2021
dde30dd
Add new collector
fengou1 Nov 27, 2021
a258380
Merge branch 'issue_27015' of https://github.com/fengou1/tidb into is…
fengou1 Nov 27, 2021
38aa7a0
add unit test for download and import back off
fengou1 Dec 20, 2021
eb7261c
fix pr review comments
fengou1 Dec 20, 2021
098cf73
Merge branch 'master' into issue_27015
fengou1 Dec 21, 2021
743e80e
Merge branch 'master' into issue_27015
ti-chi-bot Dec 21, 2021
f194873
Merge branch 'master' into issue_27015
ti-chi-bot Dec 21, 2021
cbe870c
Merge branch 'master' into issue_27015
ti-chi-bot Dec 21, 2021
b89046e
Merge branch 'master' into issue_27015
ti-chi-bot Dec 21, 2021
fa7ce87
Merge branch 'master' into issue_27015
ti-chi-bot Dec 21, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion br/pkg/backup/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -470,7 +470,13 @@ func (bc *Client) BackupRanges(
elctx := logutil.ContextWithField(ectx, logutil.RedactAny("range-sn", id))
err := bc.BackupRange(elctx, sk, ek, req, metaWriter, progressCallBack)
if err != nil {
return errors.Trace(err)
// The error due to context cancel, stack trace is meaningless, the stack shall be suspended (also clear)
if errors.Cause(err) == context.Canceled {
return errors.SuspendStack(err)
} else {
return errors.Trace(err)
}
fengou1 marked this conversation as resolved.
Show resolved Hide resolved

}
return nil
})
Expand Down
11 changes: 10 additions & 1 deletion br/pkg/summary/collector.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,13 @@
package summary

import (
"context"
"strings"
"sync"
"time"

"github.com/docker/go-units"
berror "github.com/pingcap/errors"
"github.com/pingcap/log"
"go.uber.org/zap"
)
Expand Down Expand Up @@ -188,9 +190,16 @@ func (tc *logCollector) Summary(name string) {
}

if len(tc.failureReasons) != 0 || !tc.successStatus {
var canceledUnits int
for unitName, reason := range tc.failureReasons {
logFields = append(logFields, zap.String("unit-name", unitName), zap.Error(reason))
if berror.Cause(reason) != context.Canceled {
logFields = append(logFields, zap.String("unit-name", unitName), zap.Error(reason))
} else {
canceledUnits++
}
}
// only print total number of cancel unit
log.Info("units canceled", zap.Int("cancel-unit", canceledUnits))
tc.log(name+" failed summary", logFields...)
return
}
Expand Down