Add Lightning struct to replace loader
#427
Conversation
generate-dm.sh
Outdated
| @@ -3,7 +3,7 @@ | |||
| cd dm/proto | |||
|
|
|||
| echo "generate dm protobuf code..." | |||
| GOGO_ROOT=${GOPATH}/src/github.com/gogo/protobuf | |||
| GOGO_ROOT=/Users/imtbkcat/go/src/github.com/gogo/protobuf | |||
There was a problem hiding this comment.
I use gvm to manage local go version, so GOPATH can not accessed by this shell script, I will fix this right now. 😂
imtbkcat
force-pushed
the
AddLightning
branch
from
f908c76
to
e615a2d
Compare
imtbkcat
added
priority/normal
| l.logCtx.L().Error("create MyDumperLoader failed", log.ShortError(err)) | ||
| pr <- pb.ProcessResult{ | ||
| Errors: []*pb.ProcessError{unit.NewProcessError(pb.ErrorType_UnknownError, err)}, | ||
| } |
There was a problem hiding this comment.
The code would be simpler if we could extract a function to log and send the error?
|
/run-all-tests tidb=release-3.0 |
Codecov Report
@@ Coverage Diff @@
## master pingcap/dm#427 +/- ##
================================================
+ Coverage 56.2905% 57.4794% +1.1888%
================================================
Files 166 165 -1
Lines 17248 16893 -355
================================================
+ Hits 9709 9710 +1
+ Misses 6585 6230 -355
+ Partials 954 953 -1 |
| @@ -0,0 +1,467 @@ | |||
| <?xml version="1.0" encoding="UTF-8"?> | |||
There was a problem hiding this comment.
Do not commit this file to the repo?
| @@ -330,6 +330,7 @@ enum UnitType { | |||
| Dump = 2; | |||
| Load = 3; | |||
| Sync = 4; | |||
| Lightning = 5; | |||
There was a problem hiding this comment.
Which one do you think is better:
- Add a new
Lightningunit. - Replace the implementation of
Loadunit with lightning.
| lightningCf.App.TableConcurrency = cfg.PoolSize | ||
|
|
||
| // Set checkpoint config. | ||
| lightningCf.Checkpoint.Driver = lightningConfig.CheckpointDriverMySQL |
There was a problem hiding this comment.
How about adjusting more config items like Checkpoint.Schema (check all items one by one)?
|
imtbkcat seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
|
@imtbkcat: PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
What problem does this PR solve?
fix pingcap/tiflow#5833
What is changed and how it works?
Add
Lightningstruct implementUnitinterface.Implement
InitandProcessmethod.Check List
Tests
Code changes
Side effects
Related changes