DM job master meets unexpected end of JSON input error can't recover

[amyangfei]

What did you do?

• run a DM full mode job on single executor

What did you expect to see?

No response

What did you see instead?

job master keeps meeting following error

[2022/11/22 11:41:55.873 +00:00] [INFO] [task_runner.go:187] ["Task Closed"] [id=41207] [error="unexpected end of JSON input"] [errorVerbose="unexpected end of JSON input\ngithub.com/pingcap/errors.AddStack\n\tgithub.com/pingcap/errors@v0.11.5-0.20220729040631-518f63d66278/errors.go:174\ngithub.com/pingcap/errors.Trace\n\tgithub.com/pingcap/errors@v0.11.5-0.20220729040631-518f63d66278/juju_adaptor.go:15\ngithub.com/pingcap/tiflow/engine/framework.(*DefaultBaseJobMaster).Poll\n\tgithub.com/pingcap/tiflow/engine/framework/base_jobmaster.go:229\ngithub.com/pingcap/tiflow/engine/framework/internal/eventloop.(*Runner[...]).doRun\n\tgithub.com/pingcap/tiflow/engine/framework/internal/eventloop/runner.go:94\ngithub.com/pingcap/tiflow/engine/framework/internal/eventloop.(*Runner[...]).Run\n\tgithub.com/pingcap/tiflow/engine/framework/internal/eventloop/runner.go:57\ngithub.com/pingcap/tiflow/engine/executor/worker.(*TaskRunner).launchTask.func1\n\tgithub.com/pingcap/tiflow/engine/executor/worker/task_runner.go:203\nruntime.goexit\n\truntime/asm_amd64.s:1594"] [runtime-task-count=2]                                                                                                              [2022/11/22 11:41:55.873 +00:00] [INFO] [task_runner.go:204] ["Task stopped"] [id=41207] [error="unexpected end of JSON input"] [errorVerbose="unexpected end of JSON input\ngithub.com/pingcap/errors.AddStack\n\tgithub.com/pingcap/errors@v0.11.5-0.20220729040631-518f63d66278/errors.go:174\ngithub.com/pingcap/errors.Trace\n\tgithub.com/pingcap/errors@v0.11.5-0.20220729040631-518f63d66278/juju_adaptor.go:15\ngithub.com/pingcap/tiflow/engine/framework.(*DefaultBaseJobMaster).Poll\n\tgithub.com/pingcap/tiflow/engine/framework/base_jobmaster.go:229\ngithub.com/pingcap/tiflow/engine/framework/internal/eventloop.(*Runner[...]).doRun\n\tgithub.com/pingcap/tiflow/engine/framework/internal/eventloop/runner.go:94\ngithub.com/pingcap/tiflow/engine/framework/internal/eventloop.(*Runner[...]).Run\n\tgithub.com/pingcap/tiflow/engine/framework/internal/eventloop/runner.go:57\ngithub.com/pingcap/tiflow/engine/executor/worker.(*TaskRunner).launchTask.func1\n\tgithub.com/pingcap/tiflow/engine/executor/worker/task_runner.go:203\nruntime.goexit\n\truntime/asm_amd64.s:1594"]

detail logs:
unexpected_json.log

Versions of the cluster

Dataflow Engine version (run tiflow version):

master@01ac379

[amyangfei]

unexpected end of JSON input was caused by OnWorkerOnline was called before UpdateStatus