Unit test TestCleanUpStaleResourcesOnStartUp failed

[amyangfei]

Which jobs are flaking?

dataflow engine unit test

Which test(s) are flaking?

[2022-06-01T06:51:55.555Z] === FAIL: engine/pkg/externalresource/integration_test TestCleanUpStaleResourcesOnStartUp (1.03s)

Jenkins logs or GitHub Actions link

https://ci.pingcap.net/blue/organizations/jenkins/ut-check/detail/ut-check/14465/pipeline/

[2022-06-01T06:51:55.555Z] === Failed

[2022-06-01T06:51:55.555Z] === FAIL: engine/pkg/externalresource/integration_test TestCleanUpStaleResourcesOnStartUp (1.03s)

[2022-06-01T06:51:55.555Z] 

[2022-06-01T06:51:55.555Z] 2022/06/01 14:50:51 /home/jenkins/agent/workspace/ut-check/tiflow/engine/pkg/orm/client.go:599 record not found

[2022-06-01T06:51:55.555Z] [0.226ms] [rows:0] SELECT * FROM `resource_meta` WHERE id = "/local/resource-1" ORDER BY `resource_meta`.`seq_id` LIMIT 1

[2022-06-01T06:51:55.555Z] [2022/06/01 14:50:51.867 +08:00] [INFO] [gc_runner.go:107] ["start gc'ing resource"] [resource="{\"seq-id\":1,\"created-at\":\"2022-06-01T14:50:51.862303733+08:00\",\"updated-at\":\"2022-06-01T14:50:51.865354049+08:00\",\"project-id\":\"\",\"id\":\"/local/resource-1\",\"job\":\"non-existent-job\",\"worker\":\"worker-1\",\"executor\":\"executor-1\",\"gc-pending\":true,\"deleted\":false}"]

[2022-06-01T06:51:55.555Z] 

[2022-06-01T06:51:55.555Z] 2022/06/01 14:50:51 /home/jenkins/agent/workspace/ut-check/tiflow/engine/pkg/orm/client.go:578 database table is locked: resource_meta

[2022-06-01T06:51:55.555Z] [0.600ms] [rows:0] DELETE FROM `resource_meta` WHERE id = "/local/resource-1"

[2022-06-01T06:51:55.555Z] [2022/06/01 14:50:51.875 +08:00] [WARN] [gc_runner.go:133] ["Failed to delete resource after GC"] [resource="{\"seq-id\":1,\"created-at\":\"2022-06-01T14:50:51.862303733+08:00\",\"updated-at\":\"2022-06-01T14:50:51.865354049+08:00\",\"project-id\":\"\",\"id\":\"/local/resource-1\",\"job\":\"non-existent-job\",\"worker\":\"worker-1\",\"executor\":\"executor-1\",\"gc-pending\":true,\"deleted\":false}"]

[2022-06-01T06:51:55.556Z]     gc_test.go:143: 

[2022-06-01T06:51:55.556Z]         	Error Trace:	gc_test.go:143

[2022-06-01T06:51:55.556Z]         	Error:      	Condition never satisfied

[2022-06-01T06:51:55.556Z]         	Test:       	TestCleanUpStaleResourcesOnStartUp

[2022-06-01T06:51:55.556Z] 

[2022-06-01T06:51:55.556Z] DONE 265 tests, 3 skipped, 1 failure in 329.032s

[2022-06-01T06:51:55.556Z] make: *** [engine_unit_test_in_verify_ci] Error 1

script returned exit code 2

Anything else we need to know

• Does this test exist for other branches as well?

• Has there been a high frequency of failure lately?

[maxshuang]

/assign maxshuang