[DSIP-67] Use command to trigger workflow instance rather generate workflow instance

[ruanwenjun]

Search before asking

• I had searched in the DSIP and found no similar DSIP.

Motivation

Right now once we trigger or backfill a workflow, the logic will be like below

1. We will send a request to the API server
2. The API server will generate a trigger command to the DB
3. The Master will fetch the command from DB and fire them
4. If fire success, then will generate a workflow instance, if fire failed will move the command to t_ds_error_command.

The workflow instance generation is async, which means once we trigger a workflow, we don't know whether the trigger is validated, maybe it will generate a workflow instance, or maybe not. It's hard for third-party integration.

Design Detail

To solve this problem, we should make workflow instance generation sync.

This will change the workflow instance lifecycle, once we trigger a workflow, the API will call master to generate a workflow instance and generate a trigger command.

The newly generated workflow instance state is SUBMITTED.

Once the command is fired, then the workflow instance state will be RUNNING, so all commands no matter the command type, are used to fire a workflow instance.

And once we call the API to trigger a workflow, we can get the workflow instance id from the response.

Compatibility, Deprecation, and Migration Plan

Compatibility with current logic.

Test Plan

Test with E2E/IT

Code of Conduct

• I agree to follow this project's Code of Conduct