Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhancement(flow): add heartbeat for flow task #2468

Merged
merged 19 commits into from
May 27, 2024

Conversation

yhilmare
Copy link
Contributor

@yhilmare yhilmare commented May 22, 2024

What type of PR is this?

type-enhancement

What this PR does / why we need it:

ticket's status will not be changed when the odc node is crashed and there are flow task instance running on it and the user will find that the ticket will keep 'RUNNING' status without any changes.

I use heartbeat to solve this problem, the ticket will be cancelled if the related heartbeat is timeout

Which issue(s) this PR fixes:

Fixes #2476

Special notes for your reviewer:

Additional documentation e.g., usage docs, etc.:


@yhilmare yhilmare added the enhancement New feature or request label May 22, 2024
@yhilmare yhilmare added this to the ODC 4.3.0 milestone May 22, 2024
@yhilmare yhilmare self-assigned this May 22, 2024
@yhilmare yhilmare changed the title enhancement(flow): add heartbeat timeout for flow task [WIP] enhancement(flow): add heartbeat timeout for flow task May 22, 2024
@yhilmare yhilmare marked this pull request as draft May 22, 2024 09:02
@yhilmare yhilmare changed the title [WIP] enhancement(flow): add heartbeat timeout for flow task [WIP] enhancement(flow): add heartbeat for flow task May 22, 2024
@yhilmare yhilmare changed the title [WIP] enhancement(flow): add heartbeat for flow task enhancement(flow): add heartbeat for flow task May 22, 2024
@yhilmare yhilmare marked this pull request as ready for review May 22, 2024 12:38
@yhilmare yhilmare linked an issue May 22, 2024 that may be closed by this pull request
Copy link
Contributor

@yizhouxw yizhouxw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't see the initial value of lastHeartbeatTime while task started, what is the behavior if lastHeatbeatTime is null ?

@yhilmare
Copy link
Contributor Author

I didn't see the initial value of lastHeartbeatTime while task started, what is the behavior if lastHeatbeatTime is null ?

added, we will update heartbeat before task instance's status is updated to 'EXECUTING'

try {
                flowTaskInstance = getFlowTaskInstance(flowInstanceId, activityId);
                BaseRuntimeFlowableDelegate<?> delegate = getDelegateInstance(flowTaskInstance);
                // we init heartbeat time here
                delegate.updateHeartbeatTime();
                List<Class<? extends ExecutionListener>> list = delegate.getExecutionListenerClasses();
                if (CollectionUtils.isNotEmpty(list)) {
                    // we update flow task instance's status here, 'CREATED' -> 'EXECUTING'
                    list.forEach(c -> doCallListener(FlowConstants.EXECUTION_START_EVENT_NAME, executionFacade, c));
                }
                delegate.execute(executionFacade);
                ...
            } catch (Exception e) {
                ...
            } finally {
                ...
            }

@yhilmare
Copy link
Contributor Author

yhilmare commented May 23, 2024

I tested the performance with this data size:
截屏2024-05-23 15 00 00

SQL duration Return rows
select * from flow_instance_node_task where status='EXECUTING' 746.16ms 10w
select * from task_task where id in (?) and last_heartbeat_time < ? 703ms 5233
select * from flow_instance_node_task where status='EXECUTING' and id in (?) for update 90.89ms 2000
update flow_instance_node_task set status='CANCELLED' where id in (?) 51.85ms 2000
update flow_instance set status='CANCELLED' where id in (?) 9.67ms 130

Copy link
Contributor

@yizhouxw yizhouxw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Contributor

@smallsheeeep smallsheeeep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@MarkPotato777 MarkPotato777 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@krihy krihy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yhilmare yhilmare merged commit 042bf99 into dev/4.3.0 May 27, 2024
7 checks passed
@yhilmare yhilmare deleted the feat/shanlu_add_heartbeat_for_flow_task branch May 27, 2024 03:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

[Bug]: ticket's status is illegal when odc node is crashed
5 participants