Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Command reordering in Core #315

Merged
merged 3 commits into from
Jul 25, 2024
Merged

Conversation

cretz
Copy link
Member

@cretz cretz commented Jul 25, 2024

What was changed

  • Update core
  • Have .NET command-reorder behavior only apply if replaying with that flag (i.e. no longer apply and set the flag for new executions)
  • Removed internal-for-test-use-only option to disable .NET reordering
  • Add tests for post-completion commands including replays for pre-.NET-flag, post-.NET-flag, and current (post-Core-flag)
  • Add tests for multi-completion success first including replays for pre-.NET-flag, post-.NET-flag, and current (post-Core-flag)
  • Add tests for multi-completion failure first including replays for pre-.NET-flag, post-.NET-flag, and current (post-Core-flag)

Checklist

  1. Closes [Feature Request] Update core and confirm new post-complete command reordering #312

@cretz cretz requested a review from a team July 25, 2024 15:12
Copy link
Contributor

@dandavison dandavison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I'll look at the tests later when I work on Python.

// This only applies for successful activations that have a completion not at the end,
// don't have completion reordering disabled, and that are either not replaying or have
// the flag set.
// In a previous version of .NET, if this was a successful activation completion with
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: (there might be a few other places in the repo)

Suggested change
// In a previous version of .NET, if this was a successful activation completion with
// In a previous version of .NET SDK, if this was a successful activation completion with

Copy link
Member Author

@cretz cretz Jul 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit pedantic for a code comment, but I understand the ambiguity to the reader. Will change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a reviewer prefixes their comment with "nit", you can't complain it's pedantic :) Universal law of code review!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very true, I definitely missed that prefacing. My bad (and fixed anyways).

//
// Note this only applies for successful activations that don't have completion
// reordering disabled and that are either not replaying or have the flag set.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be worth leaving a comment for any future readers explaining that the code below assumes there is only one completion command. Otherwise having "the completion" in comments could confuse readers, making them forget that there could be multiple.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can change the variable name to lastCompletionCommandIndex if that helps

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure yes, and ideally "the last completion" in comments rather than "the completion" but, that's prefixed by "nit"!

Copy link
Member Author

@cretz cretz Jul 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated var name, yeah "the completion in question" is hopefully implied now

// Now we know the completion is in the wrong spot and we're on a newer SDK, so set the
// SDK flag and move it
// Now we know that we're replaying w/ the flag set and the completion in the wrong
// spot, so set the SDK flag and move it
completion.Successful.UsedInternalFlags.Add((uint)WorkflowLogicFlag.ReorderWorkflowCompletion);
var compCmd = completion.Successful.Commands[completionCommandIndex];
completion.Successful.Commands.RemoveAt(completionCommandIndex);
Copy link
Contributor

@dandavison dandavison Jul 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So just to try to recap this logic for myself and anyone else. It would require a fairly complicated diagram to set it all out, but here's one story:

  • This release of .NET will also be the first .NET release that has the new core command reordering code.

  • So, for example, let C be a completion command and N be a non-completion, and consider a workflow being processed today by a .NET worker on the recently released code. Suppose the raw lang sequence is N C1 N C2 N. The .NET flag is set, so .NET will change that to N C1 N N C2. And core will change that to N C1, since it's old core.

  • Now, if we release .NET workers based on this PR, the same thing will happen on replay, because the .NET flag will be set, and the new core internal flag will not be set, so both .NET and core will do their legacy behavior.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now consider a workflow born after this PR is released. Again the raw commands collected in lang are N C1 N C2 N. lang does not do anything to them, since lang's flag is not set, and core receives them unchanged. Core then does its new behavior, and changes them to N N N C1, and sets core's flag, so that this will also be done on replay.

Copy link
Member Author

@cretz cretz Jul 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, everything in your statements is correct

@cretz cretz merged commit df3b47a into temporalio:main Jul 25, 2024
7 checks passed
@cretz cretz deleted the command-reorder-in-core branch July 25, 2024 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] Update core and confirm new post-complete command reordering
3 participants