Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation of Temporalio.Workflows.Semaphore #287

Merged
merged 1 commit into from
Jun 26, 2024

Conversation

cretz
Copy link
Member

@cretz cretz commented Jun 25, 2024

What was changed

  • Added Temporalio.Workflows.Semaphore deterministic semaphore implementation
  • Added tests for that implementation
  • Added tests to confirm when existing standard library semaphore does/doesn't work
  • Updated README with guidance

Opening as draft due to a bug that is occurring when cancelling the workflow while waiting on this semaphore (see TODO in test).

Checklist

  1. Relates to Workflow-friendly concurrency control #281

@cretz cretz force-pushed the semaphore-tests branch from 63e64c0 to a99fac1 Compare June 26, 2024 12:19
@cretz cretz requested a review from a team June 26, 2024 12:22
@cretz cretz marked this pull request as ready for review June 26, 2024 12:22
@cretz cretz mentioned this pull request Jun 26, 2024
}
completion.Result.Completed.Result =

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add an explicit test to cover this? I guess the semaphore tests cover it as well?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the semaphore test hit this because it was our first Task-no-result activity (as opposed to just void). I can add an explicit test.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrmm, so ExecuteActivityAsync_SimpleAsyncVoidMethod_Succeeds tested this and succeeds in CI but I don't know why, digging...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, there are different routes to creating an activity, and that other test didn't test this route. This semaphore test should be sufficient.

@cretz cretz merged commit 8b7bd2a into temporalio:main Jun 26, 2024
7 checks passed
@cretz cretz deleted the semaphore-tests branch June 26, 2024 16:48
Copy link
Contributor

@dandavison dandavison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all LGTM. Just submitting some minor comments I made while reviewing.

Comment on lines +707 to 709
* _Technically_ `SemaphoreSlim` does work if only the async form of `WaitAsync` is used without no timeouts and
`Release` is used. But anything else can deadlock the workflow and its use is cumbersome since it must be disposed.
* Be wary of additional libraries' implicit use of the default scheduler.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* _Technically_ `SemaphoreSlim` does work if only the async form of `WaitAsync` is used without no timeouts and
`Release` is used. But anything else can deadlock the workflow and its use is cumbersome since it must be disposed.
* Be wary of additional libraries' implicit use of the default scheduler.
* _Technically_ `SemaphoreSlim` does work if only the async form of `WaitAsync` is used without timeouts and
`Release` is used. But any other usage pattern can deadlock the workflow and its use is cumbersome since it must be disposed.
* Be wary of additional libraries' implicit use of the default scheduler.


// Send handle back to check
handleCompletion.SetResult(handle);
// This will never complete
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because the update handlers attempt to execute an activity with a semaphore permit held. Since we are attempting to perform more updates than there are available semaphore permits, this will deadlock.

a comment like that would have saved me time reading the tests.

public async Task ExecuteWorkflowAsync_Semaphore_MultipleWaiters()
{
// This is a basic test of semaphore usage. We will make a semaphore with 2 allowed and 5
// provided and confirm as we complete some that others can go.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice comment, would appreciate one of those on the stdlib semaphore tests.

Assert.DoesNotContain(waiting, waiting1.Contains);
return waiting;
});
Assert.Contains(await handle.QueryAsync(wf => wf.Completed), waiting1.Contains);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion seems to be saying

Assert that at least one of the waiting1 items is now completed.

If so, it seems that we could make it stronger and assert that both of them are completed. Clearer to use set operations?

Copy link
Member Author

@cretz cretz Jun 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems that we could make it stronger and assert that both of them are completed

Yes, I meant this to be Assert.All or Assert.True w/ .All on the query result (set equality doesn't have as simple of assertion tools in .NET). Will set reminder to change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assert.True(completed.Count == 5);
return completed;
});
Assert.Contains(completed.GetRange(0, 2), waiting1.Contains);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above, I think we really want to assert that the set of waiting1 items equals the set of items in GetRange(0, 2), but we're asserting something weaker IIUC.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same deal here, I misunderstood Contains(collection, predicate), the intent was to check all

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants