Revise serialization of entitymessages #972
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jviau
reviewed
Oct 2, 2023
| string serializedEventData; | ||
| if (eventData is EntityMessageEvent entityMessageEvent) | ||
| { | ||
| serializedEventData = entityMessageEvent.AsSerializedString(); // bypass application-defined serializer |
There was a problem hiding this comment.
If you merge main, I have introduced a RawInput object - maybe you can leverage that instead of special casing this type. So instead of passing in EntityMessageEvent, the caller would pre-serialize EntityMessageEvent, construct RawInput from that, and pass into the send event method.
There was a problem hiding this comment.
That might work. I will give it a try.
There was a problem hiding this comment.
RawInput is marked Obsolete on purpose - just suppress the message. It is done that way to have an "I acknowledge" before using it.
…ization-of-entitymessages # Conflicts: # src/DurableTask.Core/TaskOrchestrationContext.cs
jviau
approved these changes
Oct 6, 2023
| /// Returns the content as an already-serialized string. Can be used to bypass the application-defined serializer. | ||
| /// </summary> | ||
| /// <returns></returns> | ||
| public RawInput AsRawInput() |
There was a problem hiding this comment.
Where is AsRawInput() expected to be called? By the dotnet isolated SDK shim layer?
There was a problem hiding this comment.
Yes. This is the corresponding PR: microsoft/durabletask-dotnet#207
sebastianburckhardt
added a commit
that referenced
this pull request
Oct 17, 2023
* Bring entity logic into DurableTask.Core (first milestone) (#887) * implementaton of entity mechanics, compatible with existing DF SDK, but without a user-facing entity SDK for DTFx * address PR feedback. * fix usings and namespaces * address PR feedback * address PR feedback (remove NameObjectManager), fix breaking change in TaskHubWorker, fix some comments * address PR feedback (fix CustomExceptionsTest, remove public property) * add #nullable enable to most new classes * address PR feedback * try to fix compiler errors * add a configuration setting that disables separate dispatch by default * address PR feedback * address PR feedback * fix semantic merge conflict. * Revise entity state and status format and access (#955) * update scheduler state and entity status format and helpers * fix mess-up caused by merge conflict * Revise entity message format and external access (#956) * revise how event messages are represented and used * fix merge anomaly. * make current critical section id publicly visible (#958) * remove orchestration tags from entity action (#952) * Rename OperationBatchRequest and OperationBatchResponse (#953) * rename OperationBatch to EntityBatch * fix accidentally commited change from another PR * Revise how entity batches are executed and handle failures (#954) * revise task entity definition * commit change that had been accidentally committed to a different PR. * Apply suggestions from code review Co-authored-by: David Justo <david.justo.1996@gmail.com> * Apply suggestions from code review Co-authored-by: Jacob Viau <javia@microsoft.com> --------- Co-authored-by: David Justo <david.justo.1996@gmail.com> Co-authored-by: Jacob Viau <javia@microsoft.com> * revise operation result encoding and add more comments. (#965) * revise entity backend properties and implement entity backend queries (#957) * revise entity backend properties and implement entity backend queries. * Minor revisions to querries and properties, and improved comments. * fix validation of which LockNext methods are being called. * improve comments * fix useage of IEntityOrchestrationService. * revise how to exclude entity results from queries. * address PR feedback * Update versions for ADO feed (#973) * Add no-warn for NU5104 (#974) * revise propagation path for entity parameters (#971) * fix propagation path for entity parameters that need to reach the orchestration executor. * address PR feedback. * Revise entity queries (#981) * rename includeDeletedEntities to includeStatelessEntities and add comment explaining the meaning * add backlogQueueSize and lockedBy to entity metadata * fix bugs in tracking store implementation (#979) * add scheduled start time parameter to the start-new-orchestration operation action. (#980) * Revise serialization of entitymessages (#972) * revise how entity messages are serialized when sent by orchestrators. * address PR feedback (use RawInput) * Rename includeStateless to includeTransient in entity queries (#985) * rename includeStateless to includeTransient * rename variable also * Rev to entities-preview.2 (#986) * fix null reference exception when running on older backends (#989) * Prepare for public preview (#994) --------- Co-authored-by: David Justo <david.justo.1996@gmail.com> Co-authored-by: Jacob Viau <javia@microsoft.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Revises how to avoid double serialization, and exposure to application-defined serialization settings, for entity message serialization. This is a challenge because orchestrators that send messages to entities have to do so by calling
SendEvent, which serializes the content using the application-defined serializer.Previously, double serialization was avoided by serializing to
JObject. But that is hacky and does not actually work with STJ.Instead we now revise the
SendEventimplementation in DurableTask.Core directly, so it can detect EntityMessageEvents and use the internally defined serialization mechanism.