Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MLA-1767] Refactor communicator connection exceptions #4935

Merged
merged 6 commits into from
Feb 11, 2021

Conversation

chriselion
Copy link
Contributor

@chriselion chriselion commented Feb 10, 2021

Proposed change(s)

Previously we were

  • Using exceptions for control flow, since RpcCommunicator.Initialize returns a struct (so it can't return null to indicate normal lack of trainer).
  • Hiding all exceptions, so that actual bugs aren't surfaced. One example of this was a SideChannel throwing an exception on the initial message; this would silently stop training and go to inference.

The main change here is to make ICommunicator.Initialize() return a bool. Unexpected exceptions (basically anything but a RpcException with Unavailable status, which is what happens when there's no communicator listening) are logged.

Also, exceptions during SideChannel processing are wrapped in a try/catch, so that a single SideChannel can't bring down the rest of the system.

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

  • Code refactor

Checklist

  • Added tests that prove my fix is effective or that my feature works
  • Updated the changelog (if applicable)

Other comments

@@ -64,8 +64,8 @@ public RpcCommunicator(CommunicatorInitParameters communicatorInitParameters)

internal static bool CheckCommunicationVersionsAreCompatible(
string unityCommunicationVersion,
string pythonApiVersion,
string pythonLibraryVersion)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this since it was unused.

@@ -113,62 +113,75 @@ public UnityRLInitParameters Initialize(CommunicatorInitParameters initParameter
{
RlInitializationOutput = academyParameters
},
out input);

var pythonPackageVersion = initializationInput.RlInitializationInput.PackageVersion;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved this logic outside of the try. It no longer throws (throw new UnityAgentsException("ICommunicator.Initialize() failed.");) and instead returns false.

}
catch
{
var exceptionMessage = "The Communicator was unable to connect. Please make sure the External " +
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this messaging - I'm not sure it was ever useful, since the exception was swallowed by the calling code.

@@ -456,33 +467,35 @@ UnityInputProto Exchange(UnityOutputProto unityOutput)
QuitCommandReceived?.Invoke();
return message.UnityInput;
}
catch (RpcException rpcException)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reorganized this to avoid duplicating some of the logic (e.g. m_IsOpen = false). Not sure if we have a preference; it's not mentioned in the style guide.

@@ -34,9 +35,18 @@ public Guid ChannelId

internal void ProcessMessage(byte[] msg)
{
using (var incomingMsg = new IncomingMessage(msg))
try
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This threw me off when I was working on the Training Analytics side channel. If the SideChannel throws here, we disconnect the trainer. In general, I think we should wrap user code in bubble wrap.

@chriselion
Copy link
Contributor Author

You may want to hide whitespace changes when reviewing this
image

Debug.Log($"Unexpected exception when trying to initialize communication: {ex}");
}

if (!initSuccessful)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not an else after the first if (initSuccessful) {}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's meant to handle both Communicator.Initialize returning false and throwing. But I think you're right, it's cleaner to handle each separately.

@chriselion chriselion merged commit 54fb8f6 into master Feb 11, 2021
@delete-merged-branch delete-merged-branch bot deleted the MLA-1767-connection-exceptions branch February 11, 2021 00:08
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 11, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants