You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
On all Apple platforms (iOS, tvOS, MacCatalyst), we have problems detecting app's exit code. The original (Xamarin) way of testing these platforms is always via the XHarness TestRunner that is included in the apps.
For all scenarios where we don't have the test runner (HelloiOS, functional tests), where we only fire up the app and wait for it to quit, XHarness is scanning MacOS syslogs for the event where the app terminated with non-zero code and parse the exit code from there. Unfortunately, this is not 100% reliable and sometimes we just don't see this line in the logs. We have no power over flushing of this log and there might be other reasons like MacOS cycling the log away when it reaches some size and starting a new one.
We have also tried to resolve this via DevWF test retries but those won't help as we have no failed tests.
Since many tests rely on detection of a previously agreed non-zero exit code that the app should return in case of success, we shouldn't treat it as GENERAL_FAILURE when we fail to detect the exit code from logs but we should add a new exit code NO_EXIT_CODE that we can then retry and get a successful run.
The text was updated successfully, but these errors were encountered:
Context
On all Apple platforms (iOS, tvOS, MacCatalyst), we have problems detecting app's exit code. The original (Xamarin) way of testing these platforms is always via the XHarness TestRunner that is included in the apps.
For all scenarios where we don't have the test runner (HelloiOS, functional tests), where we only fire up the app and wait for it to quit, XHarness is scanning MacOS syslogs for the event where the app terminated with non-zero code and parse the exit code from there. Unfortunately, this is not 100% reliable and sometimes we just don't see this line in the logs. We have no power over flushing of this log and there might be other reasons like MacOS cycling the log away when it reaches some size and starting a new one.
We have also tried to resolve this via DevWF test retries but those won't help as we have no failed tests.
More context: dotnet/runtime#64452
Goal
Since many tests rely on detection of a previously agreed non-zero exit code that the app should return in case of success, we shouldn't treat it as
GENERAL_FAILURE
when we fail to detect the exit code from logs but we should add a new exit codeNO_EXIT_CODE
that we can then retry and get a successful run.The text was updated successfully, but these errors were encountered: