Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new exit code for failure of app's exit code detection #812

Closed
Tracked by #9136
premun opened this issue Feb 2, 2022 · 0 comments · Fixed by #813
Closed
Tracked by #9136

Add a new exit code for failure of app's exit code detection #812

premun opened this issue Feb 2, 2022 · 0 comments · Fixed by #813
Assignees
Labels
apple iOS/tvOS/WatchOS/Mac Catalyst area bug enhancement New feature or request

Comments

@premun
Copy link
Member

premun commented Feb 2, 2022

Context

On all Apple platforms (iOS, tvOS, MacCatalyst), we have problems detecting app's exit code. The original (Xamarin) way of testing these platforms is always via the XHarness TestRunner that is included in the apps.

For all scenarios where we don't have the test runner (HelloiOS, functional tests), where we only fire up the app and wait for it to quit, XHarness is scanning MacOS syslogs for the event where the app terminated with non-zero code and parse the exit code from there. Unfortunately, this is not 100% reliable and sometimes we just don't see this line in the logs. We have no power over flushing of this log and there might be other reasons like MacOS cycling the log away when it reaches some size and starting a new one.

We have also tried to resolve this via DevWF test retries but those won't help as we have no failed tests.

More context: dotnet/runtime#64452

Goal

Since many tests rely on detection of a previously agreed non-zero exit code that the app should return in case of success, we shouldn't treat it as GENERAL_FAILURE when we fail to detect the exit code from logs but we should add a new exit code NO_EXIT_CODE that we can then retry and get a successful run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
apple iOS/tvOS/WatchOS/Mac Catalyst area bug enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant