Xamarin.UITest support fo iOS 13 and Xcode 11 #1129

GeorgeDarakchiev · 2019-10-03T07:14:17Z

I would like to be able to run UI tests on device\simulator running iOS13 (coming with Xcode 11).

tjsomasundaram-zz · 2019-10-03T23:19:14Z

Hi, iOS 13 devices are currently selectable for testing. If you use Xamarin.UITest, you may need to update to Xcode 11 and the latest dev release of Xamarin.UITest. Please let us know if you attempt to use the Test service and are unable to select iOS 13 devices.

Oddj0b · 2019-10-07T07:30:25Z

@GeorgeDarakchiev Just to clarify is this locally or in App Center Test. Our engineers are still working on iOS 13 support for App Center Test as there are some unexpected challenges.

j-hat · 2019-10-09T11:21:45Z

@Oddj0b

Hey, running UI tests on any of the new iOS 13 simulators doesn't work for me since upgrading to Xcode 11.1

I get this error mentioning the iPhone 6 simulator and containing an absolute file path for a developer named Ivan. This same error was talked about here for building a Xamarin.iOS project, however for me this problem only occurs for running UI tests on Xamarin.iOS, building and deploying the app to the new simulators is working fine. The fix they recommended on that thread doesn't fix the UI tests.

Xamarin.UITest.XDB.Exceptions.DeviceAgentException : Failed to launch simulator

ExitCode: 134
	0   CoreFoundation                      0x00007fff4237e2fd __exceptionPreprocess + 256
	1   libobjc.A.dylib                     0x00007fff6ca78a17 objc_exception_throw + 48
	10  FBSimulatorControl                  0x0000000109505457 +[FBSimulator fromSimDevice:configuration:launchdSimProcess:containerApplicationProcess:set:] + 311
	11  FBSimulatorControl                  0x0000000109508aad -[FBSimulatorInflationStrategy inflateSimulators:availableDevices:previouslyIdentifiedContainerApplications:] + 1085
	12  FBSimulatorControl                  0x0000000109508588 -[FBSimulatorInflationStrategy inflateFromDevices:exitingSimulators:] + 1208
	13  FBSimulatorControl                  0x00000001094cd6f1 -[FBSimulatorSet allSimulators] + 145
	14  FBSimulatorControl                  0x00000001094caf8d -[FBSimulatorSet query:] + 173
	15  iOSDeviceManager                    0x00000001092a5e8b +[Simulator withID:] + 246
	16  iOSDeviceManager                    0x000000010929e300 +[Device withID:] + 118
	17  iOSDeviceManager                    0x00000001092a172b +[Command simulatorFromArgs:] + 163
	18  iOSDeviceManager                    0x00000001092adacc +[LaunchSimulatorCommand execute:] + 20
	19  iOSDeviceManager                    0x00000001092a4171 +[CLI process:] + 1111
	2   CoreFoundation                      0x00007fff42399016 +[NSException raise:format:arguments:] + 98
	20  iOSDeviceManager                    0x000000010928f539 main + 121
	21  libdyld.dylib                       0x00007fff6e2463d5 start + 1
	3   Foundation                          0x00007fff44638791 -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 194
	4   FBSimulatorControl                  0x00000001094c561e +[FBSimulatorConfiguration makeDefaultConfiguration] + 558
	5   FBSimulatorControl                  0x00000001094c53a4 __48+[FBSimulatorConfiguration defaultConfiguration]_block_invoke + 36
	6   libdispatch.dylib                   0x00007fff6e1f963d _dispatch_client_callout + 8
	7   libdispatch.dylib                   0x00007fff6e1fad4b _dispatch_once_callout + 20
	8   FBSimulatorControl                  0x00000001094c5358 +[FBSimulatorConfiguration defaultConfiguration] + 136
	9   FBSimulatorControl                  0x000000010950beb0 +[FBSimulatorConfiguration(CoreSimulator) inferSimulatorConfigurationFromDevice:error:] + 704
(
)
*** First throw call stack:
2019-10-09 12:01:54.629 iOSDeviceManager[16737:439713] *** Assertion failure in +[FBSimulatorConfiguration makeDefaultConfiguration], /Users/Ivan/Documents/repos/Xamarin.UITest/FBSimulatorControl/FBSimulatorControl/Configuration/FBSimulatorConfiguration.m:65
2019-10-09 12:01:54.630 iOSDeviceManager[16737:439713] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Could not obtain OS for Default Device 'Model 'iPhone 6''. Available OS Versions []'
libc++abi.dylib: terminating with uncaught exception of type NSException

King-of-Spades · 2019-10-09T17:43:35Z

Version 3.0.4-dev2 has an update for local simulator tests using Xcode 11: https://www.nuget.org/packages/Xamarin.UITest/3.0.4-dev2

It might be worth giving it a shot @j-hat, though in the past, Xcode's minor version bumps (11.0 vs. 11.1, in this case) have sometimes required additional updates to Xamarin.UITest and/or the Test Cloud Agent; because despite being minor version bumps they contain changes which are breaking the test flow somehow.

Oddj0b · 2019-10-14T12:37:18Z

@GeorgeDarakchiev @j-hat does it work on the dev branch?

GeorgeDarakchiev · 2019-10-14T13:30:41Z

@Oddj0b we are running the tests locally and I can confirm that version 3.0.4-dev2 works with Xcode 11.0. and Xcode 11.1 and I am not getting the error @j-hat reported (I was not able to create iPhone 6 with iOS 13 simulator, but I have tested it on iPhone 11, iOS 13.1).

Sadly, after upgrading to 3.0.4-dev2 (from 3.0.0-dev1) I am getting the following exception on Android (running on Windows, not tested on Mac):

"Cannot access a disposed object.\r\nObject name: 'System.Net.Http.StringContent'."
at Xamarin.UITest.Shared.Http.HttpClient.SendData(String endpoint, String method, HttpContent content, ExceptionPolicy exceptionPolicy, Nullable`1 timeOut)
   at Xamarin.UITest.Shared.Http.HttpClient.PostUntilExpectedResultReceived(String endpoint, String arguments, String expectedContent, Int32 retries)
   at Xamarin.UITest.Shared.Android.HttpApplicationStarter.Execute(String intentJson)
   at Xamarin.UITest.Shared.Android.AndroidAppLifeCycle.LaunchApp(String appPackageName, ApkFile testServerApkFile, Int32 testServerPort)
   at Xamarin.UITest.Android.AndroidApp..ctor(IAndroidAppConfiguration appConfiguration, IExecutor executor)
   at Xamarin.UITest.Configuration.AndroidAppConfigurator.StartApp(AppDataMode appDataMode)
   at TestInitialize.TestInitializer.StartApp()

This can be reproduce by starting the app few times (in my measurements the crash occurs around the 12th start of the app, but it is not always the same):

                    for (int i = 0; i < 25; i++)
                    {
                        try
                        {
                            currentApp = appconfig.StartApp();
                        }
                        catch (Exception e)
                        {
                            Thread.Sleep(1000);
                        }
                    }

Here is my appconfig:

 var appconfig = ConfigureApp
                     .Android
                     .ApkFile(apk)
                     .EnableLocalScreenshots()
                     .DeviceSerial(currentDevice.Serial);

P.S. I am sorry for the late response.

gziebold · 2019-10-15T15:47:04Z

Just a FYI...using xcode 11.1 and the newish Version 3.0.4-dev2 I still get:

Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Could not obtain OS for Default Device 'Model 'iPhone 6''. Available OS Versions []'

olenka93 · 2019-10-24T08:44:38Z

Can't start Exit code '134' 0 CoreFoundation 1 libobjc.A.dylib 10 FBSimulatorControl 11 FBSimulatorControl 12 FBSimulatorControl 13 FBSimulatorControl 14 FBSimulatorControl 15 iOSDeviceManager 16 iOSDeviceManager 17 iOSDeviceManager 18 iOSDeviceManager 19 iOSDeviceManager 2 CoreFoundation 20 iOSDeviceManager 21 libdyld.dylib 3 Foundation 4 FBSimulatorControl 5 FBSimulatorControl 6 libdispatch.dylib 7 libdispatch.dylib 8 FBSimulatorControl 9 FBSimulatorControl (
)
*** First throw call stack:
2019-10-23 12:40:05.785 2019-10-23 12:40:05.787 libc++abi.dylib: any test with new version of Xcode 11.1 and Visual Studio for mac 8.3.4, the problem is still reproducible and really critical. DeviceAgent app could not be installed on simulator and tests fail with error
received from '/var/folders/r9/bh9m7hpj3p1_nkj3lr06ly900000gq/T/xdb/DeviceAgent.iOS.Dependencies/4b3724d8104d6f340f7db14b4f047186/bin/iOSDeviceManager launch-simulator -d 7F65F7AC-37C9-45C5-8FCC-6579F03E01A6'
0x00007fff33698cfd __exceptionPreprocess + 256
0x00007fff5dd60a17 objc_exception_throw + 48
0x0000000100b08457 +[FBSimulator fromSimDevice:configuration:launchdSimProcess:containerApplicationProcess:set:] + 311
0x0000000100b0baad -[FBSimulatorInflationStrategy inflateSimulators:availableDevices:previouslyIdentifiedContainerApplications:] + 1085
0x0000000100b0b588 -[FBSimulatorInflationStrategy inflateFromDevices:exitingSimulators:] + 1208
0x0000000100ad06f1 -[FBSimulatorSet allSimulators] + 145
0x0000000100acdf8d -[FBSimulatorSet query:] + 173
0x00000001008a4e7f +[Simulator withID:] + 246
0x000000010089d2f4 +[Device withID:] + 118
0x00000001008a071f +[Command simulatorFromArgs:] + 163
0x00000001008acac0 +[LaunchSimulatorCommand execute:] + 20
0x00000001008a3165 +[CLI process:] + 1111
0x00007fff336b3a1a +[NSException raise:format:arguments:] + 98
0x000000010088e52d main + 121
0x00007fff5f52e3d5 start + 1
0x00007fff35952c85 -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 194
0x0000000100ac861e +[FBSimulatorConfiguration makeDefaultConfiguration] + 558
0x0000000100ac83a4 __48+[FBSimulatorConfiguration defaultConfiguration]_block_invoke + 36
0x00007fff5f4e163d _dispatch_client_callout + 8
0x00007fff5f4e2d4c _dispatch_once_callout + 20
0x0000000100ac8358 +[FBSimulatorConfiguration defaultConfiguration] + 136
0x0000000100b0eeb0 +[FBSimulatorConfiguration(CoreSimulator) inferSimulatorConfigurationFromDevice:error:] + 704
iOSDeviceManager[4716:138712] *** Assertion failure in +[FBSimulatorConfiguration makeDefaultConfiguration], /Users/Ivan/Documents/repos/Xamarin.UITest/FBSimulatorControl/FBSimulatorControl/Configuration/FBSimulatorConfiguration.m:65
iOSDeviceManager[4716:138712] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Could not obtain OS for Default Device 'Model 'iPhone 6''. Available OS Versions []'
terminating with uncaught exception of type NSException

jeromelaban · 2019-10-25T15:30:33Z

Same issue on my end. Visual Studio 8.4 Preview, XCode 11.1 installed, running local tests.

sebastianferrari · 2019-10-31T17:30:09Z

Same issue here too. Any workaround available?

gziebold · 2019-10-31T21:17:55Z

@sebastianferrari my workaround is to have all my UI tests working is to have my xcode at v 11.0

sebastianferrari · 2019-10-31T21:26:40Z

Thank you @gziebold I'll downgrade it to v11.0. As it is now, it works fine on App Center Tests Platform.

jdeff · 2019-11-04T15:52:49Z

@gziebold I was also seeing issues on dev2. After updating to 3.0.4-dev4 everything seems to work for me with Xcode 11.1.

King-of-Spades · 2019-11-04T16:24:11Z

My teammate @ahanag22 has a theory to reproduce this issue, that might line up with the observations of those who can reproduce it vs. can't reproduce it:

I can reproduce the same with our sample. However, it was working fine just before I’ve updated the visual studio version to 8.4 Preview (8.4 build 1936). Previously, it was running fine on the latest stable version.

Can some of you who have reported that the dev version still doesn't work confirm your Visual Studio version? I'm not sure if we've confirmed that it's the VSMac version itself that causes the issue, or if it could be other tooling linked to that version / update channel. (Other common candidates are Mono & Xamarin.iOS, just because they're the most likely to have some type of interaction.)

Oddj0b · 2019-11-14T14:01:35Z

PSA: 3.0.5-dev5 has been released.

https://www.nuget.org/packages/Xamarin.UITest/3.0.4-dev5

Oddj0b · 2019-12-03T14:02:47Z

3.0.4 have been released! 🎆

GeorgeDarakchiev added the feature request New feature request label Oct 3, 2019

tjsomasundaram-zz assigned Oddj0b Oct 3, 2019

nilofer mentioned this issue Oct 8, 2019

Iteration Plan for September 2019 #970

Closed

28 tasks

nilofer mentioned this issue Oct 10, 2019

Iteration Plan for October 2019 #1114

Closed

21 tasks

Oddj0b added the test Related to the App Center Test service label Nov 4, 2019

Oddj0b mentioned this issue Nov 5, 2019

Iteration Plan for November 2019 #1322

Closed

16 tasks

GeorgeDarakchiev mentioned this issue Nov 14, 2019

Unable to start UI Test on ios simulator #274

Closed

Oddj0b closed this as completed Dec 3, 2019

This was referenced Sep 23, 2020

Xamarin UI Test support for Xcode 12 (iOS 14) #1992

Closed

Xamarin UI Test Not Supporting Xcode 12 & iOS 14 #1998

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Xamarin.UITest support fo iOS 13 and Xcode 11 #1129

Xamarin.UITest support fo iOS 13 and Xcode 11 #1129

GeorgeDarakchiev commented Oct 3, 2019

tjsomasundaram-zz commented Oct 3, 2019

Oddj0b commented Oct 7, 2019

j-hat commented Oct 9, 2019 •

edited

Loading

King-of-Spades commented Oct 9, 2019 •

edited

Loading

Oddj0b commented Oct 14, 2019

GeorgeDarakchiev commented Oct 14, 2019

gziebold commented Oct 15, 2019

olenka93 commented Oct 24, 2019 •

edited

Loading

jeromelaban commented Oct 25, 2019 •

edited

Loading

sebastianferrari commented Oct 31, 2019

gziebold commented Oct 31, 2019

sebastianferrari commented Oct 31, 2019

jdeff commented Nov 4, 2019

King-of-Spades commented Nov 4, 2019

Oddj0b commented Nov 14, 2019 •

edited

Loading

Oddj0b commented Dec 3, 2019

Xamarin.UITest support fo iOS 13 and Xcode 11 #1129

Xamarin.UITest support fo iOS 13 and Xcode 11 #1129

Comments

GeorgeDarakchiev commented Oct 3, 2019

tjsomasundaram-zz commented Oct 3, 2019

Oddj0b commented Oct 7, 2019

j-hat commented Oct 9, 2019 • edited Loading

King-of-Spades commented Oct 9, 2019 • edited Loading

Oddj0b commented Oct 14, 2019

GeorgeDarakchiev commented Oct 14, 2019

gziebold commented Oct 15, 2019

olenka93 commented Oct 24, 2019 • edited Loading

jeromelaban commented Oct 25, 2019 • edited Loading

sebastianferrari commented Oct 31, 2019

gziebold commented Oct 31, 2019

sebastianferrari commented Oct 31, 2019

jdeff commented Nov 4, 2019

King-of-Spades commented Nov 4, 2019

Oddj0b commented Nov 14, 2019 • edited Loading

Oddj0b commented Dec 3, 2019

j-hat commented Oct 9, 2019 •

edited

Loading

King-of-Spades commented Oct 9, 2019 •

edited

Loading

olenka93 commented Oct 24, 2019 •

edited

Loading

jeromelaban commented Oct 25, 2019 •

edited

Loading

Oddj0b commented Nov 14, 2019 •

edited

Loading