Skip to content
This repository has been archived by the owner on Aug 8, 2023. It is now read-only.

iOS testing failures around location permissions #2734

Closed
friedbunny opened this issue Oct 22, 2015 · 5 comments
Closed

iOS testing failures around location permissions #2734

friedbunny opened this issue Oct 22, 2015 · 5 comments
Assignees
Labels
bug iOS Mapbox Maps SDK for iOS tests

Comments

@friedbunny
Copy link
Contributor

We had hangs on Travis during tests because the location permission dialog was going untapped (since #2612), which we addressed in #2645. This was tweaked somewhat in #2685 to speed things up (as waiting for the permission dialog takes 10 seconds).

There have been some different failures since #2685:

Test Case '-[MapViewTests testDelegatesStartStopLocatingUser]' started.
2015-10-22 13:47:54.330 Mapbox GL Tests[3963:13037] _BSMachError: (os/kern) invalid capability (20)
2015-10-22 13:47:54.334 Mapbox GL Tests[3963:13093] _BSMachError: (os/kern) invalid name (15)
2015-10-22 13:47:54.385 Mapbox GL Tests[3963:13037] _BSMachError: (os/kern) invalid capability (20)
2015-10-22 13:47:57.477 Mapbox GL Tests[3963:13037] tap point is not within the bounds of the screen
/Users/vagrant/git/test/ios/KIF/Classes/UIAutomationHelper.m:61: error: -[MapViewTests testDelegatesStartStopLocatingUser] : failed: caught "UIAExceptionBadPoint", "tap point is not within the bounds of the screen"
(
    0   CoreFoundation                      0x0000000105e4bf65 __exceptionPreprocess + 165
    1   libobjc.A.dylib                     0x00000001058c3deb objc_exception_throw + 48
    2   CoreFoundation                      0x0000000105e4be9d +[NSException raise:format:] + 205
    3   UIAutomation                        0x000000011c0992a3 -[UIATarget _tapOptionalObject:tapCount:touchCount:tapOffset:] + 466
    4   UIAutomation                        0x000000011c087f4b -[UIAElement tap] + 319
    5   Test Bundle                         0x00000001155d2835 -[UIAutomationHelper acknowledgeSystemAlert] + 203
    6   Test Bundle                         0x00000001155d26e4 +[UIAutomationHelper acknowledgeSystemAlert] + 48
    7   Test Bundle                         0x00000001155b739c -[MapViewTests approveLocationIfNeeded] + 72
    8   Test Bundle                         0x00000001155de186 -[KIFSystemTestActor waitForNotificationName:object:whileExecutingBlock:complete:] + 360
    9   Test Bundle                         0x00000001155ddfea -[KIFSystemTestActor waitForNotificationName:object:whileExecutingBlock:] + 74
    10  Test Bundle                         0x00000001155cc77b -[MapViewTests testDelegatesStartStopLocatingUser] + 313
    11  CoreFoundation                      0x0000000105d3a85c __invoking___ + 140
    12  CoreFoundation                      0x0000000105d3a6ae -[NSInvocation invoke] + 286
    13  XCTest                              0x000000010fa2c608 __24-[XCTestCase invokeTest]_block_invoke_2 + 159
    14  XCTest                              0x000000010fa5bc90 -[XCTestContext performInScope:] + 184
    15  XCTest                              0x000000010fa2c558 -[XCTestCase invokeTest] + 169
    16  XCTest                              0x000000010fa2c9f3 -[XCTestCase performTest:] + 443
    17  XCTest                              0x000000010fa2a6b9 -[XCTestSuite performTest:] + 377
    18  XCTest                              0x000000010fa2a6b9 -[XCTestSuite performTest:] + 377
    19  XCTest                              0x000000010fa2a6b9 -[XCTestSuite performTest:] + 377
    20  XCTest                              0x000000010fa17ec2 __25-[XCTestDriver _runSuite]_block_invoke + 51
    21  XCTest                              0x000000010fa380cb -[XCTestObservationCenter _observeTestExecutionForBlock:] + 615
    22  XCTest                              0x000000010fa17e0e -[XCTestDriver _runSuite] + 408
    23  XCTest                              0x000000010fa18b6d -[XCTestDriver _checkForTestManager] + 263
    24  XCTest                              0x000000010fa5cff1 _XCTestMain + 628
    25  CoreFoundation                      0x0000000105d7808c __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    26  CoreFoundation                      0x0000000105d6dd15 __CFRunLoopDoBlocks + 341
    27  CoreFoundation                      0x0000000105d6d472 __CFRunLoopRun + 850
    28  CoreFoundation                      0x0000000105d6ce98 CFRunLoopRunSpecific + 488
    29  GraphicsServices                    0x000000010bd73ad2 GSEventRunModal + 161
    30  UIKit                               0x00000001067b7676 UIApplicationMain + 171
    31  Mapbox GL Tests                     0x000000010483a0d3 main + 80
    32  libdyld.dylib                       0x000000010883a92d start + 1
)
Test Case '-[MapViewTests testDelegatesStartStopLocatingUser]' failed (7.664 seconds).

and

MapViewTests
  testUserTrackingModeFollow, ((tester.mapView.userLocationVisible) equal to (YES)) failed: ("NO") is not equal to ("YES") - user location should be visible
  /Users/travis/build/mapbox/mapbox-gl-native/test/ios/MapViewTests.m:630
                   YES,
                   @"user location should be visible");
    XCTAssertEqual(tester.mapView.userLocation.coordinate.latitude,

and

MapViewTests
  testDelegatesStartStopLocatingUser, The step timed out after 10.00 seconds: Application is ignoring interaction events
  /Users/travis/build/mapbox/mapbox-gl-native/test/ios/KIFTestActor+MapboxGL.m:18

- (MGLMapView *)mapView {
    return (MGLMapView *)[tester waitForViewWithAccessibilityLabel:@"Map"];
}

  testDelegatesStartStopLocatingUser, Failure in child step: The step timed out after 10.00 seconds: Application is ignoring interaction events
  /Users/travis/build/mapbox/mapbox-gl-native/test/ios/KIFTestActor+MapboxGL.m:14

- (MGLTViewController *)viewController {
    return (MGLTViewController *)[[tester.mapView nextResponder] nextResponder];
}

  testDelegatesStartStopLocatingUser, Failure in child step: The step timed out after 10.00 seconds: Application is ignoring interaction events
  /Users/travis/build/mapbox/mapbox-gl-native/test/ios/MapViewTests.m:27

    [system simulateDeviceRotationToOrientation:UIDeviceOrientationPortrait];
    [tester.viewController resetMapView];

     Executed 16 tests, with 3 failures (3 unexpected) in 32.161 (32.182) seconds

/cc @incanus @1ec5

@friedbunny friedbunny added bug iOS Mapbox Maps SDK for iOS tests labels Oct 22, 2015
@incanus
Copy link
Contributor

incanus commented Oct 23, 2015

Taking another swipe at this now.

@incanus incanus self-assigned this Oct 23, 2015
@friedbunny
Copy link
Contributor Author

Doin' work in #2769.

@1ec5
Copy link
Contributor

1ec5 commented Nov 3, 2015

I found in #2920 that waiting for animations to complete inside -beforeEach helped with the compass tests. Perhaps it matters here too.

@jfirebaugh
Copy link
Contributor

Can we disable the problematic tests for now? They're still causing spurious failed builds.

@1ec5
Copy link
Contributor

1ec5 commented Apr 22, 2016

Once we replace KIF with XCUITest in #4794, we’ll be able to get past this alert quite easily.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug iOS Mapbox Maps SDK for iOS tests
Projects
None yet
Development

No branches or pull requests

4 participants