Xcode bot tests stalling in waitUntil block

[abbeycode]

• I have read CONTRIBUTING and have done my best to follow them.

What did you do?

I've got a bunch of tests covering NSOperation subclasses, which are written with dependency injection so I can use simulated network operations with them that don't actually talk to any servers. To test them, I have specs that look like this:

it("fails when no search results returned") {
    let lookupOp = LookupOperation(title: "Title", year: 1977) // This is an NSOperation subclass

    waitUntil(timeout: 2) { done in
        lookupOp.completionBlock = {
            done()
        }
    
        lookupOp.start()
    } // <-- This is the line indicated in the sample

    expect(lookupOp.customResult).to(beNil())
    expect(lookupOp.error) == LookupError.noResultsInList
}

My tests run most often in an Xcode bot, and have been stalling about once out of every 2-5 runs (and I was also able to reproduce this hang locally too). I downloaded the logs for a few different occurrences of the stall, and looked at the xctest process samples, which consistently show a stack trace containing the line above (in different test cases, but always the closing brace of a waitUntil block), with these Nimble lines concluding the stack trace:

+ 8852 waitUntil(timeout:file:line:action:)  (in Nimble) + 150  [0x1177f9e96]  DSL+Wait.swift:115
+   8852 static NMBWait.until(timeout:file:line:action:)  (in Nimble) + 220  [0x1177f7a5c]  DSL+Wait.swift:26
+     8852 static NMBWait.throwableUntil(timeout:file:line:action:)  (in Nimble) + 657  [0x1177f8021]  DSL+Wait.swift:67
+       8852 AwaitPromiseBuilder.wait(_:file:line:)  (in Nimble) + 639  [0x1177a269f]  Async.swift:273
+         8852 -[NMBExceptionCapture tryBlock:]  (in Nimble) + 73  [0x117795b79]  NMBExceptionCapture.m:21
+           8852 thunk for @callee_owned () -> ()  (in Nimble) + 41  [0x11779c2a9]  AssertionRecorder.swift:0
+             8852 partial apply for closure #3 in AwaitPromiseBuilder.wait(_:file:line:)  (in Nimble) + 12  [0x1177a2ffc]  Async.swift:0
+               8852 closure #3 in AwaitPromiseBuilder.wait(_:file:line:)  (in Nimble) + 1017  [0x1177a2d69]  Async.swift:266
+                 8852 -[NSRunLoop(NSRunLoop) runMode:beforeDate:]  (in Foundation) + 274  [0x1017786de]
+                   8852 CFRunLoopRunSpecific  (in CoreFoundation) + 409  [0x1027b2889]
+                     8852 __CFRunLoopRun  (in CoreFoundation) + 1730  [0x1027b31c2]
+                       8852 __CFRunLoopServiceMachPort  (in CoreFoundation) + 213  [0x1027b3c85]
+                         8852 mach_msg  (in libsystem_kernel.dylib) + 60  [0x109741cdc]
+                           8852 mach_msg_trap  (in libsystem_kernel.dylib) + 10  [0x1097427c2]

These are the errors Xcode shows for the integrations that time out:

What did you expect to happen?

Each test using a waitUntil block either completes successfully, or fails after the specified 2-second timeout

What actually happened instead?

The tests sometimes stall until the Bot times out (after 10 minutes)

Environment

• Quick: 1.2.0

• Nimble: 7.1.0

• Xcode Version: 9.3 (9E145)

• Swift Version: Xcode Default

• macOS Version: 10.13.4 (though this was happening on 10.12 as well)

• Cocoapods: 1.5.0

Project that demonstrates the issue

Let me know if you absolutely need this, but the issue isn't reproducible every time in my own project, so it would be difficult to distill it down to a sample project.

[abbeycode]

@jeffh Is there anything I can do to help track this down, and maybe start working on a fix?

[abbeycode]

I've added some updates. Specifically, I'm seeing this locally too (as well as still in my Bot builds), and I've updated Xcode, Nimble, CocoaPods, and macOS. Still seeing this.

[AnthonyMillerSF]

I'm seeing this locally right now as of today as well.

[AnthonyMDev]

I've figured out the underlying issue here, created a reproducible example case, and implemented a fix. I'm exhausted now, so I'll go ahead and clean it all up and make a PR tomorrow for you guys on this one.

[AnthonyMDev]

Fix has been pushed up for code review!

[AnthonyMDev]

@abbeycode Any chance someone can look at this, merge it in, and push a new patch version to Cocoapods. I'd really love to have this fix in, since it's killing my CI builds.

[abbeycode]

@AnthonyMDev I reviewed your PR and it looks good to me, but someone else (@jeffh) will have to merge it (and I'm not sure what branch he'd want it to go into). That said, you don't need to wait for it to be merged to fix your builds - just point CocoaPods to your branch in the meantime, while you wait for it to be merged.

Thanks, by the way, for taking the initiative to fix this. I haven't been motivated, since I haven't been working on my project that uses Nimble for a while.

[AnthonyMDev]

Thanks! I am doing that for now. I just like to remove my forked pods as quickly as possible to ensure I get other bug fixes and new features.

[ikesyo]

Finally fixed by #721 👏

[AnthonyMillerSF]

Any idea when this fix will be included in a new release?

[ikesyo]

I'm sorry for the delay, v8.0.6 has been released including the fix 🚀