Skip to content

Commit

Permalink
Fix flaky tests (#784)
Browse files Browse the repository at this point in the history
This PR fixes 3 common source of failure for our test :

1. ~Deadlock : sometimes the test runner freeze and needs a total clean to be run. We fix this by removing the need to call the main thread in SegmentIntegration~ Moved to #784
2. Missing iOS simulator : Adding a command to list the simulators seems to magically fix the issue.
3. Flaky maxQueueSize test : I rewrote this one to be more stable

We ran 21+ successful and consecutive test iterations : https://circleci.com/gh/segmentio/workflows/analytics-ios/tree/fix%2Fflaky-test

Ref : LIB-543
  • Loading branch information
fathyb authored Sep 21, 2018
1 parent 00c075a commit 9970131
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 12 deletions.
1 change: 1 addition & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ jobs:
xcode: "9.4.1"
steps:
- checkout
- run: xcrun simctl list
- run:
name: Install build dependencies
command: |
Expand Down
19 changes: 9 additions & 10 deletions AnalyticsTests/AnalyticsTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -121,19 +121,18 @@ class AnalyticsTests: QuickSpec {
let integration = analytics.test_integrationsManager()?.test_segmentIntegration()
expect(integration).notTo(beNil())

var sent = 0

analytics.flush()
integration?.test_dispatchBackground {
if let count = integration?.test_queue()?.count {
sent = count
}
else {
sent = -1
waitUntil(timeout: 60) {done in
let queue = DispatchQueue(label: "test")

queue.async {
while(integration?.test_queue()?.count != max) {
sleep(1)
}

done()
}
}

expect(sent).toEventually(equal(max))
}

it("protocol conformance should not interfere with UIApplication interface") {
Expand Down
4 changes: 2 additions & 2 deletions AnalyticsTests/EndToEndTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,9 @@ class AnalyticsE2ETests: QuickSpec {

analytics.track("E2E Test", properties: ["id": uuid])

self.waitForExpectations(timeout: 20)
self.waitForExpectations(timeout: 3 * 60)

for _ in 1...5 {
for _ in 1...3 * 30 {
sleep(2)
if hasMatchingId(messageId: uuid) {
return
Expand Down

0 comments on commit 9970131

Please sign in to comment.