Rewrite Swift runner using XCTestObservation

[kazk]

Swift runner is huge (#328) and takes forever to build (#422) because it's not following the best practices and compiling the entire Swift project from source.

This can be avoided by using the official builds, writing our own XCTMain and CodewarsObserver implementing XCTestObservation protocol.
Existing Kata should be able to migrate by replacing XCTMain with _XCTMain, but I'm not sure.

// `XCTMain` without filtering, using `CodewarsObserver`.
@_exported import Foundation
import XCTest
import Glibc

public func _XCTMain(_ testCases: [XCTestCaseEntry]) -> Never {
  XCTestObservationCenter.shared().addTestObserver(CodewarsObserver())
  let root = XCTestSuite(name: "All tests")
  let current = XCTestSuite(name: "\(Bundle.main.bundleURL.lastPathComponent).xctest")
  root.addTest(current)
  testCases.map(TestCaseSuite.init).forEach(current.addTest)
  root.run()
  exit(root.testRun!.totalFailureCount == 0 ? 0 : 1)
}

fileprivate class TestCaseSuite: XCTestSuite {
  // ...
}

import XCTest

public class CodewarsObserver: XCTestObservation {
  func testBundleWillStart(_ testBundle: Bundle) {}
  func testSuiteWillStart(_ testSuite: XCTestSuite) {}
  func testCaseWillStart(_ testCase: XCTestCase) {}
  func testCase(_ testCase: XCTestCase, didFailWithDescription description: String, inFile filePath: String?, atLine lineNumber: UInt) {}
  func testCaseDidFinish(_ testCase: XCTestCase) {}
  func testSuiteDidFinish(_ testSuite: XCTestSuite) {}
  func testBundleDidFinish(_ testBundle: Bundle) {}
}

// entry point
import XCTest

_XCTMain([testCase(T1.allTests), testCase(T2.allTests)])

<DESCRIBE::>T1
<IT::>test_foo
<PASSED::>Test Passed
<COMPLETEDIN::>0.000
<COMPLETEDIN::>0.000
<DESCRIBE::>T2
<IT::>test_bar
<PASSED::>Test Passed
<COMPLETEDIN::>0.000
<COMPLETEDIN::>0.000

[kazk]

@jhoffner what do you think? Using this approach:

• Swift version can be easily locked
• Image size is much smaller
• Build time is much shorter since we're no longer compiling Swift

This is from my side project and should be compatible with this project.
Maybe some issues with Clang version on Trusty.

If using _XCTMain is an issue, I can try to look into some other ways.

[jhoffner]

I'm all for it. The swift image size needs to come down. I honestly haven't had the time to wrap my head fully around this change and I don't know Swift, but I don't see any downsides to this approach from what I can tell.

[kazk]

XCTMain accepts a list of test cases and runs them. The test results are printed out using PrintObserver registered in the shared XCTestObservationCenter.

// Simplified `XCTMain`
public func XCTMain(_ testCases: [XCTestCaseEntry]) -> Never {
  XCTestObservationCenter.shared().addTestObserver(PrintObserver())

  let root = XCTestSuite(name: "All tests")
  let current = XCTestSuite(name: "\(Bundle.main.bundleURL.lastPathComponent).xctest")
  root.addTest(current)
  testCases.map(TestCaseSuite.init).forEach(current.addTest)
  root.run()
  exit(root.testRun!.totalFailureCount == 0 ? 0 : 1)
}

swift-runner makes use of XCTMain by replacing the source of Private/PrintObserver.swift. This is why XCTest library needs to be compiled. Compiling XCTest seems to require some scripts in the Swift project which leads to compiling the entire Swift ecosystem :(

XCTestObservationCenter's addTestObserver method accepts objects implementing XCTestObservation protocol (protocol describes set of requirements). This means that we can define CodewarsObserver implementing the protocol and pass the instance to addTestObserver.

import XCTest

XCTestObservationCenter.shared().addTestObserver(CodewarsObserver())
XCTMain([testCase(Tests.allTests)])

Unfortunately this won't work for us because PrintObserver is also registered by XCTMain causing the output to be interleaved.

The easiest way, and the only way I've found so far, to workaround this is to define our own _XCTMain that registers CodewarsObserver. This can be done only using public API so we don't have to mess with the source.

---

The swift image size needs to come down.

My swift-runner:3.1 derived from ubuntu:16.10 is 900 MB uncompressed.
So applying this approach should make the image size go down close to 2 GB uncompressed (around codewars/node-runner and codewars/python-runner). I'd like to work on improving other images for this project as well.