Skip to content
This repository was archived by the owner on Oct 14, 2020. It is now read-only.

Rewrite Swift runner #451

Merged
merged 1 commit into from
Jul 12, 2017
Merged

Rewrite Swift runner #451

merged 1 commit into from
Jul 12, 2017

Conversation

kazk
Copy link
Member

@kazk kazk commented Jul 12, 2017

Rewrite swift runner using XCTestObservation protocol.
Test results are now reported through CodewarsObserver which is registered by _XCTMain.

This way, we no longer need to hack the source.

  • Use Swift version 3.1.1, official build for Ubuntu 14.04
  • Files are now compiled with swiftc and are no longer concatenated
    • This passes existing tests, but requires further testing
    • solutionOnly:
      • opts.solution is written to main.swift
      • opts.setup is compiled together if provided as setup.swift
    • testIntegration:
      • opts.fixture is written to main.swift
      • opts.solution is compiled together as solution.swift
      • opts.setup is compiled together if provided as setup.swift
  • Make Travis build and test swift-runner
  • Remove cw-2 test framework
  • Update examples

The new image is less than 2 GB uncompressed.

image

Required Changes

  • Use _XCTMain instead of XCTMain
  • import XCTest is required

Output Compatibility

Outputs are mostly compatible with the original.
The failure messages are now reported with <FAILED::>description not <FAILED::>Test Failed followed by <ERROR::>description.
The output for test cases with multiple assertions changed. IT only contains single PASSED, FAILED or ERROR.

Closes #448

Rewrite swift runner using `XCTestObservation` protocol.
Test results are now reported through `CodewarsObserver` which is
registered by `_XCTMain`.

This way, we no longer need to hack the source.

- Use Swift version 3.1.1, official build for Ubuntu 14.04
- Files are now compiled with `swiftc` and are no longer concatenated
  - This passes existing tests, but requires further testing
  - `solutionOnly`:
    - `opts.solution` is written to `main.swift`
    - `opts.setup` is compiled together if provided as `setup.swift`
  - `testIntegration`:
    - `opts.fixture` is written to `main.swift`
    - `opts.solution` is compiled together as `solution.swift`
    - `opts.setup` is compiled together if provided as `setup.swift`
- Make Travis build and test swift-runner
- Remove cw-2 test framework
- Update examples

Required Changes

- Use `_XCTMain` instead of `XCTMain`
- `import XCTest` is required

Output Compatibility

Outputs are mostly compatible with the original.
The failure messages are now reported with `<FAILED::>description` not
`<FAILED::>Test Failed` followed by `<ERROR::>description`.
The output for test cases with multiple assertions changed.
`IT` only contains single `PASSED`, `FAILED` or `ERROR`.

Closes codewars#448
@OverZealous
Copy link
Contributor

This is awesome! I love commits with way more lines removed than added!

@jhoffner
Copy link
Member

This is great @kazk! Funny enough I was just troubleshooting Swift build issues on our new build server, so I'm eager to merge this in.

It's a little bit of a bummer that we lost resolution on how many assertions passed within a failing spec, but it's not a big deal. Other languages work the same. Sometimes test creators get a little too happy with creating only 1 or 2 tests with far too many assertions within each, I'm not really sure how helpful it was to see the additional assertions (so that they at least see they are making progress on a spec).

Beyond that though this looks like a huge improvement!

@kazk
Copy link
Member Author

kazk commented Jul 12, 2017

@jhoffner

Is this valid?

<IT::>testWithMultiple
<PASSED::>Test Passed
<FAILED::>some test failed
<COMPLETED::>0.000

I can try to look into how it used to work if this is valid format.

@jhoffner
Copy link
Member

Yes, that is valid.

@jhoffner jhoffner merged commit 241a65b into codewars:master Jul 12, 2017
@jhoffner
Copy link
Member

@kazk I went ahead and merged since I am in the process of building out the new images and testing them. The multiple assertions tweaks can be a separate PR if you end up finding a solution.

@kazk
Copy link
Member Author

kazk commented Jul 12, 2017

I'll open another PR if I find a way to do this without messing with XCTest's source.

I think the original one had output like this for 1 passing and 1 failing (guessing from the tests):

<IT::>testWithMultiple
<PASSED::>Test Passed
<FAILED::>Test Failed
<ERROR::>failure description
<COMPLETED::>1

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rewrite Swift runner using XCTestObservation
3 participants