Spike: BDD DSL on top of Swift Testing

[younata]

Swift Testing is the new, mostly open source testing framework for Swift. It provides a pytest-style approach to testing, with native support for nested tests, like what Quick and other BDD-style frameworks provide.

This PR is an attempt to create a Quick-Style DSL on top of Swift Testing. This work explicitly does not support Objective-C, using resultbuilders to build its DSL.

This PR, for now, adds a new library target, QuickTesting, just to provide a separate space for these new type names. I intend to later merge this new target back in to the main Quick target.

Design Decisions

Spec protocol vs. Spec Macro

Currently, Specs are defined by creating a type that conforms to the Spec protocol:

struct MySpec: Spec {
    var body: Behavior {
        let ocean = ["whales", "dolphins"]
        describe("some example group") {
            it("does the thing") {
                #expect(ocean.contains("dolphins"))
            }
        }

        describe("some other example group") {
            it("does the other thing") {
                #expect(ocean.contains("whales"))
            }
        }
    }
}

I can easily reduce the level of indentation by one by using a macro instead:

@Spec
func MySpec() {
    let ocean = ["whales", "dolphins"]
    describe("some example group") {
        it("does the thing") {
            #expect(ocean.contains("dolphins"))
        }
    }

    describe("some other example group") {
        it("does the other thing") {
            #expect(ocean.contains("whales"))
        }
    }
}

But, I'm not doing that (yet?). This is because macros (currently) come with a number of drawbacks:

• Longer build times & Larger packages
  • Using Swift-Syntax dramatically increases the size of the package download, and having to compile it is a very large (but 1-time) performance issue.
  • However, because macros are sandboxed and run as separate processes, that adds additional, noticeable time to building test, which increases with each macro used.
• Additional maintenance burden
  • Macros require adding a macro target to the package, and have their own rather complicated implementation.

Most importantly, they're rather daunting to implement. Using a protocol is something I'm very familiar with, while macros are still new to me.

But, gosh, I'd love to remove as much horizontal indentations as I can.

Issues

• Testing.Test needs to have a public initializer so that we can generate tests Add a public initializer for Testing.Test swiftlang/swift-testing#521

Previous Work

I previously tried to implement Quick's Async support using resultbuilders, which was quite the learning experience: #1194

[github-actions]

	1 Warning
⚠️	Big PR

Generated by 🚫 Danger

[sushant-here]

Thanks for giving this a go @younata - Ive started looking at SwiftTesting and can say its definitely a step up from XCTest! Keen to use it. However I really like BDD and the test structure that Quick provides. So i am stuck between a rock and a hard place!

I am keen to see where you get with this. Happy to get involved with any beta testing/feedback you may be looking for.