Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.0 release preparations #102

Merged
merged 8 commits into from
Sep 8, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -50,57 +50,15 @@
</BuildActionEntry>
<BuildActionEntry
buildForTesting = "YES"
buildForRunning = "YES"
buildForProfiling = "YES"
buildForArchiving = "YES"
buildForAnalyzing = "YES">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "swift-collections-benchmark"
BuildableName = "swift-collections-benchmark"
BlueprintName = "swift-collections-benchmark"
ReferencedContainer = "container:">
</BuildableReference>
</BuildActionEntry>
<BuildActionEntry
buildForTesting = "YES"
buildForRunning = "YES"
buildForProfiling = "YES"
buildForArchiving = "YES"
buildForAnalyzing = "YES">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "CollectionsTestSupport"
BuildableName = "CollectionsTestSupport"
BlueprintName = "CollectionsTestSupport"
ReferencedContainer = "container:">
</BuildableReference>
</BuildActionEntry>
<BuildActionEntry
buildForTesting = "YES"
buildForRunning = "YES"
buildForProfiling = "YES"
buildForArchiving = "YES"
buildForAnalyzing = "YES">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "Benchmarks"
BuildableName = "Benchmarks"
BlueprintName = "Benchmarks"
ReferencedContainer = "container:">
</BuildableReference>
</BuildActionEntry>
<BuildActionEntry
buildForTesting = "YES"
buildForRunning = "YES"
buildForProfiling = "YES"
buildForArchiving = "YES"
buildForAnalyzing = "YES">
buildForRunning = "NO"
buildForProfiling = "NO"
buildForArchiving = "NO"
buildForAnalyzing = "NO">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "CppBenchmarks"
BuildableName = "CppBenchmarks"
BlueprintName = "CppBenchmarks"
BlueprintIdentifier = "DequeTests"
BuildableName = "DequeTests"
BlueprintName = "DequeTests"
ReferencedContainer = "container:">
</BuildableReference>
</BuildActionEntry>
Expand All @@ -112,9 +70,9 @@
buildForAnalyzing = "NO">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "DequeTests"
BuildableName = "DequeTests"
BlueprintName = "DequeTests"
BlueprintIdentifier = "CollectionsTestSupportTests"
BuildableName = "CollectionsTestSupportTests"
BlueprintName = "CollectionsTestSupportTests"
ReferencedContainer = "container:">
</BuildableReference>
</BuildActionEntry>
Expand All @@ -126,9 +84,9 @@
buildForAnalyzing = "NO">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "CollectionsTestSupportTests"
BuildableName = "CollectionsTestSupportTests"
BlueprintName = "CollectionsTestSupportTests"
BlueprintIdentifier = "_CollectionsTestSupport"
BuildableName = "_CollectionsTestSupport"
BlueprintName = "_CollectionsTestSupport"
ReferencedContainer = "container:">
</BuildableReference>
</BuildActionEntry>
Expand Down Expand Up @@ -182,33 +140,22 @@
debugDocumentVersioning = "YES"
debugServiceExtension = "internal"
allowLocationSimulation = "YES">
<BuildableProductRunnable
runnableDebuggingMode = "0">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "swift-collections-benchmark"
BuildableName = "swift-collections-benchmark"
BlueprintName = "swift-collections-benchmark"
ReferencedContainer = "container:">
</BuildableReference>
</BuildableProductRunnable>
</LaunchAction>
<ProfileAction
buildConfiguration = "Release"
shouldUseLaunchSchemeArgsEnv = "YES"
savedToolIdentifier = ""
useCustomWorkingDirectory = "NO"
debugDocumentVersioning = "YES">
<BuildableProductRunnable
runnableDebuggingMode = "0">
<MacroExpansion>
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "swift-collections-benchmark"
BuildableName = "swift-collections-benchmark"
BlueprintName = "swift-collections-benchmark"
BlueprintIdentifier = "Collections"
BuildableName = "Collections"
BlueprintName = "Collections"
ReferencedContainer = "container:">
</BuildableReference>
</BuildableProductRunnable>
</MacroExpansion>
</ProfileAction>
<AnalyzeAction
buildConfiguration = "Debug">
Expand Down
119 changes: 119 additions & 0 deletions Documentation/Internals/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# Package Internals

## Benchmarks

The package includes an extensive library of benchmarks in the [Benchmarks](./Benchmarks) directory, driven by a command-line executable target, called `benchmark`. These benchmarks, the executable target, and its command-line interface are not considered part of the public interface of the package. As such, new releases may break them without any special ceremony. We do expect the benchmarks will stabilize with time.

For more information on our benchmarking tool, please see its dedicated package, [**Swift Collections Benchmark**][swift-collections-benchmark].

[swift-collections-benchmark]: https://github.com/apple/swift-collections-benchmark

## Test Support Library

The package comes with a rich test support library in the [Sources/_CollectionsTestSupport](./Sources/_CollectionsTestSupport) directory. These were loosely adapted from the contents of the `StdlibUnittest*` modules in the [Swift compiler repository](https://github.com/apple/swift/tree/main/stdlib/private), with some custom additions.

These components would likely be of interest to the wider Swift community, but they aren't yet stable enough (or documented enough) to publish them. Accordingly, these testing helpers are currently considered implementation details of this package, and are subject to change at whim.

The test support library currently provides the following functionality:

- [`AssertionContexts`](./Sources/CollectionsTestSupport/AssertionContexts): Custom test assertions with support for keeping track of nested context information, including stopping execution when the current context matches a particular value. (Useful for debugging combinatorial tests.)

<details>
<summary><strong>Click here for a short demonstration</strong></summary>

```swift
import XCTest
import _CollectionsTestSupport

final class DequeTests: CollectionTestCase {
func test_demo() {
let values = [0, 10, 20, 30, 42, 50, 60]
for i in values.indices {
context.withTrace("i: \(i)") {
expectEqual(values[i], 10 * i)
}
}
}
}
```

```
DemoTests.swift:21: error: -[DemoTests.DemoTests test_demo] : XCTAssertEqual failed: ("42") is not equal to ("40") -
Trace:
- i: 4
```

To debug issues, copy the trace message into a `context.failIfTraceMatches(_:)` invocation and set a breakpoint on test failures.

```swift
let values = [0, 10, 20, 30, 42, 50, 60]
for i in values.indices {
context.withTrace("i: \(i)") {
// This will report a test failure before executing the `i == 4` case,
// letting us investigate what's going on.
context.failIfTraceMatches("""
Trace:
- i: 4
""")
expectEqual(values[i], 10 * i)
}
}
```

</details>

- [`Combinatorics`](./Sources/CollectionsTestSupport/AssertionContexts/Combinatorics.swift): Basic support for exhaustive combinatorial testing. This allows us to easily verify that a collection operation works correctly on all possible instances up to a certain size, including behavioral variations such as unique/shared storage.

<details>
<summary><strong>Click here for an example</strong></summary>

```swift
func test_popFirst() {
withEveryDeque("deque", ofCapacities: [0, 1, 2, 3, 5, 10]) { layout in
withEvery("isShared", in: [false, true]) { isShared in
withLifetimeTracking { tracker in
var (deque, contents) = tracker.deque(with: layout)
withHiddenCopies(if: isShared, of: &deque) { deque in
let expected = contents[...].popFirst()
let actual = deque.popFirst()
expectEqual(actual, expected)
expectEqualElements(deque, contents)
}
}
}
}
}
```

</details>

- [`ConformanceCheckers`](./Sources/CollectionsTestSupport/ConformanceCheckers): A set of generic, semi-automated protocol conformance tests for some Standard Library protocols. These can be used to easily validate the custom protocol conformances provided by this package. These checks aren't (can't be) complete -- but when used correctly, they are able to detect most accidental mistakes.

We currently have conformance checkers for the following protocols:

- [`Sequence`](./Sources/CollectionsTestSupport/ConformanceCheckers/CheckSequence.swift)
- [`Collection`](./Sources/CollectionsTestSupport/ConformanceCheckers/CheckCollection.swift)
- [`BidirectionalCollection`](./Sources/CollectionsTestSupport/ConformanceCheckers/CheckBidirectionalCollection.swift)
- [`Equatable`](./Sources/CollectionsTestSupport/ConformanceCheckers/CheckEquatable.swift)
- [`Hashable`](./Sources/CollectionsTestSupport/ConformanceCheckers/CheckHashable.swift)
- [`Comparable`](./Sources/CollectionsTestSupport/ConformanceCheckers/CheckComparable.swift)

- [`MinimalTypes`](./Sources/CollectionsTestSupport/MinimalTypes): Minimally conforming implementations for standard protocols. These types conform to various standard protocols by implementing the requirements in as narrow-minded way as possible -- sometimes going to extreme lengths to, say, implement collection index invalidation logic in the most unhelpful way possible.

- [`MinimalSequence`](./Sources/CollectionsTestSupport/MinimalTypes/MinimalSequence.swift)
- [`MinimalCollection`](./Sources/CollectionsTestSupport/MinimalTypes/MinimalCollection.swift)
- [`MinimalBidirectionalCollection`](./Sources/CollectionsTestSupport/MinimalTypes/MinimalBidirectionalCollection.swift)
- [`MinimalRandomAccessCollection`](./Sources/CollectionsTestSupport/MinimalTypes/MinimalRandomAccessCollection.swift)
- [`MinimalMutableRandomAccessCollection`](./Sources/CollectionsTestSupport/MinimalTypes/MinimalMutableRandomAccessCollection.swift)
- [`MinimalRangeReplaceableRandomAccessCollection`](./Sources/CollectionsTestSupport/MinimalTypes/MinimalRangeReplaceableRandomAccessCollection.swift)
- [`MinimalMutableRangeReplaceableRandomAccessCollection`](./Sources/CollectionsTestSupport/MinimalTypes/MinimalMutableRangeReplaceableRandomAccessCollection.swift)
- [`MinimalIterator`](./Sources/CollectionsTestSupport/MinimalTypes/MinimalIterator.swift)
- [`MinimalIndex`](./Sources/CollectionsTestSupport/MinimalTypes/MinimalIndex.swift)
- [`MinimalEncoder`](./Sources/CollectionsTestSupport/MinimalTypes/MinimalEncoder.swift)
- [`MinimalDecoder`](./Sources/CollectionsTestSupport/MinimalTypes/MinimalDecoder.swift)

- [`Utilities`](./Sources/CollectionsTestSupport/Utilities): Utility types. Wrapper types for boxed values, a simple deterministic random number generator, and a lifetime tracker for catching simple memory management issues such as memory leaks. (The [Address Sanitizer][asan] can be used to catch more serious problems.)

[asan]: https://developer.apple.com/documentation/xcode/diagnosing_memory_thread_and_crash_issues_early?language=objc


58 changes: 58 additions & 0 deletions Documentation/Internals/ReleaseChecklist.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Swift Collections Release Checklist

1. Create a milestone for the new version (if one doesn't exist yet).
2. Collect all issues & PRs that are going to be included in the new tag under the new milestone.
3. If the new release moves code between source files or adds new API that has the potential to cause mutual dependencies between source files, then run the [shuffle-sources.sh](./Utils/shuffle-sources.sh) script for at least a few hundred iterations on the affected module to help catch [nondeterministic build issues with the compiler's MergeModules phase](https://github.com/apple/swift-collections/issues/7). (Note: it's best to do this on a fresh clone. We can stop doing this when MergeModules is no longer used to build debug configurations in SPM.)
4. Run the [full tests script](./Utils/run-full-tests.sh) on the commit that you intend to tag, with all supported (major) toolchain releases, and on as many supported platforms as are practical. (At minimum, run the script with the latest stable toolchain releases on macOS and one Linux distribution.)
The script exercises a tiny subset of the environments this package can be built and run on.

The full matrix includes the following axes:

- Toolchains
- All major Swift releases supported by the package, plus a recent toolchain snapshot from swift.org.
E.g., for the Swift Collection 1.0 release, this included:
- Swift 5.3 (from swift.org and Xcode 12)
- Swift 5.4 (from swift.org and Xcode 12.5)
- Prerelease builds of Swift 5.5 (using the latest swift.org snapshot and the latest Xcode 13 beta)
- The latest `main` development snapshot from swift.org.
- Platforms & architectures
- macOS (using SPM, xcodebuild, cmake)
- Intel, Apple Silicon
- Deploying on any macOS release starting from macOS 10.10
- Mac Catalyst (using xcodebuild)
- Intel, Apple Silicon
- Deploying on any macOS release starting from macOS 10.15
- iOS (using xcodebuild)
- Device (arm64, armv7), simulator (x86_64, i386, arm64)
- Deploying on any iOS release starting from iOS 8
- watchOS (using xcodebuild)
- Device (arm64_32, arm7k), simulator (x86_64, i386, arm64)
- Deploying on any watchOS release starting from watchOS 2
- tvOS (using xcodebuild)
- Device (arm64), simulator (x86_64, arm64)
- Deploying on any tvOS 9 release starting from tvOS 10
- Linux (using SPM, cmake)
- All supported distributions
- Build systems
- Swift Package Manager
- xcodebuild
- cmake & ninja (note: this support isn't source stable)
- Configurations
- Debug
- Release
- Build settings
- `COLLECTIONS_INTERNAL_CHECKS`
- `COLLECTIONS_DETERMINISTIC_HASHING`
- `BUILD_LIBRARY_FOR_DISTRIBUTION=YES` with xcodebuild (this is unsupported, but why break it)
- `-warnings-as-errors`
- Any combination of the above
- Components
- The swift-collections package
- The private `benchmark` executable under `Benchmarks/`

5. Submit PRs to fix any issues that pop up, then try again until no issues are found.
6. Update the README if necessary.
7. Draft a new release on GitHub, following the template established by [previous releases](https://github.com/apple/swift-collections/releases/tag/0.0.7).
8. As a final chance to catch issues, generate a diff between the last tagged release and the release candidate commit. Review it in case we landed something that isn't appropriate to include; watch out for potential issues such as source compatibility problems, or new public API without adequate documentation.
9. Double check that the new tag will have the right version number and it will be on the correct branch.
10. Hit publish.
8 changes: 4 additions & 4 deletions Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ let package = Package(

// Testing support module
.target(
name: "CollectionsTestSupport",
name: "_CollectionsTestSupport",
dependencies: [],
swiftSettings: settings,
linkerSettings: [
Expand All @@ -79,7 +79,7 @@ let package = Package(
),
.testTarget(
name: "CollectionsTestSupportTests",
dependencies: ["CollectionsTestSupport"],
dependencies: ["_CollectionsTestSupport"],
swiftSettings: settings),

// Deque<Element>
Expand All @@ -89,7 +89,7 @@ let package = Package(
swiftSettings: settings),
.testTarget(
name: "DequeTests",
dependencies: ["DequeModule", "CollectionsTestSupport"],
dependencies: ["DequeModule", "_CollectionsTestSupport"],
swiftSettings: settings),

// OrderedSet<Element>, OrderedDictionary<Key, Value>
Expand All @@ -99,7 +99,7 @@ let package = Package(
swiftSettings: settings),
.testTarget(
name: "OrderedCollectionsTests",
dependencies: ["OrderedCollections", "CollectionsTestSupport"],
dependencies: ["OrderedCollections", "_CollectionsTestSupport"],
swiftSettings: settings),
]
)
Loading