[SR-16008] Crash in NativeSet.subtracting in Swift 5.6

[bjhomer]

Previous ID	SR-16008
Radar	rdar://problem/90450460
Original Reporter	@bjhomer
Type	Bug
Status	Resolved
Resolution	Done

Attachment: Download

Environment

Swift 5.6 (Xcode 13.3 RC)

Additional Detail from JIRA

Votes	0
Component/s
Labels	Bug
Assignee	@eeckstein
Priority	Medium

md5: 674f6e7f2c1d562dd01cd5928d2907d0

Issue Description:

We've recently upgraded to Swift 5.6 (with Xcode 13.3), and we're seeing reports of crashes in NativeSet.subtracting. We haven't changed any of this code in months; the only change is that we're now building with Swift 5.6.

I see that there were some changes to NativeSet.subtracting in Swift 5.6, so I'm wondering if our crashes are related to these changes.

Our code looks like this:

// These sets could have 1000s of items in them. Or zero.
// Depends on user behavior.
let a: Set<String> = potentiallyLargeSetOfStrings()
let b: Set<String> = anotherLargeSetOfStrings()
let itemsNoLongerUsed = b.subtracting(a) // This line is crashing

Full crash logs attached below, but the relevant part looks like this:

 Thread 11 name:
 Thread 11 Crashed:
 0 ??? 0x0000000000000000 0x0 + 0
 1 DOCore 0x00000001059f477c $ss10_NativeSetV11subtractingyAByxGqd__7ElementQyd__RszSTRd__lFSS_ShySSGTg5 + 8 (<compiler-generated>:0)
 2 DOCore 0x00000001059f477c $sSh12_subtractingyShyxGqd__7ElementQyd__RszSTRd__lFSS_ShySSGTg5 + 8 (<compiler-generated>:0)

swift-demangle reports those frames as this:

1   generic specialization <Swift.String, Swift.Set<Swift.String>> of Swift._NativeSet.subtracting<A where A == A1.Element, A1: Swift.Sequence>(A1) -> Swift._NativeSet<A>

2   generic specialization <Swift.String, Swift.Set<Swift.String>> of Swift.Set._subtracting<A where A == A1.Element, A1: Swift.Sequence>(A1) -> Swift.Set<A>

I have not been able to reproduce this crash locally yet, but we've got a bunch of reports from 8 beta testers in just a few hours, so it seems like it's a reasonably common thing to hit, and not likely dependent on having very particularly crafted set.

[lorentey]

Unexpected! That jump to NULL happens on a suspiciously early offset; I wonder if this this turns out to be some sort of undiagnosed availability issue. (If so, I'm surprised it didn't show up in our back deployment tests.)

It's too early to tell if the fix will require an SDK update, but if you're urgently blocked on this, a quick-and-dirty workaround would be to define something like this in affected modules:

#if swift(>=5.6) && swift(<5.6.99)
extension Set {
  func subtracting<S: Sequence>(_ other: S) -> Set<Element>
  where S.Element == Element {
    var result = self
    for item in other { result.remove(item) }
    return result
  }
}
#endif

(This is considerably slower than the actual implementation, but it will be most likely free of this issue.)

[lorentey]

@swift-ci create

[bjhomer]

Oh, interesting. I haven't seen a crash yet from iOS 15.4, but I've seen a lot from OS versions prior to that. (Particularly iOS 15.3.1). That could just be indicative that we don't have many users on 15.4 yet, though.

[lorentey]

Erik's fix is up: #41954 It did turn out to be an availability issue, caused by a misfiring optimization in the compiler.

[eeckstein]

fix for 5.6: #41954
for main: #41026

[swift-ci]

Comment by Spencer Transier (JIRA)

@eeckstein Do you know if the fix for this bug was included in Xcode 13.3.1?

[belkadan]

It looks like 13.3.1 has the same version of Swift as 13.3. But there has been a 5.6.1 tagged, so we can be hopeful that there'll be a 13.3.2…

(Signal also ran into this bug, generating many crashes a day.)