Skip to content

[Distributed] Support complex generic distributed actors in thunk gen #70842

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 28, 2024

Conversation

ktoso
Copy link
Contributor

@ktoso ktoso commented Jan 11, 2024

Finally complete our adventure with generics and distributed thunks.

Description: Distributed thunks still incorrectly handled more complex generics cases, where the distributed func would use a generic type that had requirements on the enclosing actor type.

This corrects how distributed thunks use generics, i.e. we don't carry generic parameters over at all. A distributed thunk does not have to have the same generic signature as the target; all the types are fully substituted and the only generic parameters we need to carry over are the InvocationDecoder and the actor type.

Risk: Low, affects only more complex distributed actors with generics. The fix is well understood and tested.
Review by: @xedin @slavapestov
Testing: CI testing, added exact reproducer test for the complex scenarios.
Radar: rdar://115497090
Resolves: the final part of #68517

@ktoso ktoso force-pushed the wip-verifying-distributed-generics branch from 033d1ae to 7fc1ac4 Compare January 11, 2024 14:21
@ktoso ktoso requested a review from slavapestov as a code owner January 11, 2024 14:48
@ktoso ktoso force-pushed the wip-verifying-distributed-generics branch from 6f77e86 to 20b1f6d Compare January 11, 2024 14:49
@ktoso ktoso force-pushed the wip-verifying-distributed-generics branch from a1c890c to 76c9e19 Compare January 12, 2024 02:46
@ktoso
Copy link
Contributor Author

ktoso commented Jan 12, 2024

Okey finally managed to have this all correct -- thank you for the help @xedin @slavapestov !

@ktoso
Copy link
Contributor Author

ktoso commented Jan 12, 2024

@swift-ci please smoke test

@ktoso ktoso changed the title [Distributed] Test case for generics issue with distributed methods [Distributed] Support complex generic distributed actors in thunk gen Jan 12, 2024
@ktoso
Copy link
Contributor Author

ktoso commented Jan 12, 2024

Heh, while this works locally on apple silicon it seems both linux and mac CI are not happy and we hit:

 7365
 7366 *** Program crashed: Bad pointer dereference at 0x0000000000000000 ***
 7367
 7368 Thread 0:
 7369
 7370  0               0x00007ff818d625a2 mach_msg2_trap + 10 in libsystem_kernel.dylib
 7371  1 [ra]          0x00007ff818d695d4 mach_msg_overwrite + 692 in libsystem_kernel.dylib
 7372  2 [ra]          0x00007ff818d6288a mach_msg + 19 in libsystem_kernel.dylib
 7373  3 [ra]          0x00007ff818e7d00f __CFRunLoopServiceMachPort + 145 in CoreFoundation
 7374  4 [ra]          0x00007ff818e7ba90 __CFRunLoopRun + 1365 in CoreFoundation
 7375  5 [ra]          0x00007ff818e7aed1 CFRunLoopRunSpecific + 560 in CoreFoundation
 7376  6 [ra]          0x00007ff818efe5f5 CFRunLoopRun + 40 in CoreFoundation
 7377  7 [ra]          0x000000010120cf63 swift_task_asyncMainDrainQueueImpl() + 35 in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/public/Concurrency/Task.cpp:1669:5
 7378  8 [ra]          0x000000010120cf2f swift_task_asyncMainDrainQueue + 63 in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/public/Concurrency/../CompatibilityOverride/../CompatibilityOverride/CompatibilityOverrideConcurrency.def:204:1
 7379  9 [ra]          0x000000010104f756 main + 102 in a.out
 7380 10 [ra] [system] 0x00007ff818a4741f start + 1903 in dyld
 7381
 7382 Thread 1 crashed:
 7383
 7384  0 [inlined]        0x0000000101207bbc swift::TargetMetadata<swift::InProcess>::getKind() const in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/include/swift/ABI/Metadata.h:270:38
 7385  1 [inlined]        0x0000000101207bbc swift::TargetClassMetadata<swift::InProcess, swift::TargetAnyClassMetadataObjCInterop<swift::InProcess>>::classof(swift::TargetMetadata<swift::InProcess> const*) in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/include/swift/ABI/Metadata.h:1129:22
 7386  2 [inlined]        0x0000000101207bbc __swift::__runtime::llvm::isa_impl<swift::TargetClassMetadata<swift::InProcess, swift::TargetAnyClassMetadataObjCInterop<swift::InProcess>>, swift::TargetHeapMetadata<swift::InProcess>, void>::doit(swift::TargetHeapMetadata<swift::InProcess> const&) in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/include/llvm/Support/Casting.h:59:12
 7387  3 [inlined]        0x0000000101207bbc __swift::__runtime::llvm::isa_impl_cl<swift::TargetClassMetadata<swift::InProcess, swift::TargetAnyClassMetadataObjCInterop<swift::InProcess>>, swift::TargetHeapMetadata<swift::InProcess> const*>::doit(swift::TargetHeapMetadata<swift::InProcess> const*) + 5 in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/include/llvm/Support/Casting.h:106:12
 7388  4 [inlined]        0x0000000101207bbc __swift::__runtime::llvm::isa_impl_wrap<swift::TargetClassMetadata<swift::InProcess, swift::TargetAnyClassMetadataObjCInterop<swift::InProcess>>, swift::TargetHeapMetadata<swift::InProcess> const*, swift::TargetHeapMetadata<swift::InProcess> const*>::doit(swift::TargetHeapMetadata<swift::InProcess> const* const&) + 5 in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/include/      llvm/Support/Casting.h:132:12
 7389  5 [inlined]        0x0000000101207bbc __swift::__runtime::llvm::isa_impl_wrap<swift::TargetClassMetadata<swift::InProcess, swift::TargetAnyClassMetadataObjCInterop<swift::InProcess>>, swift::TargetHeapMetadata<swift::InProcess> const* const, swift::TargetHeapMetadata<swift::InProcess> const*>::doit(swift::TargetHeapMetadata<swift::InProcess> const* const&) + 5 in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/in      clude/llvm/Support/Casting.h:122:12
 7390  6 [inlined]        0x0000000101207bbc bool __swift::__runtime::llvm::isa<swift::TargetClassMetadata<swift::InProcess, swift::TargetAnyClassMetadataObjCInterop<swift::InProcess>>, swift::TargetHeapMetadata<swift::InProcess> const*>(swift::TargetHeapMetadata<swift::InProcess> const* const&) + 5 in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/include/llvm/Support/Casting.h:143:10
 7391  7 [inlined]        0x0000000101207bbc __swift::__runtime::llvm::cast_retty<swift::TargetClassMetadata<swift::InProcess, swift::TargetAnyClassMetadataObjCInterop<swift::InProcess>>, swift::TargetHeapMetadata<swift::InProcess> const*>::ret_type __swift::__runtime::llvm::cast<swift::TargetClassMetadata<swift::InProcess, swift::TargetAnyClassMetadataObjCInterop<swift::InProcess>>, swift::TargetHeapMetadata<swift::InProcess> const>(swift::TargetHeapMetadata<swift::InProcess> const*)       + 5 in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/include/llvm/Support/Casting.h:270:3
 7392  8                  0x0000000101207bbc swift_distributed_actor_is_remote + 12 in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/public/Concurrency/Actor.cpp:2193:35
 7393  9 [ra]             0x000000010120891c (anonymous namespace)::DefaultActorImpl::unlock(bool) + 28 in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/public/Concurrency/Actor.cpp:1655:35
 7394 10 [ra]             0x00000001012079ed swift_task_switchImpl(swift::AsyncContext*, void (swift::AsyncContext* swift_async_context) swiftasynccall*, swift::SerialExecutorRef) + 509 in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/public/Concurrency/Actor.cpp:2027:5
 7395 11 [async]          0x000000010103d410 TheWorker.dist_asyncThrows(work:) in a.out
 7396 12 [async]          0x000000010118b880 swift_distributed_execute_target_resume(swift::AsyncContext*, swift::SwiftError*) in libswiftDistributed.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/public/Distributed/DistributedActor.cpp:101
 7397 13 [async]          0x0000000101181120 DistributedActorSystem.executeDistributedTarget<A>(on:target:invocationDecoder:handler:) in libswiftDistributed.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/public/Distributed/DistributedActorSystem.swift:608
 7398 14 [async]          0x0000000101056090 doIt #1 <A, B, C><A1>(active:) in FakeRoundtripActorSystem.remoteCall<A, B, C>(on:target:invocation:throwing:returning:) in a.out
 7399 15 [async]          0x0000000101055440 FakeRoundtripActorSystem.remoteCall<A, B, C>(on:target:invocation:throwing:returning:) in a.out
 7400 16 [async]          0x000000010103cf30 TheWorker.dist_asyncThrows(work:) in a.out
 7401 17 [async]          0x0000000101042380 protocol witness for DistributedWorker.dist_asyncThrows(work:) in conformance TheWorker in a.out
 7402 18 [async]          0x0000000101048550 call_dist_asyncThrows #1 <A>(w:) in test_generic(system:) in a.out
 7403 19 [async]          0x0000000101045a90 test_generic(system:) in a.out
 7404 20 [async]          0x000000010104ef30 static Main.main() in a.out
 7405 21 [async] [system] 0x000000010104f560 static Main.$main() in a.out
 7406 22 [async] [system] 0x000000010104f6a0 async_MainTQ0_ in a.out
 7407 23 [async] [thunk]  0x000000010104f7d0 thunk for @escaping @convention(thin) @async () -> () in a.out
 7408 24 [async] [thunk]  0x000000010104fe70 partial apply for thunk for @escaping @convention(thin) @async () -> () in a.out
 7409 25 [async] [system] 0x000000010120d730 completeTaskWithClosure(swift::AsyncContext*, swift::SwiftError*) in libswift_Concurrency.dylib at /Users/ec2-user/jenkins/workspace/swift-PR-macos-smoke-test/branch-main/swift/stdlib/public/Concurrency/Task.cpp:471
 7410

Which seems like a null type -- probably the actor type; since yes we do not pass it a witness, perhaps that is the issue?

Back to the investigation...

@ktoso ktoso force-pushed the wip-verifying-distributed-generics branch 3 times, most recently from 5aa85cc to 8bcf07c Compare January 26, 2024 10:36
/*decoderWitnessTable=*/void **),
/*actorType=*/Metadata *,
/*decoderWitnessTable=*/void **,
/*distributedActorWitnessTable=*/void **
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This mismatch was a lot of "fun" (not) to debug.

Thank you @xedin for the help debugging here 🙏

@ktoso ktoso force-pushed the wip-verifying-distributed-generics branch from 8bcf07c to 79005d5 Compare January 26, 2024 10:41
@ktoso
Copy link
Contributor Author

ktoso commented Jan 26, 2024

@swift-ci please test

@ktoso
Copy link
Contributor Author

ktoso commented Jan 26, 2024

macos had a git failure

@ktoso
Copy link
Contributor Author

ktoso commented Jan 26, 2024

@swift-ci please test macOS

@ktoso ktoso requested a review from slavapestov January 26, 2024 11:03
@ktoso ktoso added concurrency Feature: umbrella label for concurrency language features distributed Feature → concurrency: distributed actor and removed concurrency Feature: umbrella label for concurrency language features labels Jan 26, 2024
@ktoso
Copy link
Contributor Author

ktoso commented Jan 26, 2024

@swift-ci please smoke test

@xedin
Copy link
Contributor

xedin commented Jan 26, 2024

@swift-ci please test

@ktoso
Copy link
Contributor Author

ktoso commented Jan 26, 2024

@swift-ci please test

@ktoso ktoso enabled auto-merge (squash) January 26, 2024 21:27
@ktoso ktoso disabled auto-merge January 26, 2024 21:27
@ktoso
Copy link
Contributor Author

ktoso commented Jan 27, 2024

@swift-ci please smoke test

@ktoso ktoso enabled auto-merge (squash) January 27, 2024 08:00
@ktoso
Copy link
Contributor Author

ktoso commented Jan 27, 2024

@swift-ci please smoke test Windows

1 similar comment
@ktoso
Copy link
Contributor Author

ktoso commented Jan 28, 2024

@swift-ci please smoke test Windows

@ktoso
Copy link
Contributor Author

ktoso commented Jan 28, 2024

@swift-ci please smoke test

@ktoso ktoso merged commit dfdbe45 into swiftlang:main Jan 28, 2024
@ktoso ktoso deleted the wip-verifying-distributed-generics branch January 29, 2024 01:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
distributed Feature → concurrency: distributed actor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants