Skip to content

[6.0][Macros] In-process plugin server #74740

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

rintaro
Copy link
Member

@rintaro rintaro commented Jun 26, 2024

Cherry-pick #73725 #74734 #74785 #74941 into release/6.0

  • Explanation: swift-syntax libraries in compiler has been complied with -enable-library-evolution because the toolchain provides them as a public interface for building plugins (i.e. lib/swift/host/*.swiftmodule). But that was not great for the performance in the compiler. This PR changes that by separating swift-syntax libs for the compiler from the one for the plugins. So the compiler and the plugin now can use different version/ABI of swift-syntax libraries from the compiler. Now swift-syntax for the compiler are compiled without library evolution.
  • Scope: Macro plugins
  • Risk: Mid. This changes the mechanism of in-process plugins significantly. Now in-process plugins are communicate with the compiler using the same messaging serialization with other plugins (i.e. swift-plugin-server, and other executable plugins), instead of directly calling the Macro.Type expansion method. Also, to separate the swift-syntax libraries, this introduces another set of shared swift-syntax libraries in the toolchain.
  • Testing: Passes current test suite.
  • Issue: rdar://130532628
  • Reviewer: Hamish Knight (@hamishknight) Alex Hoppen (@ahoppen) Ben Barham (@bnbarham) and Daniel Rodríguez Troitiño (@drodriguez) as the author of [Macros] In-process plugin server library tied to compiler host, not target #74785

rintaro added 3 commits June 25, 2024 09:31
Separate swift-syntax libs for the compiler and for the library plugins.
Compiler communicates with library plugins using serialized messages
just like executable plugins.

* `lib/swift/host/compiler/lib_Compiler*.dylib`(`lib/CompilerSwiftSyntax`):
  swift-syntax libraries for compiler. Library evolution is disabled.
* Compiler (`ASTGen` and `swiftIDEUtilsBridging`) only depends on
  `lib/swift/host/compiler` libraries.
* `SwiftInProcPluginServer`: In-process plugin server shared library.
  This has one `swift_inproc_plugins_handle_message` entry point that
  receives a message and return the response.
* In the compiler
  * Add `-in-process-plugin-server-path` front-end option, which specifies
    the `SwiftInProcPluginServer` shared library path.
  * Remove `LoadedLibraryPlugin`, because all library plugins are managed
    by `SwiftInProcPluginServer`
  * Introduce abstract `CompilerPlugin` class that has 2 subclasses:
    * `LoadedExecutablePlugin` existing class that represents an
      executable plugin
    * `InProcessPlugins` wraps `dlopen`ed `SwiftInProcPluginServer`
  * Unified the code path in `TypeCheckMacros.cpp` and `ASTGen`, the
    difference between executable plugins and library plugins are now
    abstracted by `CompilerPlugin`

(cherry picked from commit 2f7aa42)
Not all driver can send '-in-process-plugin-server-path'. To keep
existing '-plugin-path' working, infer default server path in the
frontend.

(cherry picked from commit f08d69c)
@rintaro rintaro requested a review from a team as a code owner June 26, 2024 18:10
…target

PR swiftlang#73725 introduced the in-process plugin server library, but the
selection of the library depends on the selected toolchain, which
depends on the compiler target, not the host. When cross-compiling (for
example from macOS to a embedded Unix target), the compiler will
incorrectly chose the `.so` file, not find it, and fail to compile
things like the `@debugDescription` macro.

Move the in-process plugin server library code from the platform
toolchains into the parent type, and code it so it uses the right name
depending on the compiler host at compilation time. This discards the
target and only relies on the compiler host for selecting the right
library.

(cherry picked from commit 55d9e74)
@rintaro
Copy link
Member Author

rintaro commented Jul 1, 2024

A `std::function` instance remained in memory.

rdar://131048379
(cherry picked from commit 432d5bb)
@rintaro
Copy link
Member Author

rintaro commented Jul 3, 2024

@rintaro
Copy link
Member Author

rintaro commented Jul 3, 2024

swiftlang/swift-syntax#2702
swiftlang/swift-driver#1642
swiftlang/llvm-project#8929
swiftlang/swift-installer-scripts#309
preset=buildbot_incremental_linux,lsan,tools=RDA,stdlib=DA,test=no
@swift-ci Please test with preset Linux Platform

@rintaro
Copy link
Member Author

rintaro commented Jul 20, 2024

Decided not to merge

@rintaro rintaro closed this Jul 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants