[Driver] Add -static flag for generating static archives #25202

troughton · 2019-06-02T06:52:30Z

Add a -static flag to the driver that, when combined with -emit-library, generates a static library using the platform archiver.

In the future, this -static flag will also be used on Windows to determine whether a Swift module should have DLLImport linkage applied.

compnerd · 2019-06-02T18:01:58Z

include/swift/Driver/Action.h

@@ -48,6 +48,7 @@ class Action {
    AutolinkExtractJob,
    REPLJob,
    LinkJob,
+    ArchiveJob,


I would like to get @jrose-apple's opinion on naming these: DyamicLinkJob and StaticLinkJob

Yeah, I think I like @compnerd's names better (sans typo).

Doing it in a separate commit (but still the same PR) might be nice for reviewing purposes, but not necessary.

compnerd · 2019-06-02T18:04:33Z

lib/Driver/DarwinToolChains.cpp

+  ArgStringList Arguments;
+
+  // Configure the toolchain.
+  const char *LibTool = "libtool";


You should be able to use ar to do the operation. The only time that libtool is really required is for fat libraries.

I don't really have an opinion here, but from the previous review: #25088 (comment)

Commented on that - I think that we can safely get away with it with llvm-ar at least

Commented against it. :-(

I think that we can get away with it using the stock Xcode tools as well (do we really care about the original AT&T Unix?).

compnerd · 2019-06-02T18:05:18Z

lib/Driver/DarwinToolChains.cpp

@@ -284,6 +284,8 @@ toolchains::Darwin::constructInvocation(const LinkJobAction &job,
  case LinkKind::DynamicLibrary:
    Arguments.push_back("-dylib");
    break;
+  case LinkKind::StaticLibrary:
+    llvm_unreachable("invalid link kind");


Rather than "invalid link kind" can you clarify that cannot perform archiving with the dynamic linker?

compnerd · 2019-06-02T18:06:26Z

lib/Driver/Driver.cpp

@@ -2177,6 +2196,19 @@ static StringRef baseNameForImage(const JobAction *JA, const OutputInfo &OI,
                                  StringRef BaseInput, StringRef BaseName) {
  if (JA->size() == 1 && OI.ModuleNameIsFallback && BaseInput != "-")
    return llvm::sys::path::stem(BaseInput);
+
+  if (auto link = dyn_cast<ArchiveJobAction>(JA)) {
+    Buffer = Triple.isOSWindows() ? "" : "lib";


Actually, please prepend lib on Windows for the static archives.

compnerd · 2019-06-02T18:06:39Z

lib/Driver/Driver.cpp

+    if (Triple.isOSWindows())
+      Buffer.append(".lib");
+    else
+      Buffer.append(".a");


Just use a ternary here.

compnerd · 2019-06-02T18:10:19Z

lib/Driver/UnixToolChains.cpp

+    }
+  }
+
+  if (AR == nullptr) {


You can combine this with the previous clause with an else:

if (const char *ar = llvm::sys::findProgramByName("llvm-ar", {toolchainPath})) Arguments.push_back(context.Args.MakeArgString(ar)); else if (const char *ar = llvm::sys::findProgramByName("ar", {toolchainPath})) Arguments.push_back(context.Args.MakeArgString(ar));

compnerd · 2019-06-02T18:11:13Z

lib/Driver/UnixToolChains.cpp

+  addInputsOfType(Arguments, context.InputActions, file_types::TY_Object);
+
+  InvocationInfo II{AR, Arguments};
+  II.allowsResponseFiles = true;


mmm I don't think that the traditional archiver permits response files. This should be equivalent to whether llvm-ar is in use.

compnerd · 2019-06-02T18:11:28Z

lib/Driver/WindowsToolChains.cpp

@@ -61,6 +61,8 @@ toolchains::Windows::constructInvocation(const LinkJobAction &job,
  case LinkKind::DynamicLibrary:
    Arguments.push_back("-shared");
    break;
+  case LinkKind::StaticLibrary:
+    llvm_unreachable("invalid link kind");


Similar to Darwin and Linux cases

compnerd · 2019-06-02T18:13:22Z

lib/Driver/WindowsToolChains.cpp

+  ArgStringList Arguments;
+
+  // Configure the toolchain.
+  const char *Lib = "lib";


Please use llvm-ar if available, otherwise fall back to lld-link (lld-link -lib), otherwise try lib. I don't think that we need to bother with link /LIB as a means of invoking the librarian (lib).

That seems a lot of complexity – three tools with different invocations. I assume you want llvm-ar and lld-link for cross-compilation, so would it be enough to just pick one of those and say it's required for cross-compilation?

I don't want to depend on lld-link yet. Until we have swift-lld created and tracking, please do not depend on lld-link. However, lld-link and link should be compatible enough, and link -lib is the same as lib.exe. So if you want to just use link to invoke the librarian, I am fine with that for Windows. You can use either lld-link or link then to invoke lib.

compnerd · 2019-06-02T18:14:17Z

test/Driver/static-archive.swift

+// INFERRED_NAME_DARWIN: libtool -static
+// INFERRED_NAME_DARWIN:  -o libARCHIVER.a
+// INFERRED_NAME_LINUX:   libARCHIVER.a
+// INFERRED_NAME_WINDOWS: ARCHIVER.lib


Please add a test case for the invalid combination (-emit-executable -static) as well as tests for the fallback path using mocked toolchains.

Could you clarify what you mean by "tests for the fallback path using mocked toolchains", or point to existing similar tests? I assume you mean ensure that we use the correct tools for e.g. lld-link, but I can't see where that's actually tested for the dynamic linker path – there's a line referencing config.link in lit.cfg but I'm not sure how that feeds into the tests.

jrose-apple · 2019-06-03T16:34:12Z

lib/Driver/DarwinToolChains.cpp

+      context.Args.MakeArgString(context.Output.getPrimaryOutputFilename()));
+
+  InvocationInfo II{LibTool, Arguments};
+  II.allowsResponseFiles = true;


I don't think libtool does allow response files. It does have a form of filelist, though, which you could borrow from the dynamic linking logic on Darwin.

jrose-apple · 2019-06-03T16:34:42Z

lib/Driver/Driver.cpp

@@ -1550,7 +1561,8 @@ void Driver::buildOutputInfo(const ToolChain &TC, const DerivedArgList &Args,
    OI.ModuleName = "REPL";
  } else if (const Arg *A = Args.getLastArg(options::OPT_o)) {
    OI.ModuleName = llvm::sys::path::stem(A->getValue());
-    if (OI.LinkAction == LinkKind::DynamicLibrary &&
+    if ((OI.LinkAction == LinkKind::DynamicLibrary ||
+        OI.LinkAction == LinkKind::StaticLibrary) &&


Nitpick: indentation should align by the open-paren.

…kJob # Conflicts: # lib/Driver/DarwinToolChains.cpp

troughton · 2019-06-17T00:55:37Z

I've removed the parts of this PR which used non-system-default linkers. @compnerd, I know that won't meet your needs, but it's probably best to get the uncontroversial parts of this PR in before dealing with the complexity of customising the choice of linker.

Provided this looks okay with you, @jrose-apple, this should be okay to merge. (I've rebased off master but can't build locally since I don't have Catalina installed – hopefully all the tests still pass CI.)

jrose-apple · 2019-06-17T20:41:58Z

I'll take a look soon, but note that you shouldn't need Catalina installed to build, just Xcode 11.

jrose-apple · 2019-06-18T00:41:23Z

@swift-ci Please test

jrose-apple · 2019-06-18T21:42:28Z

@compnerd, are you okay with this base thing going in, and we can discuss libtool/ar/llvm-ar later?

compnerd · 2019-06-20T04:33:37Z

@jrose-apple - okay, lets do that. At least we have a starting point then to make improvements.

troughton mentioned this pull request Jun 2, 2019

Add '-emit-archive' option to mark modules as being part of static libraries #25088

Closed

compnerd reviewed Jun 2, 2019

View reviewed changes

troughton force-pushed the static-libraries-driver branch from cf350b2 to 4f391b4 Compare June 2, 2019 22:57

jrose-apple reviewed Jun 3, 2019

View reviewed changes

troughton force-pushed the static-libraries-driver branch 3 times, most recently from 19585e2 to 78b593f Compare June 17, 2019 00:46

troughton added 2 commits June 17, 2019 12:49

Driver: add -static flag for generating static archives

b34a1cb

Driver: renamic LinkJob to DynamicLinkJob and ArchiveJob to StaticLin…

e5ea42d

…kJob # Conflicts: # lib/Driver/DarwinToolChains.cpp

troughton force-pushed the static-libraries-driver branch from 78b593f to e5ea42d Compare June 17, 2019 00:49

jrose-apple approved these changes Jun 18, 2019

View reviewed changes

jrose-apple self-assigned this Jun 18, 2019

jrose-apple merged commit 543632d into swiftlang:master Jun 20, 2019

troughton deleted the static-libraries-driver branch April 15, 2020 00:10

kateinoigakukun mentioned this pull request Apr 27, 2020

[LTO] Support LLVM level link time optimization on Darwin, Linux and Windows #31146

Merged

finagolfin mentioned this pull request Jan 24, 2023

[Driver] Use llvm-ar by default on Unix and copy it over into the build directory #62800

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Driver] Add -static flag for generating static archives #25202

[Driver] Add -static flag for generating static archives #25202

troughton commented Jun 2, 2019

compnerd Jun 2, 2019

jrose-apple Jun 3, 2019

jrose-apple Jun 3, 2019

compnerd Jun 2, 2019

troughton Jun 2, 2019

compnerd Jun 3, 2019

jrose-apple Jun 3, 2019

compnerd Jun 4, 2019

compnerd Jun 2, 2019

compnerd Jun 2, 2019

compnerd Jun 2, 2019

compnerd Jun 2, 2019

compnerd Jun 2, 2019

compnerd Jun 2, 2019

compnerd Jun 2, 2019

troughton Jun 2, 2019

compnerd Jun 3, 2019

compnerd Jun 2, 2019

troughton Jun 12, 2019

jrose-apple Jun 3, 2019

jrose-apple Jun 3, 2019

troughton commented Jun 17, 2019

jrose-apple commented Jun 17, 2019

jrose-apple commented Jun 18, 2019

jrose-apple commented Jun 18, 2019

compnerd commented Jun 20, 2019

[Driver] Add -static flag for generating static archives #25202

[Driver] Add -static flag for generating static archives #25202

Conversation

troughton commented Jun 2, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

troughton commented Jun 17, 2019

jrose-apple commented Jun 17, 2019

jrose-apple commented Jun 18, 2019

jrose-apple commented Jun 18, 2019

compnerd commented Jun 20, 2019