SwiftPM on macOS produces "error: manifest parse error(s)"

[MaxDesiatov]

I think this exact issue will be fixed in tonight's snapshot, at least not able to reproduce it locally anymore thanks to c7557be. Nevertheless, there's a different issue with SwiftPM on macOS, which prevents it from working properly. I think it has something to be with how libPackageDescription.dylibthat belongs to SwiftPM is compiled. If you completely clean your .build directory (for any package, not just Tokamak) you get this error:

 error: manifest parse error(s):
<unknown>

The breakage happens when the Package.swift manifest path is compiled, you can infer by passing the -v flag to swift build and checking where exactly the error occurs:

/Users/maxd/.swiftenv/versions/wasm-DEVELOPMENT-SNAPSHOT-2020-06-13-a/usr/bin/swiftc -L /Users/maxd/.swiftenv/versions/wasm-DEVELOPMENT-SNAPSHOT-2020-06-13-a/usr/lib/swift/pm/4_2 -lPackageDescription -Xlinker -rpath -Xlinker /Users/maxd/.swiftenv/versions/wasm-DEVELOPMENT-SNAPSHOT-2020-06-13-a/usr/lib/swift/pm/4_2 -target x86_64-apple-macosx10.15 -swift-version 5 -I /Users/maxd/.swiftenv/versions/wasm-DEVELOPMENT-SNAPSHOT-2020-06-13-a/usr/lib/swift/pm/4_2 -sdk /Applications/Xcode-11.5.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -package-description-version 5.2.0 /Users/maxd/Documents/Tokamak/Package.swift -o /var/folders/r4/5d00qxmn0rz1v00hb9lv3fdr0000gn/T/TemporaryDirectory.MvuBHz/tokamak-manifest
/var/folders/r4/5d00qxmn0rz1v00hb9lv3fdr0000gn/T/TemporaryDirectory.MvuBHz/tokamak-manifest -fileno 7 sandbox-exec -p '(version 1)
(deny default)
(import "system.sb")
(allow file-read*)
(allow process*)
(allow sysctl*)
(allow file-write*
    (regex #"^/private/var/tmp/org\.llvm\.clang.*")
    (regex #"^/var/folders/r4/5d00qxmn0rz1v00hb9lv3fdr0000gn/T/org\.llvm\.clang.*")
    (regex #"^/private/var/folders/r4/5d00qxmn0rz1v00hb9lv3fdr0000gn/T/org\.llvm\.clang.*")
    (regex #"^/private/var/folders/r4/5d00qxmn0rz1v00hb9lv3fdr0000gn/C/org\.llvm\.clang.*")
    (subpath "/Users/maxd/Documents/Tokamak/.build")
)
'
/Users/maxd/Documents/Tokamak: error: manifest parse error(s):
<unknown>

Note that I'm using 2020-06-13-a snapshot in the above output, that's the oldest snapshot where the issue is reproduced. 2020-06-12-a snapshot is working fine. I've also tried to compile the manifest file separately by 2020-06-13-a and 2020-06-12-a versions of the compiler, but using the same version of the libPackageDescription.dylib module from either 2020-06-13-a or 2020-06-12-a. Testing these different combinations, I think that the issues is caused by libPackageDescription.dylib that became broken in 2020-06-13-a and is still broken in the latest snapshots for both swiftwasm-release/5.3 and swiftwasm branches. Looking at file sizes, we can also see that something happened to the library that significantly changed its binary size starting with the 2020-06-13-a snapshot:

892K	/Users/maxd/.swiftenv/versions/wasm-DEVELOPMENT-SNAPSHOT-2020-06-11-a/usr/lib/swift/pm/4_2/libPackageDescription.dylib
892K	/Users/maxd/.swiftenv/versions/wasm-DEVELOPMENT-SNAPSHOT-2020-06-12-a/usr/lib/swift/pm/4_2/libPackageDescription.dylib
752K	/Users/maxd/.swiftenv/versions/wasm-DEVELOPMENT-SNAPSHOT-2020-06-13-a/usr/lib/swift/pm/4_2/libPackageDescription.dylib
756K	/Users/maxd/.swiftenv/versions/swift-wasm-5.3-SNAPSHOT-2020-07-03-a/usr/lib/swift/pm/4_2/libPackageDescription.dylib
752K	/Users/maxd/.swiftenv/versions/wasm-DEVELOPMENT-SNAPSHOT-2020-06-22-a/usr/lib/swift/pm/4_2/libPackageDescription.dylib

There don't seem to be any changes between 2020-06-12-a and 2020-06-13-a in the SwiftPM itself, I think something changed in the compiler that made it miscompile the library.

I'm going to research this further, need to investigate what exactly is different between those two compiler versions and how to fix it.

Originally posted by @MaxDesiatov in #1362 (comment)