Non-zero return codes from command plugins aren't propagated to the shell

[hassila]

Is it reproducible with SwiftPM command-line tools: swift build, swift test, swift package etc?

• Confirmed reproduction steps with SwiftPM CLI.

Description

For a command plugin that returns a non-zero exit code (either by exit, or my throwing an exception), SwiftPM currently returns a binary 0 for success or 1 for failure, "eating up" the actual exit code by the command plugin for failures.

This is problematic for our benchmark plugin (https://github.com/ordo-one/package-benchmark) when run in CI, as we want to be able to differentiate between success, regressions and improvements in benchmark runs.

This is the location where the error is "swallowed" and transformed from the exit code to a boolean basically:
https://github.com/apple/swift-package-manager/blob/64adc92bd6db10f885aec2dd0655d7407de29ff6/Sources/SPMBuildCore/Plugins/PluginInvocation.swift#L351

Expected behavior

I expect any exit code from a command plugin to be surfaced to the shell and not transformed to a "1" to allow for more complex CI workflows that doesn't just have success/failure modes.

Actual behavior

The exit code is transformed to "1"

Steps to reproduce

Use the attached sample project:
swiftpm-exitcode.zip

And simply run:
swift package crash

This should return 555 as the exit code, not 1.

Sample output:

hassila@ice ~/e/s/swiftpm-exitcode (main)> swift package crash
error: pluginCrashed
hassila@ice ~/e/s/swiftpm-exitcode (main) [1]> echo $status
1
hassila@ice ~/e/s/swiftpm-exitcode (main)> swift package crash nocrash
hassila@ice ~/e/s/swiftpm-exitcode (main)> echo $status
0
hassila@ice ~/e/s/swiftpm-exitcode (main)> 

Swift Package Manager version/commit hash

5.10

Swift & OS version (output of swift --version ; uname -a)

swift-driver version: 1.90.11.1 Apple Swift version 5.10 (swiftlang-5.10.0.13 clang-1500.3.9.4)
Target: arm64-apple-macosx14.0

Darwin ice.local 23.3.0 Darwin Kernel Version 23.3.0: Wed Dec 20 21:30:44 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6000 arm64