Multi-destination snapshot strategy blocked by xcodebuild test slowness

[gohanlon]

This isn’t a SnapshotTesting bug, but I’m hoping someone else has bumped into this xcodebuild test issue and maybe even knows a workaround. (Feedback filed: FB9972000.)

I’ve written a basic multi-destination snapshotting strategy to capture snapshots across multiple devices and OS’s in parallel. But, I've run into an unfortunate problem with xcodebuild test taking a very long time to complete whenever the test run includes a test failure. (Test runs without any test failures complete quickly, as expected.)

Starting with a new project from the default app template, having a single test containing just XCTFail() takes over 23 seconds to complete using xcodebuild test (warmed up by running xcodebuild test a couple times prior):

% time xcodebuild test -scheme "XcodebuildTestSlowAtFailingTests" \
  -destination "platform=iOS Simulator,OS=15.4,name=iPod touch (7th generation)"
…
xcodebuild test …   16.84s user 1.31s system 79% cpu 22.816 total

After making the test succeed by removing the XCTFail(), the test run now completes in about 2 seconds.

When running on 9 Simulators in parallel, a run with a failing test takes 77 seconds vs. 6 seconds for a run where the test succeeds:

# 77 seconds vs 6 seconds: Failing test vs. empty succeeding test
% time xcodebuild test -scheme "XcodebuildTestSlowAtFailingTests" \
  -destination "platform=iOS Simulator,OS=15.4,name=iPhone 8" \
  -destination "platform=iOS Simulator,OS=15.4,name=iPhone 13 mini" \
  -destination "platform=iOS Simulator,OS=15.4,name=iPhone 13" \
  -destination "platform=iOS Simulator,OS=15.4,name=iPhone 13 Pro" \
  -destination "platform=iOS Simulator,OS=15.4,name=iPhone 13 Pro Max" \
  -destination "platform=iOS Simulator,OS=15.4,name=iPad mini (6th generation)" \
  -destination "platform=iOS Simulator,OS=15.4,name=iPad Pro (12.9-inch) (5th generation)" \
  -destination "platform=iOS Simulator,OS=15.4,name=iPhone SE (3rd generation)" \
  -destination "platform=iOS Simulator,OS=15.4,name=iPod touch (7th generation)" \
  -maximum-concurrent-test-simulator-destinations 9

I’d like to generate snapshots frequently while developing using a build trigger or even by watching changes to files—something a bit heavier than Live Previews but packing a much bigger punch—but that seems rather pointless unless I can fix this.

Has anyone experienced this? Any known workarounds or ideas?

Thank you :-)