[ios_xctestrun_runner] codesigning/entitlements blockers

[iainsmith]

Summary

ios_xctestrun_runner doesn't match codesigning/entitlements from Xcodes XCTRunner.app, which blocks certain API's (in our case a Network.framework based HTTP Server). This Apple forums post, seems to indicate this as the likely cause.

• ios_xctestrun_runner code, copies the supporting frameworks directly
• google/xctestrunner code, codesigns and matches XCTRunner.app/RunnerEntitlements.plist.

More details

We tried migrating to the new ios_xctestrunner for our UITests and hit this issue. We did a bit of digging, and think it's related to the codesigning/entitlements. Our UITest try to start a (Network.framework) based MockServer, which fails to launch.

There is a previous issue #504 about moving the codesigning responsibilities outside of the test runner from a couple of years ago.

I was wondering if the test runner can call tools/codesigningtool.py or if it's better to just shell out to codesign -f directly in the ios_xctestrun_runner.template.sh

cc @mattrobmattrob

[keith]

@maxwellE

[maxwellE]

So I assume that you are running on device? Or even using Network.framework on sim crashes?

Feel free to assign to me. Ideally we have a test case for this scenario as well

[iainsmith]

So I assume that you are running on device? Or even using Network.framework on sim crashes?

It happens on the simulator.

We probably crash in our test because we assume that server always starts. I think the underlying issue is NWListener can't bind to the port, from within XTRunner on the simulator.

I will try and post a minimal reproducer later.

[maxwellE]

@keith you can assign to me, should be able to take a look today

[maxwellE]

@iainsmith Can you check out #1932 and see if that works for you?

[iainsmith]

Hey @maxwellE I tried this out locally, but I'm hitting the same error where we can't start the NWListener.

I can see these lines in the log,

entitlements_path=/var/folders/9j/_v7frzl550x84_qn_8d6rrm00000gq/T//test_tmp_dir.D7dg5F/XCTRunner.app/RunnerEntitlements.plist
find /var/folders/9j/_v7frzl550x84_qn_8d6rrm00000gq/T//test_tmp_dir.D7dg5F/XCTRunner.app/Frameworks -name '*.framework' -exec codesign -f --entitlements /var/folders/9j/_v7frzl550x84_qn_8d6rrm00000gq/T//test_tmp_dir.D7dg5F/XCTRunner.app/RunnerEntitlements.plist --timestamp=none -s - '{}' ';'
codesign -f --entitlements /var/folders/9j/_v7frzl550x84_qn_8d6rrm00000gq/T//test_tmp_dir.D7dg5F/XCTRunner.app/RunnerEntitlements.plist --timestamp=none -s - /var/folders/9j/_v7frzl550x84_qn_8d6rrm00000gq/T//test_tmp_dir.D7dg5F/XCTRunner.app

Unable to start mock server on port: 11704 (some random port)

[maxwellE]

@iainsmith Can you push up a small repro to this branch for me to test with?

[maxwellE]

I am starting to investigate this again

[maxwellE]

@iainsmith I am ready to test again with #1968

[iainsmith]

Fixed in 2.4.0