Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Xcodebuild command can not launch test on device/simulator #234

Closed
lucasromanomr opened this issue Apr 6, 2021 · 6 comments
Closed

Xcodebuild command can not launch test on device/simulator #234

lucasromanomr opened this issue Apr 6, 2021 · 6 comments

Comments

@lucasromanomr
Copy link

lucasromanomr commented Apr 6, 2021

I was having this problem many times in the CI, but now I started running the bazel locally too, it only happens when I run the tests, and these certain tests need the simulator to run

I was thinking that it could be something about the processing on top of the machine, I did some tests limiting the number of jobs I had on the tests, but still giving the same thing.

Note: I am running several tests at the same time, and a command that runs the tests of all modules in the project. Does anyone have any idea what they can help solve?

FAIL: //Modules/Core:CoreTests (see /private/var/tmp/_bazel_lucasromano/6ae06e63d3f59df6a568ce87d763d2c7/execroot/__main__/bazel-out/darwin-dbg/testlogs/Modules/Core/CoreTests/test.log)
INFO: From Testing //Modules/Core:CoreTests:
==================== Test output for //Modules/Core:CoreTests:
2021-04-06 04:56:36,286 Will consider the test as test type xctest to run.
2021-04-06 04:56:36,291 Generating xctestrun file.
2021-04-06 04:56:39,458 Failed to import biplist module. Will use tool /usr/libexec/PlistBuddy to handle the binary format plist.
2021-04-06 04:56:40,276 Failed to import biplist module. Will use tool /usr/libexec/PlistBuddy to handle the binary format plist.
2021-04-06 04:56:40,774 Failed to import biplist module. Will use tool /usr/libexec/PlistBuddy to handle the binary format plist.
2021-04-06 04:56:40,783 Creating a new simulator:
Name: New-iPhone 12 Pro Max-14.4
OS: iOS 14.4
Type: iPhone 12 Pro Max
2021-04-06 04:56:42,161 Created new simulator EA08A20B-0DB7-4653-8079-84C2A98A45EF.
2021-04-06 04:56:42,162 Running test-without-building with device EA08A20B-0DB7-4653-8079-84C2A98A45EF
Command line invocation:
    /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild test-without-building -xctestrun /var/folders/12/t1f_q1cn2p7dq0v9xvlmx7hm0000gn/T/test_runner_work_dir.VkgZsW/TEST_ROOT/xctestrun.plist -destination id=EA08A20B-0DB7-4653-8079-84C2A98A45EF -derivedDataPath /private/var/tmp/_bazel_lucasromano/6ae06e63d3f59df6a568ce87d763d2c7/execroot/__main__/bazel-out/darwin-dbg/testlogs/Modules/Core/CoreTests/test.outputs -resultBundlePath /private/var/tmp/_bazel_lucasromano/6ae06e63d3f59df6a568ce87d763d2c7/execroot/__main__/bazel-out/darwin-dbg/testlogs/Modules/Core/CoreTests/test.outputs/test.xcresult

User defaults from command line:
    IDEBuildOperationResultBundlePath = /private/var/tmp/_bazel_lucasromano/6ae06e63d3f59df6a568ce87d763d2c7/execroot/__main__/bazel-out/darwin-dbg/testlogs/Modules/Core/CoreTests/test.outputs/test.xcresult
    IDEDerivedDataPathOverride = /var/tmp/_bazel_lucasromano/6ae06e63d3f59df6a568ce87d763d2c7/execroot/__main__/bazel-out/darwin-dbg/testlogs/Modules/Core/CoreTests/test.outputs
    IDETestRunSpecificationPath = /var/folders/12/t1f_q1cn2p7dq0v9xvlmx7hm0000gn/T/test_runner_work_dir.VkgZsW/TEST_ROOT/xctestrun.plist

Writing result bundle at path:
	/private/var/tmp/_bazel_lucasromano/6ae06e63d3f59df6a568ce87d763d2c7/execroot/__main__/bazel-out/darwin-dbg/testlogs/Modules/Core/CoreTests/test.outputs/test.xcresult

Testing started
2021-04-06 04:56:48.622 xcodebuild[23230:3304179]  IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
/var/tmp/_bazel_lucasromano/6ae06e63d3f59df6a568ce87d763d2c7/execroot/__main__/bazel-out/darwin-dbg/testlogs/Modules/Core/CoreTests/test.outputs/test.xcresult/Staging/1_Test/Diagnostics/CoreTests-2BAC9DC9-836C-45E6-AD79-61A096F8E6A3/CoreTests-3AEDF200-7F64-4C85-9596-C2A510C0B5F3/Session-CoreTests-2021-04-06_045648-2ZyJ1O.log
2021-04-06 04:56:48.624 xcodebuild[23230:3303950] [MT] IDETestOperationsObserverDebug: (24BD3FD9-16F6-4D20-B01F-449316436AEB) Beginning test session CoreTests-24BD3FD9-16F6-4D20-B01F-449316436AEB at 2021-04-06 04:56:48.624 with Xcode 12D4e on target <DVTiPhoneSimulator: 0x7f859bf76cf0> {
		SimDevice: New-iPhone 12 Pro Max-14.4 (EA08A20B-0DB7-4653-8079-84C2A98A45EF, iOS 14.4, Shutdown)
} (14.4 (18D46))
2021-04-06 04:59:12,395 The xcodebuild command got stuck and has not started test in 150. Will kill the command directly.
2021-04-06 04:59:12.899 xcodebuild[23230:3303950] [MT] IDETestOperationsObserverDebug: (24BD3FD9-16F6-4D20-B01F-449316436AEB) Failed to make test runner session:
Error Domain=DTXProxyChannel Code=1 "(null)" UserInfo={DTXProxyChannelErrorMessage=Channel disconnected}


*** If you believe this error represents a bug, please attach the result bundle at /var/tmp/_bazel_lucasromano/6ae06e63d3f59df6a568ce87d763d2c7/execroot/__main__/bazel-out/darwin-dbg/testlogs/Modules/Core/CoreTests/test.outputs/test.xcresult

** BUILD INTERRUPTED **
2021-04-06 04:59:18,977 xcodebuild command can not launch test on device/simulator in 150s.
error: Info.plist at /private/var/tmp/_bazel_lucasromano/6ae06e63d3f59df6a568ce87d763d2c7/execroot/__main__/bazel-out/darwin-dbg/testlogs/Modules/Core/CoreTests/test.outputs/test.xcresult/Info.plist does not exist, the result bundle might be corrupted or the provided path is not a result bundle
2021-04-06 04:59:19,758
2021-04-06 04:59:19,760 Deleting simulator EA08A20B-0DB7-4653-8079-84C2A98A45EF asynchronously.
2021-04-06 04:59:19,776 Done.

Xcode: 12.4
OS: Bir Sur 11.2.3

@thiagohmcruz
Copy link
Contributor

Hi @lucasromanomr! Hard to tell without some more context. Mind sharing a small project or repo that reproduces the issue?

@lucasromanomr
Copy link
Author

lucasromanomr commented Apr 6, 2021

Hi @lucasromanomr! Hard to tell without some more context. Mind sharing a small project or repo that reproduces the issue?

I cannot share the project because it is a private project of the company.

I am running the module and application tests. All at once.

Example:

bazel test --config=ci --remote_upload_local_results=true //AppOne:AppOneTests //AppTwo:AppTwoTest //Module1: Module1Tests //Module2:Module2Tests ....

Some tests are for everyone on top of the host, or better saying, run on top of the simulator:

apple_framework(
    name = "Core",
    module_name = "Core",
    platforms = {"ios": "10.3"},
    srcs = glob([
        "Core/**/*.h",
        "Core/**/*.m",
        "Core/**/*.swift",
    ]),
    resource_bundles = {
        "CoreResources": glob([
            "Core/**/*.xcassets/**",
            "Core/**/*.strings",
            "Core/**/*.cer",
            "Core/**/*.png",
        ]), 
    },        
    deps = ["@//Pods/KeychainAccess"],
    
    visibility = ["//visibility:public"],
)


apple_framework(
    name = "CoreTestsLib",    
    deps = [
        ":Core",
    ],
    srcs = glob(["CoreTests/**/*.swift"]),
    data = glob([
        "CoreTests/**/*.json",
        "CoreTests/**/*.xcassets/**",
        "CoreTests/**/*.cer"
    ]),
)

ios_unit_test(
    name = "CoreTests",
    minimum_os_version = "12.0",
    deps = [":CoreTestsLib"],
    test_host = True
)

As well as applications.

Apparently it happens that when it is time to run or call the simulator, it is waiting for the simulator to finish initializing so that afterwards the tests can start, but after the timeout on top of the simulator, interrupting the tests. As it says in the log line that I sent earlier:

** BUILD INTERRUPTED **
2021-04-06 04: 59: 18,977 xcodebuild command can not launch test on device / simulator in 150s.

@dostrander
Copy link
Collaborator

@lucasromanomr We've had (and still are having) this problem. To get around it we've done a few things.

First we have upped the amount of time to run to 300s, you can do this by passing in startup_timeout_sec in the launch options JSON.

Second we have a wrapper script that is caled from CI that callls the bazel test invocation and will detect if a simulator launch failure occurred and then retry.

The last thing we do is we do when we detect that it was a simulator launch failure we do some of the suggestions here: bazelbuild/rules_apple#185 (comment) which often will fix the problem. We've thought about even calling it before calling bazel test to see if it cuts down on the issue.

There are a few things we also have in mind to do that we haven't gotten to yet, one being start using a cloning mechanism rather than a creating mechanism from within the xctestrunner. This would require us to update it however, according to apple cloning is a much faster and less resource intensive action than creating. The other thing we've thought about doing it using bluepill which seems to do a good job of managing simulators.

I hope this helps!

@lucasromanomr
Copy link
Author

@dostrander Thank you very much for the return, I had seen this issue there, I will return with the exclusive and kill the simulators as it was before (as I was still having a problem I ended up removing it).

About setting the timeout (startup_timeout_sec), where would I do this configuration in JSON? It is a kind of layman's question.

@thiagohmcruz
Copy link
Contributor

@dostrander Thank you very much for the return, I had seen this issue there, I will return with the exclusive and kill the simulators as it was before (as I was still having a problem I ended up removing it).

About setting the timeout (startup_timeout_sec), where would I do this configuration in JSON? It is a kind of layman's question.

@lucasromanomr that would go in the JSON you can pass to the test runner (via --launch_options_json_path), see this: https://github.com/google/xctestrunner/blob/64a9be0b6fa833b4b2371729c5c8cdd2c6f7775b/test_runner/ios_test_runner.py#L79-L81

@lucasromanomr
Copy link
Author

@thiagohmcruz, @dostrander Hello, sorry, i'm starting in bazel in some parts.

How would I go about passing those arguments on? it would go through ios_unit_test or it would be somewhere else. Do I have to create a specific runner? Put in the bazel settings?

Would you like to put an example of how to pass these arguments on to bazel?

mattrobmattrob added a commit to reddit/xctestrunner that referenced this issue Dec 2, 2022
Allow simulator more time to start running a test bundle.

Aims to fix:
```
2022-12-01 20:12:58,627 The xcodebuild command got stuck and has not started test in 150. Will kill the command directly.
```

Related to bazel-ios/rules_ios#234.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants