Strange issues in CI

[jwaldrip]

I am getting the following errors on CI. What would be causing this?

#!/bin/bash -leo pipefail
detox test -c ios.sim.release -a ~/artifacts/ --cleanup --record-logs=all --take-screenshots=all --record-videos=all --workers=10

^D��^D��configuration="ios.sim.release" cleanup=true artifactsLocation="/Users/distiller/artifacts/" recordLogs="all" takeScreenshots="all" recordVideos="all" node_modules/.bin/jest e2e --config=e2e/config.json --maxWorkers=10 '--testNamePattern=^((?!:android:).)*$'
detox INFO:  [DetoxServer.js] server listening on localhost:49288...
detox INFO:  [DetoxServer.js] server listening on localhost:49289...
detox INFO:  [DetoxServer.js] server listening on localhost:49292...
detox ERROR: [exec.js/EXEC_FAIL, #8] "/usr/bin/xcrun simctl io 91B8F46D-D7B5-4CC4-BC1D-7274FC3A79A9 screenshot "/dev/null"" failed with code = 2, stdout and stderr:

detox ERROR: [exec.js/EXEC_FAIL, #8] 
detox ERROR: [exec.js/EXEC_FAIL, #8] An error was encountered processing the command (domain=SimulatorKit.SimDisplayScreenshotWriter.ScreenshotError, code=2):
Error creating the image

detox ERROR: [exec.js/EXEC_FAIL, #6] "/usr/bin/xcrun simctl io 2F30D0C5-2DF8-4894-B6AF-6B6D62A5E06B screenshot "/dev/null"" failed with code = 2, stdout and stderr:

detox ERROR: [exec.js/EXEC_FAIL, #6] 
detox ERROR: [exec.js/EXEC_FAIL, #6] An error was encountered processing the command (domain=SimulatorKit.SimDisplayScreenshotWriter.ScreenshotError, code=2):
Error creating the image

detox ERROR: [exec.js/EXEC_FAIL, #8] "/usr/bin/xcrun simctl io DD748A52-7165-4926-B7EB-1930F02188B3 screenshot "/dev/null"" failed with code = 2, stdout and stderr:

detox ERROR: [exec.js/EXEC_FAIL, #8] 
detox ERROR: [exec.js/EXEC_FAIL, #8] An error was encountered processing the command (domain=SimulatorKit.SimDisplayScreenshotWriter.ScreenshotError, code=2):
Error creating the image

detox ERROR: [SimulatorRecordVideoPlugin.js/MOVE_FILE_ERROR] could not find temporary file at: "/private/var/folders/gk/lkr9pm5x039fx6d3j9r52rv80000gn/T/216e4fc4-4b33-4419-aef7-e9e679bab127.mp4"
detox ERROR: [SimulatorRecordVideoPlugin.js/MOVE_FILE_ERROR] could not find temporary file at: "/private/var/folders/gk/lkr9pm5x039fx6d3j9r52rv80000gn/T/77d81d37-f8c5-4762-a715-2c916d587a38.mp4"
detox ERROR: [SimulatorRecordVideoPlugin.js/MOVE_FILE_ERROR] could not find temporary file at: "/private/var/folders/gk/lkr9pm5x039fx6d3j9r52rv80000gn/T/1cf19446-b04b-42c2-8b7d-e178f97a9674.mp4"
detox ERROR: [SimulatorRecordVideoPlugin.js/MOVE_FILE_ERROR] could not find temporary file at: "/private/var/folders/gk/lkr9pm5x039fx6d3j9r52rv80000gn/T/05ea836a-8819-4b4b-bf62-8ef51e33bfca.mp4"
detox ERROR: [SimulatorRecordVideoPlugin.js/MOVE_FILE_ERROR] could not find temporary file at: "/private/var/folders/gk/lkr9pm5x039fx6d3j9r52rv80000gn/T/124e3614-be13-4237-ae8f-48e833f20438.mp4"
 PASS  e2e/profile.test.js (167.145s)
 PASS  e2e/permissions.test.js (172.993s)
detox ERROR: [SimulatorRecordVideoPlugin.js/MOVE_FILE_ERROR] could not find temporary file at: "/private/var/folders/gk/lkr9pm5x039fx6d3j9r52rv80000gn/T/715f46ad-18fb-4c0d-bd7a-04fa706ab602.mp4"
detox ERROR: [SimulatorRecordVideoPlugin.js/MOVE_FILE_ERROR] could not find temporary file at: "/private/var/folders/gk/lkr9pm5x039fx6d3j9r52rv80000gn/T/14564a35-5570-493b-9c0f-dde3bbf17a12.mp4"
detox ERROR: [SimulatorRecordVideoPlugin.js/MOVE_FILE_ERROR] could not find temporary file at: "/private/var/folders/gk/lkr9pm5x039fx6d3j9r52rv80000gn/T/20728cfb-ae15-4dfb-9339-87df61df93e8.mp4"
detox ERROR: [SimulatorRecordVideoPlugin.js/MOVE_FILE_ERROR] could not find temporary file at: "/private/var/folders/gk/lkr9pm5x039fx6d3j9r52rv80000gn/T/06e5dcaf-f502-4d56-83b8-de9ac508392a.mp4"
 PASS  e2e/onboarding.test.js (410.563s)

Test Suites: 3 passed, 3 total
Tests:       1 skipped, 12 passed, 13 total
Snapshots:   0 total
Time:        412.201s
Ran all test suites matching /e2e/i with tests matching "^((?!:android:).)*$".
Force exiting Jest

Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?

[noomorph]

The first three screenshot failures are something expected (it is a workaround), albeit we might conceal them as the work gets done on #848.

However, the video errors are less expected. Sure, I'll smoke-test SimulatorRecordVideoPlugin in the next couple of days to eliminate the possibility it got broken. We don't have it enabled on CI due to the error below, so I can check only locally.

Error: Error Domain=NSPOSIXErrorDomain Code=22 "Invalid argument" UserInfo={NSLocalizedDescription=Video recording requires hardware Metal capability.}.

Meanwhile, my theory is that either video recording commands did not produce files at all (was the test duration too short?), or the hooks moving the videos were called too early (concurrency issue).

If that's possible, could you post logs in --loglevel trace mode? Also try out the recommendation from Jest - add CLI --detectOpenHandles if the issue with Force exiting Jest is regularly reproduced:

configuration="ios.sim.release" cleanup=true artifactsLocation="/Users/distiller/artifacts/" recordLogs="all" takeScreenshots="all" recordVideos="all" loglevel="trace" node_modules/.bin/jest e2e --config=e2e/config.json --detectOpenHandles --maxWorkers=10 '--testNamePattern=^((?!:android:).)*$'

Otherwise, as you see, I won't have much input to proceed with your issue.

[noomorph]

@jwaldrip , just so you know, I was not lucky enough to reproduce your issue, so the more input you give the better.

Most likely, Force exiting Jest message has to do with your own tests code. Maybe you forgot to shutdown the mock server or anything similar.

Regarding, the MOVE_FILE_ERROR - please post what CI do you use, what the versions of MacOS, XCode, applesimutils are there, and everything else that might be relevant, please.

[jwaldrip]

Hey @noomorph I was able to get additional information. You can see the gist capturing info here:

https://gist.github.com/jwaldrip/8d2aa505d4321aa26ad1bd9c6e1793ae

Also I set the jest config to do detectOpenHandles true. Does this not work in the config?

{
  "setupTestFrameworkScriptFile": "./init.js",
  "transformIgnorePatterns": ["/node_modules/(?!@gigsmart)"],
  "forceExit": true,
  "detectOpenHandles": true
}

[jwaldrip]

CI is Circle CI. Latest xcode 9.4, not sure the OS version.

[noomorph]

You don't have Metal on CI. That's why videos are not taken. It is not an issue of Detox. It is beyond our reach.

I think that the only Detox's fault here is that in a regular log level mode, it didn't manage to raise this issue to you.

I'll accept this as a regression bug related to log levels in exec.js.

Thanks.