Support parallel test execution #609

doronpr · 2018-03-06T20:16:13Z

fixed merge issues
fixed missing coverage warnings

# Conflicts: # detox/src/devices/AppleSimUtils.js # detox/test/package.json

rotemmiz

Were you able to attach to debugger when using jest as test runner ?

rotemmiz · 2018-03-06T20:35:51Z

detox/test/e2e/config.json

+  "setupTestFrameworkScriptFile" : "./helpers/init.js",
+  "bail": true,
+  "verbose": true,
+  "forceExit": true


We should validate Detox doesn't leave anything open, and that everything is being closed correctly.
Please remove forceExit

rotemmiz · 2018-03-06T20:36:23Z

detox/test/e2e/config.json

+  "testEnvironment": "node",
+  "bail": true,
+  "verbose": true,
+  "forceExit": true


We should validate Detox doesn't leave anything open, and that everything is being closed correctly.
Please remove forceExit

rotemmiz · 2018-03-06T20:42:56Z

detox/local-cli/detox-test.js

  cp.execSync(command, {stdio: 'inherit'});
 }

 function runJest() {
+  const currentConfiguration = config.configurations && config.configurations[program.configuration];
+  const maxTestWorkers = _.get(currentConfiguration, 'maxTestWorkers', 1);


Please use the same way to get configuration on all variables, either move all to _.get, use getConfigFor, or "vanilla"

rotemmiz · 2018-03-06T20:49:22Z

detox/src/devices/DeviceRegistry.js

+  }
+};
+
+const writeLockedDevices = lockedDevices => fs.writeFileSync(LOCK_FILE, JSON.stringify(lockedDevices));


please use functions instead of consts when declaring functions ☺️

rotemmiz · 2018-03-06T20:51:26Z

detox/src/devices/AppleSimUtils.js

+    const stdout = _.get(result, 'stdout');
+    const runtimes = JSON.parse(stdout);
+    const newestRuntime = _.maxBy(runtimes.runtimes, r => Number(r.version));
+    if (newestRuntime) {


What happens if there no newestRuntime ?

good catch. added handling

rotemmiz · 2018-03-06T20:53:33Z

detox/src/devices/DeviceRegistry.js

+
+  async _createDeviceIfNecessary ({deviceIds, deviceType}) {
+    const numberOfDevicesNeededToCreate = Math.max(this.maxTestRunners - deviceIds.length, 0);
+    _.times(numberOfDevicesNeededToCreate, async () => await this.createDevice(deviceType));


if xcrun simctl supports it, we may be able to create all devices simultaneously and use Promise.all to wait for all responses instead of doing them serially.

I'm not sure. It's Xcode after all.

LeoNatan

What was changed in Earl Grey?

LeoNatan · 2018-03-06T21:22:39Z

detox/test/package.json

-    "react": "16.2.0",
-    "react-native": "0.53.3"
+    "react": "16.0.0",
+    "react-native": "0.51.0"


I think this change is not necessary.

iOS RN 53 support is supposedly broken.

In what way?

LeoNatan · 2018-03-06T21:23:35Z

Also, if possible, I would like to understand what the strategy for device creation and reuse is. Perhaps you've discussed it, but I am not in the know.

rotemmiz · 2018-03-07T09:06:24Z

detox/src/devices/AppleSimUtils.js

+    const newestRuntime = _.maxBy(runtimes.runtimes, r => Number(r.version));
+    if (newestRuntime) {
+      console.log('Creating simulator', `create "${type}" "${type}" "${newestRuntime.identifier}"`)
+      await this._execSimctl({cmd: `create "${type}" "${type}" "${newestRuntime.identifier}"`});


name should not be the type:

const name = `detox_${type}_${count}` await this._execSimctl({cmd: `create "${name}" "${type}" "${newestRuntime.identifier}"`});

Now, when you change the name, you won't be able to query applesimutils by name anymore. @LeoNatan added query byType and byId. please check the implementation in:
https://github.com/wix/detox/compare/parallelSupport#diff-2ec04c0d120ceb57a189afbc6a59a9e8R27
https://github.com/wix/detox/compare/parallelSupport#diff-2ec04c0d120ceb57a189afbc6a59a9e8R39

rotemmiz · 2018-03-07T09:22:23Z

detox/src/devices/DeviceRegistry.js

+  }
+
+  async getDevice(deviceType) {
+    await retry(() => plockfile.lockSync(LOCK_FILE));


we may want to add Number.MAX_SAFE_INTEGER as retry amount, and reduce the exponential backoff timeouts to minimum.

rotemmiz · 2018-03-07T09:27:28Z

detox/src/devices/DeviceRegistry.js

+const plockfile = require('proper-lockfile');
+const _ = require('lodash');
+const retry = require('../utils/retry');
+const LOCK_FILE = './device.registry.state.lock';


To support multiple Detox runs (e.g. in a non virtualized CI machine) LOCK_FILE should be added globally. We already create a ~/Library/Detox/ when compiling the framework, I think LOCK_FILE should be there.

This reverts commit f1d8b48.

This reverts commit 87c357f.

This reverts commit 0c73826.

This reverts commit 00cd14b.

# Conflicts: # detox/local-cli/detox-test.js

rotemmiz · 2018-05-28T21:41:09Z

Still missing - DOCS!

rotemmiz · 2018-05-28T21:43:47Z

@doronpr, time to stress test some build agents.

Doron Pearl added 11 commits March 6, 2018 17:39

switch to e2e tests to jest

ae9c471

downgrade test project to RN51

57ef873

wait for device based on bootstatus

c54109e

introduce findDevicesUDID and use in findDeviceUDID

f5cd1b9

remove redundant check

3c26649

support multiple workers. for now set to 1

bdf4eb8

device registry

fb96349

# Conflicts: # detox/src/devices/AppleSimUtils.js # detox/test/package.json

device registry

bf31acb

# Conflicts: # detox/src/devices/AppleSimUtils.js # detox/test/package.json

fix device registry race condition

5c5d923

move maxTestWorkers param to detox config

f18c34c

clean up

5004898

doronpr requested review from LeoNatan and rotemmiz as code owners March 6, 2018 20:16

doronpr mentioned this pull request Mar 6, 2018

test sharding/parallelization #598

Closed

-forceExit

74849ae

rotemmiz requested changes Mar 6, 2018

View reviewed changes

LeoNatan added type: enhancement ⚡️ status: blocked labels Mar 6, 2018

LeoNatan reviewed Mar 6, 2018

View reviewed changes

LeoNatan suggested changes Mar 6, 2018

View reviewed changes

LeoNatan mentioned this pull request Mar 6, 2018

createPushNotificationJson creates a notification.json file under a constant path #601

Closed

rotemmiz reviewed Mar 7, 2018

View reviewed changes

rotemmiz requested changes Mar 7, 2018

View reviewed changes

Doron Pearl added 6 commits March 7, 2018 12:02

error when no runtime is available

4251edc

use dot notation

3bc6518

move lock file to ~/Library/Detox

f839c3d

adjust lock file retry options

fa268cd

use functions instead of consts when declaring functions

d5e7dab

use const for detox library root path

06b917e

rotemmiz and others added 22 commits May 12, 2018 17:27

added unit test for a new edge case

57a7b38

better visiblity on messages passing on a closed ws

efb39a6

WAT

a3c9e1e

only call currentStatus if ws is open

c9c4089

increase verbosity for debug purposes

9b51f72

per platfrom app data path

3a082bc

ensure file exists, create including path if needed

d76a43a

always query applesimutils byOS

3f14a38

Triggering PR build

f1d8b48

print stderrs of after all retries failed

c0393b0

print stderrs of after all retries failed

86b1b8a

Revert "Triggering PR build"

1084c20

This reverts commit f1d8b48.

Trigger PR after fixed jenkins

87c357f

Revert "Trigger PR after fixed jenkins"

e6743da

This reverts commit 87c357f.

Last try trigger PR

0c73826

Revert "Last try trigger PR"

00cd14b

This reverts commit 0c73826.

Revert "Revert "Last try trigger PR""

dd79249

This reverts commit 00cd14b.

Add empty line to package.json for triggering build

d91bd9c

Merge branch 'master' into test-parallelization

f7923af

update xcode

c6c90f4

Merge branch 'master' into test-parallelization

5830c96

# Conflicts: # detox/local-cli/detox-test.js

MOAR retries

c6357ae

rotemmiz merged commit 94348fc into master May 28, 2018

rotemmiz deleted the test-parallelization branch May 28, 2018 21:40

rotemmiz changed the title ~~test sharding/parallelization v2~~ Support parallel test execution May 28, 2018

rotemmiz mentioned this pull request May 31, 2018

Parallel Simulator support #97

Closed

dat00ste mentioned this pull request Jun 5, 2018

error in clearDeviceRegistryLockFile() when running detox test #771

Closed

wix locked and limited conversation to collaborators Jul 23, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support parallel test execution #609

Support parallel test execution #609

doronpr commented Mar 6, 2018

rotemmiz left a comment

rotemmiz Mar 6, 2018

rotemmiz Mar 6, 2018

rotemmiz Mar 6, 2018

doronpr Mar 7, 2018

rotemmiz Mar 6, 2018

doronpr Mar 7, 2018

rotemmiz Mar 6, 2018

doronpr Mar 7, 2018

rotemmiz Mar 6, 2018

LeoNatan Mar 6, 2018

LeoNatan left a comment

LeoNatan Mar 6, 2018

rotemmiz Mar 7, 2018

LeoNatan Mar 7, 2018

LeoNatan commented Mar 6, 2018

rotemmiz Mar 7, 2018

rotemmiz Mar 7, 2018

doronpr Mar 7, 2018

rotemmiz Mar 7, 2018

doronpr Mar 7, 2018

rotemmiz commented May 28, 2018

rotemmiz commented May 28, 2018

Support parallel test execution #609

Support parallel test execution #609

Conversation

doronpr commented Mar 6, 2018

rotemmiz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LeoNatan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LeoNatan commented Mar 6, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rotemmiz commented May 28, 2018

rotemmiz commented May 28, 2018