Add --shard-all and --shard-split options to test command

[tokou]

Proposed changes

• Add broadcast option to test command
• This supersedes --shards but respects --device
• It runs all the flows on all the available devices
• This PR will be conflicting with Fix device selection #1953 quite a lot

Testing

• Launch two devices and run maestro test -b on a folder with multiple flows
• Make sure all flows run once on each device

Issues fixed

Fixes #1844

[bartekpacia]

@tokou Which PR do you think we should merge first? This one or #1953?

[tokou]

Let's start with this one.

[bartekpacia]

This LGTM, I'm happy to see such a useful change taking so little LoC! huge kudos:)

• what do you mean by "--broadcast options supersedes --shards"? I'd prefer to not introduce breaking changes.

• --broadcast sounds a bit confusing to me. I'd prefer sth different. Here are some ideas from chatgpt:

  

[tokou]

what do you mean by "--broadcast options supersedes --shards"? I'd prefer to not introduce breaking changes.

It's not breaking anything as it only affects cases where both --shards and --replicate are present (which can't already be the case)
In that case, as those are conflicting, we honor --replicate and ignore --shards

Here's a recap. Let's say we have 3 launched devices.

Arguments	Effect
<no args>	Prompts the user to choose 1 out of the 3 devices
--shards 2	Splits all tests across 2 out of the 3 devices
--replicate --shards 2	Runs all tests on all 3 devices
--device emulator-5554	Runs all tests on the specified device
--device emulator-5554 --shards 2	Runs all tests on the specified device
--device emulator-5554 --replicate	Runs all tests on the specified device
--device emulator-5554 --replicate --shards 2	Runs all tests on the specified device

[bartekpacia]

I see, this makes sense! Perhaps it'd be great to add this to the docs.

Also heads up that the E2E test is failing.

[bartekpacia]

Test failing is false positive from #2005

Details

[bartekpacia]

Another false positive from #2005. They are getting more and more crazy:

Details

[simakovroman23]

http://morykam.ru