Flank Scripts 1.9.18

flank-scripts-1.9.18

build: Dependencies updates [2021-06-14] (#2028)

Flank Scripts 1.9.17

flank-scripts-1.9.17

build: Dependencies updates [2021-06-07] (#2008)

Flank v21.06.0

v21.06.0

Bug Fixes

• #1994 Disable ASCII doc generation (release blocker) ()
• #1990 Puml diagram links (jan-gogo)
• #1981 Enable dump shards IT (Sloox)
• #1982 CIOEngineContainer not found (jan-gogo)
• #1967 Filter devices without supported versions (pawelpasterz)
• #1956 Annotation filtering for parameterized classes (asadsalman)
• #1955 Service account as default (adamfilipow92)
• #1943 Several minor corellium changes - part 3. (jan-gogo)
• #1936 Several minor corellium changes - part 2. (jan-gogo)
• #1933 Remove deprecated corellium adapters example (jan-gogo)
• #1900 Several minor issues in corellium scope (jan-gogo)

Features

• #1992 Disable window animation in tests by default (jan-gogo)
• #1991 Enable GPU acceleration for new instances (jan-gogo)
• #1968 Added support for JUnit Theory tests (asadsalman)
• #1970 Add test file column to result table (pawelpasterz)
• #1897 Domain layer implementation for android test run Corellium MVP (jan-gogo)
• #1947 Added max-test-shards and client-details to additional-app-test-apks (asadsalman)
• #1945 Integrate flank.jar with corellium CLI (jan-gogo)
• #1940 Corellium MVP CLI layer (jan-gogo)
• #1935 Public API of corellium domain layer (jan-gogo)
• #1923 Obfuscate shards (jan-gogo)
• #1921 Dump shards (jan-gogo)
• #1908 Add junit module (jan-gogo)
• #1880 Corellium adapters (jan-gogo)

Documentation

• #1977 Add user guide for Corellium MVP (jan-gogo)
• #1969 Add docs for Corellium modules (jan-gogo)
• #1961 Add implementation section to architecture doc (jan-gogo)
• #1954 Add architecture abstraction (jan-gogo)

CI Changes

• #1972 Update cla assitant (piotradamczyk5)
• #1949 Validate Maven version on publish (piotradamczyk5)
• #1942 Ubuntu updates (Sloox)
• #1934 Make IT results independent of os versions (Sloox, pawelpasterz)
• #1927 Remaining ubuntu-latest variables (pawelpasterz)
• #1918 Update wrong ubuntu variable name (pawelpasterz)

Refactor

• #1952 Data scratch ios (Sloox)
• #1937 Run doctor command (pawelpasterz, adamfilipow92)
• #1948 Move am instrument commad formatter to dedicated module (jan-gogo)
• #1950 Move apk parsers to dedicated module (jan-gogo)
• #1941 Refactor data scratch-Android run (piotradamczyk5)
• #1931 Add removing stack traces for simple report (piotradamczyk5)
• #1930 List android versions (Sloox)
• #1924 Structural output list ios versions (adamfilipow92)
• #1915 Structural output list network profiles (piotradamczyk5)
• #1919 Structural output list provided software (piotradamczyk5)
• #1920 Move shards calculation to separated module (jan-gogo)
• #1907 Structural output for list android models (adamfilipow92)
• #1911 Remove client classes for JUnit refactor (piotradamczyk5)
• #1914 IOS List locales (Sloox)
• #1916 Ip blocks output (pawelpasterz)
• #1909 Structural output orientations (piotradamczyk5)
• #1912 Move files for authentication data refactor (piotradamczyk5)
• #1910 Structural output for iOS model list (adamfilipow92)
• #1901 Data scratch - test matrix (adamfilipow92, piotradamczyk5)
• #1906 Describe ios versions (Sloox)
• #1902 Structural output cancel last run (piotradamczyk5)
• #1896 Logging refactor - android/ios test environment (pawelpasterz)
• #1875 Fetching artifacts (pawelpasterz)
• #1899 Android locales describes (Sloox)
• #1884 Refactor data scratch-Junit test result (piotradamczyk5)
• #1886 Describe ios locales (Sloox)
• #1895 Structural output iOS models describe (piotradamczyk5)
• #1894 List android locales (Sloox)
• #1890 Refactor android versions logging (pawelpasterz)
• #1887 Structural output Android models describe (piotradamczyk5)

Flank Scripts 1.9.16

build: update kotlin to 1.5.10 and remove warnings (#1965)

Fixes #1962

## Test Plan
> How do we know the code works?

Code compiles without any problem, there should not be any errors and warnings

Flank Scripts 1.9.15

build: Dependencies updates [2021-05-17] (#1938)

* [Automatic PR] Dependencies update

* Update flank-scripts version

* fix problem with building

Co-authored-by: piotradamczyk5 <piotradamczyk5@users.noreply.github.com>
Co-authored-by: Pawel Pasterz <pawel.pasterz@gmail.com>
Co-authored-by: Piotr Adamczyk <piotr.adamczyk@gogoapps.io>

Flank Scripts 1.9.14

flank-scripts-1.9.14

Make IT results independent from os versions (#1934)

Flank Scripts 1.9.13

flank-scripts-1.9.13

Fix remaining ubuntu-latest variables (#1927)

Flank v21.05.0

v21.05.0

Refactor

• #1885 Refactor DescribeNetworkProfiles logging (pawelpasterz)
• #1841 Data scratch file references (Sloox)
• #1878 Corellium client to functional style (jan-gogo)
• #1828 Data scratch getLocales (Sloox)
• #1840 Refactor data scratch-performance metrics (piotradamczyk5)
• #1833 Refactor os versions commands (adamfilipow92)
• #1834 Refactor data scratch-device model (piotradamczyk5)
• #1823 Refactor data scratch-softwarecatalog (Sloox)
• #1810 Refactor data scratch-remote storage (Sloox)
• #1819 Refactor data scratch network profiles (adamfilipow92)
• #1820 Refactor data scratch-orientation (piotradamczyk5)
• #1811 Refactor data scratch-ipblocks (piotradamczyk5)
• #1809 Refactor data scratch-authorization (piotradamczyk5)
• #1806 Rename corellium modules (pawelpasterz)

Bug Fixes

• #1883 Fix release job (piotradamczyk5)
• #1874 Add storage dir support (wclausen)
• #1829 CI issues on ubuntu (piotradamczyk5)
• #1807 Fix missing junit report in legacy mode (adamfilipow92)

Features

• #1877 TestApk parser (jan-gogo)
• #1879 API layer for corellium MVP (jan-gogo)
• #1835 Add sharding implementation for corellium (jan-gogo)
• #1824 Instrument test console log parser (pawelpasterz, jan-gogo)

Tests update

• #1842 Update task ordering (pawelpasterz)
• #1825 Optimize quota usage in IT (pawelpasterz)
• #1783 Add 3dmark instrumented test (jan-gogo)

CI Changes

• #1813 Disabled estimations copying (piotradamczyk5)

Documentation

• #1805 Fix information about project id (piotradamczyk5)

Flank Scripts 1.9.12

flank-scripts-1.9.12

ci: Disabled Estimation copying (#1813)

Flank Scripts 1.9.11

test: Add 3dmark instrumented test (#1783)

Fixes #1771 

The instrumented test based on ui-automator should run a benchmark on the tested device.
The benchmark must be installed on the tested device.

## Goal

Running the benchmark on FTL device and obtaining the benchmark results.

## Config

https://github.com/Flank/flank/blob/1771_ftl_benchmarks/test_runner/src/test/kotlin/ftl/fixtures/benchmark.yml

## Details

* The benchmark `apk` file is specified using `additional-apks` option
* The 3dmark benchmark results will be pulled from device dir `/sdcard/Android/data/com.futuremark.dmandroid.application/files/3DMarkAndroid/`

## Test Plan
> How do we know the code works?

copy
* `Geekbench 5_v5.3.2.apk`
* `3dmarkandroid-v2-1-4726.apk`

to `test_artifacts/1771_ftl_benchmarks/apk/benchmark/`

```bash
. .env
buildFlankScripts
flankScripts assemble app --artifact benchmark
cd test_runner
flank android run -c="./src/test/kotlin/ftl/fixtures/benchmark/*.yml"
```

# Devices

* Pixel 5e (physical) - API 30
* NexusLowRes (virtual) - API 30
* NexusLowResEmulator (emulator) - API 30

# Results

## 3dmarkandroid-v2-1-4726.apk

* Pixel 5e (physical) - API 30 - [passed](https://gist.github.com/jan-gogo/794a01631645f7a1549f82334eb2fba4) https://github.com/Flank/flank/pull/1783/commits/43dfc014fc3f23c309aa81de8ac0af104b104a0c [video](https://console.firebase.google.com/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/4704820152614748057/executions/bs.458fb96e63f99c51/videos)
* NexusLowRes (virtual) - API 30 - [failed](https://gist.github.com/jan-gogo/e3e4e7d80b1b65d550300c1ecb75498b) https://github.com/Flank/flank/pull/1783/commits/180e154e43546958d007417b4ab3bdb879880786
* NexusLowResEmulator (emulator) - API 30 - [freezed](https://gist.github.com/jan-gogo/683f1477b876649416d4366f6d1b005f) https://github.com/Flank/flank/pull/1783/commits/bc1a2f6a13b55e2cb6e3d0e6e234311b6eb2c942

## Geekbench - [all passed](https://gist.github.com/jan-gogo/64700f34a5f7511aeecffe6da44a7d07) https://github.com/Flank/flank/pull/1783/commits/836e6221df89f996887a1fa6bcdf21d31c2e4a52 [firebase](https://console.firebase.google.com/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/7399025084907015803)

* Pixel 5e (physical) - API 30 - [video](https://console.firebase.google.com/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/7399025084907015803/executions/bs.e9ea758006f16bee/videos)
* NexusLowRes (virtual) - API 30 - [video](https://console.firebase.google.com/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/7399025084907015803/executions/bs.7ee7ba87833f43ab/videos)
* NexusLowResEmulator (emulator) - API 30 - [video](https://console.firebase.google.com/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/7399025084907015803/executions/bs.2d604e5ac73159aa/videos)

## PassMark

* NexusLowRes (virtual) - API 30
* NexusLowResEmulator (emulator) - API 30 - starting but failing during the benchmark - [video](https://console.firebase.google.com/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/5356711449268917714/executions/bs.a612c1ef9c1df07/videos) https://github.com/Flank/flank/pull/1783/commits/b239766ffa471ab4055522aa992227e44302def9

## Checklist

- [ ] Documented