[NoQA] Increase number of e2e test runs

[ospfranco]

Details

It appears e2e tests are miss-detecting performance regressions. Will try to debug the issues. First idea is to increase the numbers of iterations to try to reduce statistical noice.

Fixed Issues

$ #28735 (comment)
PROPOSAL:

Tests

• Verify that no errors appear in the JS console

Offline tests

QA Steps

• Verify that no errors appear in the JS console

PR Author Checklist

• I linked the correct issue in the ### Fixed Issues section above
• I wrote clear testing steps that cover the changes made in this PR
  • I added steps for local testing in the Tests section
  • I added steps for the expected offline behavior in the Offline steps section
  • I added steps for Staging and/or Production testing in the QA steps section
  • I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
  • I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
  • I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).
• I included screenshots or videos for tests on all platforms
• I ran the tests on all platforms & verified they passed on:
  • Android / native
  • Android / Chrome
  • iOS / native
  • iOS / Safari
  • MacOS / Chrome / Safari
  • MacOS / Desktop
• I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
• I followed proper code patterns (see Reviewing the code)
  • I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
  • I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
  • I verified that comments were added to code that is not self explanatory
  • I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
  • I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
    • If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:
  • I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
  • I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
  • I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
  • I verified the JSDocs style guidelines (in STYLE.md) were followed
• If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
• I followed the guidelines as stated in the Review Guidelines
• I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
• I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
• I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
• I verified that if a function's arguments changed that all usages have also been updated correctly
• If a new component is created I verified that:
  • A similar component doesn't exist in the codebase
  • All props are defined accurately and each prop has a /** comment above it */
  • The file is named correctly
  • The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
  • The only data being stored in the state is data necessary for rendering and nothing else
  • If we are not using the full Onyx data that we loaded, I've added the proper selector in order to ensure the component only re-renders when the data it is using changes
  • For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
  • Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
  • All JSX used for rendering exists in the render method
  • The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions
• If any new file was added I verified that:
  • The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory
• If a new CSS style is added I verified that:
  • A similar style doesn't already exist
  • The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))
• If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
• If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
• If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
• If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
• If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
• If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
• I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

[melvin-bot]

@pecanoro Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

[Julesssss]

maybe I'm missing something but wouldn't this reduce accuracy?

I mistakenly thought this was reducing from 90 > 30

[mountiny]

I don think this should reduce accuracy, you will have more attempts so the average will be more accurate in theory.

However, does this mean the tests will run 3 times longer. It already runs quite a bit and we dont have that many tests

[ospfranco]

@Julesssss systems stabilize over time, it's like flipping a coin more times, over time the real tendency (e.g. 50-50) emerges from the noise.

[Julesssss]

Ignore me, I thought we were REDUCING from 90 to 30 😅

[AndrewGable]

Reviewer Checklist

• I have verified the author checklist is complete (all boxes are checked off).
• I verified the correct issue is linked in the ### Fixed Issues section above
• I verified testing steps are clear and they cover the changes made in this PR
  • I verified the steps for local testing are in the Tests section
  • I verified the steps for Staging and/or Production testing are in the QA steps section
  • I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
  • I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
• I checked that screenshots or videos are included for tests on all platforms
• I included screenshots or videos for tests on all platforms
• I verified tests pass on all platforms & I tested again on:
  • Android / native
  • Android / Chrome
  • iOS / native
  • iOS / Safari
  • MacOS / Chrome / Safari
  • MacOS / Desktop
• If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
• I verified proper code patterns were followed (see Reviewing the code)
  • I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
  • I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
  • I verified that comments were added to code that is not self explanatory
  • I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
  • I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
  • I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
  • I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
  • I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
  • I verified the JSDocs style guidelines (in STYLE.md) were followed
• If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
• I verified that this PR follows the guidelines as stated in the Review Guidelines
• I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
• I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
• I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
• If a new component is created I verified that:
  • A similar component doesn't exist in the codebase
  • All props are defined accurately and each prop has a /** comment above it */
  • The file is named correctly
  • The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
  • The only data being stored in the state is data necessary for rendering and nothing else
  • For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
  • Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
  • All JSX used for rendering exists in the render method
  • The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions
• If any new file was added I verified that:
  • The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory
• If a new CSS style is added I verified that:
  • A similar style doesn't already exist
  • The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)
• If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
• If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
• If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
• If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
• If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
• If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
• I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

N/A - Just tests

Mobile Web - Chrome

N/A - Just tests

Mobile Web - Safari

N/A - Just tests

Desktop

N/A - Just tests

iOS

N/A - Just tests

Android

N/A - Just tests

[AndrewGable]

I don't think test time should go up by any significant amount. 95% of the time related to the test is building the APK and getting AWS spun up.

[OSBotify]

✋ This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

[github-actions]

Performance Comparison Report 📊

Significant Changes To Duration

Name	Duration
App start TTI	1261.770 ms → 1347.283 ms (+85.514 ms, +6.8%) 🔴
App start runJsBundle	865.708 ms → 923.185 ms (+57.477 ms, +6.6%) 🔴

Show details

Name	Duration
App start TTI	Baseline Mean: 1261.770 ms Stdev: 44.057 ms (3.5%) Runs: 1152.571913999971 1158.073967000004 1160.8446439998224 1168.6349269999191 1184.3265789998695 1188.5767899998464 1191.8809879999608 1193.3665109998547 1207.0709600001574 1212.6236009998247 1214.6063680001535 1215.9482330000028 1220.556539000012 1220.7886459999718 1221.703306000214 1224.5385420001112 1225.2490539997816 1226.361949000042 1227.3351400000975 1233.3232399998233 1236.4474220001139 1237.269838000182 1237.3326059998944 1239.526889000088 1240.2268260000274 1241.6531489999034 1242.4520939998329 1243.6261970000342 1243.784810999874 1244.0122679998167 1246.7607040000148 1248.5233539999463 1248.9454040001146 1249.118265000172 1249.8423799998127 1250.9992410000414 1251.1696600001305 1251.7570279999636 1253.5189290000126 1253.5499459998682 1253.8342160000466 1254.1507430002093 1257.0191709999926 1257.068463999778 1260.2702270001173 1260.5165630001575 1261.2616059998982 1261.3577939998358 1261.4161649998277 1263.4494210002013 1267.9895130000077 1268.7896409998648 1269.157718999777 1270.5349809997715 1277.436089000199 1277.7927430002019 1277.8476929999888 1278.7290670000948 1278.740075000096 1279.4155689999461 1283.4272010000423 1284.2207590001635 1285.3040660000406 1285.3058449998498 1289.8008500002325 1289.8959550000727 1292.5234210002236 1293.669964000117 1294.5752579998225 1297.0070099998266 1297.7863599997945 1297.9664960000664 1299.363985999953 1299.6022569998167 1306.5012059998699 1309.3996749999933 1311.8159540002234 1312.494878000114 1314.152236000169 1318.7414899999276 1321.5376079999842 1321.698080000002 1330.3130769999698 1331.3766839997843 1339.2535520000383 1341.0796570000239 1345.6652409997769 1349.3909979998134 1357.9648270001635 Current Mean: 1347.283 ms Stdev: 75.812 ms (5.6%) Runs: 1186.7044139998034 1199.6788090001792 1200.852487000171 1219.8926789998077 1222.262219000142 1223.0677959998138 1230.7037209998816 1236.5966520002112 1236.9856289997697 1241.8411559998058 1247.4285539998673 1251.296287999954 1251.536803000141 1252.194029000122 1256.0877660000697 1258.5224689999595 1259.359501000028 1271.085735999979 1271.1530169998296 1272.3295240001753 1273.2203270001337 1275.6887480001897 1276.6143299997784 1281.677472999785 1284.7479900000617 1286.7843829998747 1290.126771000214 1293.4281580001116 1302.5839619999751 1303.7399590001442 1308.6741470000707 1309.002237000037 1313.91169800004 1319.8063300000504 1327.0241499999538 1329.2259550001472 1333.8711489997804 1334.9702519997954 1339.1849480001256 1345.0126120001078 1348.152747000102 1348.3488579997793 1349.1335109998472 1350.76495099999 1352.5820209998637 1355.3057200000621 1355.9368389998563 1359.042330000084 1367.2878760001622 1368.838746999856 1371.70804399997 1372.563360999804 1372.9720859997906 1373.204969999846 1376.3665000000037 1382.9197990000248 1385.728124000132 1385.935986999888 1387.0692799999379 1388.6266589998268 1391.6307810000144 1391.9348829998635 1394.8197960001417 1398.017930999864 1398.593063000124 1402.5463780001737 1403.0221210001037 1404.2197770001367 1407.289218999911 1409.6611410002224 1410.1263489997946 1412.2517099999823 1416.139893999789 1416.5805290001445 1416.822856000159 1416.9020569999702 1417.6227819998749 1417.9869470000267 1427.8897259999067 1431.1733470000327 1435.389756000135 1444.7738330001011 1444.7785979998298 1446.2108160001226 1447.9092450002208 1452.0977480001748 1453.7124950001016 1459.1084340000525 1459.749818999786 1468.30589699978 1488.7041659997776 1492.7343100002035
App start runJsBundle	Baseline Mean: 865.708 ms Stdev: 37.418 ms (4.3%) Runs: 795 797 799 800 805 807 808 813 815 815 818 821 823 824 827 829 832 833 836 838 839 840 841 843 844 846 849 849 852 853 854 855 855 855 857 857 858 858 858 859 859 860 861 861 862 862 864 865 867 868 868 868 868 869 869 870 873 875 877 877 880 880 884 886 887 887 887 889 890 893 894 895 899 901 904 907 912 915 918 918 920 921 926 927 929 938 952 953 956 Current Mean: 923.185 ms Stdev: 59.921 ms (6.5%) Runs: 801 818 827 833 833 834 835 837 840 843 843 844 854 855 856 857 858 859 859 860 861 865 871 872 874 881 883 886 887 887 889 894 895 895 899 900 901 903 903 905 913 914 917 926 927 930 932 932 933 936 936 939 940 940 943 944 946 948 949 951 952 955 961 961 964 966 967 969 969 970 971 973 974 977 978 982 984 986 990 992 996 997 999 1002 1003 1008 1008 1012 1015 1035 1058 1066

Meaningless Changes To Duration

Show entries

Name	Duration
Open Search Page TTI	646.311 ms → 649.920 ms (+3.609 ms, +0.6%)
App start nativeLaunch	22.722 ms → 24.045 ms (+1.323 ms, +5.8%)
App start regularAppStart	0.018 ms → 0.017 ms (-0.001 ms, -7.5%)

Show details

Name	Duration
Open Search Page TTI	Baseline Mean: 646.311 ms Stdev: 29.099 ms (4.5%) Runs: 594.4797359998338 603.0745859998278 606.5936280000024 607.4591479999945 607.7386070000939 608.1322439997457 608.1686200001277 612.3164470000193 612.7474770001136 614.1580400001258 615.443847999908 616.0246179997921 617.4674480003305 618.5134689998813 618.6749680000357 619.8400880000554 619.9148770002648 620.6921800002456 621.1623540003784 621.7905269996263 621.8811440002173 621.9268799996935 622.0540769998915 622.1523440000601 622.5838630003855 623.1387940002605 624.6427420000546 627.5242110001855 627.8362229997292 629.3308109999634 630.1888839998282 630.6628020000644 631.8779299999587 633.505858999677 633.5394699997269 633.7228600000963 634.1234130002558 636.0782469999976 636.2137449998409 636.3740240000188 636.7003579996526 637.771646999754 637.803915000055 638.975260999985 638.9976400001906 640.0999759999104 640.7900800001808 640.8569339998066 640.9216720000841 644.3904220000841 644.9907230003737 645.5239659999497 645.6376549997367 646.1580810002051 648.4335130001418 649.5994060002267 650.2223720001057 650.6821290003136 654.9318039999343 655.8695479999296 656.1217859997414 656.6114509999752 656.925333999563 657.4986169999465 660.563355000224 660.7654219996184 661.8610840002075 663.2565510002896 663.2609049999155 664.1013589999638 665.9739180002362 666.4891360001639 666.8174639996141 668.8162440001033 669.1370040001348 670.0706380000338 670.2609049999155 673.4877530001104 681.9936119997874 682.1848559998907 683.7573240003549 685.5311690000817 685.8540449999273 686.0140789998695 686.9396979999729 687.7323809997179 695.509887999855 712.1129959998652 720.394125000108 720.8723559998907 722.8704830002971 723.6796479998156 Current Mean: 649.920 ms Stdev: 28.788 ms (4.4%) Runs: 601.9095060001127 603.2066660001874 610.3475350001827 610.9448650004342 614.702432999853 618.6840419997461 618.7227779999375 619.2855630000122 620.6664630002342 620.9418950001709 622.0264900000766 625.6176759996451 626.2367760003544 626.6545819998719 626.8079430004582 627.7261560000479 628.0892749996856 628.1039229999296 628.290324000176 628.4044600003399 628.5753990001976 628.7889000000432 628.8679209998809 628.8699139999226 629.4696450000629 630.1391199999489 630.3905849996954 630.4213870000094 630.9281419999897 631.6912439996377 632.409260999877 632.9519859999418 634.0598959997296 634.9715980002657 634.9925939999521 636.2550860000774 636.2897959998809 636.3302410002798 636.503337000031 636.590046999976 637.2211099998094 637.7121179997921 638.0544030000456 638.8854980003089 639.8683280004188 639.9110929998569 640.0561529998668 640.4286700002849 641.5869139996357 643.3706459999084 643.3732099998742 643.5708010001108 644.6612140000798 645.7291259998456 647.446248000022 648.4101569997147 652.0980630000122 652.7586680003442 652.8699539997615 654.1940520000644 654.9175619999878 655.2404789999127 656.5023199999705 659.676188999787 660.0132250003517 660.6974689997733 661.2138669998385 665.2078859996982 665.9170329999179 667.9554039998911 670.0956219998188 670.7369389999658 671.5518400003202 671.7504879999906 674.0283200000413 675.846802000422 677.7058930001222 679.8201910001226 686.3337000003085 687.2789309998043 689.1316730002873 690.0615240000188 694.8969319998287 698.5316570000723 698.5660410001874 699.669637999963 703.8960370002314 707.5823169997893 713.818726000376 722.6642669998109 731.0957439998165 731.1569419996813
App start nativeLaunch	Baseline Mean: 22.722 ms Stdev: 3.310 ms (14.6%) Runs: 19 19 19 19 19 19 19 19 19 19 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 22 22 22 22 22 22 23 23 23 23 23 23 23 23 24 24 24 24 25 25 25 25 25 26 26 27 27 27 27 27 27 28 28 28 28 28 28 28 29 29 30 31 32 Current Mean: 24.045 ms Stdev: 2.727 ms (11.3%) Runs: 20 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 21 22 22 22 22 22 22 22 22 22 22 22 22 22 22 23 23 23 23 23 23 23 23 23 24 24 24 24 24 24 24 24 24 24 24 24 24 24 25 25 25 25 25 25 25 25 26 26 26 26 26 26 26 26 26 26 26 27 27 28 28 28 28 29 29 29 29 30 30 31 32
App start regularAppStart	Baseline Mean: 0.018 ms Stdev: 0.002 ms (8.6%) Runs: 0.015461999922990799 0.015705999918282032 0.01599099999293685 0.01599099999293685 0.0159919997677207 0.015992000233381987 0.01607299968600273 0.016153999604284763 0.01615400006994605 0.016276000067591667 0.016358000226318836 0.01643899967893958 0.016439000144600868 0.0165200000628829 0.016600999981164932 0.01660200022161007 0.0166830001398921 0.0166830001398921 0.01672299997881055 0.016805000137537718 0.0170889999717474 0.017089999746531248 0.017171000130474567 0.01729400036856532 0.01733400020748377 0.017414999660104513 0.0174150001257658 0.01741599990054965 0.01741599990054965 0.017456000205129385 0.017456000205129385 0.017496999818831682 0.017578000202775 0.0176189998164773 0.017780999653041363 0.01778100011870265 0.017862999811768532 0.01786300027742982 0.017943999730050564 0.01794400019571185 0.01794400019571185 0.017985000275075436 0.017985000275075436 0.018025999888777733 0.01814799988642335 0.01814799988642335 0.018188000191003084 0.018188999965786934 0.018188999965786934 0.018188999965786934 0.018188999965786934 0.018269000109285116 0.018270000349730253 0.01831099996343255 0.01831099996343255 0.01839200034737587 0.018432999961078167 0.01847400004044175 0.018635999877005816 0.0186769999563694 0.0188400000333786 0.018881000112742186 0.018881000112742186 0.018920999951660633 0.018962000031024218 0.01904299994930625 0.019204999785870314 0.019245999865233898 0.019247000105679035 0.019367999862879515 0.019368999637663364 0.019491000100970268 0.019531000405550003 0.019694000016897917 0.019816000014543533 0.019979000091552734 0.020223000086843967 0.020262999925762415 0.02038499992340803 0.0206300001591444 0.020791999995708466 0.020995999686419964 0.021525000222027302 0.021605999674648046 0.022013999987393618 0.022298000287264585 0.022338999900966883 Current Mean: 0.017 ms Stdev: 0.001 ms (7.5%) Runs: 0.01428299956023693 0.014689000323414803 0.014769999776035547 0.014769999776035547 0.015217999927699566 0.015299999620765448 0.015339999925345182 0.015381000004708767 0.015463000163435936 0.015463000163435936 0.015544000081717968 0.015625 0.015625 0.015625 0.015664999838918447 0.015664999838918447 0.015705999918282032 0.015705999918282032 0.01570699969306588 0.01570700015872717 0.015746999997645617 0.015746999997645617 0.0157880000770092 0.015829000156372786 0.015910000074654818 0.01599099999293685 0.01599099999293685 0.016030999831855297 0.016193999908864498 0.016195000149309635 0.016195000149309635 0.016234999988228083 0.01627599960193038 0.016315999906510115 0.01631700014695525 0.01635799976065755 0.016439000144600868 0.016478999983519316 0.016480000223964453 0.016560999676585197 0.01660200022161007 0.0166830001398921 0.016724000219255686 0.016764000058174133 0.01680499967187643 0.016805000137537718 0.016844999976456165 0.016846000216901302 0.016846000216901302 0.01688600005581975 0.016887000296264887 0.016927000135183334 0.016966999974101782 0.017130999825894833 0.017130999825894833 0.01713100029155612 0.017171000130474567 0.017171999905258417 0.01725299982354045 0.017253000289201736 0.017293000128120184 0.017293999902904034 0.01733399974182248 0.017375000286847353 0.017414999660104513 0.017456000205129385 0.017496999818831682 0.017537000123411417 0.017577999737113714 0.01769999973475933 0.017700000200420618 0.017700000200420618 0.0177819998934865 0.017822999972850084 0.01786300027742982 0.017943999730050564 0.018188999965786934 0.0185139998793602 0.0185139998793602 0.0185139998793602 0.018554999958723783 0.0186769999563694 0.019286999944597483 0.0197350000962615 0.019857000093907118 0.019897999707609415 0.01993800001218915 0.02030499977990985

[github-actions]

@Expensify/mobile-deployers 📣 Please look into this performance regression as it's a deploy blocker.

[OSBotify]

🚀 Deployed to staging by https://github.com/AndrewGable in version: 1.3.78-0 🚀

platform	result
🤖 android 🤖	success ✅
🖥 desktop 🖥	success ✅
🍎 iOS 🍎	success ✅
🕸 web 🕸	success ✅

[OSBotify]

🚀 Deployed to production by https://github.com/Beamanator in version: 1.3.78-4 🚀

platform	result
🤖 android 🤖	success ✅
🖥 desktop 🖥	success ✅
🍎 iOS 🍎	failure ❌
🕸 web 🕸	success ✅

[ospfranco]

@Julesssss no no, we are going from 30 → 90

@AndrewGable After further testing on my local environment I can definitely see a performance regression with the tests running on a single emulator instance. I can imagine the same is happening on the cloud instance of the device. While increasing the runs will get rid of statistical noise there is clearly an issue where running multiple tests on a single instance degrades performance of the device over time.

[OSBotify]

🚀 Deployed to staging by https://github.com/AndrewGable in version: 1.3.79-0 🚀

platform	result
🤖 android 🤖	success ✅
🖥 desktop 🖥	success ✅
🍎 iOS 🍎	success ✅
🕸 web 🕸	success ✅

[OSBotify]

🚀 Deployed to production by https://github.com/jasperhuangg in version: 1.3.79-5 🚀

platform	result
🤖 android 🤖	success ✅
🖥 desktop 🖥	failure ❌
🍎 iOS 🍎	failure ❌
🕸 web 🕸	success ✅

[OSBotify]

🚀 Deployed to production by https://github.com/jasperhuangg in version: 1.3.79-5 🚀

platform	result
🤖 android 🤖	success ✅
🖥 desktop 🖥	cancelled 🔪
🍎 iOS 🍎	failure ❌
🕸 web 🕸	success ✅

[OSBotify]

🚀 Deployed to production by https://github.com/jasperhuangg in version: 1.3.79-5 🚀

platform	result
🤖 android 🤖	success ✅
🖥 desktop 🖥	success ✅
🍎 iOS 🍎	failure ❌
🕸 web 🕸	success ✅