[Flask] bitrise pipelines #6645

owencraston · 2023-06-20T03:58:03Z

Development & PR Process

Follow MetaMask Mobile Coding Standards
Add release-xx label to identify the PR slated for a upcoming release (will be used in release discussion)
Add needs-dev-review label when work is completed
Add needs-qa label when dev review is completed
Add QA Passed label when QA has signed off

Description

Write a short description of the changes included in this pull request, also include relevant motivation and context. Have in mind the following questions,
1. What is the reason for the change?

This allows us to run flask builds/releases on bitrise
it will enable the smooth rollout of flask and should handle all the environment setup.
2. What is the improvement/solution?
modify the build.sh to build flask on android/ios in release mode. For this I followed the exact patterns that were already in place for the qa/release versions of the app.
created yarn scripts to call our build.sh with the correct arguments
created bitrise workflows and pipelines to create the builds for us
created a release pipeline that takes the builds and releases it to the stores.
updated the signing for MetaMask-Flask in xcode which modified the .xcodeproject file
Modified the sentry environment variable to handle the different types of flask releases
I was getting the error "Invalid App Store Icon. The App Store Icon in the asset catalog in 'MetaMask-Flask.app' can't be transparent nor contain an alpha channel. (ID: 9e93a511-109f-43f3-b3a5-a80ee40734cd)"; when I tried to upload to testflight so I needed to modify the ios app icons to remove the alpha.

Test Builds

https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/207891f7-53a6-4746-931e-dd2acb8f1fd1?tab=workflows
latest one with all the correct artifacts: https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/14942ea9-e976-43e5-93a2-3fd0fe87e12d

Issue

Progresses https://github.com/MetaMask/mobile-planning/issues/1044
Progresses https://app.zenhub.com/workspaces/metamask-accounts-team-62505028c3853700162f65e0/issues/gh/metamask/mobile-planning/1043

Checklist

There is a related GitHub issue
Tests are included if applicable
Any added code is fully documented

github-actions · 2023-06-20T03:58:16Z

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

owencraston · 2023-06-20T20:16:57Z

scripts/checksum.sh

@@ -8,7 +8,12 @@ FILE=./android/app/build/outputs/apk/qa/release/app-qa-release.apk
  if test -f "$FILE"; then
    shasum -a 512 "$FILE" > ./android/app/build/outputs/apk/qa/release/sha512sums.txt
  fi
+elif [ "$MODE" == "Flask" ]; then
+FILE=./android/app/build/outputs/apk/flask/release/app-flask-release.apk


Note: Is this the correct file?

scripts/build.sh

sethkfman · 2023-06-20T22:48:55Z

scripts/build.sh

+	echo "Remapping flask env variable names to match production"
+
+	# js.env variables
+	remapEnvVariable "MM_FLASK_PUBNUB_SUB_KEY" "MM_PUBNUB_SUB_KEY"


Remove PubNub keys

sethkfman · 2023-06-21T18:40:04Z

bitrise.yml

-            - pipeline_intermediate_files: ios/build/output/MetaMask-QA.ipa:BITRISE_APP_STORE_IPA_PATH
-            - deploy_path: ios/build/output/MetaMask-QA.ipa
+            - pipeline_intermediate_files: ios/build/output/MetaMask.ipa:BITRISE_APP_STORE_IPA_PATH
+            - deploy_path: ios/build/output/MetaMask.ipa


This will be MetaMask-Flask.ipa

I accidentally edited the QA steps here. I will remove these.

sethkfman · 2023-06-21T18:40:13Z

bitrise.yml

+          is_always_run: false
+          is_skippable: true
+          inputs:
+            - deploy_path: ios/build/MetaMask.xcarchive


Same name update here

sethkfman · 2023-06-21T18:40:29Z

bitrise.yml

-          inputs:     
-            - deploy_path: ios/build/MetaMask-QA.xcarchive
+          inputs:
+            - pipeline_intermediate_files: ios/build/output/MetaMask.ipa:BITRISE_APP_STORE_IPA_PATH


sethkfman · 2023-06-21T18:40:36Z

bitrise.yml

+          is_always_run: false
+          is_skippable: true
+          inputs:
+            - deploy_path: ios/build/MetaMask.xcarchive


bitrise.yml

scripts/build.sh

NicolasMassart · 2023-06-21T22:26:29Z

bitrise.yml

@@ -671,6 +690,115 @@ workflows:
            - pipeline_intermediate_files: sourcemaps/ios/index.js.map:BITRISE_APP_STORE_SOURCEMAP_PATH
            - deploy_path: sourcemaps/ios/index.js.map
          title: Deploy Source Map
+  build_ios_flask_release:


C: I wonder if we have a way to prevent duplication of these big yaml blocks and just have one called with params. they really mostly look the same.

@NicolasMassart regarding the sourcemap comment, yes they will be different. I am wondering if I need to change the sourcemap logic since it is failing on bitrise

sethkfman · 2023-06-28T22:10:58Z

app/util/sentryUtils.js

    const environment =
-      __DEV__ || !METAMASK_ENVIRONMENT ? 'development' : METAMASK_ENVIRONMENT;
+      __DEV__ || !METAMASK_ENVIRONMENT


Can you add tests with explicit output and comments? This thing is getting out of hand.

On extension we found it easier to disable Sentry entirely in development builds (see https://github.com/MetaMask/metamask-extension/blob/230c0c6fa1626bc1afd68c22321f5c166425dd12/app/scripts/lib/setupSentry.js#L88). Most of our Sentry testing was done with QA or production-like builds as part of regression testing. On the rare occasion that we wanted to test Sentry with a development build, it's easy to comment out that line.

That would reduce this condition to a single ternary:

const environment = METAMASK_BUILD_TYPE === 'main' ? METAMASK_ENVIRONMENT : `${METAMASK_ENVIRONMENT}-${METAMASK_BUILD_TYPE}`;

If we wanted to go that route here, and disable Sentry for development builds, that would be simpler to do in a separate PR. Better to keep this one focused if we can.

A more direct solution to cleaning this up for now would be to split this into two steps: determine the environment, then the sentry environment. e.g.

const metamaskEnvironment = __DEV__ || !METAMASK_ENVIRONMENT ? 'development' : METAMASK_ENVIRONMENT; const sentryEnvironment = METAMASK_BUILD_TYPE === 'main' ? METAMASK_ENVIRONMENT : `${METAMASK_ENVIRONMENT}-${METAMASK_BUILD_TYPE}`;

I have went ahead and modified this logic to live inside a function I am calling deriveSentryEnvironment.

/** * Derives the Sentry environment based on input parameters. * This function is similar to the environment logic used in MetaMask extension. * - https://github.com/MetaMask/metamask-extension/blob/34375a57e558853aab95fe35d5f278aa52b66636/app/scripts/lib/setupSentry.js#L91 * * @param {boolean} isDev - Represents if the current environment is development (__DEV__ global variable). * @param {string} [metamaskEnvironment='local'] - The environment MetaMask is running in * (process.env.METAMASK_ENVIRONMENT). * It defaults to 'local' if not provided. * @param {string} [metamaskBuildType='main'] - The build type of MetaMask * (process.env.METAMASK_BUILD_TYPE). * It defaults to 'main' if not provided. * * @returns {string} - The Sentry environment. Possible values are 'development', 'local', * 'production', or a string in the format `${metamaskEnvironment}-${metamaskBuildType}`. * 'development' is returned if 'isDev' is true or 'metamaskEnvironment' is not provided. * 'metamaskEnvironment' is returned if 'metamaskBuildType' is 'main' or undefined. * `${metamaskEnvironment}-${metamaskBuildType}` is returned for other cases, * for example 'production-flask' or 'debug-flask'. */ export function deriveSentryEnvironment( isDev, metamaskEnvironment = 'local', metamaskBuildType = 'main', ) { const environment = isDev || !metamaskEnvironment ? 'development' : metamaskBuildType === 'main' ? metamaskEnvironment : `${metamaskEnvironment}-${metamaskBuildType}`; return environment; }

This function is well tested in the new sentryUtils.test.ts file. I opted to not split up the meta mask environment and sentry environment to keep continuity with what we have already published in sentry. what would be the use case for having a separate metamaskEnvironment if we don't log it in sentry?

Gudahtt · 2023-07-07T18:27:06Z

app/util/sentry/sentryUtils.js

+  metamaskEnvironment = 'local',
+  metamaskBuildType = 'main',
+) {
+  const environment =


Nit: Thoughts on splitting this up onto multiple statements? The standard metamask ESLint config (not yet used here) disallows nested ternaries. They're a bit hard to read.

I have broken it up into multiple return statements in my latest commit.

Gudahtt · 2023-07-07T18:28:04Z

app/util/sentry/sentryUtils.js

+ *                                              (process.env.METAMASK_BUILD_TYPE).
+ *                                              It defaults to 'main' if not provided.
+ *
+ * @returns {string} - The Sentry environment. Possible values are 'development', 'local',


Nit: it might be better to include less specifics here, so we don't have to update it each time we add a new environment

Gudahtt · 2023-07-07T18:28:57Z

app/util/sentry/sentryUtils.js

+) {
+  const environment =
+    isDev || !metamaskEnvironment
+      ? 'development'


Should we include the build type even for development builds? Seems like it could be useful

Gudahtt · 2023-07-07T18:49:01Z

It's not yet clear to me whether the metric events are being flagged as being from Flask rather than from the main release. This is quite important. We rely on those metrics a lot, so we need to ensure we know precisely what they mean.

I'm not familiar with how metrics are setup on mobile at the moment, so I'm not sure how to approach this.

socket-security · 2023-07-11T23:36:50Z

🚨 Potential security issues detected. Learn more about Socket for GitHub ↗︎

To accept the risk, merge this PR and you will not be notified again.

Issue	Package	Version	Note
Filesystem access	fast-xml-parser	4.2.4	Module: fs Location: src/cli/cli.js
Mild CVE	fast-xml-parser	4.2.4	CVE: GHSA-gpv5-7x3g-ghjv fast-xml-parser regex vulnerability patch could be improved from a safety perspective (LOW)
Unmaintained	strnum	1.0.5	Last Publish: 12/4/2021, 10:58:54 AM

Next steps

What is filesystem access?

Accesses the file system, and could potentially read sensitive data.

If a package must read the file system, clarify what it will read and ensure it reads only what it claims to. If appropriate, packages can leave file system access to consumers and operate on data passed to it instead.

What is a mild CVE?

Contains a low severity Common Vulnerability and Exposure (CVE).

Remove or replace dependencies that include known low severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

What are unmaintained packages?

Package has not been updated in more than a year and may be unmaintained. Problems with the package may go unaddressed.

Package should publish periodic maintenance releases if they are maintained, or deprecate if they have no intention in further maintenance.

Take a deeper look at the dependency

Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support [AT] socket [DOT] dev.

Remove the package

If you happen to install a dependency that Socket reports as Known Malware you should immediately remove it and select a different dependency. For other alert types, you may may wish to investigate alternative packages or consider if there are other ways to mitigate the specific risk posed by the dependency.

Mark a package as acceptable risk

To ignore an alert, reply with a comment starting with @SocketSecurity ignore followed by a space separated list of package-name@version specifiers. e.g. @SocketSecurity ignore foo@1.0.0 bar@* or ignore all packages with @SocketSecurity ignore-all

@SocketSecurity ignore fast-xml-parser@4.2.4
@SocketSecurity ignore strnum@1.0.5

owencraston · 2023-07-13T00:34:56Z

The environment production-flask is working as expected

owencraston · 2023-07-13T18:17:21Z

I created a production testflight build of this branch (v 7.2.1 1145) and it appears that there are no "extra" environments in sentry with the new logic.

android/app/build.gradle

sethkfman · 2023-07-13T21:11:59Z

ios/MetaMask/MetaMask-Flask-Info.plist

@@ -128,14 +126,16 @@
 		<key>test</key>
 		<string>$(MM_BRANCH_KEY_TEST)</string>
 	</dict>
+	<key>branch_universal_link_domains</key>


we should be adding the iOS bundle identifier and android apk hash to branch or deep linking won't work

I will work with Andrea to get this working after we submit the builds.

sethkfman · 2023-07-13T21:12:42Z

scripts/build.sh

+
+	# js.env variables``
+	remapEnvVariable "FLASK_MOONPAY_API_KEY_STAGING" "MOONPAY_API_KEY_STAGING"
+    remapEnvVariable "SEGMENT_FLASK_DEV_KEY" "SEGMENT_DEV_KEY"


format error

owencraston changed the title ~~basic pipeline scaffold~~ [Flask] bitrise pipelines Jun 20, 2023

owencraston added 2 commits June 20, 2023 12:50

basic pipeline scaffold

99710ef

flask release ios

f621274

owencraston force-pushed the flask-pipelines branch from 5aa5768 to f621274 Compare June 20, 2023 19:51

owencraston added 2 commits June 20, 2023 13:11

android release scripts

95fda8a

check for sentry props on flask build

018ff06

owencraston commented Jun 20, 2023

View reviewed changes

scripts/build.sh Show resolved Hide resolved

owencraston added 6 commits June 20, 2023 13:57

bitrise certificate

3e0e16e

debug/release provisioning profiles

cd861a8

basic build steps

9b02f37

flask pipelines

47b75d2

fix yaml

50a3475

remap flask variables

5247447

sethkfman reviewed Jun 20, 2023

View reviewed changes

owencraston added 2 commits June 20, 2023 15:50

remove pubnub keys

5924596

add deploy pipelines

7b275c4

sethkfman reviewed Jun 21, 2023

View reviewed changes

bitrise.yml Show resolved Hide resolved

fix file paths and undo edits to qa build

56f03d4

owencraston force-pushed the flask-pipelines branch from a6a090c to 56f03d4 Compare June 21, 2023 19:06

owencraston commented Jun 21, 2023

View reviewed changes

bitrise.yml Show resolved Hide resolved

owencraston added 2 commits June 21, 2023 13:40

new flask release pipelines

69ccfe9

update set-versions script and try to fix sourcemaps for flask

061b507

NicolasMassart reviewed Jun 21, 2023

View reviewed changes

try a different sourcemp filepath

df55ec1

sethkfman reviewed Jun 28, 2023

View reviewed changes

pass params to derive sentry environment and tests

17b6758

Gudahtt reviewed Jul 7, 2023

View reviewed changes

owencraston and others added 2 commits July 11, 2023 14:42

new build numbers

2a71962

fix: update cookie-tough dependency (#6772)

37aeb8a

owencraston force-pushed the flask-pipelines branch from a985c90 to 37aeb8a Compare July 11, 2023 23:32

move versions into product flavours

80aa292

test prod build with flask variables

78c4a01

owencraston added 3 commits July 13, 2023 11:54

cleanup deriveSentryEnvironment function

da12ce0

cleanup comment

fe3069e

reset build numbers

f79aab3

owencraston commented Jul 13, 2023

View reviewed changes

android/app/build.gradle Show resolved Hide resolved

sethkfman reviewed Jul 13, 2023

View reviewed changes

owencraston added 7 commits July 13, 2023 15:09

enable deploy to android and pass in env variable

9490d89

new syntax

38d2dc6

is expand true

1da8ac1

no android pipeline

6dfcce6

new syntax

cbd72aa

new new syntax

889d066

new new new syntax

285b28c

owencraston merged commit 5ef0e5d into flask Jul 14, 2023

owencraston deleted the flask-pipelines branch July 14, 2023 18:20

github-actions bot locked and limited conversation to collaborators Jul 14, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Flask] bitrise pipelines #6645

[Flask] bitrise pipelines #6645

owencraston commented Jun 20, 2023 •

edited

Loading

github-actions bot commented Jun 20, 2023

owencraston Jun 20, 2023

sethkfman Jun 20, 2023

sethkfman Jun 20, 2023

sethkfman Jun 21, 2023

owencraston Jun 21, 2023

sethkfman Jun 21, 2023

sethkfman Jun 21, 2023

sethkfman Jun 21, 2023

NicolasMassart Jun 21, 2023

owencraston Jun 21, 2023

sethkfman Jun 28, 2023

Gudahtt Jun 29, 2023

Gudahtt Jun 29, 2023

owencraston Jul 4, 2023

Gudahtt Jul 7, 2023 •

edited

Loading

owencraston Jul 13, 2023

Gudahtt Jul 7, 2023

Gudahtt Jul 7, 2023

Gudahtt commented Jul 7, 2023

socket-security bot commented Jul 11, 2023

owencraston commented Jul 13, 2023

owencraston commented Jul 13, 2023

sethkfman Jul 13, 2023

owencraston Jul 14, 2023

sethkfman Jul 13, 2023

[Flask] bitrise pipelines #6645

[Flask] bitrise pipelines #6645

Conversation

owencraston commented Jun 20, 2023 • edited Loading

Test Builds

github-actions bot commented Jun 20, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Gudahtt Jul 7, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Gudahtt commented Jul 7, 2023

socket-security bot commented Jul 11, 2023

Next steps

owencraston commented Jul 13, 2023

owencraston commented Jul 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

owencraston commented Jun 20, 2023 •

edited

Loading

Gudahtt Jul 7, 2023 •

edited

Loading