feat: add rule-based filtering for messages

[philprime]

Objective

Implement a system of rules and conditions which can be configured for each message to conditionally include them when fetching messages from clients.

Motivation

App users have requested us to notify them about app updates, in case they do not have automatic app updates enabled. Therefore, we should check if the currently installed version is older than the latest available version, and include a message with an action to open the mobile App Store and update the app.

A similar mechanism has been implemented in LaunchGate and Gandalf, but they are specific solutions for this problem.

Design Proposal

During a concept meeting on May 13th 2024 between @flohoch @DamianJaeger and myself, we discussed the relevant aspects of solving this issue.

The approach of implementing this functionality as a separate component of OnLaunch was discarded in favour of a general solution, using rules to handle different versions on different platforms (i.e. 1.5.2 on iOS and 840 on Android), based on the following information:

• The machine-readable version on iOS is stored in the Info.plist for the key CFBundleVersion. This version is not shown to users but used in the system internally (in contrast to the human-readable version stored with CFBundleShortVersionString. The version can only contain numbers and dots, with up to three components major.minor.patch. When a component is missing, a component with 0 is appended, i.e. 10 is 10.0.0, 10.14 is 10.14.0 and 10.14.2 stays the same.
• Versioning on Android is done using a version code, which is a monotonic growing number (and can be seen as a semantic version with only a major component)

The goal is using this information and displaying a blocking message with an action "to update the app", or a non-blocking message with the same action as well as a "dismiss" action.

This leads to the following requirements:

• Message actions are extended with an action type OPEN_IN_APP_STORE, which will open the on-device Apple App Store or Google Play Store page of the app.
• Each message can be configured with a set of conditions in boolean logic in a RULE
• A RULE is an OR-BLOCK AND-BLOCK
• AND-BLOCKs and OR-BLOCKs can contain nested blocks or CONDITIONs
• AND-BLOCKs require all conditions to evaluate to truthy
• OR-BLOCKs require at least on condition to evaluate to truthy
• CONDITIONs consist of a pre-defined SYSTEM-VARIABLE, an OPERATOR and a pre-defined/manual USER-VARIABLE
• USER-VARIABLEs are pre-defined based on the SYSTEM-VARIABLE in the front-end
• A message without rules is not evaluated

This should allow to configure a single message for multiple platforms to be displayed. Example (platform IS iOS AND version IS_OLDER_THAN <latest_iOS_version>) OR (platform IS Android AND version IS_OLDER_THAN <latest_android_version>) is implemented as the following rule:

- Rule / AND-BLOCK:
   - OR-BLOCK
      - AND-BLOCK
         - CONDITION
            - PLATFORM      
            - IS_EQUAL
            - "iOS"
         - CONDITION     
            - VERSION
            - IS_EQUAL
            - "1.5.2"
      - AND-BLOCK
         - CONDITION
            - PLATFORM
            - IS_EQUAL
            - "Android"
         - CONDITION     
            - VERSION
            - IS_EQUAL
            - "840"

The evaluation of rules is performed on the server-side, to allow further extension of the implementation in the future, and to reduce implementation work in clients.

To perform evaluation, the context is necessary, including at least the following information from the client:

• Bundle ID or Package Name (i.e. app.kula.onlaunch)
• App Version (i.e. 1.5.2)
• Platform Name (i.e. iOS)
• Platform Version (i.e. 17.4)
• Google Play App Update Available (only for Android, see explanation below)

This context is sent by the client with the GET /messages request as HTTP Headers, i.e. X-ONLAUNCH-APP-VERSION or X-ONLAUNCH-PLATFORM-NAME. If a value is not available, or not sent by the client in older library versions, the rule-evaluation is cancelled and the message is not included in the response.

The rules can be configured in the frontend in two ways:

• Simple Rule Editor
• Advanced Rule Editor

The Simple Rule Editor implements common use-cases such as "Minimum Required App Version must be X" or "Display Only On Android". These common-use cases can be extended over time as requested by users of OnLaunch. The configured rules are translated by the frontend client to the same structure, but simplified in the UI for better UX. This UI will also be extended with platform-specific user input validation, i.e. semantic version for iOS.

The Advanced Rule Editor allows complex configuration of rules, but expects more technical expertise from the user.

To match the original motivation, this would require an OnLaunch user to update the USER-VARIABLEs with every app release, to match the latest app version. To resolve this, special USER-VARIABLE values are introduced, especially the value $latest.

If the value is set to $latest the server-side of OnLaunch will fetch the latest available app version from the App Stores and use it to evaluate the condition.

The Apple App Store is built on-top of the iTunes Store, therefore it is possible to fetch the latest version using the iTunes Lookup endpoint, as discussed on StackOverflow. The lookup is performed using the bundle identifier sent in the app context, so that OnLaunch apps can be used for multiple apps with different bundle identifiers at the same time.

For Google Play Console, it is not possible to fetch the latest version via a public endpoint. Existing solutions try to read the version using HTML parsing of the public Google Play Page, which is not sufficiently stable enough. Instead, Google offers a library to check for in-app-updates directly on device from the Google Play Store. The library will be used on the client to determine if a new app update is available, and send as a boolean flag with the app context.

During the concept meeting we decided to use a single system variable VERSION and select the relevant remote value based on the evaluation context, but now looking at it, it's probably necessary to use platform-specific variables instead

The lookup response of the Apple App Store is cached using Redis for 15 minutes, to improve response times of the messages endpoint.

The OnLaunch Admin API will also be extended with CRUD operations for rules per message.

User Impact

Developer for Apple platforms, specifically iOS, will have to configure the Apple App Store ID at the setup of OnLauch. It will not be possible to open the App Store without it, therefore showing an error message instead.

Discussion

Further discussion will be handled in this issue, and tasks will be distributed to the relevant repositories. RFC @DamianJaeger if I missed any relevant points, after that I'll create subtasks in separate issues.

Sub-Tasks

Backend:

• Define and implement rule schema and evaluation #983
• Adapt message fetching endpoint to handle HTTP headers with rule evaluation #984
• Implement latest version fetching via iTunes Lookup API #985
• Implement special value $latest as user-variable in rule system #986

Frontend:

• Create simple rule editor in message editor #987
• Create advanced rule editor in message editor #988
• Extend message editor with action OPEN_APP_IN_APP_STORE #989

Clients:

• Add app context to fetching messages for rule evaluation OnLaunch-Android-Client#12
• Add handling for action OPEN_APP_IN_APP_STORE OnLaunch-Android-Client#13
• Integrate Google Play Core Library to check for in-app updates OnLaunch-Android-Client#14
• Send a boolean flag indicating if a new app update is available in the context headers  OnLaunch-Android-Client#15
• Add app context to fetching messages for rule evaluation OnLaunch-iOS-Client#43
• Add handling for action OPEN_APP_IN_APP_STORE OnLaunch-iOS-Client#44
• Add app context to fetching messages for rule evaluation OnLaunch-Flutter-Client#8
• Add handling for action OPEN_APP_IN_APP_STORE OnLaunch-Flutter-Client#9
• Integrate Google Play Core Library to check for in-app updates OnLaunch-Flutter-Client#10
• Send a boolean flag indicating if a new app update is available in the context headers OnLaunch-Flutter-Client#11

Admin API

• Implement Create, Read, Update, and Delete operations for rules per message. #990

[philprime]

RFC @Rasakul

Would like to hear your opinion on the system design, as you mentioned before that you also need rules for messages.

[philprime]

Additional note:
For iOS, we can also implement the iTunes Store lookup as a fallback, if the App Store ID is not configured

[philprime]

After discussing the proposal with @DamianJaeger we decided that a rule is a top-level AND-BLOCK, as it seems more intuitiv for users to limit the relevant scope with an AND, rather than extending it with an OR.

We believe there is not really a right or wrong way of doing this, so this is the way

[philprime]

The current development implementation of the iOS client now includes these headers:

• X-ONLAUNCH-BUNDLE-ID
• X-ONLAUNCH-BUNDLE-VERSION
• X-ONLAUNCH-RELEASE-VERSION
• X-ONLAUNCH-PLATFORM-NAME
• X-ONLAUNCH-PLATFORM-VERSION