Sherlock - The ASO inspector

Sherlock is a program I wrote as a companion to my presentation at the ASO Conference in NYC in 2019.

It's based on top of a library I wrote called pm-prophet

It can produce curated reports about your app and it's traffic, including:

• Updates impact (both visual and textual updates)
• Seasonality
• Uplift

Sample output: is demostrated in the file examples/report.html (which is a fit of sherlock on some fake data contained in examples/example.csv).

Installation

1. Install python 3
2. Install pipenv
3. Open your terminal and run pipenv install from the folder in which you have cloned this repository

Et Voilà. You should be all set.

Preparing your data

Sherlock takes in input a csv file of different time-series. The naming convention is strict and it's case-sensitive.

Required columns:

1. date the date of the events (expressed in dd/mm/yyyy format, e.g. 12/30/2019).
2. update a column with value textual, visual or all (i.e. visual and textual) to tell sherlock what type of update was done in that particular date (in lowercase!).
3. search_downloads downloads data coming from iTunes connect filtered by App Store Search source
4. search_impressions impressions data coming from iTunes connect filtered by App Store Search source

Optional columns (if you do Apple Search Ads):

1. asa_impressions impressions coming from Apple Search Ads
2. asa downloads coming from Apple Search Ads

Extra columns: Any other time series you like (e.g. Facebook, Google, Snapchat..) for which to compute uplift.

Usage

Usage is very simple, from the terminal run:

pipenv shell

to activate the virtual environment and:

python sherlock.py -i example.csv

To run the script. There are some options available:

Options:
  -h, --help            show this help message and exit
  -a APP_NAME, --app-name=APP_NAME
                        Specify the app name if you want a more personalized
                        report
  -i FILE, --input-file=FILE
                        Input CSV file
  -o FILE, --output-file=FILE
                        Output report file (in html format)
  -s SAMPLER, --sampler=SAMPLER
                        Sampler to use ("nuts" is slower but more precise,
                        default "metropolis")
  -n, --no-asa          Do not use ASA as an additional regressor (better
                        seasonality fits)
  -w, --weekly          Run the analysis on a weekly resampling
  -r SIGMA, --remove-outliers-sigma=SIGMA
                        Remove outliers at more than X sigma from the mean
                        (suggested values range between 1.5-3.5). Default
                        value is: 0 that means that Sherlock will not remove
                        outliers
  -l ALPHA, --significance-level=ALPHA
                        The significance level for the analysis (default is
                        0.05)
  -k SEASONALITY_SCALE, --seasonality-scale=SEASONALITY_SCALE
                        The scale of the seasonality, if it fits poorly
                        because you have  great variance due to seasonality
                        increase this. By default this is automatically
                        computed

Output

Open the report.html file generated by Sherlock.

License

CC BY-SA 4.0