Cast histogram buckets into a string array.
+Generate linear buckets for a histogram.
Returns a custom build id generated from the build seconds of a FOG build.
-Returns a custom build id generated from the build hour of a Fenix build.
-buckets ARRAY<STRING>, val FLOAT64
+min FLOAT64, max FLOAT64, nBuckets FLOAT64
OUTPUTS
-FLOAT64
+ARRAY<FLOAT64>DETERMINISTIC
-Source | Edit
+Source | Edit
Create an empty histogram from an array of buckets.
-Parameters
+Parameters
INPUTS
@@ -4039,7 +4010,7 @@ Parameters
Source | Edit
histogram_cast_json (UDF)
Cast a histogram into a JSON blob.
-Parameters
+Parameters
INPUTS
histogram ARRAY<STRUCT<key STRING, value FLOAT64>>
@@ -4049,7 +4020,7 @@ Parameters
Source | Edit
histogram_fill_buckets (UDF)
Interpolate missing histogram buckets with empty buckets.
-Parameters
+Parameters
INPUTS
input_map ARRAY<STRUCT<key STRING, value FLOAT64>>, buckets ARRAY<STRING>
@@ -4057,34 +4028,18 @@ Parameters
ARRAY<STRUCT<key STRING, value FLOAT64>>
Source | Edit
-histogram_fill_buckets_dirichlet (UDF)
-Interpolate missing histogram buckets with empty buckets so it becomes a valid
-estimator for the dirichlet distribution.
-See: https://docs.google.com/document/d/1ipy1oFIKDvHr3R6Ku0goRjS11R1ZH1z2gygOGkSdqUg
-To use this, you must first:
-Aggregate the histograms to the client level, to get a histogram {k1: p1,
-k2:p2, ..., kK: pN} where the p's are proportions(and p1, p2, ... sum to 1)
-and Kis the number of buckets.
-This is then the client's estimated density, and every client has been
-reduced to one row (i.e the client's histograms are reduced to this single
-one and normalized).
-Then add all of these across clients to get {k1: P1, k2:P2, ..., kK: PK}
-where P1 = sum(p1 across N clients) and P2 = sum(p2 across N clients).
-Calculate the total number of buckets K, as well as the total number of
-profiles N reporting
-Then our estimate for final density is:
-[{k1: ((P1 + 1/K) / (nreporting+1)), k2: ((P2 + 1/K) /(nreporting+1)), ... }
-Parameters
+map_from_array_offsets_precise (UDF)
+Parameters
INPUTS
-input_map ARRAY<STRUCT<key STRING, value FLOAT64>>, buckets ARRAY<STRING>, total_users INT64
+required ARRAY<FLOAT64>, `values` ARRAY<FLOAT64>
OUTPUTS
ARRAY<STRUCT<key STRING, value FLOAT64>>
-Source | Edit
+Source | Edit
histogram_generate_scalar_buckets (UDF)
Generate scalar buckets for a histogram using a fixed number of buckets.
-Parameters
+Parameters
INPUTS
min_bucket FLOAT64, max_bucket FLOAT64, num_buckets INT64
@@ -4094,7 +4049,7 @@ Parameters
Source | Edit
histogram_generate_exponential_buckets (UDF)
Generate exponential buckets for a histogram.
-Parameters
+Parameters
INPUTS
min FLOAT64, max FLOAT64, nBuckets FLOAT64
@@ -4109,7 +4064,7 @@ histogram_generate_functional
determined with the following function:
i = $$ \lfloor{n log_{\text{base}}{(x)}}\rfloor $$
In other words, there are n buckets for each power of base
magnitude.
-Parameters
+Parameters
INPUTS
log_base INT64, buckets_per_magnitude INT64, range_max INT64
@@ -4117,19 +4072,18 @@ Parameters
ARRAY<FLOAT64>DETERMINISTIC
Source | Edit
-histogram_generate_linear_buckets (UDF)
-Generate linear buckets for a histogram.
-Parameters
+histogram_bucket_from_value (UDF)
+Parameters
INPUTS
-min FLOAT64, max FLOAT64, nBuckets FLOAT64
+buckets ARRAY<STRING>, val FLOAT64
OUTPUTS
-ARRAY<FLOAT64>DETERMINISTIC
+
-Source | Edit
+Source | Edit
build_hour_to_datetime (UDF)
Parses the custom build id used for Fenix builds in GLAM to a datetime.
-Parameters
+Parameters
INPUTS
@@ -4139,7 +4093,7 @@ Parameters
Source | Edit
percentile (UDF)
Get the value of the approximate CDF at the given percentile.
-Parameters
+Parameters
INPUTS
percentile FLOAT64, histogram ARRAY<STRUCT<key STRING, value FLOAT64>>, type STRING
@@ -4149,7 +4103,7 @@ Parameters
Source | Edit
histogram_normalized_sum (UDF)
Compute the normalized sum of an array of histograms.
-Parameters
+Parameters
INPUTS
arrs ARRAY<STRUCT<key STRING, value INT64>>, weight FLOAT64
@@ -4158,7 +4112,7 @@ Parameters
Source | Edit
map_from_array_offsets (UDF)
-Parameters
+Parameters
INPUTS
required ARRAY<FLOAT64>, `values` ARRAY<FLOAT64>
@@ -4166,15 +4120,61 @@ Parameters
ARRAY<STRUCT<key STRING, value FLOAT64>>
Source | Edit
-map_from_array_offsets_precise (UDF)
-Parameters
+histogram_fill_buckets_dirichlet (UDF)
+Interpolate missing histogram buckets with empty buckets so it becomes a valid
+estimator for the dirichlet distribution.
+See: https://docs.google.com/document/d/1ipy1oFIKDvHr3R6Ku0goRjS11R1ZH1z2gygOGkSdqUg
+To use this, you must first:
+Aggregate the histograms to the client level, to get a histogram {k1: p1,
+k2:p2, ..., kK: pN} where the p's are proportions(and p1, p2, ... sum to 1)
+and Kis the number of buckets.
+This is then the client's estimated density, and every client has been
+reduced to one row (i.e the client's histograms are reduced to this single
+one and normalized).
+Then add all of these across clients to get {k1: P1, k2:P2, ..., kK: PK}
+where P1 = sum(p1 across N clients) and P2 = sum(p2 across N clients).
+Calculate the total number of buckets K, as well as the total number of
+profiles N reporting
+Then our estimate for final density is:
+[{k1: ((P1 + 1/K) / (nreporting+1)), k2: ((P2 + 1/K) /(nreporting+1)), ... }
+Parameters
INPUTS
-required ARRAY<FLOAT64>, `values` ARRAY<FLOAT64>
+input_map ARRAY<STRUCT<key STRING, value FLOAT64>>, buckets ARRAY<STRING>, total_users INT64
OUTPUTS
ARRAY<STRUCT<key STRING, value FLOAT64>>
-Source | Edit
+Source | Edit
+histogram_buckets_cast_string_array (UDF)
+Cast histogram buckets into a string array.
+Parameters
+INPUTS
+
+OUTPUTS
+
+Source | Edit
+build_seconds_to_hour (UDF)
+Returns a custom build id generated from the build seconds of a FOG build.
+Parameters
+INPUTS
+
+OUTPUTS
+
+Source | Edit
+fenix_build_to_build_hour (UDF)
+Returns a custom build id generated from the build hour of a Fenix build.
+Parameters
+INPUTS
+
+OUTPUTS
+
+Source | Edit
diff --git a/mozfun/hist/index.html b/mozfun/hist/index.html
index 3e40fb592b2..4f28abad90e 100644
--- a/mozfun/hist/index.html
+++ b/mozfun/hist/index.html
@@ -2846,11 +2846,11 @@
-
-
- string_to_json (UDF)
+
+ merge (UDF)
-
-Parameters
+Parameters
INPUTS
@@ -3436,7 +3622,7 @@ Parameters
Source | Edit
normalize (UDF)
Normalize a histogram. Set sum to 1, and normalize to 1 the histogram bucket counts.
-Parameters
+Parameters
INPUTS
histogram STRUCT<bucket_count INT64, `sum` INT64, histogram_type INT64, `range` ARRAY<INT64>, `values` ARRAY<STRUCT<key INT64, value INT64>>>
@@ -3444,72 +3630,6 @@ Parameters
STRUCT<bucket_count INT64, `sum` INT64, histogram_type INT64, `range` ARRAY<INT64>, `values` ARRAY<STRUCT<key INT64, value FLOAT64>>>
Source | Edit
-merge (UDF)
-Merge an array of histograms into a single histogram.
-
-- The histogram values will be summed per-bucket
-- The count will be summed
-- Other fields will take the mode_last
-
-Parameters
-INPUTS
-histogram_list ANY TYPE
-
-Source | Edit
-count (UDF)
-Given histogram h, return the count of all measurements across all buckets.
-Given histogram h, return the count of all measurements across all buckets.
-Extracts the values from the histogram and sums them, returning the total_count.
-Parameters
-INPUTS
-
-OUTPUTS
-
-Source | Edit
-mean (UDF)
-Given histogram h, return floor(mean) of the measurements in the bucket.
-That is, the histogram sum divided by the number of measurements taken.
-https://github.com/mozilla/telemetry-batch-view/blob/ea0733c/src/main/scala/com/mozilla/telemetry/utils/MainPing.scala#L292-L307
-Parameters
-INPUTS
-
-OUTPUTS
-STRUCT<sum INT64, VALUES ARRAY<STRUCT<value INT64>>>
-
-Source | Edit
-threshold_count (UDF)
-Return the number of recorded observations greater than threshold for
-the histogram. CAUTION: Does not count any buckets that have any values
-less than the threshold. For example, a bucket with range (1, 10) will
-not be counted for a threshold of 2. Use threshold that are not bucket
-boundaries with caution.
-https://github.com/mozilla/telemetry-batch-view/blob/ea0733c/src/main/scala/com/mozilla/telemetry/utils/MainPing.scala#L213-L239
-Parameters
-INPUTS
-histogram STRING, threshold INT64
-
-Source | Edit
-
-Sum of a keyed histogram, across all keys it contains.
-
-Takes a keyed histogram and returns a single number:
-the sum of all keys it contains. The expected input
-type is ARRAY<STRUCT<key STRING, value STRING>>
-The return type is INT64
.
-The key
field will be ignored, and
-the `value is expected to be the compact
-histogram representation.
-Parameters
-INPUTS
-keyed_histogram ARRAY<STRUCT<key STRING, value STRING>>
-
-OUTPUTS
-
-Source | Edit
diff --git a/mozfun/iap/index.html b/mozfun/iap/index.html
index c74bf385d89..9fcc2462ea6 100644
--- a/mozfun/iap/index.html
+++ b/mozfun/iap/index.html
@@ -2866,11 +2866,11 @@
-
-
- parse_apple_receipt (UDF)
+
+ parse_apple_event (UDF)
-
+
-
@@ -2906,11 +2906,11 @@
-
-
- parse_android_receipt (UDF)
+
+ parse_apple_receipt (UDF)
-
+
-
@@ -2946,11 +2946,11 @@
-
-
- parse_apple_event (UDF)
+
+ parse_android_receipt (UDF)
-
+
-
@@ -3398,11 +3398,11 @@
-
-
- parse_apple_receipt (UDF)
+
+ parse_apple_event (UDF)
-
+
-
@@ -3438,11 +3438,11 @@
-
-
- parse_android_receipt (UDF)
+
+ parse_apple_receipt (UDF)
-
+
-
@@ -3478,11 +3478,11 @@
-
-
- parse_apple_event (UDF)
+
+ parse_android_receipt (UDF)
-
+
-
@@ -3513,16 +3513,13 @@
iap
-parse_apple_receipt (UDF)
-Used to parse provider_receipt_json in mozilla vpn subscriptions where provider is "APPLE". The content is documented at https://developer.apple.com/documentation/appstorereceipts/responsebody
+parse_apple_event (UDF)
+Used to parse data
field from firestore export of fxa dataset iap_app_store_purchases_raw. The content is documented at https://developer.apple.com/documentation/appstoreservernotifications/responsebodyv2decodedpayload and https://github.com/mozilla/fxa/blob/700ed771860da450add97d62f7e6faf2ead0c6ba/packages/fxa-shared/payments/iap/apple-app-store/subscription-purchase.ts#L115-L171
Parameters
INPUTS
-provider_receipt_json STRING
-
-OUTPUTS
-STRUCT<environment STRING, latest_receipt BYTES, latest_receipt_info ARRAY<STRUCT<cancellation_date STRING, cancellation_date_ms INT64, cancellation_date_pst STRING, cancellation_reason STRING, expires_date STRING, expires_date_ms INT64, expires_date_pst STRING, in_app_ownership_type STRING, is_in_intro_offer_period STRING, is_trial_period STRING, original_purchase_date STRING, original_purchase_date_ms INT64, original_purchase_date_pst STRING, original_transaction_id STRING, product_id STRING, promotional_offer_id STRING, purchase_date STRING, purchase_date_ms INT64, purchase_date_pst STRING, quantity INT64, subscription_group_identifier INT64, transaction_id INT64, web_order_line_item_id INT64>>, pending_renewal_info ARRAY<STRUCT<auto_renew_product_id STRING, auto_renew_status INT64, expiration_intent INT64, is_in_billing_retry_period INT64, original_transaction_id STRING, product_id STRING>>, receipt STRUCT<adam_id INT64, app_item_id INT64, application_version STRING, bundle_id STRING, download_id INT64, in_app ARRAY<STRUCT<cancellation_date STRING, cancellation_date_ms INT64, cancellation_date_pst STRING, cancellation_reason STRING, expires_date STRING, expires_date_ms INT64, expires_date_pst STRING, in_app_ownership_type STRING, is_in_intro_offer_period STRING, is_trial_period STRING, original_purchase_date STRING, original_purchase_date_ms INT64, original_purchase_date_pst STRING, original_transaction_id STRING, product_id STRING, promotional_offer_id STRING, purchase_date STRING, purchase_date_ms INT64, purchase_date_pst STRING, quantity INT64, subscription_group_identifier INT64, transaction_id INT64, web_order_line_item_id INT64>>, original_application_version STRING, original_purchase_date STRING, original_purchase_date_ms INT64, original_purchase_date_pst STRING, receipt_creation_date STRING, receipt_creation_date_ms INT64, receipt_creation_date_pst STRING, receipt_type STRING, request_date STRING, request_date_ms INT64, request_date_pst STRING, version_external_identifier INT64>, status INT64>DETERMINISTIC
+
-Source | Edit
+Source | Edit
derive_apple_subscription_interval (UDF)
Take output purchase_date and expires_date from mozfun.iap.parse_apple_receipt and return the subscription interval to use for accounting. Values must be DATETIME in America/Los_Angeles to get correct results because of how timezone and daylight savings impact the time of day and the length of a month.
Parameters
@@ -3533,13 +3530,16 @@ Parameters
STRUCT<`interval` STRING, interval_count INT64>
Source | Edit
-parse_android_receipt (UDF)
-Used to parse data
field from firestore export of fxa dataset iap_google_raw. The content is documented at https://developer.android.com/google/play/billing/subscriptions and https://developers.google.com/android-publisher/api-ref/rest/v3/purchases.subscriptions
+parse_apple_receipt (UDF)
+Used to parse provider_receipt_json in mozilla vpn subscriptions where provider is "APPLE". The content is documented at https://developer.apple.com/documentation/appstorereceipts/responsebody
Parameters
INPUTS
-input STRING
+provider_receipt_json STRING
-Source | Edit
+OUTPUTS
+STRUCT<environment STRING, latest_receipt BYTES, latest_receipt_info ARRAY<STRUCT<cancellation_date STRING, cancellation_date_ms INT64, cancellation_date_pst STRING, cancellation_reason STRING, expires_date STRING, expires_date_ms INT64, expires_date_pst STRING, in_app_ownership_type STRING, is_in_intro_offer_period STRING, is_trial_period STRING, original_purchase_date STRING, original_purchase_date_ms INT64, original_purchase_date_pst STRING, original_transaction_id STRING, product_id STRING, promotional_offer_id STRING, purchase_date STRING, purchase_date_ms INT64, purchase_date_pst STRING, quantity INT64, subscription_group_identifier INT64, transaction_id INT64, web_order_line_item_id INT64>>, pending_renewal_info ARRAY<STRUCT<auto_renew_product_id STRING, auto_renew_status INT64, expiration_intent INT64, is_in_billing_retry_period INT64, original_transaction_id STRING, product_id STRING>>, receipt STRUCT<adam_id INT64, app_item_id INT64, application_version STRING, bundle_id STRING, download_id INT64, in_app ARRAY<STRUCT<cancellation_date STRING, cancellation_date_ms INT64, cancellation_date_pst STRING, cancellation_reason STRING, expires_date STRING, expires_date_ms INT64, expires_date_pst STRING, in_app_ownership_type STRING, is_in_intro_offer_period STRING, is_trial_period STRING, original_purchase_date STRING, original_purchase_date_ms INT64, original_purchase_date_pst STRING, original_transaction_id STRING, product_id STRING, promotional_offer_id STRING, purchase_date STRING, purchase_date_ms INT64, purchase_date_pst STRING, quantity INT64, subscription_group_identifier INT64, transaction_id INT64, web_order_line_item_id INT64>>, original_application_version STRING, original_purchase_date STRING, original_purchase_date_ms INT64, original_purchase_date_pst STRING, receipt_creation_date STRING, receipt_creation_date_ms INT64, receipt_creation_date_pst STRING, receipt_type STRING, request_date STRING, request_date_ms INT64, request_date_pst STRING, version_external_identifier INT64>, status INT64>DETERMINISTIC
+
+Source | Edit
scrub_apple_receipt (UDF)
Take output from mozfun.iap.parse_apple_receipt and remove fields or reduce their granularity so that the returned value can be exposed to all employees via redash.
Parameters
@@ -3550,13 +3550,13 @@ Parameters
STRUCT<environment STRING, active_period STRUCT<start_date DATE, end_date DATE, start_time TIMESTAMP, end_time TIMESTAMP, `interval` STRING, interval_count INT64>, trial_period STRUCT<start_time TIMESTAMP, end_time TIMESTAMP>>
Source | Edit
-parse_apple_event (UDF)
-Used to parse data
field from firestore export of fxa dataset iap_app_store_purchases_raw. The content is documented at https://developer.apple.com/documentation/appstoreservernotifications/responsebodyv2decodedpayload and https://github.com/mozilla/fxa/blob/700ed771860da450add97d62f7e6faf2ead0c6ba/packages/fxa-shared/payments/iap/apple-app-store/subscription-purchase.ts#L115-L171
+parse_android_receipt (UDF)
+Used to parse data
field from firestore export of fxa dataset iap_google_raw. The content is documented at https://developer.android.com/google/play/billing/subscriptions and https://developers.google.com/android-publisher/api-ref/rest/v3/purchases.subscriptions
Parameters
INPUTS
-Source | Edit
+Source | Edit
diff --git a/mozfun/json/index.html b/mozfun/json/index.html
index bb791d15c72..0ed495c242b 100644
--- a/mozfun/json/index.html
+++ b/mozfun/json/index.html
@@ -2886,11 +2886,11 @@