Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite SW query to use blink_features #196

Merged
merged 1 commit into from
Jun 20, 2024
Merged

Rewrite SW query to use blink_features #196

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 10 additions & 15 deletions sql/timeseries/swControlledPages.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,18 @@
#standardSQL
SELECT
SUBSTR(_TABLE_SUFFIX, 0, 10) AS date,
UNIX_DATE(CAST(REPLACE(SUBSTR(_TABLE_SUFFIX, 0, 10), '_', '-') AS DATE)) * 1000 * 60 * 60 * 24 AS timestamp,
IF(ENDS_WITH(_TABLE_SUFFIX, 'desktop'), 'desktop', 'mobile') AS client,
ROUND(
SUM(
IF(IFNULL(
JSON_EXTRACT(payload, '$._blinkFeatureFirstUsed.Features.ServiceWorkerControlledPage'),
JSON_EXTRACT(payload, '$._blinkFeatureFirstUsed.Features.990')
) IS NOT NULL, 1, 0)
) * 100 / COUNT(0),
2
) AS percent
REGEXP_REPLACE(yyyymmdd, r'(\d{4})(\d{2})(\d{2})', '\\1_\\2_\\3') AS date,
UNIX_DATE(CAST(REGEXP_REPLACE(yyyymmdd, r'(\d{4})(\d{2})(\d{2})', '\\1-\\2-\\3') AS DATE)) * 1000 * 60 * 60 * 24 AS timestamp,
client,
SUM(IF(id = '990' OR feature = 'ServiceWorkerControlledPage', num_urls, 0)) AS num_urls,
ROUND(SUM(IF(id = '990' OR feature = 'ServiceWorkerControlledPage', num_urls, 0)) / total_urls * 100, 5) AS percent
FROM
`httparchive.pages.*`
`httparchive.blink_features.usage`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this table updated by the time the reports are generated? Or is it expected to lag by a month?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like all the other blink features queries this one will be delayed. But not by a month, as we rerun the reports on the 2nd of the month specifically because this data is available then.

Eventually we'll rewrite these all to use the all tables at which point this will no longer be an issue but until then I don't see why this SW query needs to be treated differently.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just asking what the expected behavior is. Sounds like the lag is WAI, and you're right that it's less than a month.

GROUP BY
date,
timestamp,
client
client,
total_urls
ORDER BY
date DESC,
client
client,
num_urls DESC