Javascript sql 2021 (#2386)

* Bump prettier from 2.4.0 to 2.4.1 in /src (#2334) Bumps [prettier](https://github.com/prettier/prettier) from 2.4.0 to 2.4.1. - [Release notes](https://github.com/prettier/prettier/releases) - [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md) - [Commits](prettier/prettier@2.4.0...2.4.1) --- updated-dependencies: - dependency-name: prettier dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Turn off uses-passive-event-listeners Lighthouse audit (#2335) * Bump node-watch from 0.7.1 to 0.7.2 in /src (#2336) Bumps [node-watch](https://github.com/yuanchuan/node-watch) from 0.7.1 to 0.7.2. - [Release notes](https://github.com/yuanchuan/node-watch/releases) - [Changelog](https://github.com/yuanchuan/node-watch/blob/master/Changelog.md) - [Commits](yuanchuan/node-watch@0.7.1...v0.7.2) --- updated-dependencies: - dependency-name: node-watch dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * PWA 2021 queries (#2272) * Initial conversion of queries to custom metrics * Better Lighthouse queries * Remove comma * Add Service Worker over time for top 1k sites query * Change adoption ranking to include all ranking categories * Formatting * Linting fixes * Spacing improvements * Missing queries * Query updates * Missed some * Remove s * Space * Misc updates * More Tweaks * Spacing fixes * Fix assestlinks query * Missing DESC * Group rankings properly * Linting fixes * Remove unnecessary extra join * Spacing * Additional queries * Linting fixes * Update Events to look at onEvents * Better array merge * Misc fixes * Better JS * Filtering false positives for install query * Fixes to events query * Adding comment for install query * Expand on comment * Review feedback * Linting fixes * Correct comment * Apply suggestions from code review Co-authored-by: David Fox <david@davidjfox.com> * Review feedback * Review feedback * Change to COUNT(DISTINCT url) Co-authored-by: demianrenzulli <demian77@gmail.com> Co-authored-by: David Fox <david@davidjfox.com> * Bump sqlfluff from 0.6.5 to 0.6.6 in /src (#2339) Bumps [sqlfluff](https://github.com/sqlfluff/sqlfluff) from 0.6.5 to 0.6.6. - [Release notes](https://github.com/sqlfluff/sqlfluff/releases) - [Changelog](https://github.com/sqlfluff/sqlfluff/blob/main/CHANGELOG.md) - [Commits](sqlfluff/sqlfluff@0.6.5...0.6.6) --- updated-dependencies: - dependency-name: sqlfluff dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add more 2 more PWA queries (#2340) * Small SQL linting fixes to old media queries (#2341) * Caching 2021 queries (#2254) * Update README.md * Top CDNs * caching queries part 1 * LH queries * caching sql part 2 * changes from review * comment * Delete sql/2021/third-parties directory (#2342) * Delete sql/2021/third-parties directory * Delete third-parties.md * frameworks, sourcemaps queries * 1p vs 3p * adds unminified, preload/prefetch queries * Bump actions/github-script from 4.1 to 5 (#2346) Bumps [actions/github-script](https://github.com/actions/github-script) from 4.1 to 5. - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@v4.1...v5) --- updated-dependencies: - dependency-name: actions/github-script dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-node from 2.4.0 to 2.4.1 (#2347) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 2.4.0 to 2.4.1. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@v2.4.0...v2.4.1) --- updated-dependencies: - dependency-name: actions/setup-node dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github/super-linter from 4.7.3 to 4.8.0 (#2350) * Bump github/super-linter from 4.7.3 to 4.8.0 Bumps [github/super-linter](https://github.com/github/super-linter) from 4.7.3 to 4.8.0. - [Release notes](https://github.com/github/super-linter/releases) - [Changelog](https://github.com/github/super-linter/blob/master/docs/release-process.md) - [Commits](github/super-linter@v4.7.3...v4.8.0) --- updated-dependencies: - dependency-name: github/super-linter dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Fix queries failing linting Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Barry <barry@tunetheweb.com> * PR feedback accomodate & fixes * link fix * Feedback II update * Fixed review comment related to web_component_specs. * update content_type and webcomponent query Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Barry Pollard <barry_pollard@hotmail.com> Co-authored-by: demianrenzulli <demian77@gmail.com> Co-authored-by: David Fox <david@davidjfox.com> Co-authored-by: Rick Viscomi <rviscomi@users.noreply.github.com> Co-authored-by: Barry <barry@tunetheweb.com> Co-authored-by: Pankaj Parkar <parkarpankaj3@gmail.com>
HTTPArchive · Oct 18, 2021 · fec59a1 · fec59a1
1 parent 5017cb6
commit fec59a1
Show file tree

Hide file tree

Showing 101 changed files with 3,173 additions and 181 deletions.
diff --git a/.github/lighthouse/lighthouse-config-dev.json b/.github/lighthouse/lighthouse-config-dev.json
@@ -34,6 +34,7 @@
         "unused-css-rules": "off",
         "unused-javascript": "off",
         "uses-long-cache-ttl": "off",
+        "uses-passive-event-listeners": "off",
         "uses-rel-preconnect": "off",
         "uses-rel-preload": "off",
         "uses-text-compression": "off",

diff --git a/.github/lighthouse/lighthouse-config-prod.json b/.github/lighthouse/lighthouse-config-prod.json
@@ -33,6 +33,7 @@
         "unused-css-rules": "off",
         "unused-javascript": "off",
         "uses-long-cache-ttl": "off",
+        "uses-passive-event-listeners": "off",
         "uses-rel-preconnect": "off",
         "uses-rel-preload": "off",
         "uses-responsive-images": "off",

diff --git a/.github/workflows/linter.yml b/.github/workflows/linter.yml
@@ -28,7 +28,7 @@ jobs:
         run: |
           echo "VALIDATE_ALL_CODEBASE=false" >> $GITHUB_ENV
       - name: Lint Code Base
-        uses: github/super-linter@v4.7.3
+        uses: github/super-linter@v4.8.0
         #uses: docker://github/super-linter:v4.7.1
         env:
           DEFAULT_BRANCH: main

diff --git a/.github/workflows/predeploy.yml b/.github/workflows/predeploy.yml
@@ -31,7 +31,7 @@ jobs:
     - name: Checkout branch
       uses: actions/checkout@v2.3.4
     - name: Setup Node.js for use with actions
-      uses: actions/setup-node@v2.4.0
+      uses: actions/setup-node@v2.4.1
       with:
         node-version: '15'
     - name: Set up Python 3.8

diff --git a/.github/workflows/progress-tracker.yml b/.github/workflows/progress-tracker.yml
@@ -18,7 +18,7 @@ jobs:
     if: github.repository == 'HTTPArchive/almanac.httparchive.org'
     runs-on: ubuntu-20.04
     steps:
-      - uses: actions/github-script@v4.1
+      - uses: actions/github-script@v5
         if: github.event_name == 'workflow_dispatch' || contains(github.event.issue.labels.*.name, env.FILTER_LABEL)
         with:
           script: |

diff --git a/.github/workflows/test-template-changes.yml b/.github/workflows/test-template-changes.yml
@@ -33,13 +33,13 @@ jobs:
       with:
         fetch-depth: 0
     - name: Setup Node.js for use with actions
-      uses: actions/setup-node@v2.4.0
+      uses: actions/setup-node@v2.4.1
       with:
         node-version:  '15'
     - name: Test Template Changes
       run: ./src/tools/scripts/test_template_changes.sh
     - name: 'Comment PR'
-      uses: actions/github-script@v4.1
+      uses: actions/github-script@v5
       if: github.event_name == 'pull_request' && env.PR_COMMENT != ''
       with:
         github-token: ${{ secrets.GITHUB_TOKEN }}

diff --git a/.github/workflows/test_website.yml b/.github/workflows/test_website.yml
@@ -23,7 +23,7 @@ jobs:
         # Full git history is needed to get a proper list of changed files within `super-linter`
         fetch-depth: 0
     - name: Setup Node.js for use with actions
-      uses: actions/setup-node@v2.4.0
+      uses: actions/setup-node@v2.4.1
       with:
         node-version:  '15'
     - name: Set up Python 3.8
@@ -37,7 +37,7 @@ jobs:
     - name: Use more complete checks for generated HTML linting
       run: cp -f .github/linters/.htmlhintrc_morechecks .github/linters/.htmlhintrc
     - name: Lint Generated HTML
-      uses: github/super-linter@v4.7.3
+      uses: github/super-linter@v4.8.0
       env:
         DEFAULT_BRANCH: main
         FILTER_REGEX_INCLUDE: src/static/html/.*

diff --git a/sql/2019/media/04_03b.sql b/sql/2019/media/04_03b.sql
@@ -43,7 +43,7 @@ FROM
           # many 404s ANDredirects show up as image/gif
           status = 200 AND
 
-        # we are trying to catch images. WPO populates the format for media but it uses a file extension guess.
+          # we are trying to catch images. WPO populates the format for media but it uses a file extension guess.
           #So we exclude mimetypes that aren't image or where the format couldn't be guessed by WPO
           (format <> '' OR mimetype LIKE 'image%') AND
 

diff --git a/sql/2019/media/04_11a.sql b/sql/2019/media/04_11a.sql
@@ -63,7 +63,7 @@ LEFT JOIN
       # many 404s and redirects show up as image/gif
       status = 200 AND
 
-        # we are trying to catch images. WPO populates the format for media but it uses a file extension guess.
+      # we are trying to catch images. WPO populates the format for media but it uses a file extension guess.
       #So we exclude mimetypes that aren't image or where the format couldn't be guessed by WPO
       (format <> '' OR mimetype LIKE 'image%') AND
 

diff --git a/sql/2020/media/bytes_per_pixel.sql b/sql/2020/media/bytes_per_pixel.sql
@@ -62,7 +62,7 @@ LEFT JOIN
       # many 404s and redirects show up as image/gif
       status = 200 AND
 
-        # we are trying to catch images. WPO populates the format for media but it uses a file extension guess.
+      # we are trying to catch images. WPO populates the format for media but it uses a file extension guess.
       #So we exclude mimetypes that aren't image or where the format couldn't be guessed by WPO
       (format <> '' OR mimetype LIKE 'image%') AND
 

diff --git a/sql/2021/caching/README.md b/sql/2021/caching/README.md
@@ -1,10 +1,8 @@
 # 2021 Caching queries
 
-<!--
-  This directory contains all of the 2021 Caching chapter queries.
+Resources:
 
-  Each query should have a corresponding `metric_name.sql` file.
-  Note that readers are linked to this directory, so try to make the SQL file names descriptive for easy browsing.
-
-  Analysts: if helpful, you can use this README to give additional info about the queries.
--->
+- [Planning doc](https://docs.google.com/document/d/1BSdljOv86CPAP5T0A9jcV6bU7NJp1f2mTFa4ZZ7xxe8/edit)
+- [Results sheet](https://docs.google.com/spreadsheets/d/1-v3yR0LZIC3t4zWtqTgR3jJsKjjRMP-HATU2caP8e2c/edit)
+- [2019 chapter](https://almanac.httparchive.org/en/2019/caching)
+- [2020 chapter](https://almanac.httparchive.org/en/2020/caching)
diff --git a/sql/2021/caching/cache_control_directives.sql b/sql/2021/caching/cache_control_directives.sql
@@ -0,0 +1,71 @@
+#standardSQL
+# Use of Cache-Control directives
+SELECT
+  client,
+  COUNT(0) AS total_requests,
+  COUNTIF(uses_cache_control) AS total_using_cache_control,
+  COUNTIF(uses_max_age) AS total_using_max_age,
+  COUNTIF(uses_no_cache) AS total_using_no_cache,
+  COUNTIF(uses_public) AS total_using_public,
+  COUNTIF(uses_must_revalidate) AS total_using_must_revalidate,
+  COUNTIF(uses_no_store) AS total_using_no_store,
+  COUNTIF(uses_private) AS total_using_private,
+  COUNTIF(uses_proxy_revalidate) AS total_using_proxy_revalidate,
+  COUNTIF(uses_s_maxage) AS total_using_s_maxage,
+  COUNTIF(uses_no_transform) AS total_using_no_transform,
+  COUNTIF(uses_immutable) AS total_using_immutable,
+  COUNTIF(uses_stale_while_revalidate) AS total_using_stale_while_revalidate,
+  COUNTIF(uses_stale_if_error) AS total_using_stale_if_error,
+  COUNTIF(uses_no_store AND uses_no_cache AND uses_max_age_zero) AS total_using_no_store_and_no_cache_and_max_age_zero,
+  COUNTIF(uses_no_store AND uses_no_cache AND NOT uses_max_age_zero) AS total_using_no_store_and_no_cache_only,
+  COUNTIF(uses_no_store AND NOT uses_no_cache AND NOT uses_max_age_zero) AS total_using_no_store_only,
+  COUNTIF(uses_max_age_zero AND NOT uses_no_store) AS total_using_max_age_zero_without_no_store,
+  COUNTIF(uses_pre_check_zero AND uses_post_check_zero) AS total_using_pre_check_zero_and_post_check_zero,
+  COUNTIF(uses_pre_check_zero) AS total_using_pre_check_zero,
+  COUNTIF(uses_post_check_zero) AS total_using_post_check_zero,
+  COUNTIF(uses_cache_control AND NOT uses_max_age AND NOT uses_no_cache AND NOT uses_public AND NOT uses_must_revalidate AND NOT uses_no_store AND NOT uses_private AND NOT uses_proxy_revalidate AND NOT uses_s_maxage AND NOT uses_no_transform AND NOT uses_immutable AND NOT uses_stale_while_revalidate AND NOT uses_stale_if_error AND NOT uses_pre_check_zero AND NOT uses_post_check_zero) AS total_erroneous_directives,
+  COUNTIF(uses_cache_control) / COUNT(0) AS pct_using_cache_control,
+  COUNTIF(uses_max_age) / COUNT(0) AS pct_using_max_age,
+  COUNTIF(uses_no_cache) / COUNT(0) AS pct_using_no_cache,
+  COUNTIF(uses_public) / COUNT(0) AS pct_using_public,
+  COUNTIF(uses_must_revalidate) / COUNT(0) AS pct_using_must_revalidate,
+  COUNTIF(uses_no_store) / COUNT(0) AS pct_using_no_store,
+  COUNTIF(uses_private) / COUNT(0) AS pct_using_private,
+  COUNTIF(uses_proxy_revalidate) / COUNT(0) AS pct_using_proxy_revalidate,
+  COUNTIF(uses_s_maxage) / COUNT(0) AS pct_using_s_maxage,
+  COUNTIF(uses_no_transform) / COUNT(0) AS pct_using_no_transform,
+  COUNTIF(uses_immutable) / COUNT(0) AS pct_using_immutable,
+  COUNTIF(uses_stale_while_revalidate) / COUNT(0) AS pct_using_stale_while_revalidate,
+  COUNTIF(uses_stale_if_error) / COUNT(0) AS pct_using_stale_if_error,
+  COUNTIF(uses_no_store AND uses_no_cache AND uses_max_age_zero) / COUNT(0) AS pct_using_no_store_and_no_cache_and_max_age_zero,
+  COUNTIF(uses_no_store AND uses_no_cache AND NOT uses_max_age_zero) / COUNT(0) AS pct_using_no_store_and_no_cache_only,
+  COUNTIF(uses_no_store AND NOT uses_no_cache AND NOT uses_max_age_zero) / COUNT(0) AS pct_using_no_store_only,
+  COUNTIF(uses_max_age_zero AND NOT uses_no_store) / COUNT(0) AS pct_using_max_age_zero_without_no_store,
+  COUNTIF(uses_pre_check_zero AND uses_post_check_zero) / COUNT(0) AS pct_using_pre_check_zero_and_post_check_zero,
+  COUNTIF(uses_pre_check_zero) / COUNT(0) AS pct_using_pre_check_zero,
+  COUNTIF(uses_post_check_zero) / COUNT(0) AS pct_using_post_check_zero,
+  COUNTIF(uses_cache_control AND NOT uses_max_age AND NOT uses_no_cache AND NOT uses_public AND NOT uses_must_revalidate AND NOT uses_no_store AND NOT uses_private AND NOT uses_proxy_revalidate AND NOT uses_s_maxage AND NOT uses_no_transform AND NOT uses_immutable AND NOT uses_stale_while_revalidate AND NOT uses_stale_if_error AND NOT uses_pre_check_zero AND NOT uses_post_check_zero) / COUNT(0) AS pct_erroneous_directives
+FROM (
+  SELECT
+    _TABLE_SUFFIX AS client,
+    TRIM(resp_cache_control) != "" AS uses_cache_control,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)max-age\s*=\s*[0-9]+') AS uses_max_age,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)max-age\s*=\s*0') AS uses_max_age_zero,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)public') AS uses_public,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)no-cache') AS uses_no_cache,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)must-revalidate') AS uses_must_revalidate,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)no-store') AS uses_no_store,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)private') AS uses_private,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)proxy-revalidate') AS uses_proxy_revalidate,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)s-maxage\s*=\s*[0-9]+') AS uses_s_maxage,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)no-transform') AS uses_no_transform,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)immutable') AS uses_immutable,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)stale-while-revalidate\s*=\s*[0-9]+') AS uses_stale_while_revalidate,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)stale-if-error\s*=\s*[0-9]+') AS uses_stale_if_error,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)pre-check\s*=\s*0') AS uses_pre_check_zero,
+    REGEXP_CONTAINS(resp_cache_control, r'(?i)post-check\s*=\s*0') AS uses_post_check_zero
+  FROM
+    `httparchive.summary_requests.2021_07_01_*`
+)
+GROUP BY
+  client
diff --git a/sql/2021/caching/cache_control_directives_invalid.sql b/sql/2021/caching/cache_control_directives_invalid.sql
@@ -0,0 +1,33 @@
+#standardSQL
+# List of invalid Cache-Control directive names.
+SELECT
+  client,
+  total_directives,
+  total_using_cache_control,
+  directive_name,
+  directive_occurrences,
+  directive_occurrences / total_using_cache_control AS pct_of_cache_control,
+  directive_occurrences / total_directives AS pct_of_total_directives
+FROM (
+  SELECT
+    _TABLE_SUFFIX AS client,
+    directive_name,
+    COUNT(0) AS directive_occurrences,
+    SUM(COUNT(0)) OVER (PARTITION BY _TABLE_SUFFIX) AS total_directives,
+    SUM(COUNTIF(TRIM(resp_cache_control) != '')) OVER (PARTITION BY _TABLE_SUFFIX) AS total_using_cache_control
+  FROM
+    `httparchive.summary_requests.2021_07_01_*`
+  LEFT JOIN
+    UNNEST(REGEXP_EXTRACT_ALL(LOWER(resp_cache_control), r'([a-z][^,\s="\']*)')) AS directive_name
+  GROUP BY
+    client,
+    directive_name)
+WHERE
+  directive_name NOT IN (
+    'max-age', 'public', 'no-cache', 'must-revalidate', 'no-store',
+    'private', 'proxy-revalidate', 's-maxage', 'no-transform',
+    'immutable', 'stale-while-revalidate', 'stale-if-error',
+    'pre-check', 'post-check')
+ORDER BY
+  client,
+  directive_occurrences DESC
diff --git a/sql/2021/caching/cache_ttl_and_content_age_diff.sql b/sql/2021/caching/cache_ttl_and_content_age_diff.sql
@@ -0,0 +1,32 @@
+#standardSQL
+# Difference between Cache TTL and the contents age
+CREATE TEMPORARY FUNCTION toTimestamp(date_string STRING)
+RETURNS INT64 LANGUAGE js AS '''
+  try {
+    var timestamp = Math.round(new Date(date_string).getTime() / 1000);
+    return isNaN(timestamp) || timestamp < 0 ? -1 : timestamp;
+  } catch (e) {
+    return null;
+  }
+''';
+
+SELECT
+  client,
+  percentile,
+  APPROX_QUANTILES(diff_in_days, 1000 IGNORE NULLS)[OFFSET(percentile * 10)] AS diff_in_days
+FROM (
+  SELECT
+    _TABLE_SUFFIX AS client,
+    ROUND((expAge - (startedDateTime - toTimestamp(resp_last_modified))) / (60 * 60 * 24), 2) AS diff_in_days
+  FROM
+    `httparchive.summary_requests.2021_07_01_*`
+  WHERE
+    resp_last_modified <> "" AND
+    expAge > 0),
+  UNNEST([10, 25, 50, 75, 90]) AS percentile
+GROUP BY
+  client,
+  percentile
+ORDER BY
+  client,
+  percentile
diff --git a/sql/2021/caching/cache_ttl_lighthouse_score.sql b/sql/2021/caching/cache_ttl_lighthouse_score.sql
@@ -0,0 +1,16 @@
+#standardSQL
+# Distribution of cache TTL Lighthouse scores
+SELECT
+  _TABLE_SUFFIX AS client,
+  JSON_EXTRACT_SCALAR(report, "$.audits.uses-long-cache-ttl.score") AS caching_score,
+  COUNT(0) AS num_pages,
+  SUM(COUNT(0)) OVER (PARTITION BY _TABLE_SUFFIX) AS total,
+  COUNT(0) / SUM(COUNT(0)) OVER (PARTITION BY _TABLE_SUFFIX) AS pct_pages
+FROM
+  `httparchive.lighthouse.2021_07_01_*`
+GROUP BY
+  client,
+  caching_score
+ORDER BY
+  client,
+  caching_score ASC
diff --git a/sql/2021/caching/cache_wastedbytes_lighthouse.sql b/sql/2021/caching/cache_wastedbytes_lighthouse.sql
@@ -0,0 +1,16 @@
+#standardSQL
+# Distribution of bytes wasted (absence of adequate caching) from Lighthouse
+SELECT
+  _TABLE_SUFFIX AS client,
+  ROUND(CAST(JSON_EXTRACT_SCALAR(report, "$.audits.uses-long-cache-ttl.details.summary.wastedBytes") AS NUMERIC) / 1024 / 1024) AS mbyte_savings,
+  COUNT(0) AS num_pages,
+  SUM(COUNT(0)) OVER (PARTITION BY _TABLE_SUFFIX) AS total,
+  COUNT(0) / SUM(COUNT(0)) OVER (PARTITION BY _TABLE_SUFFIX) AS pct_pages
+FROM
+  `httparchive.lighthouse.2021_07_01_*`
+GROUP BY
+  client,
+  mbyte_savings
+ORDER BY
+  client,
+  mbyte_savings ASC
diff --git a/sql/2021/caching/content_age_older_than_ttl.sql b/sql/2021/caching/content_age_older_than_ttl.sql
@@ -0,0 +1,30 @@
+#standardSQL
+# Requests with a content age older than its TTL
+CREATE TEMPORARY FUNCTION toTimestamp(date_string STRING)
+RETURNS INT64 LANGUAGE js AS '''
+  try {
+    var timestamp = Math.round(new Date(date_string).getTime() / 1000);
+    return isNaN(timestamp) || timestamp < 0 ? -1 : timestamp;
+  } catch (e) {
+    return null;
+  }
+''';
+
+SELECT
+  client,
+  COUNT(0) AS total_req,
+  COUNTIF(diff < 0) AS req_too_short_cache,
+  COUNTIF(diff < 0) / COUNT(0) AS perc_req_too_short_cache
+FROM (
+  SELECT
+    _TABLE_SUFFIX AS client,
+    expAge - (startedDateTime - toTimestamp(resp_last_modified)) AS diff
+  FROM
+    `httparchive.summary_requests.2021_07_01_*`
+  WHERE
+    resp_last_modified <> "" AND
+    expAge > 0)
+GROUP BY
+  client
+ORDER BY
+  client
diff --git a/sql/2021/caching/content_age_older_than_ttl_by_party.sql b/sql/2021/caching/content_age_older_than_ttl_by_party.sql
@@ -0,0 +1,36 @@
+#standardSQL
+# Difference between Cache TTL and the content age for third party request
+CREATE TEMPORARY FUNCTION toTimestamp(date_string STRING)
+RETURNS INT64 LANGUAGE js AS '''
+  try {
+    var timestamp = Math.round(new Date(date_string).getTime() / 1000);
+    return isNaN(timestamp) || timestamp < 0 ? -1 : timestamp;
+  } catch (e) {
+    return null;
+  }
+''';
+
+SELECT
+  client,
+  party,
+  COUNT(0) AS total_req,
+  COUNTIF(diff < 0) AS req_too_short_cache,
+  COUNTIF(diff < 0) / COUNT(0) AS perc_req_too_short_cache
+FROM (
+  SELECT
+    _TABLE_SUFFIX AS client,
+    IF(NET.HOST(url) IN (
+      SELECT domain FROM `httparchive.almanac.third_parties` WHERE date = '2021-07-01' AND category != 'hosting'
+    ), 'third party', 'first party') AS party,
+    requests.expAge - (requests.startedDateTime - toTimestamp(requests.resp_last_modified)) AS diff
+  FROM
+    `httparchive.summary_requests.2021_07_01_*` requests
+  WHERE
+    TRIM(requests.resp_last_modified) <> "" AND
+    expAge > 0)
+GROUP BY
+  client,
+  party
+ORDER BY
+  client,
+  party