Skip to content

Commit

Permalink
Privacy 2024 queries (#3653)
Browse files Browse the repository at this point in the history
* readme

* copied 2022 SQLs over to update/review

* fixed link

* origin trials

* Bump puppeteer from 22.7.1 to 22.8.0 in /src (#3655)

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.7.1 to 22.8.0.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](puppeteer/puppeteer@puppeteer-v22.7.1...puppeteer-v22.8.0)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* notebook + readme (#3652)

* Bump pytest from 8.1.1 to 8.2.0 in /src (#3651)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 8.1.1 to 8.2.0.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@8.1.1...8.2.0)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Translation of privacy chapter to Japanese (#3654)

* Update Timestamps (#3657)

Co-authored-by: tunetheweb <10931297+tunetheweb@users.noreply.github.com>

* 2023 Performance (#3525)

* cp 2022->2023

* 2023ify

* 2023/perf

* lint

* lint

* fix initiator

* null initiators

* Bump puppeteer from 22.8.0 to 22.9.0 in /src (#3662)

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.8.0 to 22.9.0.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](puppeteer/puppeteer@puppeteer-v22.8.0...puppeteer-v22.9.0)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Upgrade to web-vitals v4 (#3661)

* Upgrade to web-vitals v4

* Update src/static/js/send-web-vitals.js

Co-authored-by: Barry Pollard <barrypollard@google.com>

---------

Co-authored-by: Barry Pollard <barrypollard@google.com>

* Bump pytest from 8.2.0 to 8.2.1 in /src (#3664)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 8.2.0 to 8.2.1.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@8.2.0...8.2.1)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* --- (#3665)

updated-dependencies:
- dependency-name: web-vitals
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump puppeteer from 22.9.0 to 22.10.0 in /src (#3668)

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.9.0 to 22.10.0.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](puppeteer/puppeteer@puppeteer-v22.9.0...puppeteer-v22.10.0)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump jsdom from 24.0.0 to 24.1.0 in /src (#3669)

Bumps [jsdom](https://github.com/jsdom/jsdom) from 24.0.0 to 24.1.0.
- [Release notes](https://github.com/jsdom/jsdom/releases)
- [Changelog](https://github.com/jsdom/jsdom/blob/main/Changelog.md)
- [Commits](jsdom/jsdom@24.0.0...24.1.0)

---
updated-dependencies:
- dependency-name: jsdom
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Typofix (#3670)

Seems like "desktop" is mentioned twice and according to the data, the second mention is related to mobile https://docs.google.com/spreadsheets/d/1JvJMiRsL6T9m_NEBHFh-rrQmU5a-ufdOKriSJbrEN8M/edit#gid=1472139207

* SQL and MD folders the 2024 Web Almanac (#3666)

* upload 2024

* change mds

* Test update

* Revert test update

* Fix line endings

---------

Co-authored-by: Barry Pollard <barrypollard@google.com>

* Bump prettier from 3.2.5 to 3.3.0 in /src (#3672)

Bumps [prettier](https://github.com/prettier/prettier) from 3.2.5 to 3.3.0.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](prettier/prettier@3.2.5...3.3.0)

---
updated-dependencies:
- dependency-name: prettier
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pytest from 8.2.1 to 8.2.2 in /src (#3673)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 8.2.1 to 8.2.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@8.2.1...8.2.2)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump prettier from 3.3.0 to 3.3.1 in /src (#3674)

Bumps [prettier](https://github.com/prettier/prettier) from 3.3.0 to 3.3.1.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](prettier/prettier@3.3.0...3.3.1)

---
updated-dependencies:
- dependency-name: prettier
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix loaf monitoring bug (#3675)

* Fix LoAF monitoring bug

* Add semi colon

* Update Timestamps (#3677)

Co-authored-by: tunetheweb <10931297+tunetheweb@users.noreply.github.com>

* Bump web-vitals from 4.0.1 to 4.1.0 in /src (#3678)

Bumps [web-vitals](https://github.com/GoogleChrome/web-vitals) from 4.0.1 to 4.1.0.
- [Changelog](https://github.com/GoogleChrome/web-vitals/blob/main/CHANGELOG.md)
- [Commits](GoogleChrome/web-vitals@v4.0.1...v4.1.0)

---
updated-dependencies:
- dependency-name: web-vitals
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fixed link

* remove unreviewed sql

* lint test

* lint

* ads supply graph

* lint

* close file

* lint

* top_direct_sellers

* ads_txt_lines_histogram

* ads_txt_seller_accounts_by_type

* top_ads_variables

* format

* tcf2

* rename

* lint

* using custom_metrics

* most_common_cname_domains

* adguard list

* gpc

* referrer policy

* usp

* iab frameworks

* lint

* bounce trackers

* Added privacy sandbox related queries

* lint

* missed lint

* dnt

* client hints

* whotracksme update

* lint

* referrer policy

* rank filter removed

* trackers

* util deps

* limits

* Privacy 2024 queries - CCPA, fingerprinting, cookies (#3720)

* CCPA metrics

* fingerprinting metrics

* cookie metrics

* lint

* bq to sheets updates

* query optimisation

* downgrade for python 3.8

* more categories

* more categories and columns reordered

* forms and formatted logs

* Refactoring queries to produce output for queries only

* lint

* lint

* Privacy Sql Tracking Detection Using Easylist Adservers (#3730)

* Add GA4 fields to match documentation (#3679)

* Add standard GA4 web-vital fields

* Add value

* Update Timestamps (#3680)

Co-authored-by: tunetheweb <10931297+tunetheweb@users.noreply.github.com>

* Bump web-vitals from 4.1.0 to 4.1.1 in /src (#3681)

Bumps [web-vitals](https://github.com/GoogleChrome/web-vitals) from 4.1.0 to 4.1.1.
- [Changelog](https://github.com/GoogleChrome/web-vitals/blob/main/CHANGELOG.md)
- [Commits](GoogleChrome/web-vitals@v4.1.0...v4.1.1)

---
updated-dependencies:
- dependency-name: web-vitals
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump puppeteer from 22.10.0 to 22.10.1 in /src (#3682)

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.10.0 to 22.10.1.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](puppeteer/puppeteer@puppeteer-v22.10.0...puppeteer-v22.10.1)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump prettier from 3.3.1 to 3.3.2 in /src (#3683)

Bumps [prettier](https://github.com/prettier/prettier) from 3.3.1 to 3.3.2.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](prettier/prettier@3.3.1...3.3.2)

---
updated-dependencies:
- dependency-name: prettier
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump puppeteer from 22.10.1 to 22.11.0 in /src (#3684)

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.10.1 to 22.11.0.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](puppeteer/puppeteer@puppeteer-v22.10.1...puppeteer-v22.11.0)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Translation of security chapter to Japanese (#3685)

* Bump puppeteer from 22.11.0 to 22.11.2 in /src (#3688)

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.11.0 to 22.11.2.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](puppeteer/puppeteer@puppeteer-v22.11.0...puppeteer-v22.11.2)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump web-vitals from 4.1.1 to 4.2.0 in /src (#3690)

Bumps [web-vitals](https://github.com/GoogleChrome/web-vitals) from 4.1.1 to 4.2.0.
- [Changelog](https://github.com/GoogleChrome/web-vitals/blob/main/CHANGELOG.md)
- [Commits](GoogleChrome/web-vitals@v4.1.1...v4.2.0)

---
updated-dependencies:
- dependency-name: web-vitals
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump puppeteer from 22.11.2 to 22.12.0 in /src (#3689)

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.11.2 to 22.12.0.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](puppeteer/puppeteer@puppeteer-v22.11.2...puppeteer-v22.12.0)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update Timestamps (#3691)

Co-authored-by: tunetheweb <10931297+tunetheweb@users.noreply.github.com>

* Remove deploy.zip step of deployment (#3692)

* Remove deploy.zip

* Remove from ignore files

* Bump puppeteer from 22.12.0 to 22.12.1 in /src (#3694)

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.12.0 to 22.12.1.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](puppeteer/puppeteer@puppeteer-v22.12.0...puppeteer-v22.12.1)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump treosh/lighthouse-ci-action from 11.4.0 to 12.1.0 (#3693)

* Bump treosh/lighthouse-ci-action from 11.4.0 to 12.1.0

Bumps [treosh/lighthouse-ci-action](https://github.com/treosh/lighthouse-ci-action) from 11.4.0 to 12.1.0.
- [Release notes](https://github.com/treosh/lighthouse-ci-action/releases)
- [Commits](treosh/lighthouse-ci-action@11.4.0...12.1.0)

---
updated-dependencies:
- dependency-name: treosh/lighthouse-ci-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Upgrade to Node 20

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Barry Pollard <barrypollard@google.com>

* Bump web-vitals from 4.2.0 to 4.2.1 in /src (#3695)

Bumps [web-vitals](https://github.com/GoogleChrome/web-vitals) from 4.2.0 to 4.2.1.
- [Changelog](https://github.com/GoogleChrome/web-vitals/blob/main/CHANGELOG.md)
- [Commits](GoogleChrome/web-vitals@v4.2.0...v4.2.1)

---
updated-dependencies:
- dependency-name: web-vitals
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/setup-python from 5.1.0 to 5.1.1 (#3699)

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.1.0 to 5.1.1.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](actions/setup-python@v5.1.0...v5.1.1)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump puppeteer from 22.12.1 to 22.13.0 in /src (#3698)

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.12.1 to 22.13.0.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](puppeteer/puppeteer@puppeteer-v22.12.1...puppeteer-v22.13.0)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Translation of mobile-web chapter to Japanese (#3700)

* Bump puppeteer from 22.13.0 to 22.15.0 in /src (#3711)

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.13.0 to 22.15.0.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](puppeteer/puppeteer@puppeteer-v22.13.0...puppeteer-v22.15.0)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump jsdom from 24.1.0 to 24.1.1 in /src (#3707)

Bumps [jsdom](https://github.com/jsdom/jsdom) from 24.1.0 to 24.1.1.
- [Release notes](https://github.com/jsdom/jsdom/releases)
- [Changelog](https://github.com/jsdom/jsdom/blob/main/Changelog.md)
- [Commits](jsdom/jsdom@24.1.0...24.1.1)

---
updated-dependencies:
- dependency-name: jsdom
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump web-vitals from 4.2.1 to 4.2.2 in /src (#3706)

Bumps [web-vitals](https://github.com/GoogleChrome/web-vitals) from 4.2.1 to 4.2.2.
- [Changelog](https://github.com/GoogleChrome/web-vitals/blob/main/CHANGELOG.md)
- [Commits](GoogleChrome/web-vitals@v4.2.1...v4.2.2)

---
updated-dependencies:
- dependency-name: web-vitals
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump prettier from 3.3.2 to 3.3.3 in /src (#3702)

Bumps [prettier](https://github.com/prettier/prettier) from 3.3.2 to 3.3.3.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](prettier/prettier@3.3.2...3.3.3)

---
updated-dependencies:
- dependency-name: prettier
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump web-vitals from 4.2.2 to 4.2.3 in /src (#3715)

Bumps [web-vitals](https://github.com/GoogleChrome/web-vitals) from 4.2.2 to 4.2.3.
- [Changelog](https://github.com/GoogleChrome/web-vitals/blob/main/CHANGELOG.md)
- [Commits](GoogleChrome/web-vitals@v4.2.2...v4.2.3)

---
updated-dependencies:
- dependency-name: web-vitals
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update Timestamps (#3716)

Co-authored-by: rviscomi <1120896+rviscomi@users.noreply.github.com>

* tracking detection using easylist adservers

* easylist_adserver tracking detection and query

* 2022 cdn portuguese (#3725)

* add file to translation

* done translation cdn.md

Makes progress on #505

* Bump puppeteer from 22.15.0 to 23.0.2 in /src (#3719)

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 22.15.0 to 23.0.2.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](puppeteer/puppeteer@puppeteer-v22.15.0...puppeteer-v23.0.2)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update Timestamps (#3726)

Co-authored-by: tunetheweb <10931297+tunetheweb@users.noreply.github.com>

* Replace `<object>` with `<iframe>` for embedded SVG (#3727)

* Replace object with iframe for embedded SVG

* Translations

* auto upload easylist data to table

* Fix the build to ignore 2024 chapters (for now) (#3728)

* Fix the build to ignore 2024 chapters (for now)

* Remove test line

* Update Timestamps (#3729)

Co-authored-by: tunetheweb <10931297+tunetheweb@users.noreply.github.com>

* liniting

* liniting

* linting

* linting

* linting

* linting

* fixes of Simplified Chinese translation for 2020 Performance (#3734)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Barry Pollard <barrypollard@google.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: tunetheweb <10931297+tunetheweb@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sakae Kotaro <ksakae1216@gmail.com>
Co-authored-by: rviscomi <1120896+rviscomi@users.noreply.github.com>
Co-authored-by: Hadi Amjad <hadiamjad@Hadis-MacBook-Air.local>
Co-authored-by: William Constantinov <33907565+HakaCode@users.noreply.github.com>
Co-authored-by: Zuckjet <zuckjet@gmail.com>
Co-authored-by: Max Ostapenko <1611259+max-ostapenko@users.noreply.github.com>

* log query errors

* Fixed privacy sandbox attestation query bug

* maximum_bytes_billed parameter

* moved to chapter root

* postpone dryrun check

* fingerprinting_most_common_apis: improve resilience to malformed JSON (#3737)

* optional maximum_bytes_billed parameter

* formatting

* queries and notebook updates

* queries to rerun

* origin trials function fix

* optimised sellers count

* apps included in ads.txt lines

* another rerun

* lint

* no origins

* optimized perf

* more optimized perf

* graph optimization and OT expiration

* earlier grouping for performance

* graph fixes

* cookies, ccpa, fingerprinting: calculate percent of total pages

* query for top third-party cookie names

* bq writer module

* add grouping

* domain suffixes and regexes removed

* add comments

* review

* add PR link

* lint

* remove mobile filter

* lint

* lint

* disable import-error rule

* adguard not used

* linting

* pages_pct in query

* lint

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sakae Kotaro <ksakae1216@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: tunetheweb <10931297+tunetheweb@users.noreply.github.com>
Co-authored-by: Rick Viscomi <rviscomi@users.noreply.github.com>
Co-authored-by: Barry Pollard <barrypollard@google.com>
Co-authored-by: Boris Schapira <borisschapira@gmail.com>
Co-authored-by: ChrisBeeti <32492572+ChrisBeeti@users.noreply.github.com>
Co-authored-by: Yash Vekaria <yvekaria.09@gmail.com>
Co-authored-by: Ben Standaert <71239179+bstandaert-wustl@users.noreply.github.com>
Co-authored-by: Hadi Amjad <46374292+hadiamjad@users.noreply.github.com>
Co-authored-by: rviscomi <1120896+rviscomi@users.noreply.github.com>
Co-authored-by: Hadi Amjad <hadiamjad@Hadis-MacBook-Air.local>
Co-authored-by: William Constantinov <33907565+HakaCode@users.noreply.github.com>
Co-authored-by: Zuckjet <zuckjet@gmail.com>
Co-authored-by: bstandaert-wustl <b.g.standaert@wustl.edu>
  • Loading branch information
17 people authored Nov 3, 2024
1 parent 6375165 commit 324d22b
Show file tree
Hide file tree
Showing 45 changed files with 2,273 additions and 215 deletions.
64 changes: 64 additions & 0 deletions sql/2024/privacy/ads_accounts_distribution.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
WITH publishers AS (
SELECT
page,
JSON_QUERY(custom_metrics, '$.ads.ads.account_types') AS ads_account_types,
JSON_QUERY(custom_metrics, '$.ads.app_ads.account_types') AS app_ads_account_types
FROM `httparchive.all.pages`
WHERE date = '2024-06-01' AND
is_root_page = TRUE AND
(CAST(JSON_VALUE(custom_metrics, '$.ads.ads.account_count') AS INT64) > 0 OR
CAST(JSON_VALUE(custom_metrics, '$.ads.app_ads.account_count') AS INT64) > 0)
), ads_accounts AS (
SELECT
page,
CEIL(CAST(JSON_VALUE(ads_account_types, '$.direct.account_count') AS INT64) / 100) * 100 AS direct_account_count_bucket,
CEIL(CAST(JSON_VALUE(ads_account_types, '$.reseller.account_count') AS INT64) / 100) * 100 AS reseller_account_count_bucket,
COUNT(DISTINCT page) OVER () AS total_pages
FROM publishers
), app_ads_accounts AS (
SELECT
page,
CEIL(CAST(JSON_VALUE(app_ads_account_types, '$.direct.account_count') AS INT64) / 100) * 100 AS direct_account_count_bucket,
CEIL(CAST(JSON_VALUE(app_ads_account_types, '$.reseller.account_count') AS INT64) / 100) * 100 AS reseller_account_count_bucket,
COUNT(DISTINCT page) OVER () AS total_pages
FROM publishers
)

SELECT
'ads' AS source,
'direct' AS account_type,
direct_account_count_bucket AS account_count_bucket,
COUNT(DISTINCT page) / ANY_VALUE(total_pages) AS pct_pages,
COUNT(DISTINCT page) AS number_of_pages
FROM ads_accounts
GROUP BY source, direct_account_count_bucket
UNION ALL
SELECT
'ads' AS source,
'reseller' AS account_type,
reseller_account_count_bucket AS account_count_bucket,
COUNT(DISTINCT page) / ANY_VALUE(total_pages) AS pct_pages,
COUNT(DISTINCT page) AS number_of_pages
FROM ads_accounts
GROUP BY source, reseller_account_count_bucket
UNION ALL
SELECT
'app_ads' AS source,
'direct' AS account_type,
direct_account_count_bucket AS account_count_bucket,
COUNT(DISTINCT page) / ANY_VALUE(total_pages) AS pct_pages,
COUNT(DISTINCT page) AS number_of_pages
FROM app_ads_accounts
GROUP BY source, direct_account_count_bucket
UNION ALL
SELECT
'app_ads' AS source,
'reseller' AS account_type,
reseller_account_count_bucket AS account_count_bucket,
COUNT(DISTINCT page) / ANY_VALUE(total_pages) AS pct_pages,
COUNT(DISTINCT page) AS number_of_pages
FROM app_ads_accounts
GROUP BY source, reseller_account_count_bucket

ORDER BY account_count_bucket ASC
LIMIT 1000
114 changes: 114 additions & 0 deletions sql/2024/privacy/ads_and_sellers_graph.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
WITH RECURSIVE pages AS (
SELECT
CASE page -- Publisher websites may redirect to an SSP domain, and need to use redirected domain instead of page domain. CASE needs to be replaced with a more robust solution from HTTPArchive/custom-metrics#136.
WHEN 'https://www.chunkbase.com/' THEN 'cafemedia.com'
ELSE NET.REG_DOMAIN(page)
END AS page_domain,
JSON_QUERY(ANY_VALUE(custom_metrics), '$.ads') AS ads_metrics
FROM `httparchive.all.pages`
WHERE date = '2024-06-01' AND
is_root_page = TRUE
GROUP BY page_domain
), ads AS (
SELECT
page_domain,
JSON_QUERY(ads_metrics, '$.ads.account_types') AS ad_accounts
FROM pages
WHERE
CAST(JSON_VALUE(ads_metrics, '$.ads.account_count') AS INT64) > 0
), sellers AS (
SELECT
page_domain,
JSON_QUERY(ads_metrics, '$.sellers.seller_types') AS ad_sellers
FROM pages
WHERE
CAST(JSON_VALUE(ads_metrics, '$.sellers.seller_count') AS INT64) > 0
), relationships_web AS (
SELECT
NET.REG_DOMAIN(REGEXP_EXTRACT(NORMALIZE_AND_CASEFOLD(domain), r'\b[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}\b')) AS demand,
'Web' AS supply,
'direct' AS relationship,
page_domain AS publisher
FROM ads, UNNEST(JSON_VALUE_ARRAY(ad_accounts, '$.direct.domains')) AS domain
UNION ALL
SELECT
NET.REG_DOMAIN(REGEXP_EXTRACT(NORMALIZE_AND_CASEFOLD(domain), r'\b[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}\b')) AS demand,
'Web' AS supply,
'indirect' AS relationship,
page_domain AS publisher
FROM ads, UNNEST(JSON_VALUE_ARRAY(ad_accounts, '$.reseller.domains')) AS domain
UNION ALL
SELECT
page_domain AS demand,
'Web' AS supply,
'direct' AS relationship,
NET.REG_DOMAIN(REGEXP_EXTRACT(NORMALIZE_AND_CASEFOLD(domain), r'\b[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}\b')) AS publisher
FROM sellers, UNNEST(JSON_VALUE_ARRAY(ad_sellers, '$.publisher.domains')) AS domain
UNION ALL
SELECT
page_domain AS demand,
'Web' AS supply,
'direct' AS relationship,
NET.REG_DOMAIN(REGEXP_EXTRACT(NORMALIZE_AND_CASEFOLD(domain), r'\b[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}\b')) AS publisher
FROM sellers, UNNEST(JSON_VALUE_ARRAY(ad_sellers, '$.both.domains')) AS domain
), relationships_adtech AS (
SELECT
page_domain AS demand,
NET.REG_DOMAIN(REGEXP_EXTRACT(NORMALIZE_AND_CASEFOLD(domain), r'\b[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}\b')) AS supply,
'indirect' AS relationship
FROM sellers, UNNEST(JSON_VALUE_ARRAY(ad_sellers, '$.intermediary.domains')) AS domain
UNION ALL
SELECT
page_domain AS demand,
NET.REG_DOMAIN(REGEXP_EXTRACT(NORMALIZE_AND_CASEFOLD(domain), r'\b[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}\b')) AS supply,
'indirect' AS relationship
FROM sellers, UNNEST(JSON_VALUE_ARRAY(ad_sellers, '$.both.domains')) AS domain
), nodes AS (
(
SELECT
demand,
supply,
CONCAT(demand, '-', supply) AS path,
relationship,
HLL_COUNT.INIT(publisher) AS supply_sketch
FROM relationships_web
GROUP BY demand, supply, relationship
)
UNION ALL
(
SELECT
relationships_grouped.demand AS demand,
relationships_grouped.supply AS supply,
CONCAT(relationships_grouped.demand, '-', nodes.path) AS path,
relationships_grouped.relationship AS relationship,
nodes.supply_sketch AS supply_sketch
FROM (
SELECT
demand,
supply,
relationship
FROM relationships_adtech
GROUP BY
demand,
supply,
relationship
) AS relationships_grouped
INNER JOIN nodes
ON relationships_grouped.supply = nodes.demand AND
nodes.supply_sketch IS NOT NULL AND
nodes.relationship = 'indirect' AND
relationships_grouped.demand IS NOT NULL AND
STRPOS(nodes.path, relationships_grouped.demand) = 0
)
)

SELECT
supply,
demand,
HLL_COUNT.MERGE(supply_sketch) AS publishers_count,
relationship,
path
FROM nodes
GROUP BY demand, supply, relationship, path
ORDER BY publishers_count DESC
LIMIT 5000
45 changes: 45 additions & 0 deletions sql/2024/privacy/ads_lines_distribution.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
WITH RECURSIVE pages AS (
SELECT
CASE page -- publisher websites may redirect to an SSP domain, and need to use redirected domain instead of page domain
WHEN 'https://www.chunkbase.com/' THEN 'cafemedia.com'
ELSE NET.REG_DOMAIN(page)
END AS page,
CAST(JSON_VALUE(custom_metrics, '$.ads.ads.line_count') AS INT64) AS ads_line_count,
CAST(JSON_VALUE(custom_metrics, '$.ads.app_ads.line_count') AS INT64) AS app_ads_line_count
FROM `httparchive.all.pages`
WHERE date = '2024-06-01' AND
is_root_page = TRUE
), ads AS (
SELECT
page,
CEIL(ads_line_count / 100) * 100 AS line_count_bucket,
COUNT(DISTINCT page) OVER () AS total_pages
FROM pages
WHERE ads_line_count > 0
), app_ads AS (
SELECT
page,
CEIL(app_ads_line_count / 100) * 100 AS line_count_bucket,
COUNT(DISTINCT page) OVER () AS total_pages
FROM pages
WHERE app_ads_line_count > 0
)

SELECT
'ads.txt' AS type,
line_count_bucket,
COUNT(DISTINCT page) / ANY_VALUE(total_pages) AS pct_pages,
COUNT(DISTINCT page) AS number_of_pages
FROM ads
GROUP BY line_count_bucket
HAVING line_count_bucket <= 10000
UNION ALL
SELECT
'app-ads.txt' AS type,
line_count_bucket,
COUNT(DISTINCT page) / ANY_VALUE(total_pages) AS pct_pages,
COUNT(DISTINCT page) AS number_of_pages
FROM app_ads
GROUP BY line_count_bucket
HAVING line_count_bucket <= 10000
ORDER BY type, line_count_bucket ASC
31 changes: 31 additions & 0 deletions sql/2024/privacy/ccpa_most_common_phrases.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
WITH pages_with_phrase AS (
SELECT
client,
rank_grouping,
page,
COUNT(DISTINCT page) OVER (PARTITION BY client, rank_grouping) AS total_pages_with_phrase_in_rank_group,
JSON_QUERY_ARRAY(custom_metrics, '$.privacy.ccpa_link.CCPALinkPhrases') AS ccpa_link_phrases
FROM `httparchive.all.pages`, --TABLESAMPLE SYSTEM (0.01 PERCENT)
UNNEST([1000, 10000, 100000, 1000000, 10000000, 100000000]) AS rank_grouping
WHERE date = '2024-06-01' AND
is_root_page = true AND
rank <= rank_grouping AND
array_length(JSON_QUERY_ARRAY(custom_metrics, '$.privacy.ccpa_link.CCPALinkPhrases')) > 0
)

SELECT
client,
rank_grouping,
link_phrase,
COUNT(DISTINCT page) AS num_pages,
COUNT(DISTINCT page) / any_value(total_pages_with_phrase_in_rank_group) AS pct_pages
FROM pages_with_phrase,
UNNEST(ccpa_link_phrases) AS link_phrase
GROUP BY
link_phrase,
rank_grouping,
client
ORDER BY
rank_grouping,
client,
num_pages DESC
27 changes: 27 additions & 0 deletions sql/2024/privacy/ccpa_prevalence.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
WITH pages AS (
SELECT
client,
rank_grouping,
page,
JSON_VALUE(custom_metrics, '$.privacy.ccpa_link.hasCCPALink') AS has_ccpa_link
FROM `httparchive.all.pages`, -- TABLESAMPLE SYSTEM (0.0025 PERCENT)
UNNEST([1000, 10000, 100000, 1000000, 10000000, 100000000]) AS rank_grouping
WHERE date = '2024-06-01' AND
is_root_page = true AND
rank <= rank_grouping
)

SELECT
client,
rank_grouping,
has_ccpa_link,
COUNT(DISTINCT page) AS num_pages
FROM pages
GROUP BY
has_ccpa_link,
rank_grouping,
client
ORDER BY
rank_grouping,
client,
has_ccpa_link
29 changes: 29 additions & 0 deletions sql/2024/privacy/common_ads_variables.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
WITH RECURSIVE pages AS (
SELECT
page,
JSON_QUERY(custom_metrics, '$.ads.ads') AS ads_metrics
FROM `httparchive.all.pages`
WHERE
date = '2024-06-01' AND
is_root_page = TRUE AND
CAST(JSON_VALUE(custom_metrics, '$.ads.ads.account_count') AS INT64) > 0
), ads AS (
SELECT
page,
variable,
COUNT(DISTINCT page) OVER() AS total_publishers
FROM pages,
UNNEST(JSON_VALUE_ARRAY(ads_metrics, '$.variables')) AS variable
WHERE
CAST(JSON_VALUE(ads_metrics, '$.account_types.reseller.account_count') AS INT64) > 0 OR
CAST(JSON_VALUE(ads_metrics, '$.account_types.direct.account_count') AS INT64) > 0
)

SELECT
variable,
COUNT(DISTINCT page) / ANY_VALUE(total_publishers) AS pct_publishers,
COUNT(DISTINCT page) AS number_of_publishers
FROM ads
GROUP BY variable
ORDER BY pct_publishers DESC
LIMIT 100
35 changes: 35 additions & 0 deletions sql/2024/privacy/cookies_top_first_party_names.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
-- Most common cookie names, by number of domains on which they appear. Goal is to identify common trackers that use first-party cookies across sites.

WITH pages AS (
SELECT
client,
root_page,
custom_metrics,
COUNT(DISTINCT net.host(root_page)) OVER(PARTITION BY client) AS total_domains
FROM `httparchive.all.pages`
WHERE date = '2024-06-01'
), cookies AS (
SELECT
client,
cookie,
NET.HOST(JSON_VALUE(cookie, '$.domain')) AS cookie_host,
NET.HOST(root_page) AS firstparty_host,
total_domains
FROM pages,
UNNEST(JSON_QUERY_ARRAY(custom_metrics, '$.cookies')) AS cookie
)

SELECT
client,
COUNT(DISTINCT firstparty_host) AS domain_count,
COUNT(DISTINCT firstparty_host) / any_value(total_domains) AS pct_domains,
JSON_VALUE(cookie, '$.name') AS cookie_name
FROM cookies
WHERE firstparty_host LIKE '%' || cookie_host
GROUP BY
client,
cookie_name
ORDER BY
domain_count DESC,
client DESC
LIMIT 500
35 changes: 35 additions & 0 deletions sql/2024/privacy/cookies_top_third_party_domains.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
WITH pages AS (
SELECT
page,
client,
root_page,
custom_metrics,
COUNT(DISTINCT page) OVER (PARTITION BY client) AS total_pages
FROM `httparchive.all.pages`
WHERE date = '2024-06-01'
), cookies AS (
SELECT
client,
page,
cookie,
NET.HOST(JSON_VALUE(cookie, '$.domain')) AS cookie_host,
NET.HOST(root_page) AS firstparty_host,
total_pages
FROM pages,
UNNEST(JSON_QUERY_ARRAY(custom_metrics, '$.cookies')) AS cookie
)

SELECT
client,
cookie_host,
COUNT(DISTINCT page) AS page_count,
COUNT(DISTINCT page) / any_value(total_pages) AS pct_pages
FROM cookies
WHERE firstparty_host NOT LIKE '%' || cookie_host
GROUP BY
client,
cookie_host
ORDER BY
page_count DESC,
client
LIMIT 500
Loading

0 comments on commit 324d22b

Please sign in to comment.