Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calculate the blocking key needed by OONI Explorer measurement pages #326

Open
hellais opened this issue Feb 4, 2020 · 3 comments
Open
Assignees
Labels
discuss Needs discussion before proceeding ooni/backend Issues related to https://github.com/ooni/backend ooni/pipeline Issues related to https://github.com/ooni/pipeline priority/medium Normal priority issue

Comments

@hellais
Copy link
Member

hellais commented Feb 4, 2020

The web_connectivity measurement details page of OONI Explorer uses the test_keys->blocking key of the raw measurements (which is calculated on the probe) that can be mis-aligned with what the fastpath believes to be the blocking reason.

We should re-compute the blocking reason in the fastpath and serve it to OONI Explorer.

This issue superseeds: ooni/explorer#360

@hellais hellais added discuss Needs discussion before proceeding ooni/backend Issues related to https://github.com/ooni/backend ooni/pipeline Issues related to https://github.com/ooni/pipeline priority/medium Normal priority issue labels Feb 4, 2020
@hellais hellais added this to the Sprint 7 - Crown jellyfish milestone Feb 4, 2020
@azadi
Copy link

azadi commented Feb 5, 2020

I am not sure if this is related but I will hopefully try to be useful:

The snippet below is from running a query on metadb:

   test_start_time   | measurement_start_time |              measurement_id               | m_report_no | anomaly | confirmed | coalesce_2 |                                                            scores                                                            | exc | residual_no |                                  report_id                                  | probe_cc | probe_asn |    test_name     |           input           
---------------------+------------------------+-------------------------------------------+-------------+---------+-----------+------------+------------------------------------------------------------------------------------------------------------------------------+-----+-------------+-----------------------------------------------------------------------------+----------+-----------+------------------+---------------------------
 2020-02-02 20:49:59 | 2020-02-02 20:49:59    | temp-fid-00d4fe139c14b9c6a86e3d5a0f6e16ff |    12411773 | f       | f         | f          | {"blocking_general":0.0,"blocking_global":0.0,"blocking_country":0.0,"blocking_isp":0.0,"blocking_local":0.0,"accuracy":0.0} | {0} |     1716728 | 20200202T204924Z_AS56402_FFVjCWvZqfWrhAZgXuJahKRGZjikgmUyJ5zerNdcuXmdepZHVo | IR       |     56402 | web_connectivity | https://ca.wikipedia.org/

But if you see the measurement on OONI Explorer (https://explorer.ooni.org/measurement/20200202T204924Z_AS56402_FFVjCWvZqfWrhAZgXuJahKRGZjikgmUyJ5zerNdcuXmdepZHVo?input=https%3A//ca.wikipedia.org/), it says that this is an anomaly and rightfully so.

Anyway, just thought I should share in case this was related.

@hellais
Copy link
Member Author

hellais commented Jun 22, 2021

@FedericoCeratto can you check if this is currently done?

@hellais
Copy link
Member Author

hellais commented Jan 23, 2025

This is now possible thanks to the metrics from the new pipeline. In order for it to work effectively we probably have to reduce the frequency of the runs of the new pipeline and possible plug it in to process data in realtime.

See: ooni/data#111

@hellais hellais moved this to Backlog in Roadmap Jan 23, 2025
@hellais hellais removed this from Roadmap Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discuss Needs discussion before proceeding ooni/backend Issues related to https://github.com/ooni/backend ooni/pipeline Issues related to https://github.com/ooni/pipeline priority/medium Normal priority issue
Projects
None yet
Development

No branches or pull requests

6 participants