Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

webconnectivity: investigate failure=nil and blocking=nil #1511

Closed
bassosimone opened this issue Dec 21, 2020 · 2 comments · Fixed by ooni/probe-cli#1472
Closed

webconnectivity: investigate failure=nil and blocking=nil #1511

bassosimone opened this issue Dec 21, 2020 · 2 comments · Fixed by ooni/probe-cli#1472
Assignees
@bassosimone
Labels
2024-01-data-quality-cleanup Data quality issues addressed on 2024-01 bug Something isn't working data quality ooni/probe-engine priority/low

Comments

@bassosimone
Copy link
Contributor 

Starting measurement with index 2
processing http://www.latinmail.com/
dnslookup://www.latinmail.com...
dnslookup://www.latinmail.com... dns_nxdomain_error
control http://www.latinmail.com/...
control http://www.latinmail.com/... generic_timeout_error
DNS analysis result: nil
TCP/TLS endpoints: 0/0 reachable
BodyLengthMatch: nil
BodyProportion: 0
StatusCodeMatch: nil
HeadersMatch: nil
TitleMatch: nil
Blocking: nil
Accessible: nil

and

Starting measurement with index 1
processing https://8.8.4.4/dns-query?dns=q80BAAABAAAAAAAAA3d3dwdleGFtcGxlA2NvbQAAAQAB
dnslookup://8.8.4.4...
dnslookup://8.8.4.4... <nil>
control https://8.8.4.4/dns-query?dns=q80BAAABAAAAAAAAA3d3dwdleGFtcGxlA2NvbQAAAQAB...
control https://8.8.4.4/dns-query?dns=q80BAAABAAAAAAAAA3d3dwdleGFtcGxlA2NvbQAAAQAB... <nil>
DNS analysis result: consistent
TCP/TLS endpoints: 0/0 reachable
BodyLengthMatch: nil
BodyProportion: 0
StatusCodeMatch: nil
HeadersMatch: nil
TitleMatch: nil
Blocking: nil
Accessible: nil
@bassosimone bassosimone self-assigned this Dec 21, 2020
@bassosimone bassosimone transferred this issue from ooni/probe-engine Jun 7, 2021
@bassosimone bassosimone added the bug Something isn't working label Jul 6, 2021
@bassosimone
Copy link
Contributor Author

Web Connectivity v0.5 correctly flags the first website as down: https://explorer.ooni.org/m/20240124141110.292023_IT_webconnectivity_0321bf33bc821d88.

It still flags the second website as error, so there's a need to investigate more: https://explorer.ooni.org/m/20240124141213.940181_IT_webconnectivity_ec61e2543261a5fe

bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 24, 2024
@bassosimone
chore: add initial data to understand the bug
9b591d1 
See ooni/probe#1511 (comment)
@bassosimone bassosimone mentioned this issue Jan 24, 2024
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 24, 2024
@bassosimone
chore(minipipeline): add data to understand ooni/probe#1511 (#1471)
4fb9f0f 
This commit adds data collected from
ooni/probe#1511 (comment) that
helps us to understand why `minipipeline` flags the measurement as
`accessible = null` and `blocking = null`.
@bassosimone bassosimone mentioned this issue Jan 24, 2024
Merged
bassosimone added a commit to ooni/probe-cli that referenced this issue Jan 24, 2024
@bassosimone
fix(minipipeline): handle IP-addr URLs in classic linear analysis (#1472
2cc9231 
)

This diff fixes
ooni/probe#1511 (comment).

This is the rationale of the diff: we need to track the origin of IP
addresses:

* "dns" if discovered using DNS;
* "th" if discovered using the test helper;
* null otherwise.

When filtering for classic analysis, we include "dns" entries if
resolved using getaddrinfo, drop "th" entries because they're not
relevant, and include null entries under the assumption that the probe
discovered them either directly from the input URL or because a redirect
redirected to an URL containing an IP address.

We also update the minipipeline test suite and show that the only
changes are related to the test added by
#1471.
@bassosimone
Copy link
Contributor Author

Here's a measurement after we fixed the issue with the second input URL: https://explorer.ooni.org/m/20240124155344.998365_IT_webconnectivity_ae1968b0b836d2e6.

@bassosimone bassosimone added the 2024-01-data-quality-cleanup Data quality issues addressed on 2024-01 label Jan 26, 2024
Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
chore(minipipeline): add data to understand ooni/probe#1511 (ooni#1471)
63d31d2 
This commit adds data collected from
ooni/probe#1511 (comment) that
helps us to understand why `minipipeline` flags the measurement as
`accessible = null` and `blocking = null`.
Murphy-OrangeMud pushed a commit to Murphy-OrangeMud/probe-cli that referenced this issue Feb 13, 2024
@bassosimone @Murphy-OrangeMud
fix(minipipeline): handle IP-addr URLs in classic linear analysis (oo…
51f2a0a 
…ni#1472)

This diff fixes
ooni/probe#1511 (comment).

This is the rationale of the diff: we need to track the origin of IP
addresses:

* "dns" if discovered using DNS;
* "th" if discovered using the test helper;
* null otherwise.

When filtering for classic analysis, we include "dns" entries if
resolved using getaddrinfo, drop "th" entries because they're not
relevant, and include null entries under the assumption that the probe
discovered them either directly from the input URL or because a redirect
redirected to an URL containing an IP address.

We also update the minipipeline test suite and show that the only
changes are related to the test added by
ooni#1471.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bassosimone bassosimone

Labels
2024-01-data-quality-cleanup Data quality issues addressed on 2024-01 bug Something isn't working data quality ooni/probe-engine priority/low
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(minipipeline): handle IP-addr URLs in classic linear analysis ooni/probe-cli
2 participants
@bassosimone @hellais