Implement measurements using decorator pattern #529
Conversation
This also includes a new experiment called urlgetter that can be used to perform URL measurements like: ``` ./miniooni -i http://www.kernel.org urlgetter ./miniooni -OTLSServerName=x.org -i http://www.kernel.org urlgetter ./miniooni -OResolverURL=https://cloudflare-dns.com/dns-query \ -i http://www.kernel.org urlgetter ``` Part of #509
Conflicts: netx/httptransport/httptransport.go
Conflicts: netx/dialer/saver.go
Conflicts: netx/httptransport/saver.go
Conflicts: netx/httptransport/httptransport.go netx/httptransport/saver.go
This is related to https://github.com/ooni/probe-engine/issues/290, which is not done, because we're still using old netx in some places. Part of #543
| exit 1 | ||
| } | ||
|
|
||
| if [ $# -ne 1 ]; then |
There was a problem hiding this comment.
Prefer [[]] to []: https://google.github.io/styleguide/shellguide.html#s6.3-tests
BTW, that's a pretty good style guide. You may want to use it if you don't already have one.
There was a problem hiding this comment.
Thanks for letting me know about this style guide, I was not aware of that!
| fi | ||
| } | ||
|
|
||
| require jq "sudo apt install jq" |
There was a problem hiding this comment.
I recommend you put all statements that are executed inside a main() function, so it's easier to read and understand. Then you call main "$@" at the bottom.
| tail -n1 report.jsonl|jq -r ".test_keys.requests|.[0]|.response.body" | ||
| } | ||
|
|
||
| testdomain=${MINIOONI_TEST_DOMAIN:-example.org} |
There was a problem hiding this comment.
You may want to organize the code below in functions, so it's easier to read.
| # XXX: maybe make these two checks separate checks for now? | ||
| urlgetter -OResolverURL=system:/// -ORejectDNSBogons=true -i dnslookup://$domain | ||
| if [ "$(getfailure)" = "dns_bogon_error" ]; then | ||
| output "MINIOONI_DNS_BOGONS=1" |
There was a problem hiding this comment.
This doesn't really explain the censorship mechanism.
Was the bogon injected or did the resolver lie?
That's what matters the most. The bogon detection is just a way to validate the IP address, but the result should be DNS lie or DNS injection.
There was a problem hiding this comment.
I agree. I think this script was an initial experiment to understand how an higher level logic for running measurements (i.e. an improved web connectivity) could work. For this reason, I didn't bother with implementing an algorithm for providing output to users of the script. The goal is to understand a good algorithm and then implement it in Go.
bassosimone
added
priority/low
|
This pull request has served its purpose of providing us with a PoC regarding how to implement better measurements. We should spend some time as part of Sprint 21 trying to understand if there is still valuable information that we can extract from here as a set of issues, or whether we can simply close this PR, delete the associated branch, and move on. |
|
Effort is |
|
So, the remaining part of this diff that has not been merged into master only exposes my lack of good shell writing skills and does not otherwise carry much meaningful content. The problem of improving Web Connectivity has not been solved yet but the place where to do that is clearly not this pull request. This is for many reasons, including that I'd like the improved domain checking code to be implemented in Go and to be callable as a OONI experiment. So, safe to close this PR. |
This also includes a new experiment called urlgetter that can
be used to perform URL measurements like:
Part of #509