Rewrite Web Connectivity in Go (2/n): QA #810

bassosimone · 2020-07-17T13:49:46Z

We have the code since #776. Now we wanna do QA.

Previous changes made it much more difficult to run on Linux. Yet, to do QA of Web Connectivity it's more practical to perform it on Linux using also OONI Probe v2.x and Measurement Kit. For this reason, this diff implements changes that should allow us to run both on Linux and when using Docker. While it's possible to create a Docker container where we deploy compiled versions of MK and OONI Probe v2.x, it's more practical to use a Linux system where they are already installed, like my Linux desktop. Part of #810

1. we cannot write on QA/HOME when running on Linux so just use `/tmp` as the HOME directory (slower but works) 2. the `/tmp` directory may be on another device than `/home`, so we cannot move, then just perform a copy Part of #810

* QA: make sure we can run from Linux Previous changes made it much more difficult to run on Linux. Yet, to do QA of Web Connectivity it's more practical to perform it on Linux using also OONI Probe v2.x and Measurement Kit. For this reason, this diff implements changes that should allow us to run both on Linux and when using Docker. While it's possible to create a Docker container where we deploy compiled versions of MK and OONI Probe v2.x, it's more practical to use a Linux system where they are already installed, like my Linux desktop. Part of #810 * QA: fix running on Linux 1. we cannot write on QA/HOME when running on Linux so just use `/tmp` as the HOME directory (slower but works) 2. the `/tmp` directory may be on another device than `/home`, so we cannot move, then just perform a copy Part of #810

Part of #810

This starts adding a single check, to check whether the framework works for Web Connectivity. More checks later. Part of #810

This will be very helpful for #810. It will allow us, in fact, to compare the result obtained by MK to the one obtained by miniooni in specific censorship conditions.

Spotted while comparing with Measurement Kit. Part of #810.

According to the implementations, the semantics is that blocking is false when accessible is true, a nullable string otherwise. The diff documents in detail how and why this was introduced. It would probably have been better to not make this field semantics so complicate, but now it's too late. Maybe in the future we can have some extra field with a more clear and simple semantics? Part of #810

Part of ooni/probe-engine#810. See ooni/probe-engine#839 for explanation.

#200) Part of ooni/probe-engine#810. See ooni/probe-engine#839 for explanation.

* fix(webconnectivity): header_match => headers_match Spotted while comparing with Measurement Kit. Part of #810. * webconnectivity: fix the semantics of blocking According to the implementations, the semantics is that blocking is false when accessible is true, a nullable string otherwise. The diff documents in detail how and why this was introduced. It would probably have been better to not make this field semantics so complicate, but now it's too late. Maybe in the future we can have some extra field with a more clear and simple semantics? Part of #810 * webconnectivity/summary.go: better docstring * webconnectivity: fix summary_test.go

Verified locally that we behave like MK. Part of #810.

We already know that probe-engine does better than MK when there is a control failure and the website is using HTTPS and we can access it. MK would flag such case as failed measurement, where instead probe-engine would flag such case as success. What we can instead test here is what happens if the same happens but we're accessing a website using HTTP rather than HTTPS. This QA test allowed us to discover the following issues: 1. MK always sets the body proportion to zero, even when performing such a comparison is not actually possible, so do the same also in probe-engine even though probably using `null` would be a more consistent result 2. MK sets the DNSConsistency field only when the control succeded and we should do the same rather than flagging the DNS as consistent This diff includes both the diff adding the new QA check and the required fixes in probe-engine required to address the two above issues. Part of #810

This is a rather common blocking case. Let us make sure we measure this case with OPE like we do with MK. I needed to remove the checks pertaining to the in depth structure of the JSON, because fields are missing in JSON files generated by MK. We know these bugs. Part of #810

bassosimone · 2020-08-14T10:02:34Z

Also #844 was part of this issue.

Make sure we do the same MK would do in this case. Part of #810.

This impacts on https://github.com/ooni/probe-engine/issues/579. Part of #810.

Part of #810 Objective is making sure we don't do different from MK as much as possible as far as we can predict censorship cases.

Part of #810

Part of #810 Strictly speaking we don't need a redirect chain to perform this measurement but we have been hiding the initial request from TCP/IP errors, so let's keep using that pattern.

Again, likely doing the redirect chain is not necessary. But it makes things more interesting, because we are testing stuff that happens after the initial DNS, TCP, etc. checks performed by Web Connectivity. Part of #810

Part of #810

bassosimone · 2020-08-17T09:06:07Z

I've been doing a bunch of work in Sprint 19. The work is not complete, though, so I created a new issue to keep track of the additional QA efforts required before declaring the reimplementation successful: #852.

bassosimone added this to the Sprint 18 - Colomber milestone Jul 17, 2020

bassosimone self-assigned this Jul 17, 2020

bassosimone mentioned this issue Jul 17, 2020

Rewrite Web Connectivity in Go (1/n): write the code #776

Closed

bassosimone modified the milestones: Sprint 18 - Colomber, Sprint 19 - Bottlenose Jul 19, 2020

bassosimone added effort/L Large effort priority/high High priority labels Jul 19, 2020

bassosimone mentioned this issue Aug 12, 2020

QA: make sure we can run on Linux without Docker #836

Merged

bassosimone mentioned this issue Aug 12, 2020

Build static MK binary measurement-kit/script-build-linux#8

Open

bassosimone added a commit that referenced this issue Aug 12, 2020

QA: implement checks for Web Connectivity

05b4a71

Part of #810

bassosimone mentioned this issue Aug 12, 2020

QA: start implementing checks for Web Connectivity #837

Merged

bassosimone added a commit that referenced this issue Aug 12, 2020

QA: implement checks for Web Connectivity (#837)

944a390

This starts adding a single check, to check whether the framework works for Web Connectivity. More checks later. Part of #810

bassosimone added a commit that referenced this issue Aug 12, 2020

QA: add support for running measurement_kit

099573d

This will be very helpful for #810. It will allow us, in fact, to compare the result obtained by MK to the one obtained by miniooni in specific censorship conditions.

bassosimone mentioned this issue Aug 12, 2020

QA: add support for running measurement_kit #838

Merged

bassosimone added a commit that referenced this issue Aug 12, 2020

QA: add support for running measurement_kit (#838)

e403704

This will be very helpful for #810. It will allow us, in fact, to compare the result obtained by MK to the one obtained by miniooni in specific censorship conditions.

bassosimone added a commit that referenced this issue Aug 13, 2020

fix(webconnectivity): header_match => headers_match

cdb1ebc

Spotted while comparing with Measurement Kit. Part of #810.

bassosimone mentioned this issue Aug 13, 2020

QA(webconnectivity): make sure output matches MK #839

Merged

bassosimone added a commit to ooni/spec that referenced this issue Aug 13, 2020

ts-017-web-connectivity.md: document that blocking could also be false

8bc755e

Part of ooni/probe-engine#810. See ooni/probe-engine#839 for explanation.

bassosimone mentioned this issue Aug 13, 2020

ts-017-web-connectivity.md: document that blocking could also be false ooni/spec#200

Merged

bassosimone added a commit to ooni/spec that referenced this issue Aug 13, 2020

ts-017-web-connectivity.md: document that blocking could also be false (

6fd578f

#200) Part of ooni/probe-engine#810. See ooni/probe-engine#839 for explanation.

bassosimone added a commit that referenced this issue Aug 13, 2020

QA/webconnectivity: check case of DNS based hijacking

1ca923c

Verified locally that we behave like MK. Part of #810.

bassosimone mentioned this issue Aug 13, 2020

QA/webconnectivity: check case of DNS based hijacking #840

Merged

bassosimone added a commit that referenced this issue Aug 13, 2020

QA/webconnectivity: check case of DNS based hijacking (#840)

f374896

Verified locally that we behave like MK. Part of #810.

bassosimone mentioned this issue Aug 13, 2020

QA/webconnectivity: control failure w/ HTTP website #841

Merged

bassosimone mentioned this issue Aug 14, 2020

QA/webconnectivity: blocked IP w/ consistent DNS #843

Merged

bassosimone added a commit that referenced this issue Aug 14, 2020

QA/webconnectivity: connection refused in redirect chain

27d8d9b

Make sure we do the same MK would do in this case. Part of #810.

bassosimone mentioned this issue Aug 14, 2020

QA/webconnectivity: connection refused in redirect chain #845

Merged

bassosimone added a commit that referenced this issue Aug 14, 2020

QA/webconnectivity: connection refused in redirect chain (#845)

7c5be53

Make sure we do the same MK would do in this case. Part of #810.

bassosimone added a commit that referenced this issue Aug 14, 2020

QA/webconnectivity: do like MK when domain is NXDOMAIN

b597576

This impacts on https://github.com/ooni/probe-engine/issues/579. Part of #810.

bassosimone mentioned this issue Aug 14, 2020

QA/webconnectivity: do like MK when domain is NXDOMAIN #846

Merged

bassosimone added a commit that referenced this issue Aug 14, 2020

QA/webconnectivity: do like MK when domain is NXDOMAIN (#846)

0e4aebb

This impacts on https://github.com/ooni/probe-engine/issues/579. Part of #810.

bassosimone added a commit that referenced this issue Aug 14, 2020

QA/webconnectivity: RST happens after redirect

f53afa4

Part of #810 Objective is making sure we don't do different from MK as much as possible as far as we can predict censorship cases.

bassosimone mentioned this issue Aug 14, 2020

QA/webconnectivity: RST happens after redirect #847

Merged

bassosimone added a commit that referenced this issue Aug 14, 2020

QA/webconnectivity: RST happens after redirect (#847)

753c56a

Part of #810 Objective is making sure we don't do different from MK as much as possible as far as we can predict censorship cases.

bassosimone added a commit that referenced this issue Aug 14, 2020

QA/webconnectivity: DNS failure in the redirect chain

b03beb4

Part of #810

bassosimone mentioned this issue Aug 14, 2020

QA/webconnectivity: DNS failure in the redirect chain #848

Merged

bassosimone added a commit that referenced this issue Aug 14, 2020

QA/webconnectivity: DNS failure in the redirect chain (#848)

7776cd5

Part of #810

bassosimone mentioned this issue Aug 14, 2020

QA/webconnectivity: EOF in the redirect chain #849

Merged

bassosimone mentioned this issue Aug 14, 2020

QA/webconnectivity: timeout error in redirect chain #850

Merged

bassosimone added a commit that referenced this issue Aug 14, 2020

QA/webconnectivity: HTTP reset w/ inconsistent DNS

7399cb6

Part of #810

bassosimone mentioned this issue Aug 14, 2020

QA/webconnectivity: HTTP reset w/ inconsistent DNS #851

Merged

bassosimone added a commit that referenced this issue Aug 14, 2020

QA/webconnectivity: HTTP reset w/ inconsistent DNS (#851)

d7ef4a9

Part of #810

bassosimone mentioned this issue Aug 17, 2020

Rewrite Web Connectivity in Go (3/n): QA #852

Closed

10 tasks

bassosimone closed this as completed Aug 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rewrite Web Connectivity in Go (2/n): QA #810

Rewrite Web Connectivity in Go (2/n): QA #810

bassosimone commented Jul 17, 2020

bassosimone commented Aug 14, 2020

bassosimone commented Aug 17, 2020

Rewrite Web Connectivity in Go (2/n): QA #810

Rewrite Web Connectivity in Go (2/n): QA #810

Comments

bassosimone commented Jul 17, 2020

bassosimone commented Aug 14, 2020

bassosimone commented Aug 17, 2020