feat(torsf): collect tor logs, select rendezvous method, count bytes

[bassosimone]

Checklist

• I have read the contribution guidelines
• reference issue for this pull request: qa(torsf): figure out proper configuration to help snowflake devs collecting useful data probe#2004, torsf: improvements over initial proof of concept probe#1686
• if you changed anything related how experiments work and you need to reflect these changes in the ooni/spec repository, please link to the related ooni/spec pull request: torsf: document persistent datadir and rendezvous method spec#231

Description

This diff contains significant improvements over the previous
implementation of the torsf experiment.

We add support for configuring different rendezvous methods after
the convo at ooni/probe#2004. In doing
that, I've tried to use a terminology that is consistent with the
names being actually used by tor developers.

In terms of what to do next, this diff basically instruments
torsf to always rendezvous using domain fronting. Yet, it's also
possible to change the rendezvous method from the command line,
when using miniooni, which allows to experiment a bit more. In the
same vein, by default we use a persistent tor datadir, but it's
also possible to use a temporary datadir using the cmdline.

Here's how a generic invocation of torsf looks like:

./miniooni -O DisablePersistentDatadir=true \
           -O RendezvousMethod=amp \
           -O DisableProgress=true \
           torsf

(The default is DisablePersistentDatadir=false and
RendezvousMethod=domain_fronting.)

With this implementation, we can start measuring whether snowflake
and tor together can boostrap, which seems the most important thing
to focus on at the beginning. Understanding why the bootstrap most
often does not converge with a temporary datadir on Android devices
remains instead an open problem for now. (I'll also update the
relevant issues or create new issues after commit this.)

We also address some methodology improvements that were proposed
in ooni/probe#1686. Namely:

• we record the tor version;

• we include the bootstrap percentage because of the logs;

• we set the anomaly key correctly;

• we measure the bytes send and received (by tor).

What remains to be done is the possibility of including Snowflake
events into the measurement, which is not possible until the new
improvements at common/event in snowflake.git are included into a
tagged version of snowflake itself. (I'll make sure to mention
this aspect to @cohosh in ooni/probe#2004.)

[bassosimone]

First pass of self review. I've highlighted when we should improve with testing and have added comments for the reviewers.

[bassosimone]

@hellais this pull request is not complete because it's still missing to adjust tests. However, it already contains ~finished code for methodological improvements with torsf. Thus, it would be great if you could (1) take a look at the diff with particular attention on the sections that I've marked as important and (2) test from the command line using miniooni (build with go build -v ./internal/cmd/miniooni, see the initial comment for usage info) and report back on the usefulness of the data structures that I am currently collecting. Thanks a lot! 🙏

[bassosimone]

This PR is approaching readiness state but there's still need to address a couple of comments!

[hellais]

In doing some testing of this, I noticed that if I run the torsf test without any arguments in miniooni I get the following error:

miniooni torsf
[      0.000222] <info> Current time: 2022-02-07 14:57:03 CET
[      0.002505] <info> miniooni home directory: $HOME/.miniooni
[      0.002755] <info> Looking up OONI backends; please be patient...
[      0.733684] <info> session: using probe services: {Address:https://ps1.ooni.io Type:https Front:}
[      0.733706] <info> Looking up your location; please be patient...
[      4.567626] <info> - country: IT
[      4.567656] <info> - network: Vodafone Italia S.p.A. (AS30722)
[      4.567666] <info> - resolver's IP: 172.253.12.129
[      4.567670] <info> - resolver's network: Google LLC (AS15169)
[      4.567683] <warn> cannot create experiment builder: map[error:no such experiment: torsf]
[      4.568241] <info> sessionresolver: failure rate: primary: 4/4; fallback: 0/4
[      4.568447] <info> whole session: recv   7 Mbyte, sent   5 kbyte
cannot create experiment builder%

I guess it's fine to require specifying correct arguments as miniooni is an experimental client, but if it's not too hard we might want to still support running the test with default arguments or improve the returned error (the fact I read map[error:no such experiment: torsf], made me wonder if I was on the correct branch).

Ignore this, I was running the miniooni I had in the PATH and not the one I built from the branch. Ignore this noise.

[bassosimone]

I guess it's fine to require specifying correct arguments as miniooni is an experimental client, but if it's not too hard we might want to still support running the test with default arguments or improve the returned error (the fact I read map[error:no such experiment: torsf], made me wonder if I was on the correct branch).

I find this error very surprising. ./miniooni torsf should work just fine. What is your current branch and what is its tip?

[hellais]

LGTM