created a data policy for OONI

[agrabeli]

I wrote a Data Policy for OONI. All feedback welcome. :)

[bassosimone]

Perhaps it would be good to point to a piece of documentation where it is explained how to do so? My initial suggestion was to provide such information inline, except that linking is more robust in the long term to updates in the config file syntax.

[agrabeli]

Agreed. I added a link to the configuration section of the ooniprobe README. Currently this documentation is not comprehensive, but I imagine that it will get updated soon and the link here will point to that,

[bassosimone]

I really like what I've read, well done! I've added some comments in specific places where I think it would make sense to invest some more effort to improve the text!

[anadahz]

We should ensure that no web server logs are being kept.

[hellais]

FYI, this is the log configuration for ooni.tpo: LogFormat "0.0.0.1 - %u %{[%d/%b/%Y:00:00:00 %z]}t \"%r\" %>s %b \"%{Referer}i\" \"-\" %{Age}o"

[agrabeli]

Good point! @hellais just checked this and I'm editing the document now.

[anadahz]

@hellais does this applies to both access and error HTTP logs?

[anadahz]

For measurements.ooni.torproject.org and explorer.ooni.torproject.org :
Error log: /dev/null
Access log:

ap $remote_addr $anon_remote_addr {
          "~^(?<ip_a>\d+\.\d+)\.\d+\.\d+" "$ip_a";
        }
        log_format combined_anon '$anon_remote_addr.0.0 - $remote_user [$time_local] '
                                 '"$request" $status $body_bytes_sent '
                                 '"$http_referer" "$http_user_agent"';

@hellais are there more servers that we don't cover?

[hellais]

@anadahz I was told the error log may include more information for ooni.torproject.org, though it wasn't specified exactly what that could be.

[hellais]

Hum, maybe we should not be logging the http_user_agent and the first 2 octets of the IP.

[anadahz]

@hellais any suggestions where to find more info about the error log info of ooni.torproject.org does this collect the same info as in www.torproject.org?

Access.log suggestion:

log_format combined_anon '0.0.0.9 - $remote_user [$time_local] '
                                 '"$request" $status $body_bytes_sent '
                                 '"$http_referer"';

[anadahz]

It seems that there is a lot of text duplication, perhaps it will make sense to have
a section with the same text for all affected subsections that share the same
text.
It will be beneficial to add some ooni-spec wiki pages relevant to the
data policy.

Overall really good work thank you for working on this!

[bassosimone]

It seems that there is a lot of text duplication [...]

Yes, I noticed the same and was about to make a similar remarks. Then, I though that perhaps, since this document is meant to be read more by users than by developers, some redundancy inside it would have been beneficial to the reader.

[hellais]

Note to self. Fix the link in the merge.

[hellais]

Yeah I think it's ok that there is duplication as I would expect that somebody would read this not necessarily entirely, but just the specific sections they are interested in.
As such I think it's good to have things re-iterated in all relevant sections.

As soon as the following issues:

• created a data policy for OONI #57 (comment)
• created a data policy for OONI #57 (comment)
• created a data policy for OONI #57 (comment)

are resolved I will proceed in merging this.

[hellais]

This looks good to me. I think it's good to merge.

[hellais]

I think this is ready to merge. Will cleanup the broken links and integrate into master.