OONI Probe CLI v3.13.0-beta

This is what changed since v3.11.0-beta.2:

• 🐛 fix(webconnectivity): send specific user agent (#615)
• 🚧 refactor(oonimkall): merge internal/task into oonimkall (#617)
• 🐛 fix(oonimkall): don't close channel to signal end of task (#619)
• 🚧 [backport] fix(oonimkall): improve channel management pattern (#621) #622
• 🚧 fix(oonimkall): make logger used by tasks unit testable (#623)
• 🚧 feat(oonimkall): improve taskEmitter testability (#624)
• 🚧 refactor(oonimkall): make the runner unit-testable (#625)
• 🚧 feat(oonimkall): write unit tests for the runner component (#626)
• 🚧 refactor(oonimkall): merge files, use proper names, zap unneeded integration tests #627
• 🚧 refactor(oonimkall): we can simplify StartTask tests (#628)
• 🚧 Backport patches mentioned by ooni/probe#1912 (#635)
• 🗒️ release 3.11: update all the dependencies (#636)

All these changes were aiming at making ooniprobe run experimental work as intended. While there, we spent some time improving the oonimkall mobile library implementation to make it more correct and more easily unit testable.

We also updated dependencies.

OONI Probe CLI v3.11.0-beta.2

This is what changed since v3.11.0-beta.1:

• 🐛 fix(signal): use the correct string to score a measurement (#602)
• 🐛 fix(mk): do nothing if we already have psiphon config (#605)
• ✨ [backport] feat: publish miniooni binaries into releases (#607) (#608)
• ✨ ci/cd: publish binaries onto a release when we create a tag (#609)
• ❗ because of automatic cloud builds, we're not using the PRODUCT-staging branches anymore. If you were tracking those branches, now you should not track them anymore and just track release branches. (Sorry, about this change of plans!)

The most tangible advancement in this release is that now we're able to automatically produce cloud builds. The assets in this release are indeed all generated using GitHub actions.

OONI Probe CLI v3.12.0-alpha.3

This is what changed since v3.12.0-alpha.2:

• 🐛 [forwardport] fix(signal): use the correct string to score a measurement (#602) #603
• 🗒️ [ci skip] doc: explain staging branches and binary builds #604
• 🐛 ✨ fix(mk): do nothing if we already have psiphon config #605
• ✨ feat: publish miniooni binaries into releases #607
• ❗ because of automatic cloud builds, we're not using the PRODUCT-staging branches anymore. If you were tracking those branches, now you should not track them anymore and just track release branches. (Sorry, about this change of plans!)

The main change here is that we're transitioning to a model where we automatically publish release binaries build when tagging.

OONI Probe CLI v3.12.0-alpha.2

This release follows v3.12.0-alpha.1 and contains a set of improvements to repair the Android and iOS builds, as well as other data quality fixes forwardported from the release/3.11 branch.

• 🗒️ fix(CONTRIBUTING.md): require specific commit message for fwd/back ports #577
• 🗒️ cleanup(ooniprobe): remove unused send_crash_report option #578
• 🐛 fix(webconnectivity): ignore any status code <= 0 #579
• 🐛 fix(measurex): allow API user to choose parallelism #581
• 🗒️ doc(tutorial): explain the big picture #582
• 🧪 feat(QA): add test cases for websteps vs webconnectivity #583
• 🐛 fix(netxlite): ensure HTTP errors are always wrapped #584
• 🐛 [forwardport] fix(netxlite): remove android-specific errno (#586) #587
• 🗒️ [forwardport] chore: use go1.17.3 (#588) #589
• 🗒️ [forwardport] mk: update ooni/go and Android build tools (#590) #591
• 🐛 [forwardport] fix: avoid http3 for dns.google and www.google.com (#593) #594
• 🐛 fix(geolocate): use powerdns.org whoami service #592
• 🐛 fix(psiphon): increase the maximum runtime to 300s #596
• 🐛 [forwardport] fix(ios): make sure we can build an iOS release (#598) #599

This is a routine release to keep track of progress in this development sprint.

OONI Probe CLI v3.11.0-beta.1

This release follows v3.11.0-beta and contains a set of improvements to repair the Android and iOS builds, as well as other data quality fixes backported from the master branch.

• 🐛 [backport] fix(webconnectivity): ignore any status code <= 0 (#579) #580
• 🐛 [backport] fix(netxlite): ensure HTTP errors are always wrapped (#584) #585
• 🐛 fix(netxlite): remove android-specific errno #586
• 🗒️ chore: use go1.17.3 #588
• 🗒️ mk: update ooni/go and Android build tools #590
• 🐛 fix: avoid http3 for dns.google and www.google.com #593
• 🐛 [backport] fix(geolocate): use powerdns.org whoami service (#592) #595
• 🐛 [backport] fix(psiphon): increase the maximum runtime to 300s (#596) #597
• 🐛 fix(ios): make sure we can build an iOS release #598

The intention is to use this release for testing on mobile and desktop.

OONI Probe CLI v3.12.0-alpha.1

Changes since 3.12.0-alpha:

• ✨ miniooni: add support for --censor command line flag. This command line flag allows us to implement censorship policies (which are useful for QA) using a transparent proxy running inside miniooni itself. The core idea here is to perform QA on every operating system without relying on Netfilter. The general direction is described in ooni/probe#1803. The related PRs are: #561 — #571

• 🐛 webstepsx: make sure we use a data format that the pipeline can parse #572

• 🐛 webstepsx: avoid data race with http.Header when doing parallel HTTP #573

• 🐛 webstepsx: avoid redirect loop with some websites by changing Accept-Language #574

OONI Probe CLI v3.12.0-alpha

This release is the first release to adopt the new branching and releasing model sketched out in #554. As such, it includes all linear changes in the master branch since v3.10.0, which is the previous tag in the master branch.

Subsequent alpha releases occurring on the master branch will feature a shorter interval between releases, and will therefore include less changes between a release and the subsequent release.

This is what changed since v3.10.0 until 8931a36, i.e., the common ancestor of this release and v3.11.0-beta:

• ❗ require Go 1.17: #438 #440

• ❗ relicense repository to GPLv3: #446

• ✨ torsf: sketch out experiment for running snowflake over tor using the ptx package: #373 #378 #387 #415

• ✨ websteps: initial implementation of new web-measuring experiment: #376 #432 #434 #435 #436 #442 #447

• ✨ oohelperd: start improving it so we can replace the legacy test helper: #374 #450 #504

• ✨ tutorial: start adding tutorials explaining how to contribute to the OONI engine: #390

• ✨ start adding support for TLS parroting: #426 #460 #461 #462 #463 #464 #465 #466 #467

• ✨ add support for freebsd (i.e., now we recognise it instead of saying unknown): #445

• ✨ dns: add support for HTTPS queries: #514

• 🛠️ update dependencies: #449

• 🛠️ ooniprobe-cli.timer: run every 12h: #512

• 🐛 ensure we correctly map Windows errors and that, in general, we better map errors: #394 #429 #430 #473 #474 #475 #510 #511 #513 #521

• 🐛 ensure we don't leak QUIC connections: #407

• 🛠️ internal: refactoring with the objective of moving generic libraries outside of internal/engine with special focus on create a unique internal/netxlite with all the networking code inside it: #353 #354 #358 #362 #363 #364 #365 #366 #368 #391 #392 #393 #394 #396 #397 #398 #399 #400 #401 #402 #403 #404 #405 #406 #408 #409 #410 #411 #412 #413 #416 #417 #418 #419 #420 #421 #422 #423 #424 #433 #454 #455 #456 #457 #458 #459 #468 #469 #470 #471 #472 #476 #477 #478 #479 #480 #489 #496 #497 #498 #500 #501 #502 #503 #507 #515 #516 #517 #522

• 🛠️ internal/platform: remove unnecessary CGO dependency: #355

• 📚 improve documentation: #356 #437 #439 #443 #444 #481 #490 #499 #519 #523

• 🛠️ 🐛 ✨ minor cleanups, bug fixes, and improvements: #357 #359 #360 #367 #369 #370 #371 #372 #375 #379 #381 #382 #383 #384 #385 #388 #395 #427 #428 #451 #452 #453 #482 #483 #484 #485 #486 #487 #488 #491 #492 #493 #494 #495 #508 #509 #518 #520 #524 #525 #526

This is what changed exclusively in this release since 8931a36:

• ❗ ✨ allow experiment runner to return multiple measurements per run. This feature is required by websteps, which returns a measurement for each redirection. Note that this feature does not work well with some experiments (e.g., urlgetter), which is why we choose as a branching point for 3.11.0-beta the commit immediately before we introduced this feature. #527 #544

• ✨ measurex: new measurement library #528

• 📚 tutorial: add tutorial explaining how to use measurex #529 #534

• ✨ 📚 improve and document branching and releasing process #554 #556

• ✨ webstepsx: websteps implementation using measurex https://git...

OONI Probe CLI v3.11.0-beta

This release is the first release to adopt the new branching and releasing model sketched out in #554. As such, it includes all linear changes in the master branch since v3.10.0, which is the previous tag in the master branch.

Subsequent alpha releases occurring on the master branch will feature a shorter interval between releases, and will therefore include less changes between a release and the subsequent release.

This is what changed since v3.10.0 until 8931a36, i.e., the common ancestor of this release and v3.12.0-alpha:

• ❗ require Go 1.17: #438 #440

• ❗ relicense repository to GPLv3: #446

• ✨ torsf: sketch out experiment for running snowflake over tor using the ptx package: #373 #378 #387 #415

• ✨ websteps: initial implementation of new web-measuring experiment: #376 #432 #434 #435 #436 #442 #447

• ✨ oohelperd: start improving it so we can replace the legacy test helper: #374 #450 #504

• ✨ tutorial: start adding tutorials explaining how to contribute to the OONI engine: #390

• ✨ start adding support for TLS parroting: #426 #460 #461 #462 #463 #464 #465 #466 #467

• ✨ add support for freebsd (i.e., now we recognise it instead of saying unknown): #445

• ✨ dns: add support for HTTPS queries: #514

• 🛠️ update dependencies: #449

• 🛠️ ooniprobe-cli.timer: run every 12h: #512

• 🐛 ensure we correctly map Windows errors and that, in general, we better map errors: #394 #429 #430 #473 #474 #475 #510 #511 #513 #521

• 🐛 ensure we don't leak QUIC connections: #407

• 🛠️ internal: refactoring with the objective of moving generic libraries outside of internal/engine with special focus on create a unique internal/netxlite with all the networking code inside it: #353 #354 #358 #362 #363 #364 #365 #366 #368 #391 #392 #393 #394 #396 #397 #398 #399 #400 #401 #402 #403 #404 #405 #406 #408 #409 #410 #411 #412 #413 #416 #417 #418 #419 #420 #421 #422 #423 #424 #433 #454 #455 #456 #457 #458 #459 #468 #469 #470 #471 #472 #476 #477 #478 #479 #480 #489 #496 #497 #498 #500 #501 #502 #503 #507 #515 #516 #517 #522

• 🛠️ internal/platform: remove unnecessary CGO dependency: #355

• 📚 improve documentation: #356 #437 #439 #443 #444 #481 #490 #499 #519 #523

• 🛠️ 🐛 ✨ minor cleanups, bug fixes, and improvements: #357 #359 #360 #367 #369 #370 #371 #372 #375 #379 #381 #382 #383 #384 #385 #388 #395 #427 #428 #451 #452 #453 #482 #483 #484 #485 #486 #487 #488 #491 #492 #493 #494 #495 #508 #509 #518 #520 #524 #525 #526

This is what changed exclusively in this release since 8931a36 (I've marked with "[backport]" the patches backported and adapted from the master branch):

• 🛠️ [backport] update dependencies etc #547

• 🐛 [backport] quic: close conn after failed handshake #533

• ✨ [backport] annotate measurements with their architecture #540

• ✨ [backport] webconnectivity & oohelperd comparison and fixes #537 #541 #542 #543 #545 #546

• 🐛 ✨ 🛠️ [backport] minor changes 1e0fe14 #551 https://github.com/ooni/probe...

OONI Probe CLI v3.10.1

This release is like v3.10.0 with minor changes to support building and running with go1.17.

OONI Probe CLI v3.10.0

This release is v3.10.0-beta.3 with minor changes to publish Debian packages for all supported architectures.

iOS

# N/A; use v3.10.0-beta.3

Android

// N/A; use v3.10.0-beta.3

Generic build instructions

go build -v $extraBuildFlags ./cmd/ooniprobe

Changes

These are the changes since v3.10.0-beta.3:

• 🐛 make sure we can publish all the supported Debian architectures from different CI tasks

• 🐛 produce the tarball files and the signatures required for embedding in the ooni/probe-desktop app

• 🐛 make sure the ios task in mk depends on copying the Psiphon assets from ooni/probe-private