cli: new testhelper and the websteps experiment prototype #432
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bassosimone
approved these changes
Aug 17, 2021
| case "user-agent": | ||
| case "accept": | ||
| case "accept-language": | ||
| case "user-agent", "accept", "accept-language": |
There was a problem hiding this comment.
This is a bug fix for the oohelperd code (which is not used in production yet).
| if req.Header.Get("content-type") != "application/json" { | ||
| w.WriteHeader(400) | ||
| return | ||
| } |
There was a problem hiding this comment.
Here we are being less strict in terms of what we require in input.
| @@ -50,7 +47,8 @@ func (h Handler) ServeHTTP(w http.ResponseWriter, req *http.Request) { | |||
| } | |||
| // We assume that the following call cannot fail because it's a | |||
| // clearly serializable data structure. | |||
| data, _ = json.Marshal(cresp) | |||
| data, err = json.Marshal(cresp) | |||
| runtimex.PanicOnError(err, "json.Marshal failed") | |||
There was a problem hiding this comment.
Here we are asserting that the JSON serialization cannot fail.
bassosimone
added a commit
that referenced
this pull request
Aug 17, 2021
We started doing this in #432. This work is part of ooni/probe#1733.
bassosimone
added a commit
that referenced
this pull request
Aug 17, 2021
We started doing this in #432. This work is part of ooni/probe#1733.
This was referenced Aug 17, 2021
ainghazal
pushed a commit
to ainghazal/probe-cli
that referenced
this pull request
Mar 8, 2022
This is the extension of ooni#431, and my final deliverable for GSoC 2021. The diff introduces: 1) The new `testhelper` which supports testing multiple IP endpoints per domain and introduces HTTP/3 control measurements. The specification of the `testhelper` can be found at ooni/spec#219. The `testhelper` algorithm consists of three main steps: * `InitialChecks` verifies that the input URL can be parsed, has an expected scheme, and contains a valid domain name. * `Explore` enumerates all the URLs that it discovers by redirection from the original URL, or by detecting h3 support at the target host. * `Generate` performs a step-by-step measurement of each discovered URL. 2) A prototype of the corresponding new experiment `websteps` which uses the control measurement of the `testhelper` to know which URLs to measure, and what to expect. The prototype does not yet have: * unit and integration tests, * an analysis tool to compare the control and the probe measurement. This PR is my final deliverable as it is the outcome of the trials, considerations and efforts of my GSoC weeks at OONI. It fully integrates HTTP/3 (QUIC) support which has been only used in the `urlgetter` experiment until now. Related issues: ooni/probe#1729 and ooni/probe#1733.
ainghazal
pushed a commit
to ainghazal/probe-cli
that referenced
this pull request
Mar 8, 2022
We started doing this in ooni#432. This work is part of ooni/probe#1733.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the extension of #431, and my final deliverable for GSoC 2021.
The diff introduces:
The new
testhelperwhich supports testing multiple IP endpoints per domain and introduces HTTP/3 control measurements. The specification of thetesthelpercan be found at th-007-websteps: websteps test helper spec spec#219. Thetesthelperalgorithm consists of three main steps:InitialChecksverifies that the input URL can be parsed, has an expected scheme, and contains a valid domain name.Exploreenumerates all the URLs that it discovers by redirection from the original URL, or by detecting h3 support at the target host.Generateperforms a step-by-step measurement of each discovered URL.A prototype of the corresponding new experiment
webstepswhich uses the control measurement of thetesthelperto know which URLs to measure, and what to expect. The prototype does not yet have:unit and integration tests,
an analysis tool to compare the control and the probe measurement.
This PR is my final deliverable as it is the outcome of the trials, considerations and efforts of my GSoC weeks at OONI.
It fully integrates HTTP/3 (QUIC) support which has been only used in the
urlgetterexperiment until now.Related issues: ooni/probe#1729 and ooni/probe#1733.