feat(minipipeline): introduce "classic" observations filtering #1402
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We're introducing failure modes that do not exist hence it seems this is not the correct way of moving forward.
I'm doing this mainly to explore whether we could have more robust webconnectivity v0.5 analysis code
Because I am dropping the requests again, we break again the tests with the redirects. I could possibly fix it by putting requests back again but I am not super happy about doing this because that would cause the DSL to do some strange work and I'd honestly rather not do this.
what remains to be done now is to make sure we make green all the tests that are currently skipped we also need to account for differences between the two
then next step is to sort out this mess :-)
(I am thankful there's a ~comprehensive test suite.)
this happens because LTE sucessfully handshakes with the wrong address
Conflicts: internal/experiment/webconnectivitylte/cleartextflow.go internal/experiment/webconnectivitylte/secureflow.go
I am not planning on butchering lte on master, but I need to butcher it here because I really need to figure out how to align them correctly. What I have so far is good, but there's some theory/abstraction that I am stil missing.
bassosimone
changed the title
bassosimone
commented
Nov 29, 2023
bassosimone
changed the title
Murphy-OrangeMud
pushed a commit
to Murphy-OrangeMud/probe-cli
that referenced
this pull request
Feb 13, 2024
…1402) Classic filtering is an `WebObservationsContainer` filtering technique that takes in input a `WebObservationsContainer` and only keeps DNS lookups using `getaddrinfo` and endpoints whose IP address has been discovered using `getaddrinfo`. By applying this technique, we reduce the richer dataset produced by Web Connectivity LTE to a smaller dataset comparable to what Web Connectivity v0.4 would return. In turn, by focusing the analysis on the reduced dataset, we hope to emulate the results produced by v0.4 for backward compatible test keys. I named this feature "classic" because it's what we used to do and I don't want to call it legacy. Part of ooni/probe#2634.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Classic filtering is an
WebObservationsContainerfiltering technique that takes in input aWebObservationsContainerand only keeps DNS lookups usinggetaddrinfoand endpoints whose IP address has been discovered usinggetaddrinfo. By applying this technique, we reduce the richer dataset produced by Web Connectivity LTE to a smaller dataset comparable to what Web Connectivity v0.4 would return. In turn, by focusing the analysis on the reduced dataset, we hope to emulate the results produced by v0.4 for backward compatible test keys.I named this feature "classic" because it's what we used to do and I don't want to call it legacy.
Part of ooni/probe#2634.