Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider using HTTP/WebSockets for client/platform communication #464

Open
motin opened this issue Aug 16, 2019 · 1 comment
Open

Consider using HTTP/WebSockets for client/platform communication #464

motin opened this issue Aug 16, 2019 · 1 comment
Labels
backlog Not a priorty, but nice to have enhancement Not a bug or a feature request

Comments

@motin
Copy link
Contributor

motin commented Aug 16, 2019

In #221, this was briefly attempted but ultimately reverted in favor of a (raw) sockets-based approach.

Using a more standard/generic approach could be more extensible, maintainable, not add technical debt, and open up possibilities for running clients remotely and so on

It would also open up support for running OpenWPM crawls on other browsers.

@vringar
Copy link
Contributor

vringar commented Oct 14, 2020

WebSockets have a number of attractive qualities:

  1. They allow us to remove our own socket implementation both in the browser but also in the platform
  2. They can by default understand the difference between binary and unicode data meaning we can just send all files we want to save as binary data that get written directly to storage
  3. They have a well defined open and close sequence allowing us to shut down in an orderly manner
  4. All of their implementations are async by default allowing us to make more parts of our code base non blocking

Known problems:

  • Today we use our sockets to send pickled objects to different processes, which is functionality we'd need to implement for WebSockets, luckily the spec allows defining custom protocols (on the same level as the binary and unicode data) and the python WebSocket implementation allows us to define such a custom protocol. Meaning we could still send pickled objects and have them automatically unpickled on the other side.
  • We loose the ability to use the built-in Socket logger. However I think we should have a discussion about if the MPLogger really meets our needs at this point or if we should consider moving forward with something else. See Should we use structlog? #788

@englehardt englehardt added backlog Not a priorty, but nice to have enhancement Not a bug or a feature request and removed discussion labels Nov 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backlog Not a priorty, but nice to have enhancement Not a bug or a feature request
Projects
None yet
Development

No branches or pull requests

3 participants