Add status communication between extension and platform

[englehardt]

Currently the extension receives the visit_id from the platform code and sends database queries to the aggregator. The extension provides no feedback to the platform on it's status, which has some negative effects:

1. If the extension crashes during a long running crawl the platform has no way of noticing and will continue crawling with no instrumentation (until the browser restarts).
2. If database queries are stuck in a buffer within the extension (for example in the debounce method of the content script, or the buffer for the socket), the platform may kill the extension before it has a chance to drain these buffers.

Ideally, the platform should check with the extension after each page visit to ensure it's still running and has finished logging all data from the page visit before moving on to the next visit. If either of these fails or the extension fails to respond, the browser should be restarted by the platform.

[englehardt]

• Extension needs to say when it’s done and the platform needs to block on that timeout.
• How does the extension know that it’s done with a page visit? We can have Firefox close the tab and wait 5 seconds before sending the next visit ID.
• When a page gets closed, any open requests (tested both DOM with src and XHR) are invalidated and we don’t receive them.

Procedure:

• Submit visit id —> wait for OK response
• Close tab
• Ask if done —> (extension drains queue) wait for OK response

[englehardt]

[englehardt]

@vringar do you think your latest PRs cover this?

[vringar]

No, not really
What we now have is an explicit communications channel going
Taskmanager-> Extension -> Aggregator -> Callbacks
But the Taskmanager still doesn't know anything about the status of the extension