Stopping Execution of Ongoing Queries

[vGsteiger]

After a discussion with @silvanheller today I wanted to start a discussion on the interaction handling by the Websocket API of cineast. This is a rather fundamental discussion about the communication between the different modules and I don't expect any swift resolution, however, it might still be interesting to have a discussion about this.

Currently, as far as I can see, for example, a query continues running even if a WebSocket connection is closed. Thus one has to rather ungraciously restart cineast to achieve a full stop of all running queries or wait for indefinite periods depending on the scope of the query to clear the running queries.

The issue is probably a result of the general architecture of cineast as it seems that the query logic is not responsible for the WebSocket management logic. So the querying will continue because the query logic has no way of being stopped by the WebSocket management. This behavior seems to be similar to #187.

The discussion I would like to start is whether this is something that could be some sort of project for the future or not. It currently seems to slow down the servers if one overloads it with dead tasks and restarting cineast every time a WebSocket connection closes prematurely seems to be rather an ungracious solution.

[silvanheller]

Very good topic for a discussion. Independently of that, can you provide steps to reproduce the issue of not being able to open another connection? At least in my experience, Cineast can handle multiple concurrent websocket-connections.
Sidenote: Cottontail currently also has no logic to abort long-running queries, maybe @ppanopticon can weigh in on how challenging that would be to implement / whether that is on the roadmap for the future

[vGsteiger]

As usual, when one has to reproduce a bug one has encountered the whole day, the time one has to document it is actually all working as it should. Therefore I would try to focus this discussion on the more general topic of state management of query execution.

I assume that the problem also arrives from my local instance and the Java WebSocket client I have extended and I will have to do some more research into its behavior to conclude on which side the error arises.

[lucaro]

I see two issues here, one of which should be much more easily addressable than the other.

You say that when the websocket is closed from the UI side, "This currently results locally that it is not possible to open another WebSocket connection to the local server and thus one has to rather ungraciously restart cineast". If this is indeed the case, this is a bug, as all websocket connections are supposed to be handled independently. You should also be able to connect from multiple UI instances to the same cineast instance, independently of if they are all running on the same machine or on different ones. Is this not the case?

The other issue concerns state management of query execution. Currently, neither cineast nor cottontail offer any mechanism to abort a query. Adding such a mechanism is non-trivial, since cineast would not only need to be able to handle interrupts at any point during query execution, but also able to forward such interrupts to the storage layer. I'm not saying that something like this could not be done, but it would certainly require some rather fundamental changes to some abstraction layers. Do you have any suggestions how such a thing could be done?

[vGsteiger]

Thanks for the swift reply.

To your first response, similar to the response to Silvan, I have to backtrack my comments a bit as I currently can't reproduce the behavior I encountered all day after properly restarting all the systems. I will keep you informed when the bug arises again and then I would write an issue.

To your second answer/comment, I see that it would be non-trivial and would need fundamental changes. I currently have no fix-it-all solution and would have to further dig myself into the codebase to give you an informed opinion. I rather wanted to start the discussion on the topic and whether the proposed behavior of query interrupts could be a project for the future of some master students. I myself am currently just writing my Bachelor's thesis with Silvan and have thus limited resources.