add session status API #28

martinpitt · 2022-09-13T05:49:02Z

We need to keep track of the session state: "new", "pod ready", "target machine connected", and possibly "abandoned" or so.
POSTing to /api/webconsole/v1/sessions/<session_id>/status should update the status, and GETing it should retrieve it.

This API endpoint should not be exposed publicly, just internally from the appservice to the session pod.

Also wire this up so that connecting the target machine updates the status. This is not super-trivial, as currently that port is forwarded directly to cockpit-ws and its socat-session.sh, and it's not obvious where to slip in the status update. We may need to modify cockpit for that, or replace socat-session.sh+websocat with something more customized.

nginx does not get along well with unavailable proxy_pass targets -- as soon as one session pod unexpectedly goes away (crashes, idle timeouts, networking flakes), it errors out and no session routes can be resolved any more. There are workarounds [1], but I can't get them to work properly. Also, nginx does not allow us to hook into (dis)connection events to implement the session status API (see issue #28). So it's finally time to grow to something more flexible. Rewrite multiplexer.py using aiohttp [2], which now handles both our own session control API as well as the dynamic reverse proxying to the session pods. This actually behaves *slower* than with nginx, resulting in lagging/missing CSS in the browser. This should still be considered as a PoC, but at least there are fewer moving parts and an easier-to-uderstand architecture now. [1] https://sandro-keil.de/blog/let-nginx-start-if-upstream-host-is-unavailable-or-down/ [2] https://docs.aiohttp.org/en/stable/index.html

nginx does not get along well with unavailable proxy_pass targets -- as soon as one session pod unexpectedly goes away (crashes, idle timeouts, networking flakes), it errors out and no session routes can be resolved any more. There are workarounds [1], but I can't get them to work properly. Also, nginx does not allow us to hook into (dis)connection events to implement the session status API (see issue #28). So it's finally time to grow to something more flexible. Rewrite multiplexer.py using aiohttp [2], which now handles both our own session control API as well as the dynamic reverse proxying to the session pods. This actually behaves *slower* than with nginx, resulting in lagging/missing CSS in the browser. This should still be considered as a PoC, but at least there are fewer moving parts and an easier-to-uderstand architecture now. Check behaviour with broken session pods in the tests: Crash a session pod, and validate that the other sessions still behave correctly. Fixes #36 [1] https://sandro-keil.de/blog/let-nginx-start-if-upstream-host-is-unavailable-or-down/ [2] https://docs.aiohttp.org/en/stable/index.html

nginx does not get along well with unavailable proxy_pass targets -- as soon as one session pod unexpectedly goes away (crashes, idle timeouts, networking flakes), it errors out and no session routes can be resolved any more. There are workarounds [1], but I can't get them to work properly. Also, nginx does not allow us to hook into (dis)connection events to implement the session status API (see issue #28). So it's finally time to grow to something more flexible. Rewrite multiplexer.py using aiohttp [2], which now handles both our own session control API as well as the dynamic reverse proxying to the session pods. This actually behaves *slower* than with nginx, resulting in lagging/missing CSS in the browser. This should still be considered as a PoC, but at least there are fewer moving parts and an easier-to-understand architecture now. Check behaviour with broken session pods in the tests: Crash a session pod, and validate that the other sessions still behave correctly. Fixes #36 [1] https://sandro-keil.de/blog/let-nginx-start-if-upstream-host-is-unavailable-or-down/ [2] https://docs.aiohttp.org/en/stable/index.html

nginx does not get along well with unavailable proxy_pass targets -- as soon as one session pod unexpectedly goes away (crashes, idle timeouts, networking flakes), it errors out and no session routes can be resolved any more. There are workarounds [1], but I can't get them to work properly. Also, nginx does not allow us to hook into (dis)connection events to implement the session status API (see issue #28). So it's finally time to grow to something more flexible. Rewrite multiplexer.py using aiohttp [2], which now handles both our own session control API as well as the dynamic reverse proxying to the session pods. This should still be considered as a PoC, but at least there are fewer moving parts and an easier-to-understand architecture now. Check behaviour with broken session pods in the tests: Crash a session pod, and validate that the other sessions still behave correctly. Fixes #36 [1] https://sandro-keil.de/blog/let-nginx-start-if-upstream-host-is-unavailable-or-down/ [2] https://docs.aiohttp.org/en/stable/index.html

martinpitt · 2022-09-19T08:43:51Z

PR #37 unblocks this -- after that it should be fairly straightforward to implement the session API.

Track session status. Right after creation, it starts as `wait_target`. After the target machine has connected (first connection to the /ws path), it changes to `running`. We'll probably add more states in the future. Add /sessions/ID/status API to query it, so that e.g. the target machine and hte UI can poll this to know when to connect. Fixes #28

nginx does not get along well with unavailable proxy_pass targets -- as soon as one session pod unexpectedly goes away (crashes, idle timeouts, networking flakes), it errors out and no session routes can be resolved any more. There are workarounds [1], but I can't get them to work properly. Also, nginx does not allow us to hook into (dis)connection events to implement the session status API (see issue #28). So it's finally time to grow to something more flexible. Rewrite multiplexer.py using aiohttp [2], which now handles both our own session control API as well as the dynamic reverse proxying to the session pods. This should still be considered as a PoC, but at least there are fewer moving parts and an easier-to-understand architecture now. Check behaviour with broken session pods in the tests: Crash a session pod, and validate that the other sessions still behave correctly. Fixes #36 [1] https://sandro-keil.de/blog/let-nginx-start-if-upstream-host-is-unavailable-or-down/ [2] https://docs.aiohttp.org/en/stable/index.html

Track session status. Right after creation, it starts as `wait_target`. After the target machine has connected (first connection to the /ws path), it changes to `running`. We'll probably add more states in the future. Add /sessions/ID/status API to query it, so that e.g. the target machine and the UI can poll this to know when to connect. Fixes #28