multi: add a Status server

[ellemouton]

This PR introduces a Status server to Lit. This server can be queried at any time to determine the
status of any of the following servers: Lit, LND, Faraday, Loop, Pool. With this PR, Lit will also not shutdown
unless stopped purposefully. The reason being so that if any parts of Litd fail to start, the status server will
still be available for a front end to query to see what is up.

Let's say for example, that Lit is started with Loop disabled (loop-mode=disabled) & then some error causes Pool
not to start up properly, then you can query the status server to see the following:

$ litcli status 

{
        "sub_servers": {
                "faraday": {
                        "disabled": false,
                        "running": true,
                        "error": ""
                },
                "lit": {
                        "disabled": false,
                        "running": true,
                        "error": ""
                },
                "lnd": {
                        "disabled": false,
                        "running": true,
                        "error": ""
                },
                "loop": {
                        "disabled": true,
                        "running": false,
                        "error": ""
                },
                "pool": {
                        "disabled": false,
                        "running": false,
                        "error": "pool not working"
                },
                "taproot-assets": {
                        "disabled": false,
                        "running": true,
                        "error": ""
                }
        }
}

Depends on: #616

Replaces #442

Fixes #421
Fixes #420
Fixes #290

[itsrachelfish]

I was able to get this integrated into terminal but while testing I noticed that when a subserver is disabled it completely disappears from the status list. We might want to change this because it means we won't be able to use the status server for feature detection.

For example when taproot assets are added to litd, it could be useful to query the status server to see if tapd is available. Currently there is no way to differentiate between "litd hasn't been upgraded, so tapd does not exist" and "litd has been upgraded, and tapd exists, but it is disabled"

[ellemouton]

thanks for the suggestion @itsrachelfish. Updated accordingly

[guggero]

Awesome PR, great work!
I did a first pass to load all the context. I think the main thing we might want to change is merging of the two subserver managers (see inline comments).
I'll do a round of manual testing in the next round.

[ViktorTigerstrom]

Very nice PR 🚀 🔥!!! My only comments are for minor things, so really great work! I've tested the functionality manually as well, and will also conduct thorough manual testing next review round 🚀!

[ellemouton]

Thanks @guggero & @ViktorTigerstrom 🚀 updated!

[ViktorTigerstrom]

(2). Even more strangely, for some reason which I can't figure out why when looking a

Ok so this is cause currently for remote sub-servers, we dont block on the connection being successful. Instead we give it:

grpc.WithConnectParams(grpc.ConnectParams{
	Backoff:           backoff.DefaultConfig,
	MinConnectTimeout: defaultConnectTimeout,
})

and return the unconnected connection so that LiT continues to maintain the connection if the remote server gets restarted or something. I dont think we want to change this behaviour right now. And I think in most cases, the pool/faraday/loop etc servers are run in integrated mode in anycase.

I think in a follow up PR, we can add a heartbeat mechanism to the status server to give more live updates regarding sub-server statuses.

Ok makes sense, thanks for checking! Sound like a good plan

[guggero]

Amazing work. tACK, LGTM 🎉

[guggero]

I assume we'll want to wait for #620 before merging this?

[ellemouton]

@guggero - yes :)