Extend `/health` to add instances

[grahamking]

Feature request

We have endpoints to list models (OpenAI spec), endpoints (#1037 ) but not yet instances. I was expecting we'd have some kind of cluster management system that handled that, but it seems we need this sooner.

Extend the /health endpoint to include instances. Add a function in lib/runtime to list them in etcd - see component.rs list_instances for an example (that one only lists the instances of a Component).

Describe the problem you're encountering

/

Describe alternatives you've tried

No response

[jlsong01]

May I try it as my first issue, thanks
/assign

[vyalamar]

/assign

[grahamking]

Please both go for it! Even if we don't merge your contribution it will be a nice chance to explore the code base and maybe improve your Rust.

[Medyan-Naser]

Hi,

I'd like to pick up this issue (#1312) and work on extending the /health endpoint to include instance listings.

Before I dive in, could you please provide any updated information or context related to this feature request? Specifically:

• Has any preliminary work been started or design decisions made since this issue was opened?
• Are there any specific details or status fields that should be included for each instance in the /health response?

Also, what would be the recommended approach for testing this new functionality?

Thanks!

[grahamking]

Go for it. No changes since the issue was posted. Include a unit test.

[Medyan-Naser]

Hi @grahamking, I've been trying to build and run the project over the past couple of days on both my Mac and a Linux VM, but I keep encountering build errors and running into memory issues. Any guidance on possible alternatives or workarounds would be greatly appreciated.

In case I can't work on this issue, I'm still very interested in contributing and would love to help with the documentation. I've added a comment with a few questions to this DOC issue — would appreciate it if you could take a look: #559

Thanks!

[grahamking]

keep encountering build errors and running into memory issues

Make a new issue with what you're running and the output you're seeing, and someone may be able to help.

[heisenberglit]

Hey @grahamking,

Attempting to open a PR for this. I have a question what is the correct way to access runtime on http service. I saw the comment on old PR it mentioned this isn't the right way to pass the runtime.

In my case, I need access to the runtime specifically to retrieve the etcd_client which is required for querying all registered instances using a prefix-based etcd query to implement the list_all_instances functionality that aggregates all instances.

Would appreciate any guidance on whether injecting just the runtime into state is preferred or if we should expose a more generic mechanism for runtime-level operations within the HTTP layer.