[Docs] Add docs for Sky Serve

[cblmemo]

Tested (run the relevant ones):

• Code formatting: bash format.sh
• Any manual or new tests for this PR (please specify below)
• All smoke tests: pytest tests/test_smoke.py
• Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
• Backward compatibility tests: bash tests/backward_comaptibility_tests.sh

[concretevitamin]

Nice!

[concretevitamin]

How about adding a section here, e.g., "Example: Text Generation Inference (TGI)", which can consist of the two snippets in https://docs.google.com/document/d/1vVmzLF-EkG3Moj-q47DQBGvFipK4PNfkz0V6LyaPstE/edit#heading=h.gntyowdq9a18 or https://docs.google.com/document/d/1vVmzLF-EkG3Moj-q47DQBGvFipK4PNfkz0V6LyaPstE/edit#heading=h.gr15nxiws63p

The value is it's much shorter --> easier to adapt. It also quickly shows the idea of one endpoint being backed by multiple regions/clouds' replicas.

Can discuss whether to put it in the opening section of this page, like User Docs does.

[cblmemo]

If we are adding this, do you think we still need the vicuna example? Not sure if it is a little bit redundant if we include TGI...

[concretevitamin]

Some redundancy is fine. The main motivation is to make the very first impression about real, useful AI serving. Currently it's HTTP server.

How about we add a "Quickstart: TGI service" section (or change it to vLLM/FastChat etc.), but keep the user doc's concise formatting -- 1 snippet showing YAML, 1 snippet showing service status, then add 1 snippet showing how to CURL it correctly. With minimal text throughout.

[concretevitamin]

We also may want to replace some snippets (e.g., sky serve status?) with screenshots which look way better with colors than the current formattin.

[landscapepainter]

Had a chance to go thorugh the doc to learn how SkyServe works. Left some comments on what I found.

[concretevitamin]

We should add a CURL based command like the user doc. Does something like this work

Then it’s ready to accept traffic!

$ curl -L Y.Y.Y.Y:8082/generate \
    -X POST \
    -d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":20}}' \
    -H 'Content-Type: application/json'
{"generated_text":"\n nobody knows"}

[concretevitamin]

Some redundancy is fine. The main motivation is to make the very first impression about real, useful AI serving. Currently it's HTTP server.

How about we add a "Quickstart: TGI service" section (or change it to vLLM/FastChat etc.), but keep the user doc's concise formatting -- 1 snippet showing YAML, 1 snippet showing service status, then add 1 snippet showing how to CURL it correctly. With minimal text throughout.