[LLM] Provide an interface used for loading model on the Fastchat side

[Zephyr596]

Description

Similar to lm-sys/FastChat#2888, we can provide an example implementation for worker that extends the BaseModelWorker, which should override the following three methods:

def generate_stream_gate(self, params):
    raise NotImplementedError

def generate_gate(self, params):
    raise NotImplementedError

def get_embeddings(self, params):
    raise NotImplementedError

After that, we can provide a bigdl_worker and a documentation on how to use it.

The controller and worker communicates through a set of interfaces, you can check this at here https://github.com/lm-sys/FastChat/blob/main/fastchat/serve/base_model_worker.py#L196. Therefore, if bigdl_worker can successfully implement these interfaces, we can integrate BigDL-LLM into FastChat.

1. Why the change?

Similar to https://github.com/lm-sys/FastChat/blob/main/fastchat/serve/mlx_worker.py#L33, we can provide a interface at BigDL side that provide the functionalities of loading models so that we can keep the code at FastChat side stable.

User will start the controller normally, but use like python3 -m fastchat.serve.bigdl_worker --model-names "bigdl-models" --model-path lmsys/vicuna-7b-v1.5 to start the worker.

2. User API changes

from bigdl.llm.utils import load_model

3. Summary of the change

Provide an interface used for loading model on the Fastchat side

4. How to test?

• Unit test
• Application test
  .

5. New dependencies

None

[gc-fu]

Closed and moved to #10323