Integrate HF TGI with TEI

[daz-williams]

I'm interested if anyone has managed to integrate HF TGI with TEI?

I was using PrivateGPT for querying against various text documents; however it has many limitations for production, mainly single tenant.

It seems HG TGI is a good fit, especially with continuous batching of incoming requests for increased total throughput. The next step is to integrate RAG and TEI seems perfect.

Has anyone integrated them together yet?

[Narsil]

Internally we have a few stuff that use them both.

However we didn't create any integrated solution for both (yet).
You should keep an eye for : https://huggingface.co/chat/ (https://github.com/huggingface/chat-ui/) because we're likely to implement there in the open source.