Open-source RAG for Japanese LLM in low-resource settings

Llama-Index as a data framework
Embeddings from Multi-lingual E5 from Microsoft
Elyza as LLM based on LlaMA 2 (Facebook/Meta)
4Bits quantized by Bitsandbytes (served by Hugging Face Accelerate)
CJKPDFReader loader for Japanese PDF files
Open source Vector database (Postgres + PGVector)
Vanilla hybrid search
Ngrok for TCP tunneling
Q&A test
Conclusion