Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

Secure open source cloud runtime for AI apps & AI agents

Evaluating tool-augmented LLMs in conversation settings

AI driven development in your terminal. Designed for large, real-world tasks.

Awesome-LLM: a curated list of Large Language Model

Run any open-source LLMs, such as Llama, Mistral, as OpenAI compatible API endpoint in the cloud.

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

Inference code for Llama models