pip install gigax
- 🕹️ NPCs that
<speak>
,<jump>
,<attack>
and perform any other action you've defined - ⚡ <1 second GPU inference on most machines
- 🤗 Open-weights models available, fined-tuned from: Llama-3, Phi-3, Mistral, etc.
- 🔒 Structured generation with Outlines 〰️ means the output format is always respected
- 🗄️ Coming soon: Local server mode, with language-agnostic API
- 📜 Available on API: Runtime quest generation, for players and NPCs
- 😶🌫️ Available on API: Memory creation, storage and retrieval with a Vector DB
Gigax has new releases and features on the way. Make sure to ⭐ star and 👀 watch this repository!
-
We provide various models on the 🤗 Huggingface hub:
- NPC-LLM-7B (our Mistral-7B fine-tune)
- NPC-LLM-3_8B (our Phi-3 fine-tune)
- NPC-LLM-3_8B-128k (our Phi-3 128k context length fine-tune)
-
All these models are also available in gguf format to run them on CPU using llama_cpp
-
Start by instantiating one of them using outlines:
from outlines import models
from gigax.step import NPCStepper
from transformers import AutoTokenizer, AutoModelForCausalLM
# Download model from the Hub
llm = Llama.from_pretrained(
repo_id="Gigax/NPC-LLM-3_8B-GGUF",
filename="npc-llm-3_8B.gguf"
# n_gpu_layers=-1, # Uncomment to use GPU acceleration
# n_ctx=2048, # Uncomment to increase the context window
)
model = models.LlamaCpp(llm)
# Instantiate a stepper: handles prompting + output parsing
stepper = NPCStepper(model=model)
- From there, stepping an NPC is a one-liner:
action = await stepper.get_action(
context=context,
locations=locations,
NPCs=NPCs,
protagonist=protagonist,
items=items,
events=events,
)
- We provide classes to instantiate
Locations
,NPCs
, etc. :
from gigax.parse import CharacterAction
from gigax.scene import (
Character,
Item,
Location,
ProtagonistCharacter,
Skill,
ParameterType,
)
# Use sample data
context = "Medieval world"
current_location = Location(name="Old Town", description="A quiet and peaceful town.")
locations = [current_location] # you can add more locations to the scene
NPCs = [
Character(
name="John the Brave",
description="A fearless warrior",
current_location=current_location,
)
]
protagonist = ProtagonistCharacter(
name="Aldren",
description="Brave and curious",
current_location=current_location,
memories=["Saved the village", "Lost a friend"],
quests=["Find the ancient artifact", "Defeat the evil warlock"],
skills=[
Skill(
name="Attack",
description="Deliver a powerful blow",
parameter_types=[ParameterType.character],
)
],
psychological_profile="Determined and compassionate",
)
items = [Item(name="Sword", description="A sharp blade")]
events = [
CharacterAction(
command="Say",
protagonist=protagonist,
parameters=[items[0], "What a fine sword!"],
)
]
Contact us to give our NPC API a try - we'll take care of model serving, NPC memory, and more!