Lu

Chinese README / 中文说明

Lu is a local, multi-channel Agent runtime written in Go. It unifies model calls, tool execution, multi-channel message ingress, and on-device memory into an extensible runtime.

The current codebase includes a CLI, WebSocket/Web UI, macOS iMessage channel, skills system, memory search, multi-agent support, scheduler/heartbeat, and optional FaceTime voice features.

Features

• Multiple entrypoints: cmd/cli (interactive terminal) and cmd/server (WebSocket + iMessage unified server)
• Multiple model providers: OpenAI-compatible API and Anthropic, with configurable base_url
• Multi-agent runtime: per-agent provider/system/tools/work_dir/security
• Tool calling: built-ins shell/read/write/edit/apply_patch, plus optional process/schedule/facetime_call/skill/sub_agent
• Skills: auto-discover from ./skills and ~/.lu/skills, with hot reload
• Memory: Markdown chunking + vector search + FTS5 keyword search (requires -tags fts5)
• Observability: OpenTelemetry (metrics/traces) + optional full payload NDJSON sink
• Automation: scheduler (cron-like jobs) + heartbeat (periodic task file checks)
• Multimodal: image input, voice transcription (Whisper/FunASR), optional FaceTime pipeline

Architecture

Core flow:

1. A channel receives user messages (CLI / WebSocket / iMessage)
2. The agent loads conversation history (SQLite)
3. The agent calls an LLM provider
4. When the model requests tool calls, the runtime executes tools and appends results to context
5. The final response is delivered back to the channel

Key packages:

• internal/agent: agent loop, tool execution, context management
• internal/server + internal/channel: WebSocket/iMessage services and routing
• internal/provider: provider adapters and registry
• internal/tool: tool definitions, registry, filtering
• internal/skill: discovery, parsing, hot reload
• internal/memory: indexing, search, memory tools
• internal/scheduler / internal/heartbeat: scheduling and periodic wake-ups
• internal/security: workspace + shell policy
• internal/observability: OTel and full-payload sink

Layout

.
├── cmd/
│   ├── cli/                  # CLI entrypoint
│   └── server/               # unified server entrypoint
├── internal/                 # core runtime
├── web/                      # static Web UI assets
├── skills/                   # project-level skills
├── scripts/                  # helper scripts
├── tools/
│   ├── audiocap/             # audio capture tool (Xcode)
│   └── screencap/            # screen capture tool (Swift Package)
├── config.example.yaml       # config template
├── config.yaml               # local config (do not commit)
├── lu.db                     # SQLite sessions + memory index
└── docs/                     # design docs and plans

Requirements

• Go 1.25.6+
• C toolchain (required by go-sqlite3)
• macOS (only required for iMessage / FaceTime features)

Optional dependencies (enable per feature):

• imsg (iMessage send/receive)
• osascript (macOS UI automation)
• ffmpeg (audio conversion)
• whisper-cli / whisper-cpp (local transcription)
• Python environment (FunASR realtime transcription)
• AudioCapCLI (built from tools/audiocap)
• screencap (built from tools/screencap)

Quick Start

1. Create config

cp config.example.yaml config.yaml

At minimum, configure:

• provider: openai or anthropic
• corresponding API key (prefer environment variables)
• enable features you need: channels.websocket / channels.imessage / memory / facetime

2. Build and run (recommended)

make build
make run-cli
make run-server

3. Or use Go directly (FTS5 build tag is required)

go run -tags fts5 ./cmd/cli --config config.yaml
go run -tags fts5 ./cmd/server --config config.yaml

When server is running (default port 8080):

• http://localhost:8080/ (Web UI)
• ws://localhost:8080/ws (WebSocket)
• http://localhost:8080/api/conversations (conversation list)

CLI Usage

go run -tags fts5 ./cmd/cli --config config.yaml --list-models
go run -tags fts5 ./cmd/cli --config config.yaml --conversation <conversation_id>

WebSocket Message Format

Client message:

{
  "type": "message",
  "conversation_id": "uuid",
  "content": "hello"
}

Server events:

• chunk: streaming output chunk
• done: end of response
• error: error message

Configuration Notes

• providers / provider: model providers (use providers for multi-provider setups)
• agents: multi-agent definitions (provider/system/tools/work_dir/security)
• channels.websocket.agent / channels.imessage.agent: bind channels to agents
• memory.*: vector + keyword hybrid search parameters
• scheduler.enabled: cron-like jobs
• heartbeat.*: periodic checks for HEARTBEAT.md and agent wake-ups
• logging.otel.*: OTel exporter settings
• logging.full_payload.*: full payload NDJSON archive + redaction
• facetime.*: optional FaceTime voice pipeline

See config.example.yaml for the full configuration surface.

Development

make fmt
make test
make build

Equivalent:

go fmt ./...
go test -tags fts5 ./...
go build -tags fts5 ./cmd/cli ./cmd/server

Note: some tests depend on external tools (e.g., ffmpeg, whisper-cpp, osascript). They may be skipped or fail if the tools are missing.

Security

• Do not commit real API keys, tokens, or private conversation data
• Prefer environment variables for secrets
• Set channels.imessage.allow_list before enabling iMessage
• If you enable the shell tool, configure per-agent security policies

License

MIT License. See LICENSE.