Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ warp/apps/web/.env.local
warp/apps/web/.env.*

# Local data store
warp/apps/web/data/
data/

# macOS
Expand Down
123 changes: 10 additions & 113 deletions Warp.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
## 0) What this is

* **Studio Plugin (Luau)**: docked chat + proposals, reads active editor state, previews diffs, applies edits inside **ChangeHistoryService**.
* **Next.js Backend (TypeScript)**: `/api/chat`, `/api/stream`, `/api/proposals/:id/apply`, `/api/assets/search`, plus an orchestrator that exposes tools to the LLM.
* **Next.js Backend (TypeScript)**: `/api/chat`, `/api/proposals/:id/apply`, `/api/proposals`, `/api/assets/search`, plus an orchestrator that exposes tools to the LLM.
* **LLM Tool‑Calling**: one‑tool‑per‑message, approval‑first workflow (like Cline). The model proposes **proposals** (edits/object ops/asset ops); only the plugin performs writes after user approval.

---
Expand Down Expand Up @@ -276,8 +276,7 @@ end

## 6) Streaming & transport

* **Plugin**: default to **short polling** or **long‑polling** `/api/stream?cursor=…`. If SSE is available and reliable in your environment, the plugin can use it; otherwise keep long‑poll.
* **Web dashboard**: use SSE for token‑level streaming.
* Streaming: not implemented in this increment. Use request/response for now.
* Limit concurrent plugin HTTP requests (≤2) to avoid Studio’s in‑flight cap.

---
Expand Down Expand Up @@ -328,7 +327,7 @@ flowchart TD
subgraph Studio Plugin (Luau)
A[User asks] --> B[Collect context\nGetEditorSource + Selection]
B --> C[/api/chat]
J[Stream/Long-poll /api/stream] --> K[Render partial tokens]
%% Streaming omitted in this increment
C -->|proposals| D[Diff/Operations UI]
D --> E{Approve?}
E -- Yes: edits --> F[TryBeginRecording\nUpdateSourceAsync\nFinishRecording]
Expand Down Expand Up @@ -460,7 +459,7 @@ flowchart LR
end

subgraph Backend["Next.js Orchestrator (Vercel)"]
ROUTES[/api/chat • /api/stream* • /api/assets/search • /api/proposals/:id/apply/]
ROUTES[/api/chat • /api/assets/search • /api/proposals/:id/apply/ • /api/proposals]
ORCH[Tool‑Calling Orchestrator]
PROVIDERS[OpenRouter / OpenAI / Anthropic]
STORE[(DB • Cache • Audit Logs)]
Expand All @@ -480,7 +479,7 @@ flowchart LR
GEN3D --> GPU --> OPENCLOUD --> ORCH
ROUTES --> CATALOG

%% Notes: * Studio plugins cannot use SSE/WebSockets. Use short/long‑polling.
%% Notes: Streaming/SSE not implemented in this increment.
```

**Key points**
Expand Down Expand Up @@ -849,117 +848,15 @@ warp/

- Docked chat UI, context capture, HTTP permission flow — implemented
- `/api/chat` round-trip producing `edit` and `object_op` proposals — implemented
- Diff preview + one-click **Apply** (single undo step) — full unified diff renderer implemented; shows context, additions/deletions, multiple hunks, with Open/Close Diff toggle
- Asset search + insert flow — implemented (server stub + plugin Browse + Insert)
- Diff preview + one-click **Apply** (single undo step) — basic unified diff snippet attached by server; minimal preview in plugin
- Asset search API — implemented (requires CATALOG_API_URL); plugin UI not implemented in this increment
- Audit log of proposals and outcomes — implemented with file-backed JSON durability and apply acknowledgements

---

## Implementation Progress — 2025-08-31 (updated @ 21:06 UTC)

Recent updates

- Added zod-based validation of provider tool-call arguments before mapping to proposals
- Added Plan/Act scaffold: executes context tools locally and performs a second provider call for the next actionable tool
- Switched proposals store to file-backed JSON durability (apps/web/data/proposals.json)
- Introduced Catalog provider interface (env CATALOG_API_URL) for real search; stub remains fallback

Decisions captured

- AI name: Vector (assistant name in UI and comments)
- LLM provider: OpenRouter, model: moonshotai/kimi-k2:free
- Package manager: npm
- Hosting: Local dev at http://127.0.0.1:3000 for now. We will move to Vercel (or similar) later.
- Domain permissions: local-only for now; will add hosted domain later when deploying.
- Plugin install workflow: Rojo during development → export .rbxm for small tester group → publish to Marketplace for public release.
- System prompt: refined Vector prompt (one-tool-per-message, proposal-first, minimal diffs).

Working now

- Web (Next.js + npm)
- Local app scaffold (Next 14) with npm scripts; landing page at /.
- API routes wired: /api/chat persists proposals to a file-backed store for auditing; /api/proposals/[id]/apply records an audit event; /api/proposals (GET) lists stored proposals; /api/assets/search uses a provider interface (real via CATALOG_API_URL or stub); /api/assets/generate3d returns a stub jobId.
- Provider tool-call parsing (OpenRouter) behind flags: when VECTOR_USE_OPENROUTER=1, model outputs exactly one XML-like tool call which is parsed and mapped to proposals. Arguments are validated with zod before mapping. Falls back to safe proposals if parsing fails.
- Plan/Act scaffold behind VECTOR_PLAN_ACT=1: if the first tool is a context tool (get_active_script, list_selection, list_open_documents), we execute it locally from the provided context, store the result in a short session, and issue a second provider call using that result to get the next actionable tool.
- Provider adapters present: openrouter.ts (call path), openai.ts (stub).
- Plugin (Vector)
- Dock UI with input + Send. Sends context (active script + selection) to /api/chat and renders proposals.
- Approve/Reject per proposal; applies:
- edit proposals that insert text via rangeEDITS (merged, UpdateSourceAsync with undo)
- object_op rename_instance (ChangeHistoryService wrapped)
- Added "Apply & Open" for edit proposals (opens the script after a successful apply).
- Asset search flow: Browse button fetches /api/assets/search results → Inline list with Insert buttons → inserts via InsertService.
- Reports apply results back to /api/proposals/:id/apply for auditing.
- Simple diff preview snippet (shows inserted text).

Configured locally

- Environment
- warp/apps/web/.env present with OPENROUTER_API_KEY (provided) and OPENROUTER_MODEL=moonshotai/kimi-k2:free (default).
- VECTOR_USE_OPENROUTER=0 by default; set to 1 to enable provider-driven tool-call parsing.
- VECTOR_PLAN_ACT=0 by default; set to 1 to enable a second provider call after context tools (Plan/Act scaffold).
- CATALOG_API_URL optional: when set, /api/assets/search calls this URL to fetch normalized results; otherwise it uses a stub list.
- Data directory: proposals are persisted to apps/web/data/proposals.json (auto-created on write).
- Do not commit .env to version control.

Not yet implemented (next milestones)

- Provider-driven tool-call execution loop (multi-turn Plan/Act with context tools) and stricter validation/guardrails
- Robust diff merging on server (multi-edit merging) and generalized path→Instance resolution
- Asset Catalog integration (real Catalog API instead of stub) and GPU 3D generation → Open Cloud upload
- Rich plugin UI: full diff preview (line-by-line, deletions/edits), improved status/streaming, thumbnails for assets
- Persistence: move from file-backed JSON to a durable DB/schema (e.g., SQLite/Prisma), richer auditing and listing endpoints (filter by projectId)
- Packaging/deploy: Vercel hosting, domain allowlisting in Studio

Needs from you

- Approvals to proceed:
- Wire plugin context capture to POST /api/chat and display proposals in the Vector dock.
- Implement diff preview UI with Approve/Reject and undo semantics.
- Enable the provider-backed path (OpenRouter) and tool-call parsing after initial local verification.
- Later (not blocking M0/M1):
- Roblox Open Cloud credentials for 3D generation and asset uploads (generate3d flows).
- Catalog API integration details/keys if we query Roblox web APIs server‑side (optional).

Recommended next steps (proposed M0 path)

1) Verify npm install in warp/apps/web and run the local dev server (npm run dev)
2) Wire the plugin to collect context (active script + selection) and POST to /api/chat
3) Implement diff preview data shaping on server (no writes), then renderable in plugin UI
4) Integrate OpenRouter provider call path behind a feature flag and progressively build tool-call parsing

Note on cline_openai.md

- Populated with a concise Vector system prompt (Cline‑style). We will refine it as the tool registry and execution loop evolve.

### How to run locally

- Web (from a new terminal):
- cd warp/apps/web
- npm install
- Ensure your .env or .env.local contains OPENROUTER_API_KEY and OPENROUTER_MODEL=moonshotai/kimi-k2:free
- Optional: set VECTOR_USE_OPENROUTER=1 to attach provider notes
- npm run dev
- In Roblox Studio:
- Load the plugin (via Rojo or by placing the plugin folder)
- Click the Vector toolbar button to open the dock
- Click "Vector Settings" to enter your provider info (OpenRouter-compatible). Use Base URL `https://openrouter.ai/api/v1`, paste your API Key, and choose a Model ID (e.g., `moonshotai/kimi-k2:free`). Saved locally via plugin settings.
- Type a message, click Send
- Approve/Reject proposals; edits and rename ops will apply as described

### Notes and next steps

- Known limitations in this increment:
- Diff renderer falls back to a simplified mode for very large files; multi-edit diffs are supported but may be slower on very large scripts
- Edits merging handles the current insertion case well, but not complex multi‑edit scenarios yet
- Asset search uses stubbed backend data and shows a text list (no thumbnails yet)
- Plan/Act is a single additional step (context tool then actionable tool); a full multi-turn loop remains
- Recommended follow‑ups:
- Optimize diff computation for very large scripts; consider server-side precomputation for proposals with many edits
- Add real Catalog integration on the server and show thumbnails in the plugin UI
- Persist proposals/audit records in a durable store and add filtering endpoints
- Move to Vercel and add the hosted domain to Studio’s allowed list
- Expand provider tool‑call parsing into a full multi-turn Plan/Act execution loop with guardrails and retries
## Implementation Status

Moved to docs/IMPLEMENTATION_STATUS.md for clarity and maintenance.

---

Expand Down
99 changes: 99 additions & 0 deletions docs/IMPLEMENTATION_STATUS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Vector Implementation Status (Spec)

This document summarizes what’s implemented, what’s partial/dummy, what’s not implemented, and what inputs/decisions are needed.

Scope: warp/apps/web (Next.js) and warp/plugin (Roblox Studio plugin).

Implemented

- Plugin (Luau)
- Dock UI with chat input and proposal list
- Context capture: active script (path + editor source) and selection
- Send to backend: POST /api/chat with context and optional provider settings
- Proposal rendering: edit/object_op/asset_op cards
- Approve/Reject per proposal
- Apply edit proposals (rangeEDITS) with ChangeHistoryService and ScriptEditorService:UpdateSourceAsync
- Apply object_op rename_instance (ChangeHistoryService wrapped)
- Diff preview: simple unified diff snippet (on-demand Open Diff)
- Asset search UI: Browse button calling /api/assets/search and inline “Insert” using InsertService
- “Apply & Open” for edit proposals
- Apply result reporting: POST /api/proposals/:id/apply
- Settings popup (internal testing): configure OpenRouter Base URL, API Key, and Model, with Kimi K2 preset and Test button

- Backend (Next.js, TypeScript)
- Routes
- POST /api/chat: validates input, runs orchestrator, returns proposals, persists for audit
- GET /api/proposals: list stored proposals
- POST /api/proposals/[id]/apply: mark proposal applied with event payload
- GET /api/assets/search: calls external CATALOG_API_URL provider; returns normalized results
- POST /api/assets/generate3d: returns stub jobId
- Orchestrator
- Provider: OpenRouter adapter; reads API key/model from per-request provider or env
- Tool-call parsing: expects one XML-like tool call; zod-validated args; maps to proposals
- Fallback behavior when provider parsing fails: safe edit/object/asset proposals
- Plan/Act scaffold: if the first tool is a context tool and flags enabled, executes locally and performs one follow-up provider call
- Persistence
- File-backed JSON datastore for proposals and audit under warp/apps/web/data/proposals.json

Half-implemented (placeholder/dummy)

- Diff preview
- Minimal unified diff preview (single simplified render; line-by-line with limited context). Full multi-hunk, highlighted, and large-file optimized renderer pending.
- Plan/Act execution loop
- Single extra step only (context tool → one additional provider call). Full multi-turn loop with guardrails and retries pending.
- 3D generation route
- /api/assets/generate3d returns a stub jobId; no GPU job enqueueing or Open Cloud upload yet.
- Provider error handling
- Errors surfaced when provider is configured; basic bubbling to plugin UI is present, with minimal categorization.

Not implemented

- Full provider-driven execution loop
- Multi-turn Plan/Act with robust validation, retries, and proper context tool chaining.
- Robust diff merging on server
- Multi-edit merging, conflict resolution, and path→Instance resolution generalization.
- Real Catalog integration service
- Current backend requires an external CATALOG_API_URL service. No built-in Roblox web API integration or thumbnails yet.
- Durable database
- Migrate from file-backed JSON to SQLite/Prisma (or similar) with richer audit and filtering endpoints (e.g., filter by projectId).
- Packaging and deploy
- Vercel deployment, domain allowlisting in Studio, production environment plumbing.
- Streaming/transport
- SSE/long-polling for plugin status updates is not implemented.

What you need to provide / decide

- CATALOG_API_URL
- A server endpoint that returns normalized results [{ id, name, creator, type, thumbnailUrl? }]; required for /api/assets/search.
- Provider credentials path
- Choose either: enter credentials in the plugin Settings (recommended for dev) or set OPENROUTER_API_KEY (+ optional OPENROUTER_MODEL) in warp/apps/web/.env(.local) and set VECTOR_USE_OPENROUTER=1.
- Persistence and deploy
- Decide on DB (e.g., SQLite/Prisma) and hosting target (e.g., Vercel) for moving beyond local-only.
- Roblox Open Cloud credentials (later)
- Needed for 3D generation uploads and asset insertion flows.

File map (key references)

- Backend
- /warp/apps/web/app/api/chat/route.ts
- /warp/apps/web/app/api/proposals/route.ts
- /warp/apps/web/app/api/proposals/[id]/apply/route.ts
- /warp/apps/web/app/api/assets/search/route.ts
- /warp/apps/web/app/api/assets/generate3d/route.ts
- /warp/apps/web/lib/orchestrator/index.ts
- /warp/apps/web/lib/orchestrator/providers/openrouter.ts
- /warp/apps/web/lib/tools/schemas.ts
- /warp/apps/web/lib/diff/rangeEdits.ts
- /warp/apps/web/lib/store/proposals.ts
- /warp/apps/web/lib/store/persist.ts

- Plugin
- /warp/plugin/src/main.server.lua (dock UI, settings popup, proposals, apply flows)
- /warp/plugin/src/net/http.lua (HTTP helper)

Notes

- Localhost default: http://127.0.0.1:3000
- Plugin stores provider credentials locally via plugin:SetSetting; not committed.
- .gitignore excludes env files and warp/apps/web/data/ (local datastore).

12 changes: 9 additions & 3 deletions warp/apps/web/app/api/assets/search/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,13 @@ export async function GET(req: Request) {
const { searchParams } = new URL(req.url)
const query = searchParams.get('query') || searchParams.get('q') || ''
const limit = Math.max(1, Math.min(50, Number(searchParams.get('limit') || '8')))

const results = await searchRobloxCatalog(query, limit)
return Response.json({ results, query, limit })
try {
const results = await searchRobloxCatalog(query, limit)
return Response.json({ results, query, limit })
} catch (err: any) {
return new Response(JSON.stringify({ error: err?.message || 'Catalog provider error', query, limit }), {
status: 502,
headers: { 'content-type': 'application/json' },
})
}
}
12 changes: 7 additions & 5 deletions warp/apps/web/app/api/chat/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,12 @@ export async function POST(req: Request) {

return Response.json({ proposals })
} catch (err: any) {
console.error('chat handler error', err)
return new Response(
JSON.stringify({ error: err?.message || 'Invalid request' }),
{ status: 400, headers: { 'content-type': 'application/json' } },
)
const msg = err?.message || 'Unknown error'
const status = /invalid/i.test(msg) ? 400 : 500
console.error('chat handler error', msg)
return new Response(JSON.stringify({ error: msg }), {
status,
headers: { 'content-type': 'application/json' },
})
}
}
3 changes: 1 addition & 2 deletions warp/apps/web/app/api/proposals/route.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
export const runtime = 'nodejs'

import { listProposals } from "../../lib/store/proposals"
import { listProposals } from "../../../lib/store/proposals"

export async function GET() {
return Response.json({ proposals: listProposals() })
}

10 changes: 10 additions & 0 deletions warp/apps/web/app/layout.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
export const metadata = { title: 'Vector Web' }

export default function RootLayout({ children }: { children: React.ReactNode }) {
return (
<html lang="en">
<body>{children}</body>
</html>
)
}

4 changes: 3 additions & 1 deletion warp/apps/web/app/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,10 @@ export default function Page() {
<p>Local API endpoints for the Roblox Studio copilot (Vector).</p>
<ul>
<li>POST /api/chat</li>
<li>GET /api/assets/search</li>
<li>GET /api/proposals</li>
<li>POST /api/proposals/[id]/apply</li>
<li>GET /api/assets/search</li>
<li>POST /api/assets/generate3d</li>
</ul>
<p>Running on localhost:3000 for development. We will move to Vercel later.</p>
</main>
Expand Down
35 changes: 12 additions & 23 deletions warp/apps/web/lib/catalog/search.ts
Original file line number Diff line number Diff line change
@@ -1,29 +1,18 @@
export type CatalogItem = { id: number; name: string; creator: string; type: string; thumbnailUrl?: string }

export async function searchRobloxCatalog(query: string, limit: number): Promise<CatalogItem[]> {
// Real integration can be implemented by setting CATALOG_API_URL to a service that returns
// normalized results: [{ id, name, creator, type, thumbnailUrl }]
const url = process.env.CATALOG_API_URL
if (url) {
try {
const res = await fetch(`${url}?query=${encodeURIComponent(query)}&limit=${limit}`)
if (res.ok) {
const js = (await res.json()) as { results?: CatalogItem[] }
if (Array.isArray(js?.results)) return js.results
}
} catch (e) {
// fall through to stub
}
if (!url) {
throw new Error('CATALOG_API_URL not configured')
}
// Stub fallback
const samples: CatalogItem[] = [
{ id: 1111111, name: 'Simple Button', creator: 'Roblox', type: 'Model', thumbnailUrl: 'https://tr.rbxcdn.com/4f9b-150x150' },
{ id: 2222222, name: 'Sci-Fi Panel', creator: 'Builder', type: 'Model', thumbnailUrl: 'https://tr.rbxcdn.com/9c2a-150x150' },
{ id: 3333333, name: 'Wooden Crate', creator: 'AssetMaker', type: 'Model', thumbnailUrl: 'https://tr.rbxcdn.com/abcd-150x150' },
{ id: 4444444, name: 'Green Button', creator: 'CreatorX', type: 'Model', thumbnailUrl: 'https://tr.rbxcdn.com/ef12-150x150' },
{ id: 5555555, name: 'Keycard', creator: 'StudioUser', type: 'Model', thumbnailUrl: 'https://tr.rbxcdn.com/12ab-150x150' },
{ id: 6666666, name: 'Door', creator: 'Roblox', type: 'Model', thumbnailUrl: 'https://tr.rbxcdn.com/77aa-150x150' },
]
return samples
const res = await fetch(`${url}?query=${encodeURIComponent(query)}&limit=${limit}`)
if (!res.ok) {
const text = await res.text().catch(() => '')
throw new Error(`Catalog provider error ${res.status}: ${text}`)
}
const js = (await res.json()) as { results?: CatalogItem[] }
if (!Array.isArray(js?.results)) {
throw new Error('Invalid catalog response: missing results[]')
}
return js.results
}

Loading