diff --git a/docs/audits/2024-12-18-overview.md b/docs/audits/2024-12-18-overview.md new file mode 100644 index 000000000..5bba204d6 --- /dev/null +++ b/docs/audits/2024-12-18-overview.md @@ -0,0 +1,32 @@ +# AG-UI Repository Audit Overview (2024-12-18) + +## Repository Structure +- **Monorepo layout** that currently hosts the TypeScript SDK, Python SDK, documentation, and example applications.【F:README.md†L1-L120】【F:typescript-sdk/README.md†L1-L80】 +- TypeScript SDK uses **pnpm workspaces** with packages for the protocol core, client runtime, CLI, event encoder, and protobuf assets.【F:typescript-sdk/pnpm-workspace.yaml†L1-L20】【F:typescript-sdk/packages/core/package.json†L1-L32】 +- Python SDK mirrors the protocol models and encoding utilities, delivered as a Poetry project with typed packages under `ag_ui`.【F:python-sdk/pyproject.toml†L1-L44】【F:python-sdk/ag_ui/core/events.py†L1-L160】 + +## Core Protocol Modeling +- Protocol contracts are centralized in `@ag-ui/core`, defining Zod schemas for messages, events, and run lifecycle types with runtime validation to enforce ordering and payload integrity.【F:typescript-sdk/packages/core/src/types.ts†L1-L74】【F:typescript-sdk/packages/core/src/events.ts†L1-L213】 +- The Python SDK provides Pydantic models that remain aligned with the TypeScript schemas, including enumerated `EventType` values and validation constraints (e.g., non-empty deltas).【F:python-sdk/ag_ui/core/events.py†L1-L160】 + +## Client Runtime +- `@ag-ui/client` implements an `AbstractAgent` that orchestrates runs, manages subscriber hooks, applies protocol events to local state, and bridges to the legacy runtime when needed.【F:typescript-sdk/packages/client/src/agent/agent.ts†L1-L220】【F:typescript-sdk/packages/client/src/agent/agent.ts†L220-L440】 +- Event processing pipelines combine chunk transformation, protocol verification, state mutation application, and subscriber notifications using RxJS operators, ensuring order and error propagation are handled consistently.【F:typescript-sdk/packages/client/src/agent/agent.ts†L43-L103】【F:typescript-sdk/packages/client/src/apply/default.ts†L1-L120】 +- The verification layer (`verifyEvents`) enforces state-machine constraints such as message/tool call lifecycle, run start/finish gating, and step tracking to prevent malformed event streams.【F:typescript-sdk/packages/client/src/verify/verify.ts†L1-L120】 + +## Testing & Quality +- TypeScript packages use Jest with extensive coverage across verification logic, agent orchestration, transformers, and legacy compatibility, emphasizing concurrent and lifecycle edge cases.【F:typescript-sdk/packages/client/src/verify/__tests__/verify.concurrent.test.ts†L1-L40】【F:typescript-sdk/packages/client/src/agent/__tests__/agent-multiple-runs.test.ts†L1-L80】 +- Core protocol schemas include unit tests validating role defaults and schema exports to catch regressions in event definitions.【F:typescript-sdk/packages/core/src/__tests__/events-role-defaults.test.ts†L1-L75】 +- Python SDK mirrors these checks with pytest suites for type parity, event validation, encoder behavior, and text role constraints.【F:python-sdk/tests/test_events.py†L1-L120】【F:python-sdk/tests/test_text_roles.py†L1-L80】 + +## Observations & Recommendations +1. **Cross-SDK Consistency:** The parallel TypeScript and Python models reduce integration drift; consider adding automated contract tests (e.g., golden JSON fixtures) to guarantee synchronized event definitions across languages.【F:typescript-sdk/packages/core/src/events.ts†L1-L213】【F:python-sdk/ag_ui/core/events.py†L1-L160】 +2. **Runtime Extensibility:** Subscriber pattern enables custom behaviors, but lacks explicit typing around mutation side-effects; documenting expected mutation semantics (e.g., stop propagation) in public docs would improve integrator experience.【F:typescript-sdk/packages/client/src/agent/subscriber.ts†L1-L160】【F:typescript-sdk/packages/client/src/apply/default.ts†L35-L120】 +3. **Verification Coverage:** Current state machine rejects post-error events but relies on in-memory maps; for distributed deployments, consider persisting run state or emitting reconciliation hooks to tolerate reconnects.【F:typescript-sdk/packages/client/src/verify/verify.ts†L1-L160】 +4. **Legacy Bridge Deprecation:** Legacy conversion utilities remain for backward compatibility; plan to sunset once downstream adopters migrate to native AG-UI streams to reduce maintenance overhead.【F:typescript-sdk/packages/client/src/legacy/convert.ts†L1-L200】【F:typescript-sdk/packages/client/src/agent/agent.ts†L340-L420】 + +## Risk Snapshot +- **Security:** Minimal direct network surface; focus is on schema validation. Ensure future transport adapters enforce authentication and redact sensitive fields before logging (currently, debug logging may expose payloads).【F:typescript-sdk/packages/client/src/agent/agent.ts†L61-L120】【F:typescript-sdk/packages/client/src/verify/verify.ts†L17-L44】 +- **Performance:** RxJS pipelines process events sequentially; high-throughput workloads should benchmark bottlenecks in `defaultApplyEvents`, especially around repeated deep clones and subscriber callbacks.【F:typescript-sdk/packages/client/src/apply/default.ts†L1-L80】 +- **Maintainability:** Clear package separation and testing discipline aid onboarding. Documenting architectural decisions (e.g., why JSON Patch is used for state deltas) would help contributors extend the protocol confidently.【F:typescript-sdk/packages/client/src/apply/default.ts†L1-L60】【F:typescript-sdk/packages/client/src/apply/default.ts†L80-L160】 +