Skip to content

Commit edfadcc

Browse files
andrewm4894claude
andauthored
feat(ai): Add $ai_framework property for framework integrations (#347)
* Add $ai_lib_metadata to AI integrations Adds framework identification metadata to all AI events for easier filtering and analytics. Each integration now includes a $ai_lib_metadata property with schema version and framework name. - LangChain: Hardcoded to "langchain" - Native wrappers (Anthropic, OpenAI, Gemini): Uses provider name - Ready for future frameworks (pydantic-ai, crewai, llamaindex) This enables PostHog queries to easily distinguish between: - Direct SDK wrapper usage - Framework-mediated usage (LangChain, etc.) - Different framework types 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add \$ai_lib_metadata to sync/async paths and tests - Added \$ai_lib_metadata to call_llm_and_track_usage (sync) - Added \$ai_lib_metadata to call_llm_and_track_usage_async (async) - Added test assertion in test_basic_completion - Placed metadata at end of properties for consistency All tests pass successfully. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Refactor: use unified utility function for $ai_lib_metadata Creates a single `get_ai_lib_metadata(framework)` utility function to generate the $ai_lib_metadata object, replacing inline implementations across the codebase. Changes: - Add get_ai_lib_metadata() utility to utils.py - Update LangChain callbacks to use utility function - Update call_llm_and_track_usage() to use utility function - Update call_llm_and_track_usage_async() to use utility function - Update capture_streaming_event() to use utility function Benefits: - Consistency across all integrations - Single source of truth for metadata structure - Easier to extend with version detection later 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Add $ai_lib_metadata assertions to provider tests Add missing $ai_lib_metadata assertions to Anthropic, Gemini, and LangChain tests to match the validation already present in OpenAI tests. Each test now verifies the metadata field contains the correct schema version and framework name. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * Simplify to $ai_framework property, only for actual frameworks Changes: - Replace complex $ai_lib_metadata object with simple $ai_framework string - Only include $ai_framework when using actual framework (LangChain) - Remove $ai_framework from direct provider calls (OpenAI, Anthropic, Gemini) - Update all tests to reflect new behavior Before: {"schema": "v1", "frameworks": [{"name": "langchain"}]} After: "langchain" (only when using LangChain framework) This eliminates wasteful redundancy where framework=provider for direct calls. --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 13184e2 commit edfadcc

File tree

2 files changed

+3
-0
lines changed

2 files changed

+3
-0
lines changed

posthog/ai/langchain/callbacks.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -486,6 +486,7 @@ def _capture_trace_or_span(
486486
"$ai_latency": run.latency,
487487
"$ai_span_name": run.name,
488488
"$ai_span_id": run_id,
489+
"$ai_framework": "langchain",
489490
}
490491
if parent_run_id is not None:
491492
event_properties["$ai_parent_id"] = parent_run_id
@@ -556,6 +557,7 @@ def _capture_generation(
556557
"$ai_http_status": 200,
557558
"$ai_latency": run.latency,
558559
"$ai_base_url": run.base_url,
560+
"$ai_framework": "langchain",
559561
}
560562

561563
if run.tools:

posthog/test/ai/langchain/test_callbacks.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -204,6 +204,7 @@ def test_basic_chat_chain(mock_client, stream):
204204
# Generation is second
205205
assert generation_args["event"] == "$ai_generation"
206206
assert "distinct_id" in generation_args
207+
assert generation_props["$ai_framework"] == "langchain"
207208
assert "$ai_model" in generation_props
208209
assert "$ai_provider" in generation_props
209210
assert generation_props["$ai_input"] == [

0 commit comments

Comments
 (0)