[Feature] Support CrewAI for BYO agents #920

supreme-gg-gg · 2025-09-20T22:33:33Z

This PR extends BYO agents support for agents created with the CrewAI framework. The design follows kagent-langgraph.

Changes

Create new kagent-crewai package for creating A2A servers using CrewAI
Provide samples/crewai for Crew and Flow mode to show developers how to integrate (<10 lines of change from CrewAi code to setup the integration)

Next steps

Implement memory and persistence for CrewAI agents
Support OpenTelemetry CrewAI Instrumentation

Testing

To get started, follow instructions in samples/crewai/research-crew. Alternatively, follow samples/crewai/poem_flow for a Flow mode example. Both examples are taken from the CrewAI QuickStart tutorial, showcasing how to create agent, tasks, provider, crew, flow, etc. in CrewAI and integrate it seamlessly by just modifying the code entry point.

How it looks like from the Kagent UI:

Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

…mple Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

Copilot

Pull Request Overview

This PR adds support for CrewAI agents to the BYO (Bring Your Own) agent framework in KAgent. It follows a similar design pattern to the existing kagent-langgraph integration but adapts it for CrewAI's multi-agent orchestration capabilities.

Creates the kagent-crewai package to enable A2A server integration with CrewAI crews and flows
Provides two comprehensive sample implementations: a research crew with multiple agents and a poem generation flow
Maintains the standard CrewAI developer experience while adding KAgent integration with minimal code changes

Reviewed Changes

Copilot reviewed 26 out of 29 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`python/packages/kagent-crewai/`	Core package providing CrewAI integration with A2A protocol support and event streaming
`python/samples/crewai/research-crew/`	Sample research crew demonstrating multi-agent collaboration with web search capabilities
`python/samples/crewai/poem_flow/`	Sample CrewAI flow showing state management and sequential execution patterns
`python/pyproject.toml`	Workspace configuration updated to include CrewAI samples
`python/Makefile`	Build targets added for CrewAI sample containers

Comments suppressed due to low confidence (1)

python/packages/kagent-crewai/src/kagent/crewai/_executor.py:1

This line appears to be copy-pasted from the poem flow sample and doesn't belong in the generic executor. The log message should be generic or removed entirely.

import asyncio

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

python/samples/crewai/poem_flow/src/poem_flow/crews/poem_crew/config/agents.yaml

python/samples/crewai/research-crew/README.md

krisztianfekete · 2025-09-22T06:41:12Z

python/samples/crewai/poem_flow/src/poem_flow/main.py

+
+import uvicorn
+from crewai.flow import Flow, listen, start
+from kagent.core import configure_tracing


What do you think setting this up in as part of the integration, so end users can have it automatically without specifying in each of their crewai agents?

Would you be also interested (either here, or as a follow-up) seeing how leveraging this SDK would improve instrumentation?

I think it's better if it user manually enable tracing, I can put this as an argument when creating the app so the user doesn't need this import: app = KAgentApp(..., tracing=True)

For the opentelemetry CrewAI SDK would you mind elaborating on what features are you looking for? I'm down to implement this as well, but I saw that there are already tracing in kagent core

Specifically this would be enabling tracing for crewai. We bake in tracing for various LLM providers, http calls, and ADK, but we don't want to ALWAYS load tracing libraries for agent libraries we aren't actively using. I think I agree with @supreme-gg-gg about app = KAgentApp(..., tracing=True), but maybe it should be True by default.

Specifically this would be enabling tracing for crewai. We bake in tracing for various LLM providers, http calls, and ADK, but we don't want to ALWAYS load tracing libraries for agent libraries we aren't actively using. I think I agree with @supreme-gg-gg about app = KAgentApp(..., tracing=True), but maybe it should be True by default.

Would CrewAI tracing with this SDK be on top of or replace the existing provider-specific tracing (e.g. OpenAI, Anthropic) in kagent core? it would be placed in the CrewAI package, is that correct

Not sure if it would replace those, as what we have in kagent-core utils covers LLM-specific and auxiliary tracing. I'd expect the CrewAI SDK to add additional, CrewAI specific spans that can be used on top of what we have as core tracing.

Have you had a chance to take a look at what spans are you getting with the current changes? If we have LLM calls then I'd say we are fine w/r/t tracing with this PR, and can do add the CrewAI SDK as a potential follow-up, if it's useful.

@krisztianfekete Sorry for the late response, I did a few experiments with CrewAI tracing and validated that the CrewAI spans are indeed sent to the collector. CrewAI has its internal instrumentor, but I found that by adding the CrewAI Instrumentation SDK you suggested the spans will be more comprehensive.

Regarding providers, the CrewAI SDK traces all calls to LiteLLM, but provider-specific SDK is required for spans containing the actual LLM call content e.g. prompts, token count, so I will use an approach similar to Langgraph tracing where both the provider specific + framework specific SDK are used.

I will be adding this to the new follow up PR #951

Thanks for following up!

In the core library we have Gemini/Anthropic/OpenAI SDKs already, so by adding the CrewAI one, we should have support for the most popular options I think.

EItanya

This is looking great so far! Mostly my comments are about the executor and some Docker config.

python/packages/kagent-crewai/src/kagent/crewai/__init__.py

EItanya · 2025-09-22T19:38:24Z

python/packages/kagent-crewai/src/kagent/crewai/_executor.py

+        loop = asyncio.get_running_loop()
+
+        def _enqueue_event(event: Any):
+            asyncio.run_coroutine_threadsafe(event_queue.enqueue_event(event), loop)


Why do we need this helper func, why can't we just enqueue the envoy from the scoped handlers?

The handlers must be synchronous because they're invoked by the crewAI event bus. However the enqueue_event function from A2A is async, so we need this helper to execute the enqueue instead of doing it directly in the handler

What will the impact be of running it this way?

I don't think there are any major risks or downsides for this workaround, the only minor problem I speculate might be that if the enqueue fails we can't log and report it since it's somewhat like "fire and forget"

EItanya · 2025-09-22T19:46:35Z

python/samples/crewai/poem_flow/Dockerfile

+FROM python:3.13-slim
+
+WORKDIR /app
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+    build-essential \
+    && rm -rf /var/lib/apt/lists/*
+
+# Install uv for fast Python package management
+RUN pip install uv


We can use uv image to avoid installing uv separately

Should I update the langgraph example to use uv image as well?

python/packages/kagent-crewai/src/kagent/crewai/_executor.py

Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

supreme-gg-gg · 2025-09-23T20:26:25Z

@EItanya could you review my changes to the Dockerfiles for python app and the samples, thanks!

EItanya · 2025-09-25T15:02:53Z

python/samples/crewai/poem_flow/pyproject.toml

+kickoff = "poem_flow.main:kickoff"
+run_crew = "poem_flow.main:kickoff"
+plot = "poem_flow.main:plot"
+poem_flow = "poem_flow.main:main"


Do we need these to be scripts? Scripts end up being global in a uv workspace. We can fix in a follow-up

Will remove these in the follow up, they're generated by crewAI

This PR is a follow up for #920 ## Features - [x] Creates go handlers for `/crewai` routes for storing and retrieving memory items - [x] Updates the database handler to store and retrieve long term memory items from CrewAI Crew and state for CrewAI Flow - [x] Adds custom memory store to agents before execution to allow for session-based persistence - [x] Updates samples to show usage of memory by setting `memory=True` and persistence by `@persist()` - [x] Support tracing with `opentelemetry-instrumentation-crewai` for crewai specific spans as suggested in code review ## Tests e2e test is added for CrewAI Poem flow sample agents using the mock LLM server. The test case will create the agent resource, create the mock LLM server using mock response in `invoke_creawi_agent.json`, test synchronous, streaming, and persistence for the agent. It requires the agent container to be build and pushed to the registry (by running `make poem-flow-sample` or the Dockerfile directly in `samples/crewai/poem_flow`). The following changes are made to helper functions in e2e test: 1. `runSyncTest` accepts an optional `contextID` to be included in the mock message to test session persistence 2. `runSyncTest` accepts an optional `useArtifacts` argument to indicate if the expected output should be checked for in the history messages or the artifact returned by the A2A server, since the A2A protocol specifies that `Artifacts` are the standard way to convey final outputs of a task --------- Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

…gent-dev#951) This PR is a follow up for kagent-dev#920 - [x] Creates go handlers for `/crewai` routes for storing and retrieving memory items - [x] Updates the database handler to store and retrieve long term memory items from CrewAI Crew and state for CrewAI Flow - [x] Adds custom memory store to agents before execution to allow for session-based persistence - [x] Updates samples to show usage of memory by setting `memory=True` and persistence by `@persist()` - [x] Support tracing with `opentelemetry-instrumentation-crewai` for crewai specific spans as suggested in code review e2e test is added for CrewAI Poem flow sample agents using the mock LLM server. The test case will create the agent resource, create the mock LLM server using mock response in `invoke_creawi_agent.json`, test synchronous, streaming, and persistence for the agent. It requires the agent container to be build and pushed to the registry (by running `make poem-flow-sample` or the Dockerfile directly in `samples/crewai/poem_flow`). The following changes are made to helper functions in e2e test: 1. `runSyncTest` accepts an optional `contextID` to be included in the mock message to test session persistence 2. `runSyncTest` accepts an optional `useArtifacts` argument to indicate if the expected output should be checked for in the history messages or the artifact returned by the A2A server, since the A2A protocol specifies that `Artifacts` are the standard way to convey final outputs of a task --------- Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

…gent-dev#951) This PR is a follow up for kagent-dev#920 ## Features - [x] Creates go handlers for `/crewai` routes for storing and retrieving memory items - [x] Updates the database handler to store and retrieve long term memory items from CrewAI Crew and state for CrewAI Flow - [x] Adds custom memory store to agents before execution to allow for session-based persistence - [x] Updates samples to show usage of memory by setting `memory=True` and persistence by `@persist()` - [x] Support tracing with `opentelemetry-instrumentation-crewai` for crewai specific spans as suggested in code review ## Tests e2e test is added for CrewAI Poem flow sample agents using the mock LLM server. The test case will create the agent resource, create the mock LLM server using mock response in `invoke_creawi_agent.json`, test synchronous, streaming, and persistence for the agent. It requires the agent container to be build and pushed to the registry (by running `make poem-flow-sample` or the Dockerfile directly in `samples/crewai/poem_flow`). The following changes are made to helper functions in e2e test: 1. `runSyncTest` accepts an optional `contextID` to be included in the mock message to test session persistence 2. `runSyncTest` accepts an optional `useArtifacts` argument to indicate if the expected output should be checked for in the history messages or the artifact returned by the A2A server, since the A2A protocol specifies that `Artifacts` are the standard way to convey final outputs of a task --------- Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

supreme-gg-gg added 5 commits September 20, 2025 18:34

feat: setup a2a server and agent executor for crewai

cc594a4

Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

feat: provide example crewai research agent

46c9d5c

Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

feat: update pyproject makefile and uv lock for crewai package and sa…

6698b99

…mple Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

feat: add more event types and include flow method exec events

233e1f6

Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

feat: add example for creawi flow setup

fe7faa9

Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

supreme-gg-gg force-pushed the feat/crewai branch from 4ee6b3f to fe7faa9 Compare September 20, 2025 22:34

fix: cleanup naming and readme

7d196f3

Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

supreme-gg-gg marked this pull request as ready for review September 22, 2025 04:37

supreme-gg-gg requested review from EItanya, peterj and yuval-k as code owners September 22, 2025 04:37

Copilot AI review requested due to automatic review settings September 22, 2025 04:37

Copilot AI reviewed Sep 22, 2025

View reviewed changes

python/samples/crewai/poem_flow/src/poem_flow/crews/poem_crew/config/agents.yaml Outdated Show resolved Hide resolved

python/samples/crewai/research-crew/README.md Outdated Show resolved Hide resolved

krisztianfekete reviewed Sep 22, 2025

View reviewed changes

EItanya requested changes Sep 22, 2025

View reviewed changes

supreme-gg-gg added 5 commits September 22, 2025 20:35

add listners and upgrade crewai deps

7001c5f

Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

update crewai examples and dockerfile

04cf1fb

Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

remove init in crewai package

a0a8d3d

Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

install only kagent-adk and kagent-core in python app dockerfile

409a122

Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

update dockerfile for langgraph to use uv base image

6fcbfe2

Signed-off-by: Jet Chiang <jetjiang.ez@gmail.com>

supreme-gg-gg requested a review from EItanya September 23, 2025 20:24

EItanya reviewed Sep 25, 2025

View reviewed changes

EItanya approved these changes Sep 25, 2025

View reviewed changes

EItanya merged commit a3b8d1a into kagent-dev:main Sep 25, 2025
16 checks passed

This was referenced Sep 25, 2025

[Feature] Implement Persistence and Tracing for CrewAI BYO Agents #951

Merged

Add CrewAI BYO Agent docs kagent-dev/website#238

Merged

This was referenced Dec 2, 2025

[CNCF Incubation] Due Diligence Review kagent-dev/community#7

Closed

[Incubation] kagent Incubation Application cncf/toc#1978

Open

[Feature] Support CrewAI for BYO agents #920

[Feature] Support CrewAI for BYO agents #920

Uh oh!

Conversation

supreme-gg-gg commented Sep 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Next steps

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

supreme-gg-gg Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

EItanya left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

supreme-gg-gg commented Sep 23, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

supreme-gg-gg commented Sep 20, 2025 •

edited

Loading

supreme-gg-gg Oct 1, 2025 •

edited

Loading