Added TruGen Avatar Plugin. #4430
Conversation
|
tinalenguyen
left a comment
tinalenguyen
left a comment
There was a problem hiding this comment.
hi, thank you for submitting this PR! i left a few comments, but i couldn't test it out. i hit the max retries and kept receiving this error: Error creating voice to video session: concurrent session limit reached
let me know what may be causing this issue and when i could try again, thanks in advance!
| if i < self._conn_options.max_retry - 1: | ||
| await asyncio.sleep(self._conn_options.retry_interval) | ||
|
|
||
| raise APIConnectionError("Max retry exhaused; Unable to start TruGen.AI Avatar Session.") |
There was a problem hiding this comment.
| raise APIConnectionError("Max retry exhaused; Unable to start TruGen.AI Avatar Session.") | |
| raise APIConnectionError("Max retries exhausted; Unable to start TruGen.AI Avatar Session.") |
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py
Outdated
Show resolved
Hide resolved
- Added default avatar id to be used when nothing is passed. - Changed API Connection Error phrasing.
|
Hi @tinalenguyen, Cheers. |
|
Hi @tinalenguyen , Note: For a free account we have a limit of 1 concurrent session at a time, so between every session you can have about 30 seconds of break. Cheers, |
|
Hi, thanks for iterating on my feedback! When I tested the plugin again, the avatar never joined the LiveKit room, but I did debug and see that the initial API request was sent successfully. I didn't see any explicit error logs. Are you able to reproduce this? |
|
Hi @tinalenguyen, We have reset the same, and it should work just fine for you, if the issue still persist we recommend you retry with a different account. We just did an end to end test with a new account both at Livekit level and at our platform, and everything is working fine. |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
📝 WalkthroughWalkthroughAdds a new TruGen.AI LiveKit plugin and example integration: plugin package, AvatarSession implementation with HTTP + retry handling, logger and version modules, packaging config, example agent worker, and README docs. All changes are additive; no existing files were modified. Changes
Sequence Diagram(s)sequenceDiagram
participant Agent as LiveKit Agent
participant AvatarSession as AvatarSession
participant TruGenAPI as TruGen API
participant LiveKitRoom as LiveKit Room
Agent->>AvatarSession: start(agent_session, room, credentials)
AvatarSession->>AvatarSession: build LiveKit JWT (agent scope, publish-on-behalf)
AvatarSession->>TruGenAPI: POST /v1/sessions {avatar_id, livekit_url, livekit_token}
alt 200 OK
TruGenAPI-->>AvatarSession: session info
AvatarSession->>LiveKitRoom: stream audio on behalf of avatar (DataStreamAudioOutput)
AvatarSession-->>Agent: mark session started
else Error
TruGenAPI-->>AvatarSession: APIStatusError / APIConnectionError
AvatarSession->>AvatarSession: log error, retry per conn_options
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧹 Recent nitpick comments
📜 Recent review detailsConfiguration used: Organization UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🧰 Additional context used📓 Path-based instructions (1)**/*.py📄 CodeRabbit inference engine (AGENTS.md)
Files:
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
🔇 Additional comments (3)
✏️ Tip: You can disable this entire section by setting Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Fix all issues with AI agents
In `@livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/__init__.py`:
- Around line 31-36: Add a Google-style docstring to the TrugenPlugin class
describing its purpose and the threading requirement, and remove the
module-level Plugin.register_plugin(TrugenPlugin()) call; instead create a
function like register_trugen_plugin() that instantiates TrugenPlugin and calls
Plugin.register_plugin(…) only after verifying it’s running on the main thread
(e.g., check threading.current_thread() or raise a clear RuntimeError), so
imports from worker threads won’t crash; references: TrugenPlugin and
Plugin.register_plugin.
In `@livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py`:
- Around line 109-142: The retry loop currently iterates
range(self._conn_options.max_retry) which performs only max_retry total attempts
and zero attempts when max_retry=0; change the loop to perform the initial
attempt plus the configured retries by iterating
range(self._conn_options.max_retry + 1) (or otherwise compute attempts = 1 +
max_retry) so that at least one attempt is made and the intended number of
retries occur; update the loop variable name (e.g., i -> attempt) and keep the
existing backoff/sleep logic and final raise of APIConnectionError unchanged.
- Around line 79-83: Replace the generic Exception with the project-specific
TrugenException when validating livekit credentials (checks for livekit_url,
livekit_api_key, livekit_api_secret) and correct the error message typos: change
"either pass then as arguments" to "either pass them as arguments" and
"enviroment" to "environment"; ensure this change occurs in the credential
validation block that references livekit_url/livekit_api_key/livekit_api_secret
(e.g., in the avatar initialization or factory function that currently raises
the Exception).
In `@livekit-plugins/livekit-plugins-trugen/README.md`:
- Line 13: Replace the non-descriptive anchor text "link" in the sentence
"Generate an API key from our Developer Studio [link](https://app.trugen.ai) and
set the `TRUGEN_API_KEY` environment variable with it" with a clear label such
as "Trugen Developer Studio" (e.g., use [Trugen Developer
Studio](https://app.trugen.ai)), updating the markdown so the anchor text is
descriptive while keeping the `TRUGEN_API_KEY` environment variable mention
unchanged.
🧹 Nitpick comments (3)
examples/avatar_agents/trugen/agent_worker.py (1)
1-19: Add Google-style docstrings for module and entrypoint.The example is public-facing and should follow the repo’s Python docstring guidance. As per coding guidelines, please add Google-style docstrings.
♻️ Proposed update
+"""TruGen avatar agent worker example. + +This example starts a LiveKit Agent session and a TruGen avatar session in the same room. +""" import logging import os @@ `@server.rtc_session`() async def entrypoint(ctx: JobContext): + """Starts the TruGen avatar and agent sessions for the given LiveKit room. + + Args: + ctx: LiveKit job context containing the room to join. + """ session = AgentSession( llm=google.realtime.RealtimeModel(), resume_false_interruption=False, )livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/__init__.py (1)
31-33: Add a Google-style class docstring forTrugenPlugin.This is a public plugin class and should follow the repo’s docstring guidance. As per coding guidelines, please add a class docstring.
♻️ Proposed update
class TrugenPlugin(Plugin): + """LiveKit Agents plugin registration for TruGen.AI.""" def __init__(self) -> None: super().__init__(__name__, __version__, __package__, logger)examples/avatar_agents/trugen/README.md (1)
11-22: DocumentTRUGEN_AVATAR_ID(or state the default).The example code reads
TRUGEN_AVATAR_ID, but the README doesn’t mention it. Adding it avoids confusion.📝 Suggested doc tweak
# TruGen Config export TRUGEN_API_KEY="..." +export TRUGEN_AVATAR_ID="..." # Optional if a default avatar is used
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (11)
examples/avatar_agents/README.mdexamples/avatar_agents/trugen/README.mdexamples/avatar_agents/trugen/agent_worker.pylivekit-plugins/livekit-plugins-trugen/README.mdlivekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/__init__.pylivekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.pylivekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/log.pylivekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/py.typedlivekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/version.pylivekit-plugins/livekit-plugins-trugen/pyproject.tomlpyproject.toml
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format code with ruff
Run ruff linter and auto-fix issues
Run mypy type checker in strict mode
Maintain line length of 100 characters maximum
Ensure Python 3.9+ compatibility
Use Google-style docstrings
Files:
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/version.pylivekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/log.pyexamples/avatar_agents/trugen/agent_worker.pylivekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/__init__.pylivekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py
🧠 Learnings (1)
📚 Learning: 2026-01-16T07:44:56.353Z
Learnt from: CR
Repo: livekit/agents PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-01-16T07:44:56.353Z
Learning: Follow the Plugin System pattern where plugins in livekit-plugins/ are separate packages registered via the Plugin base class
Applied to files:
pyproject.toml
🧬 Code graph analysis (2)
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/__init__.py (2)
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py (1)
TrugenException(31-32)livekit-agents/livekit/agents/plugin.py (2)
Plugin(13-56)register_plugin(31-36)
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py (5)
livekit-agents/livekit/agents/job.py (3)
api(288-296)get_job_context(56-63)local_participant_identity(327-331)livekit-agents/livekit/agents/_exceptions.py (1)
APIStatusError(45-81)livekit-agents/livekit/agents/types.py (1)
APIConnectOptions(54-88)livekit-agents/livekit/agents/utils/http_context.py (1)
http_session(40-51)livekit-agents/livekit/agents/utils/aio/sleep.py (1)
sleep(68-69)
🪛 markdownlint-cli2 (0.18.1)
livekit-plugins/livekit-plugins-trugen/README.md
13-13: Link text should be descriptive
(MD059, descriptive-link-text)
🔇 Additional comments (7)
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/version.py (1)
15-15: Version export looks good.pyproject.toml (1)
48-48: Workspace registration looks correct.livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/log.py (1)
1-3: Logger setup is clean and consistent.livekit-plugins/livekit-plugins-trugen/pyproject.toml (1)
1-39: Package metadata/build configuration looks good.livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py (1)
85-94: No action needed —local_participant_identityis already a property.The code correctly accesses
local_participant_identityas a property without calling it. The@propertydecorator is defined onJobContext.local_participant_identityinlivekit-agents/livekit/agents/job.py:326, so the current usage is correct.Likely an incorrect or invalid review comment.
examples/avatar_agents/README.md (1)
17-17: Nice addition to the providers list.examples/avatar_agents/trugen/agent_worker.py (1)
28-32: The original review comment is incorrect.AgentSession.generate_replyis a synchronous method (not async) and returns aSpeechHandleobject. Calling it withoutawaitis the correct usage.Likely an incorrect or invalid review comment.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/__init__.py
Show resolved
Hide resolved
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py
Outdated
Show resolved
Hide resolved
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py
Outdated
Show resolved
Hide resolved
- Implemented all the changes recommended.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py`:
- Around line 41-57: The constructor currently uses truthy "or" with the
NOT_GIVEN sentinel which can leave fields set to NOT_GIVEN; change assignments
to explicitly check "is NOT_GIVEN" for avatar_id, api_key,
avatar_participant_identity, and avatar_participant_name (e.g., if api_key is
NOT_GIVEN then read os.getenv("TRUGEN_API_KEY"), else use provided value) so
env/default fallbacks are applied correctly, then set self._api_key and raise
TrugenException only if the resolved api_key is None; update references to
_avatar_id, _api_key, _avatar_participant_identity, and _avatar_participant_name
accordingly.
- Around line 72-83: The constructor currently uses "livekit_url = livekit_url
or (os.getenv(... ) )" which treats the NOT_GIVEN sentinel as truthy and skips
env fallbacks; change the assignment logic in the __init__ (avatar.py) so that
if each param (livekit_url, livekit_api_key, livekit_api_secret) equals
NOT_GIVEN you read from os.getenv, otherwise keep the provided value, then
validate that the resulting values are truthy and raise TrugenException if any
are missing; reference the NOT_GIVEN sentinel and the variables livekit_url,
livekit_api_key, livekit_api_secret to locate and update the code.
🧹 Nitpick comments (3)
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py (2)
107-140: Respect non‑retryable API errors and use the retry-interval helper.
APIStatusErroralready marks 4xx as non‑retryable, andAPIConnectOptionsprovides_interval_for_retryfor immediate first retry.♻️ Proposed update
- except Exception as e: + except Exception as e: + if isinstance(e, APIStatusError) and not e.retryable: + raise if isinstance(e, APIConnectionError): logger.warning( "API Error; Unable to trigger TruGen.AI API backend.", extra={"error": str(e)}, ) else: logger.exception("API Error; Unable to trigger TruGen.AI API backend.") if i < self._conn_options.max_retry: - await asyncio.sleep(self._conn_options.retry_interval) + await asyncio.sleep(self._conn_options._interval_for_retry(i))
67-107: Add Google‑style docstrings for public methods.
AvatarSession.start(and__init__) are public API and should have Google‑style docstrings.As per coding guidelines, "Use Google-style docstrings".
examples/avatar_agents/trugen/agent_worker.py (1)
17-34: Add a Google‑style docstring and explicit return type forentrypoint.This helps satisfy the project’s docstring requirement and mypy strict settings.
As per coding guidelines, "Use Google-style docstrings" and "Run mypy type checker in strict mode".✍️ Proposed update
`@server.rtc_session`() -async def entrypoint(ctx: JobContext): +async def entrypoint(ctx: JobContext) -> None: + """Run the TruGen avatar example. + + Args: + ctx: LiveKit job context provided by the AgentServer. + """
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (4)
examples/avatar_agents/trugen/agent_worker.pylivekit-plugins/livekit-plugins-trugen/README.mdlivekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.pylivekit-plugins/livekit-plugins-trugen/pyproject.toml
🚧 Files skipped from review as they are similar to previous changes (2)
- livekit-plugins/livekit-plugins-trugen/pyproject.toml
- livekit-plugins/livekit-plugins-trugen/README.md
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format code with ruff
Run ruff linter and auto-fix issues
Run mypy type checker in strict mode
Maintain line length of 100 characters maximum
Ensure Python 3.9+ compatibility
Use Google-style docstrings
Files:
examples/avatar_agents/trugen/agent_worker.pylivekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py
🧬 Code graph analysis (1)
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py (5)
livekit-agents/livekit/agents/job.py (2)
api(288-296)get_job_context(56-63)livekit-agents/livekit/agents/_exceptions.py (2)
APIConnectionError(84-88)APIStatusError(45-81)livekit-agents/livekit/agents/types.py (1)
APIConnectOptions(54-88)livekit-agents/livekit/agents/voice/avatar/_datastream_io.py (1)
DataStreamAudioOutput(23-260)livekit-agents/livekit/agents/utils/http_context.py (1)
http_session(40-51)
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py`:
- Around line 147-158: The isinstance(e, APIConnectionError) check is
ineffective and the loop currently ignores APIStatusError.retryable; update the
except block around the TruGen API call in avatar.py to: determine whether the
caught exception is retryable by checking if it's an APIConnectionError or an
APIStatusError with retryable==True (reference APIConnectionError and
APIStatusError.retryable), log a warning via logger.warning for retryable errors
and then await retry (respecting self._conn_options.max_retry and
retry_interval), but for non-retryable errors (e.g., APIStatusError with
retryable==False or other exceptions) log with logger.exception and break/return
without retrying; remove the ineffective isinstance(e, APIConnectionError) sole
check and ensure the retry loop honors APIStatusError.retryable.
🧹 Nitpick comments (1)
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py (1)
138-138: Consider adding a total timeout for the HTTP request.Only
sock_connecttimeout is configured. Long-running or stalled responses won't be terminated. Consider also settingtotalorsock_readtimeouts.♻️ Proposed enhancement
- timeout=aiohttp.ClientTimeout(sock_connect=self._conn_options.timeout), + timeout=aiohttp.ClientTimeout( + total=self._conn_options.timeout * 3, + sock_connect=self._conn_options.timeout, + ),
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
examples/avatar_agents/trugen/agent_worker.pylivekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format code with ruff
Run ruff linter and auto-fix issues
Run mypy type checker in strict mode
Maintain line length of 100 characters maximum
Ensure Python 3.9+ compatibility
Use Google-style docstrings
Files:
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.pyexamples/avatar_agents/trugen/agent_worker.py
🧬 Code graph analysis (2)
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py (4)
livekit-agents/livekit/agents/job.py (2)
api(288-296)get_job_context(56-63)livekit-agents/livekit/agents/_exceptions.py (2)
APIConnectionError(84-88)APIStatusError(45-81)livekit-agents/livekit/agents/types.py (1)
APIConnectOptions(54-88)livekit-agents/livekit/agents/utils/http_context.py (1)
http_session(40-51)
examples/avatar_agents/trugen/agent_worker.py (1)
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py (2)
AvatarSession(35-159)start(73-122)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: unit-tests
- GitHub Check: type-check (3.9)
- GitHub Check: type-check (3.13)
🔇 Additional comments (7)
examples/avatar_agents/trugen/agent_worker.py (3)
1-11: LGTM!Imports are well-organized and appropriate for the example use case.
35-36: LGTM!Standard CLI entry point pattern for LiveKit agents.
17-32: No changes needed.AgentSession.generate_reply()is a synchronous method that returnsSpeechHandle, not a coroutine. The method call withoutawaitis correct and expected.Likely an incorrect or invalid review comment.
livekit-plugins/livekit-plugins-trugen/livekit/plugins/trugen/avatar.py (4)
31-32: LGTM!Clean custom exception class for TruGen-specific errors.
38-65: LGTM!The
NOT_GIVENsentinel handling has been properly implemented with explicitis NOT_GIVENchecks, ensuring environment variable fallbacks work correctly.
67-71: LGTM!Lazy HTTP session initialization using the framework's
http_contextutility is appropriate.
73-122: LGTM!Credential validation, JWT token generation with proper grants, and
DataStreamAudioOutputwiring are correctly implemented.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
+ misc fixes
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| __version__ = "1.3.10" |
There was a problem hiding this comment.
🔴 Plugin version 1.3.10 is below the >=1.4.1 requirement declared in livekit-agents optional deps, making installation impossible
The trugen plugin's own version is 1.3.10 (version.py:15), but livekit-agents/pyproject.toml:110 declares the optional dependency as trugen = ["livekit-plugins-trugen>=1.4.1"]. Since 1.3.10 < 1.4.1, running pip install livekit-agents[trugen] will fail with a version resolution error — pip cannot find a livekit-plugins-trugen version satisfying >=1.4.1.
Root Cause and Comparison with Other Plugins
Every other avatar plugin in the repo has its version.py set to 1.4.1 (e.g. livekit-plugins/livekit-plugins-bey/livekit/plugins/bey/version.py:15 has __version__ = "1.4.1") and their corresponding optional dependency in livekit-agents/pyproject.toml also requires >=1.4.1. The trugen plugin appears to have been created from an older template without updating the version.
Additionally, the trugen pyproject.toml:25 declares dependencies = ["livekit-agents>=1.3.11"] instead of >=1.4.1 like all other plugins, which is a related inconsistency that could allow installation alongside an incompatible older agents version.
Impact: pip install livekit-agents[trugen] will fail outright. Even manual installation with pip install livekit-plugins-trugen would install a package advertising version 1.3.10, which is inconsistent with the rest of the monorepo's 1.4.1 release cadence.
| __version__ = "1.3.10" | |
| __version__ = "1.4.1" | |
Was this helpful? React with 👍 or 👎 to provide feedback.
3 participants
Authored by: Hari Govind hari@trugen.ai
Summary by CodeRabbit
New Features
Documentation
Chores
✏️ Tip: You can customize this high-level summary in your review settings.