feat: dyn namespace scoping for trtllm

[biswapanda]

Overview:

Scope trtllm worker to DYN_NAMESPACE for namespace isolation

Related PR: #2475

closes: DYN-838

fixes : https://nvbugspro.nvidia.com/bug/5507011

Supports scoping backend to a specific dynamo namespace based on env variable DYN_NAMESPACE

Details:

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• closes GitHub issue: #xxx

Summary by CodeRabbit

• New Features
  • Endpoints now adapt to a configurable namespace set via environment variables, enabling flexible deployment across different environments and multi-tenant setups without code changes.
  • Defaults remain unchanged if no environment configuration is provided, preserving existing behavior.
  • Improves portability for staging, testing, and production by allowing namespace overrides at runtime.

[coderabbitai]

Walkthrough

Introduces an environment-configurable namespace (DYN_NAMESPACE) and updates three default endpoint constants to interpolate this namespace, falling back to "dynamo" if the environment variable is unset. No other logic or constants are changed.

Changes

Cohort / File(s)	Summary of Changes
Namespace-driven endpoints components/backends/trtllm/src/dynamo/trtllm/utils/trtllm_utils.py	Added import os and global DYN_NAMESPACE = os.environ.get("DYN_NAMESPACE", "dynamo"). Updated DEFAULT_ENDPOINT, DEFAULT_NEXT_ENDPOINT, and DEFAULT_ENCODE_ENDPOINT to f"dyn://{DYN_NAMESPACE}.…". All other constants unchanged.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Env as Environment
  participant Utils as trtllm_utils.py
  participant Client as Caller

  Env->>Utils: DYN_NAMESPACE (optional)
  Note over Utils: On import
  Utils->>Utils: DYN_NAMESPACE = os.getenv("DYN_NAMESPACE", "dynamo")
  Utils->>Utils: DEFAULT_ENDPOINT = dyn://{DYN_NAMESPACE}.tensorrt_llm.generate
  Utils->>Utils: DEFAULT_NEXT_ENDPOINT = dyn://{DYN_NAMESPACE}.tensorrt_llm_next.generate
  Utils->>Utils: DEFAULT_ENCODE_ENDPOINT = dyn://{DYN_NAMESPACE}.tensorrt_llm_encode.generate

  Client->>Utils: Import constants
  Utils-->>Client: Resolved endpoints (namespace-aware)

Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

In burrows of code where endpoints dwell,
I nudge a namespace—now it’s set by shell.
From dyn:// paths I twitch my nose,
Env-var winds decide where it goes.
Carrot-quick swap, defaults still fine—
Hop, hop, deploy—aligned by design. 🥕🐇

Tip

👮 Agentic pre-merge checks are now available in preview!

Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.

• Built-in checks – Quickly apply ready-made checks to enforce title conventions, require pull request descriptions that follow templates, validate linked issues for compliance, and more.
• Custom agentic checks – Define your own rules using CodeRabbit’s advanced agentic capabilities to enforce organization-specific policies and workflows. For example, you can instruct CodeRabbit’s agent to verify that API documentation is updated whenever API schema files are modified in a PR. Note: Upto 5 custom checks are currently allowed during the preview period. Pricing for this feature will be announced in a few weeks.

Example:

reviews:
  pre_merge_checks:
    custom_checks:
      - name: "Undocumented Breaking Changes"
        mode: "warning"
        instructions: |
          Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).

Please share your feedback with us on this Discord post.

Pre-merge checks (2 passed, 1 warning)

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description Check	⚠️ Warning	The pull request description includes the required section headings but the Details and Where should the reviewer start sections are left empty and the Related Issues section still contains a placeholder rather than actual issue references. The Overview is concise but does not follow through into the Details section, and the closes/fixes lines are not correctly placed under Related Issues. As a result, the description does not meet the template requirements for completeness and clarity.	Please populate the Details section with a description of the specific code changes, list the key files or components for review in the Where should the reviewer start section, and update the Related Issues section with the real issue numbers using the correct action keywords (e.g., “closes #123”).

✅ Passed checks (2 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.
Title Check	✅ Passed	The title “feat: dyn namespace scoping for trtllm” accurately and concisely captures the primary change of adding dynamic namespace scoping for the TRTLLM component and is directly related to the modifications described in the changeset.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

components/backends/trtllm/src/dynamo/trtllm/utils/trtllm_utils.py (1)

20-23: Deduplicate endpoint construction in trtllm_utils
Replace the three hard-coded DEFAULT_* constants in components/backends/trtllm/src/dynamo/trtllm/utils/trtllm_utils.py with the _build_endpoint helper. Similar hard-coded dyn://… defaults still exist in examples, docs, and launch scripts—consider extending this helper globally for consistency.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8c665c1 and 13992c8.

📒 Files selected for processing (1)

• components/backends/trtllm/src/dynamo/trtllm/utils/trtllm_utils.py (2 hunks)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: Build and Test - dynamo

🔇 Additional comments (1)

components/backends/trtllm/src/dynamo/trtllm/utils/trtllm_utils.py (1)

5-5: Import for env-based config — OK

No issues adding os; needed for env-driven defaults.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[saturley-hall]

/ok to test 680254b