remove dependency of API agents: inline::meta-reference

[raptorsun]

Description

remove unused dependency from API agents: inline::meta-reference

RAG and MCP server functionalities are not affected.

Type of change

• Refactor
• New feature
• Bug fix
• CVE fix
• Optimization
• Documentation Update
• Configuration Update
• Bump-up service version
• Bump-up dependent library
• Bump-up library or tool used for development (does not change the final image)
• CI configuration change
• Konflux configuration change
• Unit tests improvement
• Integration tests improvement
• End to end tests improvement

Tools used to create PR

Identify any AI code assistants used in this PR (for transparency and review context)

• Assisted-by: N/A
• Generated by: N/A

Related Tickets & Documents

• Related Issue #
• Closes #

Checklist before requesting a review

• I have performed a self-review of my code.
• PR has passed all pre-merge test jobs.
• If it is a core feature, I have added thorough tests.

Testing

• Please provide detailed steps to perform tests related to this code change.
• How were the fix/results from this change verified? Please provide relevant screenshots or results.

Summary by CodeRabbit

• Chores
  • Updated project dependencies in configuration and lock files
  • Removed matplotlib, pandas, scikit-learn, and pillow from active dependencies
  • Added new packages for natural language processing and machine learning, including sentence transformers, language detection, NLTK, and neural network libraries
  • Streamlined transitive dependency declarations

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

This pull request updates the project's dependency configuration by commenting out five packages in pyproject.toml's llslibdev group, introducing a new expanded set of runtime and tooling dependencies, and removing numerous pinned transitive dependencies and their associated hashes from platform-specific requirements files.

Changes

Cohort / File(s)	Summary
Project configuration pyproject.toml	Comments out five unused packages (matplotlib, pillow, pandas, scikit-learn, psycopg2-binary) in llslibdev group; adds comprehensive new dependency block including tree_sitter, pythainlp, langdetect, emoji, nltk, sentence-transformers, faiss-cpu, requests, aiosqlite, datasets, opentelemetry components, transformers, numpy, mcp, torch, trl, peft, autoevals, fire, blobfile, psutil, and others.
Platform-specific lock files requirements.aarch64.txt, requirements.x86_64.txt	Removes numerous pinned transitive dependency entries and their associated hash blocks, including contourpy, cycler, matplotlib, kiwisolver, and lightspeed-stack references; simplifies or eliminates "via" dependency chain annotations; reduces overall declared transitive dependency graph.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~30 minutes

Possibly related PRs

• #938: Updates to platform-specific dependency manifests (requirements.aarch64.txt and requirements.x86_64.txt) with similar scope to this PR.
• #852: Modifies pyproject.toml and platform-specific requirements files with torch-related dependency reorganization and pinning.
• #292: Updates llslibdev dependency group in pyproject.toml and adjusts CI/build steps for dependency synchronization.

Suggested reviewers

• tisnik
• radofuchs

Pre-merge checks

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title 'remove dependency of API agents: inline::meta-reference' is vague and does not clearly describe the actual changes, which involve commenting out five packages in pyproject.toml and removing numerous dependencies from requirements files.	Revise the title to clearly describe the main changes, such as 'Clean up dependencies: comment out unused packages and update requirements' or 'Consolidate dependencies: replace matplotlib and related packages with new tooling stack'.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

pyproject.toml (1)

132-137: Clarify the comment: these packages are not used by inline::meta-reference agents, not unused in the project

The commented-out dependencies at lines 132–137 are safe to remove from llslibdev: matplotlib, pillow, pandas, and scikit-learn have no direct imports in the codebase, and psycopg2-binary is correctly retained as a top-level dependency since it's actively used in src/utils/quota.py, src/quota/quota_limiter.py, src/quota/connect_pg.py, src/cache/postgres_cache.py, and src/quota/token_usage_history.py.

However, the comment "not used in the project" is misleading. Since psycopg2-binary is directly imported and required by llama-stack providers, clarify the reason for commenting them out:

-    # commented out because they are not used in the project
+    # commented out because inline::meta-reference agents are no longer used

This avoids confusion about whether the packages are still required elsewhere (which they are, transitively or directly).

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b24cdaf and fcdf6fe.

⛔ Files ignored due to path filters (1)

• uv.lock is excluded by !**/*.lock

📒 Files selected for processing (3)

• pyproject.toml
• requirements.aarch64.txt
• requirements.x86_64.txt

🧰 Additional context used

📓 Path-based instructions (1)

pyproject.toml

📄 CodeRabbit inference engine (CLAUDE.md)

pyproject.toml: Configure pylint with source-roots = "src"
Exclude src/auth/k8s.py from pyright type checking

Files:

• pyproject.toml

🧠 Learnings (3)

📚 Learning: 2025-11-24T16:58:04.410Z

Learnt from: CR
Repo: lightspeed-core/lightspeed-stack PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-24T16:58:04.410Z
Learning: Always check `pyproject.toml` for existing dependencies and versions before adding new ones

Applied to files:

• pyproject.toml

📚 Learning: 2025-08-18T10:58:14.951Z

Learnt from: matysek
Repo: lightspeed-core/lightspeed-stack PR: 292
File: pyproject.toml:47-47
Timestamp: 2025-08-18T10:58:14.951Z
Learning: psycopg2-binary is required by some llama-stack providers in the lightspeed-stack project, so it cannot be replaced with psycopg v3 or moved to optional dependencies without breaking llama-stack functionality.

Applied to files:

• pyproject.toml
• requirements.x86_64.txt
• requirements.aarch64.txt

📚 Learning: 2025-08-18T10:57:39.266Z

Learnt from: matysek
Repo: lightspeed-core/lightspeed-stack PR: 292
File: pyproject.toml:59-59
Timestamp: 2025-08-18T10:57:39.266Z
Learning: In the lightspeed-stack project, transitive dependencies like faiss-cpu are intentionally pinned as top-level dependencies to maintain better control over the dependency graph and avoid version conflicts when bundling ML/LLM tooling packages.

Applied to files:

• requirements.x86_64.txt
• requirements.aarch64.txt

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)

• GitHub Check: build-pr
• GitHub Check: Konflux kflux-prd-rh02 / lightspeed-stack-on-pull-request
• GitHub Check: E2E: server mode / vertexai
• GitHub Check: E2E: library mode / azure
• GitHub Check: E2E: server mode / ci
• GitHub Check: E2E: server mode / azure
• GitHub Check: E2E: library mode / ci
• GitHub Check: E2E: library mode / vertexai

🔇 Additional comments (4)

requirements.x86_64.txt (1)

1902-1902: uv “via” annotations correctly reflect the updated dependency graph

The new # via ... comments for pandas, psycopg2-binary, and scikit-learn line up with the current pyproject.toml setup (direct vs transitive sources), so the regenerate-from-uv flow looks consistent and there’s nothing to fix in this file for these changes. Please just ensure this file was produced by uv pip compile after the pyproject changes, not hand-edited.

Also applies to: 2208-2208, 2921-2921

requirements.aarch64.txt (3)

1898-1904: Pandas still present transitively (now attributed to llama-stack)

This change just updates the “via” metadata; pandas remains in the lock as a transitive dep (now correctly attributed to llama-stack). No functional impact, but be aware this environment still carries pandas even though it’s no longer top‑level in llslibdev.

If your intention was to fully drop pandas from this profile rather than just from pyproject.toml, please confirm that no remaining deps (e.g., llama-stack) require it, or consider regenerating constraints without it.

---

2204-2210: psycopg2-binary attribution matches project constraints

The updated “via” annotation for psycopg2-binary now points at lightspeed-stack (pyproject.toml), which aligns with the requirement to keep this package for some llama-stack providers (and not move it to optional deps). No issues here.

Given this is a critical DB driver, please double‑check that pyproject.toml still keeps psycopg2-binary in a non‑optional dependency group used by all relevant providers. Based on learnings, this is important for llama-stack compatibility.

---

2916-2922: scikit-learn remains as a transitive dep via sentence-transformers

Although scikit-learn was removed from the llslibdev group in pyproject.toml, this lock file still pins it because sentence-transformers depends on it (as reflected by the new “via” comment). This is consistent with pinning heavy transitives for control over the graph, but it means the runtime still includes scikit-learn.

If the goal of the PR is to slim the API agents’ dependency surface, consider whether keeping sentence-transformers (and thus scikit-learn) in this env is desired, or if those usages can be isolated to another group. Based on learnings, transitive pins are intentional, but worth reconfirming here.

[tisnik]

LGTM

[tisnik]

ah this one is leftover, as we already need psycopg2-binary for other part of lcore

[raptorsun]

yup, psycopg2 is already in the main dependency list, this removal has no effect.

[raptorsun]

/retest

[radofuchs]

LGTM