Skip to content

perf: use module-scoped fixture for RAG tests#337

Merged
nrfulton merged 3 commits intogenerative-computing:mainfrom
planetf1:perf/rag-module-scope
Jan 23, 2026
Merged

perf: use module-scoped fixture for RAG tests#337
nrfulton merged 3 commits intogenerative-computing:mainfrom
planetf1:perf/rag-module-scope

Conversation

@planetf1
Copy link
Contributor

@planetf1 planetf1 commented Jan 22, 2026

Optimize RAG Test Performance

Type of PR

  • Bug Fix
  • New Feature
  • Documentation
  • Other (Performance Optimization)

Description

This PR optimizes the RAG intrinsic test suite by changing the backend fixture from function-scoped to module-scoped. The backend fixture loads a 3B parameter model (granite-4.0-micro) which was being reloaded for each of the 7 tests in the file, causing significant overhead (a big proportion of the overal tests runnable on this system as per #326)

Fixes #336

Performance Impact:

  • Before: 17 minutes
  • After: 3.5 minutes
  • Improvement: ~5x faster

Safety Justification:

  1. Tests remain isolated: All 7 tests perform read-only operations on the backend
  2. No state mutation: Each test uses different intrinsics (different adapters)
  3. Cleanup preserved: Teardown code still executes once at module end
  4. Assertions unchanged: All test logic and expected outputs remain identical
  5. Verified: All 7 tests pass with same results

The change is a single line addition (scope="module") to the fixture decorator, so should be a low-risk optimization with significant benefits.

Testing

  • Tests added to the respective file if code was changed (N/A - no new tests needed)
  • New code has 100% coverage if code was added (N/A - only fixture scope changed)
  • Ensure existing tests and github automation passes
    • All 7 RAG tests pass: uv run pytest test/stdlib/components/intrinsic/test_rag.py -v
    • Pre-commit hooks pass (ruff, mypy, codespell)
    • Test execution verified:

NOTE: Testing was done on a base of #326 , but have created a clean delta here - will rebase/retest once that merged

@github-actions
Copy link
Contributor

The PR description has been updated. Please fill out the template for your PR to be reviewed.

@mergify
Copy link

mergify bot commented Jan 22, 2026

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert|release)(?:\(.+\))?:

@planetf1
Copy link
Contributor Author

Rebased and retested with the pytest markers fix (now in main)

  • overall time down from ~20 mins to ~6 mins - some tests still a bit of a stretch for 32GB, but getting better.
=================== 218 passed, 59 skipped, 2 xpassed, 104 warnings in 367.86s (0:06:07) ====================

for full run.

@planetf1 planetf1 force-pushed the perf/rag-module-scope branch from 6ae3b5a to 219b5d7 Compare January 22, 2026 18:47
@planetf1 planetf1 marked this pull request as ready for review January 22, 2026 18:48
@nrfulton nrfulton self-requested a review January 23, 2026 14:58
@nrfulton nrfulton merged commit d72ecfd into generative-computing:main Jan 23, 2026
1 check was pending
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tests: speed up RAG tests

3 participants

Comments