Skip to content

Commit 8c82979

Browse files
waynesun09waynesun09
committed
feat: add temporal query parameters to MCP tools
Add comprehensive temporal filtering capabilities with new search_commits tool, enhanced list_repos activity filtering, and improved search_code temporal support. **New Features:** - Add search_commits tool for Git commit history search with time range, author, and query filtering - Add dateUtils.ts with comprehensive date parsing supporting ISO 8601 and relative formats ("30 days ago", "last week", "yesterday") - Add date range validation to prevent invalid ranges (since > until) - Add activeAfter/activeBefore parameters to list_repos for commit activity filtering - Add gitRevision, since/until, and includeDeletedFiles parameters to search_code - Add 30-second timeout for git operations to handle large repositories - Add detailed error messages for common git failures - Move cache directory constants to @sourcebot/shared package **Fixes:** - Fix getRepos() pagination bug where take limit was applied before activity filtering, returning fewer results than requested **Testing & Documentation:** - Add comprehensive test coverage: 106 tests (59 dateUtils, 24 gitApi, 23 searchApi) - Clarify temporal filtering semantics: search_code filters by INDEX time (when Sourcebot indexed), while list_repos and search_commits filter by COMMIT time - Clarify that repositories are cloned on Sourcebot server disk, not user's local disk, and cloning process may not be finished when search_commits is called - Update MCP tool descriptions with temporal parameter examples and date format documentation - Add "Date Format Examples" section to README - Update CHANGELOG with all improvements All changes are backward compatible with optional parameters. Resolves #511 Signed-off-by: Wayne Sun <gsun@redhat.com>
1 parent 09507d3 commit 8c82979

File tree

18 files changed

+1932
-44
lines changed

18 files changed

+1932
-44
lines changed

‎packages/backend/src/constants.ts‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
import { CodeHostType } from "@sourcebot/db";
2-
import { env } from "@sourcebot/shared";
3-
import path from "path";
2+
43

54
export const SINGLE_TENANT_ORG_ID = 1;
65

@@ -9,8 +8,7 @@ export const PERMISSION_SYNC_SUPPORTED_CODE_HOST_TYPES: CodeHostType[] = [
98
'gitlab',
109
];
1110

12-
export const REPOS_CACHE_DIR = path.join(env.DATA_CACHE_DIR, 'repos');
13-
export const INDEX_CACHE_DIR = path.join(env.DATA_CACHE_DIR, 'index');
11+
export { REPOS_CACHE_DIR, INDEX_CACHE_DIR } from "@sourcebot/shared";
1412

1513
// Maximum time to wait for current job to finish
1614
export const GROUPMQ_WORKER_STOP_GRACEFUL_TIMEOUT_MS = 5 * 1000; // 5 seconds

‎packages/mcp/CHANGELOG.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Added
11+
- Added comprehensive relative date support for all temporal parameters (e.g., "30 days ago", "last week", "yesterday")
12+
- Added `search_commits` tool to search commits by actual commit time with full temporal filtering
13+
- Added `since`/`until` parameters to `search_code` (filters by index time - when Sourcebot indexed the repo)
14+
- Added `gitRevision`, and `includeDeletedFiles` parameters to `search_code`
15+
- Added `activeAfter`/`activeBefore` parameters to `list_repos` (filters by commit time - actual git commit dates)
16+
- Added date range validation to prevent invalid date ranges (since > until)
17+
- Added 30-second timeout for git operations to handle large repositories
18+
- Added enhanced error messages for git operations (timeout, repository not found, invalid git repository, ambiguous arguments)
19+
- Added clarification that repositories must be cloned on Sourcebot server disk for `search_commits` to work
20+
- Added comprehensive temporal parameter documentation to README with clear distinction between index time and commit time filtering
21+
- Added comprehensive unit tests for date parsing utilities (90+ test cases)
22+
- Added unit tests for git commit search functionality with mocking
23+
- Added integration tests for temporal parameter validation
24+
25+
### Changed
26+
- Enhanced `list_repos` tool with better pagination feedback messages
27+
28+
### Fixed
29+
- Fixed `list_repos` pagination bug where `take` limit was applied before activity filtering, returning fewer results than requested
30+
1031
## [1.0.9] - 2025-11-17
1132

1233
### Added

‎packages/mcp/README.md‎

Lines changed: 48 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -166,6 +166,8 @@ For a more detailed guide, checkout [the docs](https://docs.sourcebot.dev/docs/f
166166

167167
Fetches code that matches the provided regex pattern in `query`.
168168

169+
**Temporal Filtering**: Use `since` and `until` to filter by repository index time (when Sourcebot last indexed the repo). This is different from commit time. See `search_commits` for commit-time filtering.
170+
169171
<details>
170172
<summary>Parameters</summary>
171173

@@ -176,6 +178,10 @@ Fetches code that matches the provided regex pattern in `query`.
176178
| `filterByLanguages` | no | Restrict search to specific languages (GitHub linguist format, e.g., Python, JavaScript). |
177179
| `caseSensitive` | no | Case sensitive search (default: false). |
178180
| `includeCodeSnippets` | no | Include code snippets in results (default: false). |
181+
| `gitRevision` | no | Git revision to search (e.g., 'main', 'develop', 'v1.0.0'). Defaults to HEAD. |
182+
| `since` | no | Only search repos indexed after this date. Supports ISO 8601 or relative (e.g., "30 days ago"). |
183+
| `until` | no | Only search repos indexed before this date. Supports ISO 8601 or relative (e.g., "yesterday"). |
184+
| `includeDeletedFiles` | no | Include deleted files in search results (default: false). |
179185
| `maxTokens` | no | Max tokens to return (default: env.DEFAULT_MINIMUM_TOKENS). |
180186
</details>
181187

@@ -184,14 +190,18 @@ Fetches code that matches the provided regex pattern in `query`.
184190

185191
Lists repositories indexed by Sourcebot with optional filtering and pagination.
186192

193+
**Temporal Filtering**: Use `activeAfter` and `activeBefore` to filter repositories by actual commit activity (requires repositories to be cloned locally).
194+
187195
<details>
188196
<summary>Parameters</summary>
189197

190-
| Name | Required | Description |
191-
|:-------------|:---------|:--------------------------------------------------------------------|
192-
| `query` | no | Filter repositories by name (case-insensitive). |
193-
| `pageNumber` | no | Page number (1-indexed, default: 1). |
194-
| `limit` | no | Number of repositories per page (default: 50). |
198+
| Name | Required | Description |
199+
|:----------------|:---------|:-----------------------------------------------------------------------------------------------|
200+
| `query` | no | Filter repositories by name (case-insensitive). |
201+
| `pageNumber` | no | Page number (1-indexed, default: 1). |
202+
| `limit` | no | Number of repositories per page (default: 50). |
203+
| `activeAfter` | no | Only return repos with commits after this date. Supports ISO 8601 or relative (e.g., "30 days ago"). |
204+
| `activeBefore` | no | Only return repos with commits before this date. Supports ISO 8601 or relative (e.g., "yesterday"). |
195205

196206
</details>
197207

@@ -208,6 +218,39 @@ Fetches the source code for a given file.
208218
| `repoId` | yes | The Sourcebot repository ID. |
209219
</details>
210220

221+
### search_commits
222+
223+
Searches for commits in a specific repository based on actual commit time (NOT index time).
224+
225+
**Requirements**: Repository must be cloned on the Sourcebot server disk. Sourcebot automatically clones repositories during indexing, but the cloning process may not be finished when this query is executed. Use `list_repos` first to get the repository ID.
226+
227+
**Date Formats**: Supports ISO 8601 dates (e.g., "2024-01-01") and relative formats (e.g., "30 days ago", "last week", "yesterday").
228+
229+
<details>
230+
<summary>Parameters</summary>
231+
232+
| Name | Required | Description |
233+
|:-----------|:---------|:-----------------------------------------------------------------------------------------------|
234+
| `repoId` | yes | Repository ID (obtain from `list_repos`). |
235+
| `query` | no | Search query to filter commits by message (case-insensitive). |
236+
| `since` | no | Show commits after this date (by commit time). Supports ISO 8601 or relative formats. |
237+
| `until` | no | Show commits before this date (by commit time). Supports ISO 8601 or relative formats. |
238+
| `author` | no | Filter by author name or email (supports partial matches). |
239+
| `maxCount` | no | Maximum number of commits to return (default: 50). |
240+
241+
</details>
242+
243+
## Date Format Examples
244+
245+
All temporal parameters support:
246+
- **ISO 8601**: `"2024-01-01"`, `"2024-12-31T23:59:59Z"`
247+
- **Relative dates**: `"30 days ago"`, `"1 week ago"`, `"last month"`, `"yesterday"`
248+
249+
**Important**: `search_code` and `list_repos` temporal filters work differently:
250+
- `search_code` `since`/`until`: Filters by **index time** (when Sourcebot indexed the repo)
251+
- `list_repos` `activeAfter`/`activeBefore`: Filters by **commit time** (actual git commit dates)
252+
- `search_commits` `since`/`until`: Filters by **commit time** (actual git commit dates)
253+
211254

212255
## Supported Code Hosts
213256
Sourcebot supports the following code hosts:

‎packages/mcp/src/client.ts‎

Lines changed: 30 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import { env } from './env.js';
2-
import { listRepositoriesResponseSchema, searchResponseSchema, fileSourceResponseSchema } from './schemas.js';
3-
import { FileSourceRequest, FileSourceResponse, ListRepositoriesResponse, SearchRequest, SearchResponse, ServiceError } from './types.js';
2+
import { listRepositoriesResponseSchema, searchResponseSchema, fileSourceResponseSchema, searchCommitsResponseSchema } from './schemas.js';
3+
import { FileSourceRequest, FileSourceResponse, ListRepositoriesResponse, SearchRequest, SearchResponse, ServiceError, SearchCommitsRequest, SearchCommitsResponse } from './types.js';
44
import { isServiceError } from './utils.js';
55

66
export const search = async (request: SearchRequest): Promise<SearchResponse | ServiceError> => {
@@ -21,8 +21,16 @@ export const search = async (request: SearchRequest): Promise<SearchResponse | S
2121
return searchResponseSchema.parse(result);
2222
}
2323

24-
export const listRepos = async (): Promise<ListRepositoriesResponse | ServiceError> => {
25-
const result = await fetch(`${env.SOURCEBOT_HOST}/api/repos`, {
24+
export const listRepos = async (params?: { activeAfter?: string, activeBefore?: string }): Promise<ListRepositoriesResponse | ServiceError> => {
25+
const url = new URL(`${env.SOURCEBOT_HOST}/api/repos`);
26+
if (params?.activeAfter) {
27+
url.searchParams.append('activeAfter', params.activeAfter);
28+
}
29+
if (params?.activeBefore) {
30+
url.searchParams.append('activeBefore', params.activeBefore);
31+
}
32+
33+
const result = await fetch(url.toString(), {
2634
method: 'GET',
2735
headers: {
2836
'Content-Type': 'application/json',
@@ -55,3 +63,21 @@ export const getFileSource = async (request: FileSourceRequest): Promise<FileSou
5563

5664
return fileSourceResponseSchema.parse(result);
5765
}
66+
67+
export const searchCommits = async (request: SearchCommitsRequest): Promise<SearchCommitsResponse | ServiceError> => {
68+
const result = await fetch(`${env.SOURCEBOT_HOST}/api/commits`, {
69+
method: 'POST',
70+
headers: {
71+
'Content-Type': 'application/json',
72+
'X-Org-Domain': '~',
73+
...(env.SOURCEBOT_API_KEY ? { 'X-Sourcebot-Api-Key': env.SOURCEBOT_API_KEY } : {})
74+
},
75+
body: JSON.stringify(request)
76+
}).then(response => response.json());
77+
78+
if (isServiceError(result)) {
79+
return result;
80+
}
81+
82+
return searchCommitsResponseSchema.parse(result);
83+
}

0 commit comments

Comments
 (0)