-
Notifications
You must be signed in to change notification settings - Fork 7
Add test coverage assessment prompt #10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| # Test Coverage Assessment | ||
|
|
||
| You are an expert Java developer tasked with analyzing test coverage for this Java SDK. Your goal is to produce a comprehensive report of what is tested and what gaps exist. | ||
|
|
||
| ## Objective | ||
|
|
||
| Analyze the test coverage of the SDK by examining: | ||
| 1. **Event types** - All session events defined in `SessionEventParser` | ||
| 2. **Hook types** - All hooks defined in `SessionHooks` | ||
| 3. **Core functionality** - Session management, tools, permissions, etc. | ||
|
|
||
| ## Assessment Process | ||
|
|
||
| ### Step 1: Identify All Testable Components | ||
|
|
||
| First, examine the source code to identify all components that should be tested: | ||
|
|
||
| ```bash | ||
| # List all event classes | ||
| ls src/main/java/com/github/copilot/sdk/events/ | ||
|
|
||
| # Check the event type mapping in SessionEventParser | ||
| grep -n "TYPE_MAP.put" src/main/java/com/github/copilot/sdk/events/SessionEventParser.java | ||
| ``` | ||
|
|
||
| Extract the list of all registered event types from `SessionEventParser.java`. | ||
|
|
||
| ### Step 2: Identify All Hook Types | ||
|
|
||
| Check `SessionHooks.java` for all available hook handlers: | ||
|
|
||
| ```bash | ||
| grep -E "private.*Handler" src/main/java/com/github/copilot/sdk/json/SessionHooks.java | ||
| ``` | ||
|
|
||
| ### Step 3: Analyze Existing Tests | ||
|
|
||
| Examine the test files to understand current coverage: | ||
|
|
||
| ```bash | ||
| # List all test files | ||
| ls src/test/java/com/github/copilot/sdk/ | ||
|
|
||
| # Check for event-related tests | ||
| grep -r "import.*events\." src/test/java/com/github/copilot/sdk/ | grep -v "\.class" | ||
|
|
||
| # Check for hook tests | ||
| grep -l "SessionHooks\|Hook" src/test/java/com/github/copilot/sdk/*.java | ||
| ``` | ||
|
|
||
| ### Step 4: Categorize Test Coverage | ||
|
|
||
| For each component, determine: | ||
| - **Unit Test Coverage**: Tests that verify JSON parsing/serialization without E2E flow | ||
| - **E2E Test Coverage**: Tests that verify the component works in a real session flow | ||
|
|
||
| ## Report Format | ||
|
|
||
| Generate a comprehensive report in the following format: | ||
|
|
||
| ### Event Types Coverage | ||
|
|
||
| | Event Type | Event Class | Unit Test | E2E Test | Notes | | ||
| |------------|-------------|:---------:|:--------:|-------| | ||
| | `session.start` | `SessionStartEvent` | ✅/❌ | ✅/❌ | Any notes | | ||
| | ... | ... | ... | ... | ... | | ||
|
|
||
| ### Hook Types Coverage | ||
|
|
||
| | Hook Type | Handler Interface | Unit Test | E2E Test | Notes | | ||
| |-----------|-------------------|:---------:|:--------:|-------| | ||
| | `preToolUse` | `PreToolUseHandler` | ✅/❌ | ✅/❌ | Any notes | | ||
| | ... | ... | ... | ... | ... | | ||
|
|
||
| ### Coverage Summary | ||
|
|
||
| | Category | Total | Unit Tested | E2E Tested | Coverage % | | ||
| |----------|-------|-------------|------------|------------| | ||
| | Events | X | X | X | X% | | ||
| | Hooks | X | X | X | X% | | ||
|
|
||
| ### Gaps Identified | ||
|
|
||
| List components that lack tests: | ||
| 1. **Missing Unit Tests**: Components without JSON parsing tests | ||
| 2. **Missing E2E Tests**: Components not exercised in integration tests | ||
| 3. **Partially Tested**: Components with incomplete test coverage | ||
|
|
||
| ### Recommendations | ||
|
|
||
| Prioritized list of tests to add: | ||
| 1. High priority: Critical paths without coverage | ||
| 2. Medium priority: Common use cases without coverage | ||
| 3. Low priority: Edge cases and rare events | ||
|
|
||
| ## Key Files to Examine | ||
|
|
||
| - `src/main/java/com/github/copilot/sdk/events/SessionEventParser.java` - Event type registry | ||
| - `src/main/java/com/github/copilot/sdk/json/SessionHooks.java` - Hook definitions | ||
| - `src/main/java/com/github/copilot/sdk/CopilotSession.java` - Hook handling logic | ||
| - `src/test/java/com/github/copilot/sdk/SessionEventParserTest.java` - Event parsing tests | ||
| - `src/test/java/com/github/copilot/sdk/SessionEventsE2ETest.java` - Event E2E tests | ||
| - `src/test/java/com/github/copilot/sdk/HooksTest.java` - Hook tests | ||
| - `src/test/java/com/github/copilot/sdk/SessionEventHandlingTest.java` - Event handling tests | ||
|
|
||
| ## Verification | ||
|
|
||
| After producing the report, verify by running: | ||
|
|
||
| ```bash | ||
| # Count total tests | ||
| mvn test 2>&1 | grep "Tests run:" | ||
|
|
||
| # Run specific test categories | ||
| mvn test -Dtest=SessionEventParserTest | ||
| mvn test -Dtest=SessionEventsE2ETest | ||
| mvn test -Dtest=HooksTest | ||
| ``` | ||
|
|
||
| ## Output | ||
|
|
||
| Provide: | ||
| 1. The complete coverage report in markdown table format | ||
| 2. A prioritized list of recommended improvements | ||
| 3. Optionally: Skeleton test code for missing high-priority tests | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These Markdown table examples start each row with
||, which creates an extra empty first column and breaks the header separator alignment in most renderers. Use a single leading|for each row so the tables render correctly.This issue also appears in the following locations of the same file: