diff --git a/.github/ISSUE_TEMPLATE/bug-report.yml b/.github/ISSUE_TEMPLATE/bug-report.yml index 4c7b3111c4c6..f6553db6b794 100644 --- a/.github/ISSUE_TEMPLATE/bug-report.yml +++ b/.github/ISSUE_TEMPLATE/bug-report.yml @@ -42,6 +42,7 @@ body: label: Version with bug description: In what version do you see this issue? Run `dotnet workload list` to find your version. options: + - 10.0.11 - 10.0.10 - 10.0.0 - 10.0.0-rc.2 @@ -151,6 +152,7 @@ body: - 10.0.0-rc.2 - 10.0.0 - 10.0.10 + - 10.0.11 validations: required: true - type: dropdown diff --git a/.github/README-AI.md b/.github/README-AI.md index 637bc29b376b..fe2b7aea00ad 100644 --- a/.github/README-AI.md +++ b/.github/README-AI.md @@ -4,26 +4,46 @@ This folder contains instructions and configurations for AI coding assistants wo ## Available Agents -### PR Reviewer Agent -The PR reviewer agent conducts thorough, constructive code reviews of .NET MAUI pull requests with hands-on testing and validation. +### Sandbox Agent +The sandbox agent is your general-purpose tool for working with the .NET MAUI Sandbox app. Use it for manual testing, PR validation, issue reproduction, and experimentation with MAUI features. + +### UI Test Coding Agent +The UI test coding agent writes and runs automated UI tests following .NET MAUI conventions. Use it for creating new test coverage, running existing tests from PRs, and validating UI test correctness. ### Issue Resolver Agent The issue resolver agent investigates, reproduces, and fixes reported issues in the .NET MAUI repository with comprehensive testing and validation. +### PR Reviewer Agent (Inline) +The PR reviewer agent conducts thorough, constructive code reviews of .NET MAUI pull requests with hands-on testing and validation. This agent uses inline instructions rather than a separate file. + ## How to Use ### Option 1: GitHub Copilot CLI (Local) -**PR Reviewer Agent:** +**Sandbox Agent:** ```bash # Start GitHub Copilot CLI with agent support copilot --allow-all-tools --allow-all-paths -# Invoke the pr-reviewer agent -/agent pr-reviewer +# Invoke the sandbox-agent +/agent sandbox-agent -# Request a review -please review https://github.com/dotnet/maui/pull/XXXXX +# Test a PR or reproduce an issue +please test PR #32479 +please reproduce issue #12345 +``` + +**UI Test Coding Agent:** +```bash +# Start GitHub Copilot CLI with agent support +copilot --allow-all-tools --allow-all-paths + +# Invoke the uitest-coding-agent +/agent uitest-coding-agent + +# Write or run UI tests +please write UI tests for issue #12345 +please run the UI tests from PR #32479 ``` **Issue Resolver Agent:** @@ -38,6 +58,18 @@ copilot --allow-all-tools --allow-all-paths please investigate and fix https://github.com/dotnet/maui/issues/XXXXX ``` +**PR Reviewer Agent:** +```bash +# Start GitHub Copilot CLI with agent support +copilot --allow-all-tools --allow-all-paths + +# Invoke the pr-reviewer agent +/agent pr-reviewer + +# Request a review +please review https://github.com/dotnet/maui/pull/XXXXX +``` + ### Option 2: GitHub Copilot Agents (Web) 1. **Navigate to the agents tab** at https://github.com/copilot/agents @@ -45,12 +77,16 @@ please investigate and fix https://github.com/dotnet/maui/issues/XXXXX 2. **Select your repository and branch** using the dropdown menus in the text box 3. **Choose your agent** from the dropdown: - - `pr-reviewer` for PR reviews + - `sandbox-agent` for manual testing and experimentation + - `uitest-coding-agent` for writing and running UI tests - `issue-resolver` for investigating and fixing issues + - `pr-reviewer` for PR reviews 4. **Enter a task** in the text box: - - For PR reviews: `Please review this PR: https://github.com/dotnet/maui/pull/XXXXX` + - For sandbox testing: `Please test PR #32479` + - For UI tests: `Please write UI tests for issue #12345` - For issue resolution: `Please investigate and fix: https://github.com/dotnet/maui/issues/XXXXX` + - For PR reviews: `Please review this PR: https://github.com/dotnet/maui/pull/XXXXX` 5. **Click Start task** or press Return @@ -58,20 +94,29 @@ please investigate and fix https://github.com/dotnet/maui/issues/XXXXX ## What the Agents Do -### PR Reviewer Agent +### Sandbox Agent -Every PR review includes: +Your go-to agent for hands-on work with the Sandbox app: -1. **Code Analysis** - Reviews code for correctness, style, and best practices -2. **Build & Deploy** - Builds the Sandbox app and deploys to simulator/emulator -3. **Real Testing** - Tests PR changes on actual devices with measurements -4. **Before/After Comparison** - Compares behavior with and without PR changes -5. **Edge Case Testing** - Tests scenarios not mentioned by the PR author -6. **Documented Results** - Provides review with actual test data and evidence +1. **PR Testing** - Manually validates PRs by running them in the Sandbox app +2. **Issue Reproduction** - Creates minimal reproduction cases for reported issues +3. **Feature Experimentation** - Try out MAUI features in a clean environment +4. **Quick Validation** - Fast manual testing without writing automated tests +5. **Automated Logging** - Uses `BuildAndRunSandbox.ps1` to capture all logs to `CustomAgentLogsTmp/Sandbox/` + +### UI Test Coding Agent + +Automated testing specialist for the .NET MAUI test suite: + +1. **Test Creation** - Writes new UI tests following .NET MAUI conventions +2. **Test Execution** - Runs existing UI tests from PRs or repository +3. **Test Validation** - Verifies tests are correct and follow best practices +4. **Cross-Platform** - Tests on iOS, Android, Windows, and MacCatalyst +5. **Automated Workflow** - Uses `BuildAndRunHostApp.ps1` to handle building, deployment, and logging to `CustomAgentLogsTmp/UITests/` ### Issue Resolver Agent -Every issue resolution includes: +Comprehensive issue investigation and resolution: 1. **Issue Investigation** - Analyzes the reported issue and gathers context 2. **Reproduction** - Creates minimal reproduction case in Sandbox app @@ -81,6 +126,17 @@ Every issue resolution includes: 6. **UI Test Creation** - Adds automated UI test to prevent regression 7. **Documentation** - Provides detailed explanation of the issue and fix +### PR Reviewer Agent + +Thorough PR review with hands-on testing: + +1. **Code Analysis** - Reviews code for correctness, style, and best practices +2. **Build & Deploy** - Builds the Sandbox app and deploys to simulator/emulator +3. **Real Testing** - Tests PR changes on actual devices with measurements +4. **Before/After Comparison** - Compares behavior with and without PR changes +5. **Edge Case Testing** - Tests scenarios not mentioned by the PR author +6. **Documented Results** - Provides review with actual test data and evidence + ### When Agents Pause Both agents will pause and ask for help if they encounter: @@ -126,8 +182,11 @@ Agents work with **time budgets as estimates for planning**, not hard deadlines: ## File Structure ### Agent Definitions -- **`agents/pr-reviewer.md`** - PR reviewer agent entry point (72 lines) -- **`agents/issue-resolver.md`** - Issue resolver agent entry point (77 lines) +- **`agents/sandbox-agent.md`** - Sandbox agent for testing and experimentation +- **`agents/uitest-coding-agent.md`** - UI test agent for writing and running tests +- **`agents/issue-resolver.md`** - Issue resolver agent for investigating and fixing issues +- **`agents/pr-reviewer.md`** - PR reviewer agent (inline instructions) +- **`agents/README.md`** - Agent selection guide and quick reference ### Agent Instruction Packages @@ -153,23 +212,52 @@ Each agent has a progressive disclosure structure for optimal learning: - `issue-resolver-agent/platforms/` - Platform-specific guides - `issue-resolver-agent/output-format.md` - PR description formatting -### Shared Instruction Files (4 files, 2,224 lines) +### Shared Instruction Files -These provide specialized guidance for specific scenarios used by both agents: +These provide specialized guidance for specific scenarios used by all agents: - **`instructions/common-testing-patterns.md`** - Command sequences (UDID extraction, builds, deploys, error checking) - **`instructions/uitests.instructions.md`** - UI testing guidelines (when to use HostApp vs Sandbox) -- **`instructions/safearea-testing.instructions.md`** - SafeArea testing patterns (measure children, not parents) -- **`instructions/instrumentation.instructions.md`** - Code instrumentation for debugging and testing -- **`instructions/appium-control.instructions.md`** - Standalone Appium scripts for manual debugging +- **`instructions/safearea-testing.md`** - SafeArea testing patterns (measure children, not parents) +- **`instructions/instrumentation.md`** - Code instrumentation for debugging and testing +- **`instructions/appium-control.md`** - Standalone Appium scripts for manual debugging - **`instructions/templates.instructions.md`** - Template modification rules +- **`instructions/sandbox-testing-patterns.md`** - Sandbox app testing patterns +- **`instructions/edge-case-testing.md`** - Edge case testing strategies + +### Shared Scripts -### Recent Improvements (Phase 1 - November 2025) +Automated PowerShell scripts for testing workflows: -**PR Reviewer Agent enhancements:** +- **`scripts/BuildAndRunSandbox.ps1`** - Build, deploy, and test Sandbox app (Android/iOS) +- **`scripts/BuildAndRunHostApp.ps1`** - Build, deploy, and run UI tests (Android/iOS) +- **`scripts/shared/Start-Emulator.ps1`** - Detect and start Android emulators or iOS simulators +- **`scripts/shared/Build-AndDeploy.ps1`** - Build and deploy apps to devices +- **`scripts/shared/shared-utils.ps1`** - Common output formatting functions +- **`scripts/templates/RunWithAppiumTest.template.cs`** - Template for Appium test scripts + +### Log Directories + +All agent logs are consolidated under `CustomAgentLogsTmp/`: + +- **`CustomAgentLogsTmp/Sandbox/`** - Sandbox agent logs (appium.log, android-device.log, ios-device.log, RunWithAppiumTest.cs) +- **`CustomAgentLogsTmp/UITests/`** - UI test agent logs (appium.log, android-device.log, ios-device.log, test-output.log) + +### Recent Improvements (Phase 2 - November 2025) + +**Infrastructure Consolidation:** +1. **Unified Log Structure** - All logs now under `CustomAgentLogsTmp/` with subdirectories for Sandbox and UITests +2. **Shared Script Library** - Created reusable PowerShell scripts for device startup, build, and deployment +3. **Agent Simplification** - Consolidated `uitest-pr-validator` into `uitest-coding-agent` for clarity +4. **Agent Rename** - `sandbox-pr-tester` → `sandbox-agent` to reflect broader purpose (testing, validation, experimentation) +5. **Automated Testing Scripts** - All agents now use PowerShell scripts instead of manual commands +6. **noReset Capability Added** - Android Appium tests now include `noReset: true` to prevent app reinstalls +7. **Complete Link Validation** - All 53 markdown files validated and updated with correct paths + +**Phase 1 Improvements (November 2025):** 1. **Mandatory pre-work moved to top** - Critical requirements now at line 6 instead of line 43 2. **Reading order & stopping points** - Explicit "STOP after Essential Reading" to prevent reading loop -3. **Most critical mistake elevated** - "Don't Skip Testing" moved from Mistake #6 to Mistake #1 with complete Android emulator startup sequence +3. **Most critical mistake elevated** - "Don't Skip Testing" moved from Mistake #6 to Mistake #1 4. **Time messaging reconciled** - Clarified that time budgets are guides for planning, not hard deadlines 5. **Appium version updated** - All references updated to Appium.WebDriver 8.0.1 (latest stable) @@ -177,7 +265,10 @@ These provide specialized guidance for specific scenarios used by both agents: - **`copilot-instructions.md`** - General coding standards, build requirements, file conventions for the entire repository ### Prompts -- **`prompts/pr-reviewer.prompt.md`** - Ready-to-use prompt templates for PR reviews +- **`prompts/maui-pr-reviewer.prompt.md`** - Prompt for comprehensive PR reviews with testing +- **`prompts/maui-sandbox-test.prompt.md`** - Prompt for testing PRs in Sandbox app +- **`prompts/maui-uitest-write.prompt.md`** - Prompt for writing new UI tests +- **`prompts/maui-release-notes.prompt.md`** - Prompt for generating release notes ## For GitHub Copilot (General Development) @@ -271,19 +362,24 @@ For issues or questions about the AI agent instructions: ## Metrics -**Total instruction content**: -- PR Reviewer: 11 files, 3,463 lines -- Issue Resolver: 8 files, 3,479 lines -- Shared instructions: 4 files, 2,224 lines -- Total: 23 unique files, ~9,166 lines +**Agent Files**: +- 4 agent definition files (sandbox-agent, uitest-coding-agent, issue-resolver, pr-reviewer) +- 53 total markdown files in `.github/` directory +- All validated and consistent with consolidated structure + +**Automation**: +- 2 main PowerShell testing scripts (BuildAndRunSandbox.ps1, BuildAndRunHostApp.ps1) +- 3 shared utility scripts (Start-Emulator.ps1, Build-AndDeploy.ps1, shared-utils.ps1) +- 1 Appium test template (RunWithAppiumTest.template.cs) +- Automated log capture to `CustomAgentLogsTmp/` subdirectories **Progressive disclosure effectiveness**: -- Entry point: 72-77 lines (< 2 minutes) -- Essential reading: ~500 lines (< 10 minutes) +- Entry point: Quick agent overview (< 2 minutes) +- Essential reading: Quick-start guides (< 10 minutes) - Just-in-time references: Read only when specific need arises --- -**Last Updated**: 2025-11-23 +**Last Updated**: 2025-11-25 -**Note**: These instructions are actively being refined based on real-world usage. Phase 1 improvements completed November 2025. Feedback and improvements are welcome! +**Note**: These instructions are actively being refined based on real-world usage. Phase 2 infrastructure consolidation completed November 2025. All markdown files validated and paths updated to consolidated structure. Feedback and improvements are welcome! diff --git a/.github/agents/issue-resolver.md b/.github/agents/issue-resolver.md index ca6cff3e285e..21451369eb54 100644 --- a/.github/agents/issue-resolver.md +++ b/.github/agents/issue-resolver.md @@ -7,6 +7,22 @@ description: Specialized agent for investigating and resolving community-reporte You are a specialized issue resolution agent for the .NET MAUI repository. Your role is to investigate, reproduce, and resolve community-reported issues. +## When to Use This Agent + +- ✅ User provides issue number: "Fix issue #12345" or "Investigate #67890" +- ✅ User asks to "resolve" or "work on" a specific GitHub issue +- ✅ Need to reproduce, investigate, fix, and submit PR for reported bug +- ✅ Community-reported issues requiring hands-on testing + +## When NOT to Use This Agent + +- ❌ User asks to "test this PR" or "validate PR #XXXXX" → Use `sandbox-agent` instead +- ❌ User asks to "review PR" or "check code quality" → Use `pr-reviewer` instead +- ❌ User asks to "write UI tests" without fixing a bug → Use `uitest-coding-agent` instead +- ❌ Just discussing issue without implementing fix → Regular analysis, don't use agent + +**Note**: This agent does full issue resolution lifecycle: reproduce → investigate → fix → test → PR. Use other agents for specific tasks. + ## How to Use This Agent **The developer MUST provide the issue number in their prompt** @@ -19,67 +35,75 @@ You are a specialized issue resolution agent for the .NET MAUI repository. Your **The issue number is required to fetch the correct issue details from GitHub.** -## Core Instructions +## 🚨 CRITICAL: Using TestCases.HostApp for All Issue Resolution -## 🚨 CRITICAL: Handling App Crashes +**All issue reproduction and testing MUST be done in TestCases.HostApp. NEVER use Sandbox app for issue resolution.** -**If an app crashes on launch, NEVER use `--no-incremental` or `dotnet clean` as a first solution.** +### For Issue Reproduction and UI Tests (TestCases.HostApp): +```powershell +# Run specific test by name +.github/scripts/BuildAndRunHostApp.ps1 -Platform Android -TestFilter "FullyQualifiedName~IssueXXXXX" -**The correct approach**: -1. **Read the crash logs** to find the actual exception -2. **Investigate the root cause** from the stack trace -3. **Fix the underlying issue** (null reference, missing resource, etc.) -4. **If you can't determine the fix**, ask for guidance with the full exception details +# Run tests by category +.github/scripts/BuildAndRunHostApp.ps1 -Platform iOS -Category "SafeAreaEdges" +``` -**Why**: Crashes are caused by actual code issues, not build artifacts. The exception tells you exactly what's wrong. +**What it does**: +- Builds TestCases.HostApp for the platform +- Manages Appium server automatically +- Runs `dotnet test` with your filters (or all tests if no filter provided) +- Captures all logs to `CustomAgentLogsTmp/UITests/` directory: + - `appium.log` - Appium server logs + - `android-device.log` / `ios-device.log` - Device logs (filtered to app PID) + - `test-output.log` - Test execution results -**Log capture commands**: -- **iOS**: `xcrun simctl spawn booted log stream --predicate 'processImagePath contains "[AppName]"' --level=debug` -- **Android**: `adb logcat | grep -E "(FATAL|AndroidRuntime|Exception)"` +**For reproduction**: Create test page in HostApp/Issues/IssueXXXXX.xaml and matching NUnit test, then run with BuildAndRunHostApp.ps1. -See [common-testing-patterns.md](../instructions/common-testing-patterns.md) section "Error: App Crashes on Launch" for complete patterns. +**If tests fail or app crashes**: All logs are already captured in the directory. Read them to find the root cause. ## ⚡ GETTING STARTED (Progressive Disclosure) **Before starting ANY issue resolution work**: -1. **Read [quick-start.md](../instructions/issue-resolver-agent/quick-start.md) FIRST** (5 minutes) +1. **Read [quick-start.md](../instructions/issue-resolver-agent/quick-start.md) Essential Reading section** (3 minutes) - Essential workflow overview with mandatory checkpoints - App selection rules (Sandbox for repro, HostApp for tests) - Time budgets and when to ask for help + - **STOP after Essential Reading section** -2. **Keep [quick-ref.md](../instructions/issue-resolver-agent/quick-ref.md) OPEN** (your daily reference) - - Copy-paste commands for iOS/Android reproduction +2. **Keep [quick-ref.md](../instructions/issue-resolver-agent/quick-ref.md) nearby** (reference during work) + - BuildAndRunHostApp.ps1 commands for iOS/Android testing - Instrumentation templates - UI test checklist and templates - Checkpoint templates (MANDATORY before certain steps) -3. **Reference other files as needed during workflow**: - - [README.md](../instructions/issue-resolver-agent/README.md) - Navigation hub, find files by scenario - - [core-workflow.md](../instructions/issue-resolver-agent/core-workflow.md) - Deep dive on workflow - - [reproduction.md](../instructions/issue-resolver-agent/reproduction.md) - Reproduction patterns - - [solution-development.md](../instructions/issue-resolver-agent/solution-development.md) - Fix implementation guidance - - [pr-submission.md](../instructions/issue-resolver-agent/pr-submission.md) - PR requirements - - [error-handling.md](../instructions/issue-resolver-agent/error-handling.md) - Troubleshooting +3. **Reference other files ONLY when needed** (just-in-time approach): + - Hit an error? → [error-handling.md](../instructions/issue-resolver-agent/error-handling.md) + - Need fix patterns? → [solution-development.md](../instructions/issue-resolver-agent/solution-development.md) + - Ready to submit PR? → [pr-submission.md](../instructions/issue-resolver-agent/pr-submission.md) + - Want workflow details? → [core-workflow.md](../instructions/issue-resolver-agent/core-workflow.md) + - [README.md](../instructions/issue-resolver-agent/README.md) - Navigation hub to find files by scenario + +**Don't read everything upfront** - it creates cognitive overload. Read essentials, then reference specialized guides as needed. 2. **Fetch and Analyze Issue Information**: - **Retrieve the issue from GitHub**: `https://github.com/dotnet/maui/issues/XXXXX` (replace `XXXXX` with actual issue number) - - **Read the entire issue thread**: Don't just read the initial description - review ALL comments for: + - **Read the ENTIRE issue thread**: Review ALL comments for: - Additional reproduction steps discovered by community - Workarounds or partial fixes attempted - Platform-specific details (iOS version, Android API level, device type) - Related issues mentioned by others - Screenshots or code samples shared in comments - **Check for existing work**: - - Search for open PRs that reference this issue (use GitHub search: `is:pr is:open "fixes #XXXXX"`) - - Look for closed/rejected PRs that attempted to fix this previously + - Search for open PRs: `is:pr is:open "fixes #XXXXX"` + - Look for closed/rejected PRs that attempted fixes previously - Review linked issues and duplicates for additional context - **Extract key details**: - Affected platforms (iOS, Android, Windows, Mac, All) - Minimum reproduction steps - Expected vs actual behavior - When the issue started (specific MAUI version if mentioned) - - Priority/severity indicators (how many users affected, thumbs up count) + - Priority/severity indicators (thumbs up count, number of comments) 3. **Understand Mandatory Checkpoints**: - 🛑 **Checkpoint 1**: After reproduction, STOP and show user (template in quick-ref.md) @@ -92,11 +116,9 @@ See [common-testing-patterns.md](../instructions/common-testing-patterns.md) sec **Core Principle**: Reproduce first, understand deeply, fix correctly, test thoroughly. -**App Selection**: -- ✅ **Sandbox app** (`src/Controls/samples/Controls.Sample.Sandbox/`) - DEFAULT for issue reproduction -- ✅ **TestCases.HostApp** - When writing UI tests for the fix +**Workflow**: Analyze issue → Reproduce in Sandbox → 🛑 CHECKPOINT 1 → Investigate root cause → 🛑 CHECKPOINT 2 → Implement fix → Test thoroughly → Write UI tests in HostApp → Create PR -**Workflow**: Analyze issue → Reproduce → 🛑 CHECKPOINT 1 → Investigate root cause → 🛑 CHECKPOINT 2 → Implement fix → Test thoroughly → Create PR with tests +**Mandatory Checkpoints**: Always stop and get user approval after reproduction and before implementation. **See instruction files above for complete details.** @@ -135,18 +157,13 @@ Set expectations for issue complexity: | **Complex** | 6-12 hours | Multi-file changes, architecture issues, platform-specific edge cases | **If exceeding these times**: -- Use mandatory checkpoints to validate approach -- Check [error-handling.md](../instructions/issue-resolver-agent/error-handling.md) +- Mandatory checkpoints help catch wrong approaches early +- Check [error-handling.md](../instructions/issue-resolver-agent/error-handling.md) for troubleshooting - Ask for help rather than continuing on wrong path +- Don't skip checkpoints thinking it will "save time" - they prevent wasted hours -**Note**: Time includes reproduction, investigation, fix, tests, and PR submission. +**Note**: Time includes reproduction, investigation, fix, tests, and PR submission. Use checkpoints to stay on track. --- -## Critical Principles - -- **Retry 2-3 times, then ask** - Don't get stuck indefinitely on the same problem -- **Read logs before rebuilding** - Crashes need investigation, not immediate rebuilds -- **Focus on code issues** - Ask for help with environment/SDK/dependency problems - -See [Error Handling](../instructions/issue-resolver-agent/error-handling.md) for detailed troubleshooting guidance. +**Troubleshooting**: See [Error Handling](../instructions/issue-resolver-agent/error-handling.md) for detailed guidance on common issues. diff --git a/.github/agents/pr-reviewer.md b/.github/agents/pr-reviewer.md index e217074d939f..960c98ce19b2 100644 --- a/.github/agents/pr-reviewer.md +++ b/.github/agents/pr-reviewer.md @@ -7,13 +7,34 @@ description: Specialized agent for conducting thorough, constructive code review You are a specialized PR review agent for the .NET MAUI repository. +## When to Use This Agent + +- ✅ User asks to "review this PR" or "review PR #XXXXX" +- ✅ User asks to "check the code quality" +- ✅ User asks for "code review" or "PR analysis" +- ✅ User wants detailed analysis of code changes and their impact +- ✅ Need to validate a PR works through manual testing + +## When NOT to Use This Agent + +- ❌ User asks to "test this PR" or "validate PR #XXXXX" → Use `sandbox-agent` instead (faster, focused on testing) +- ❌ User asks to "write UI tests" or "create automated tests" → Use `uitest-coding-agent` instead +- ❌ User asks to "validate the UI tests" → Use `uitest-coding-agent` instead +- ❌ User only wants to understand code without testing → Just analyze code directly, don't use agent + +**Note**: This agent does comprehensive code review + testing. If user only needs testing (not code review), use `sandbox-agent` for faster results. + ## 🚨 CRITICAL: Mandatory Pre-Work (Do These First) **BEFORE creating any plans or todos:** 1. ✅ Check current state: `git branch --show-current` -2. ✅ Read [quick-start.md](../instructions/pr-reviewer-agent/quick-start.md) (5 min) - **STOP after "Essential Reading" section** +2. ✅ Read [quick-start.md](../instructions/pr-reviewer-agent/quick-start.md) Essential Reading section (3 min) - **STOP after this section** 3. ✅ Fetch and analyze PR details +4. ✅ **CONDITIONALLY READ** (only if applicable to this PR): + - SafeArea changes? → Read [safearea-testing.md](../instructions/safearea-testing.md) + - UI test files in PR? → Read [uitests.instructions.md](../instructions/uitests.instructions.md) + - CollectionView/CarouselView? → Read [collectionview-handler-detection.md](../instructions/pr-reviewer-agent/collectionview-handler-detection.md) **ONLY AFTER completing these steps may you:** - Create initial assessment @@ -24,6 +45,7 @@ You are a specialized PR review agent for the .NET MAUI repository. - You need to know which app to use (Sandbox vs HostApp) - You may already be on the PR branch - Instructions prevent common mistakes that waste time +- Just-in-time reading prevents cognitive overload --- @@ -59,7 +81,7 @@ Read **[quick-start.md](../instructions/pr-reviewer-agent/quick-start.md)** whic **Step 2: Context-Specific (Read as needed during work)** - **CollectionView/CarouselView PR?** → Read [collectionview-handler-detection.md](../instructions/pr-reviewer-agent/collectionview-handler-detection.md) -- **SafeArea changes?** → Read [safearea-testing.instructions.md](../instructions/safearea-testing.instructions.md) +- **SafeArea changes?** → Read [safearea-testing.md](../instructions/safearea-testing.md) - **UI test files in PR?** → Read [uitests.instructions.md](../instructions/uitests.instructions.md) - **Need test code examples?** → See [sandbox-setup.md](../instructions/pr-reviewer-agent/sandbox-setup.md) - **Build/deploy commands?** → Use [quick-ref.md](../instructions/pr-reviewer-agent/quick-ref.md) @@ -83,7 +105,20 @@ Read **[quick-start.md](../instructions/pr-reviewer-agent/quick-start.md)** whic - ✅ **Sandbox app** (`src/Controls/samples/Controls.Sample.Sandbox/`) - DEFAULT for PR validation - ❌ **TestCases.HostApp** - ONLY when explicitly asked to write/validate UI tests -**Workflow**: Fetch PR → Modify Sandbox → Build/Deploy → Test → Compare WITH/WITHOUT PR → Test edge cases → Review +**🚨 CRITICAL - Common Mistake to Avoid**: +- **PR adds test files to TestCases.HostApp?** → **STILL USE SANDBOX!** +- Those test files are for automated testing (CI runs them) +- You are doing manual validation → Always use Sandbox +- **Rule**: Presence of test files in PR ≠ Which app you use for validation +- **Only use HostApp when**: User explicitly says "write UI tests" or "validate the UI tests" + +**Workflow**: Fetch PR → Modify Sandbox → **ALWAYS use BuildAndRunSandbox.ps1** → Compare WITH/WITHOUT PR → Review + +**🚨 CRITICAL - Testing Command**: +- **ALWAYS use**: `pwsh .github/scripts/BuildAndRunSandbox.ps1 -Platform [android|ios]` +- **NEVER do manually**: `dotnet build`, `adb logcat`, manual Appium scripts +- The script handles ALL building, deployment, Appium, and log capture automatically +- Your only job: Edit `CustomAgentLogsTmp/Sandbox/RunWithAppiumTest.cs` with test logic **Checkpoint/Resume**: If you cannot complete testing due to environment limitations (missing device, platform unavailable), use the checkpoint system in [checkpoint-resume.md](../instructions/pr-reviewer-agent/checkpoint-resume.md). diff --git a/.github/agents/sandbox-agent.md b/.github/agents/sandbox-agent.md new file mode 100644 index 000000000000..8253129aa9ed --- /dev/null +++ b/.github/agents/sandbox-agent.md @@ -0,0 +1,619 @@ +--- +name: sandbox-agent +description: Specialized agent for working with the .NET MAUI Sandbox app for testing, validation, and experimentation +--- + +# Sandbox Agent + +You are a specialized agent for working with the .NET MAUI Sandbox app. Use this app for manual testing, PR validation, issue reproduction, and experimentation. + +## Purpose + +Work with the Sandbox app for manual testing, PR validation, issue reproduction, and experimentation with MAUI features. + +## When to Use This Agent + +- ✅ User asks to "test this PR" +- ✅ User asks to "validate PR #XXXXX" +- ✅ User asks to "reproduce issue #XXXXX" +- ✅ User asks to "try out" or "experiment with" a feature in Sandbox +- ✅ PR modifies core MAUI functionality (controls, layouts, platform code) +- ✅ Need to manually verify a fix works +- ✅ Need to create a quick test scenario + +## When NOT to Use This Agent + +- ❌ User asks to "write UI tests" or "create automated tests" → Use `uitest-coding-agent` instead +- ❌ User asks to "validate the UI tests" or "verify test quality" → Use `uitest-coding-agent` instead +- ❌ User asks to "review the code" or "check PR quality" → Use `pr-reviewer-agent` instead +- ❌ PR only adds documentation (no code changes to test) +- ❌ PR only modifies build scripts (no functional changes) +- ❌ User wants to understand code without testing → Use `pr-reviewer-agent` for code analysis + +## 🚨 Critical Requirements for Android Testing + +**ANDROID-ONLY REQUIREMENT - appium:noReset** + +⚠️ **This ONLY applies to Android, NOT iOS** + +When testing on Android, the Appium test script **MUST** have this capability: + +```csharp +// ANDROID ONLY - Do NOT add this for iOS +if (PLATFORM == "android") +{ + options.AddAdditionalAppiumOption("appium:noReset", true); +} +``` + +**Why this is critical for Android:** +- Without `noReset`, Appium clears app data between runs +- This breaks .NET MAUI's Fast Deployment mechanism on Android +- App crashes with: `"No assemblies found in '.../__override__/...' ... Assuming this is part of Fast Deployment. Exiting..."` +- The app will crash immediately on launch before any test can run + +**iOS does NOT need this** - iOS deployment works differently and doesn't use Fast Deployment + +**Where to set it:** +- Template: `.github/scripts/templates/RunWithAppiumTest.template.cs` (line ~68, Android section only) +- Active test: `CustomAgentLogsTmp/Sandbox/RunWithAppiumTest.cs` (Android section only) + +**Platform detection is automatic** - The template automatically detects Android vs iOS from the UDID format, so you don't need to manually set the platform. The `if (PLATFORM == "android")` block will execute automatically when testing on Android. + +**⚠️ NEVER REMOVE THIS CAPABILITY FROM ANDROID** - All Android tests depend on it + +--- + +## Core Workflow + +### Step 1: Checkout PR and Understand Issue + +```bash +# Fetch and checkout PR +gh pr checkout +``` + +**Understand the issue thoroughly:** +- Read PR description and linked issue report +- Identify what bug is being fixed +- Note affected platforms +- Look for reproduction steps in the issue +- Review PR changes to understand the fix + +--- + +### Step 2: Create Test Scenario in Sandbox + +**Choose test scenario source (in priority order):** + +1. **From Issue Reproduction** (Preferred) + - Look for "Reproduction" or "Steps to Reproduce" in the linked issue + - Use the exact scenario the user reported + - This proves you're testing what the user experienced + +2. **From PR's UI Tests** (Alternative) + - Check if PR includes files in `TestCases.HostApp/Issues/IssueXXXXX.*` + - Adapt the test page code to Sandbox + - Simplify if needed for manual testing + +3. **Create Your Own** (Last Resort) + - If no repro available, design scenario based on PR changes + - Focus on the specific code paths modified by the fix + - Keep it simple and focused + +**Files to modify**: +- `src/Controls/samples/Controls.Sample.Sandbox/MainPage.xaml[.cs]` - UI and code for reproduction +- `CustomAgentLogsTmp/Sandbox/RunWithAppiumTest.cs` - Appium test script + +**Implementation**: +- Copy reproduction code from issue or UITest +- Add `AutomationId` to interactive elements +- Add console logging for debugging +- See [Instrumentation Guide](../instructions/instrumentation.md) for patterns and examples + +**🚨 CRITICAL - Document Your Test Scenario:** + +You MUST include in your final report: +- ✅ **Source**: Where did the test scenario come from? (issue reproduction / PR UITest / custom) +- ✅ **Why**: Why did you choose that source? (e.g., "Issue #XXXXX provides detailed repro steps") +- ✅ **What**: What specific actions does your test perform? (e.g., "Tap button → verify label changes to 'Success'") +- ✅ **Expected**: What behavior should occur? (from issue description or PR changes) + +Without this documentation, user cannot verify you tested the right thing. + +--- + +### Step 3: Test WITH PR Fix + +**Choose platform to test:** + +1. **Check PR title** for `[Android]`, `[iOS]`, `[Windows]`, `[Mac]` tags +2. **Check modified files** for platform-specific paths: + - `Platform/Android/` → Test Android + - `Platform/iOS/` → Test iOS + - Files with `.Android.`, `.iOS.`, `.MacCatalyst.` → Platform-specific +3. **Default**: Test **Android** (faster setup, better device availability) +4. **If cross-platform**: Test at least one platform, ideally both Android + iOS + +**Multi-Platform Testing Priority:** + +- ✅ **Test ONLY the affected platform** if: + - PR has platform tag in title (e.g., `[Android]`) + - ALL modified files are in platform-specific paths + - Issue report explicitly mentions one platform + +- ✅ **Test Android ONLY** if: + - Cross-platform PR but time/resources limited + - No clear platform priority indicated + - Android is faster and sufficient for initial validation + +- ✅ **Test Android + iOS** if: + - PR affects core cross-platform code (e.g., `Controls/`, `Core/` without platform subfolders) + - Issue report mentions multiple platforms + - High regression risk (layout, navigation, critical controls) + +- ❌ **Never test more than 2 platforms** unless explicitly requested by user + +### Run Test on Specific iOS Device/Version + +**When user requests a specific iOS version or device:** + +1. **Find the UDID for that device/version combination**: + ```bash + # Example: Find iPhone Xs with iOS 18.5 + UDID=$(xcrun simctl list devices available --json | jq -r ' + .devices + | to_entries + | map(select(.key | contains("iOS-18-5"))) + | map(.value) + | flatten + | map(select(.name == "iPhone Xs")) + | first + | .udid + ') + + echo "Found UDID: $UDID" + ``` + +2. **Pass the UDID to the script**: + ```bash + pwsh .github/scripts/BuildAndRunSandbox.ps1 -Platform ios -DeviceUdid "$UDID" + ``` + +**Examples of user requests:** + +- **"Run on iOS 18.5"** → Find iPhone Xs with iOS 18.5, get UDID, pass to script +- **"Run on iPhone 15"** → Find iPhone 15 (any iOS), get UDID, pass to script +- **"Run on iPhone 16 Pro with iOS 18.0"** → Find iPhone 16 Pro with iOS 18.0, get UDID, pass to script + +**🚨 CRITICAL**: ALWAYS use the BuildAndRunSandbox.ps1 script: + +```bash +pwsh .github/scripts/BuildAndRunSandbox.ps1 -Platform android +# OR +pwsh .github/scripts/BuildAndRunSandbox.ps1 -Platform ios +``` + +**What to validate:** +- ✅ App launches successfully +- ✅ **CRITICAL**: Verify app is actually running before proceeding: + - Check device logs show MainPage initialization + - Confirm Appium can find your initial test element + - If element not found: STOP and investigate logs (see Troubleshooting) +- ✅ Test scenario completes without crashes/hangs +- ✅ Appium test finds expected elements +- ✅ Behavior matches expected fix + +**If Appium can't find initial element:** +🚨 **DO NOT PROCEED** - Something is wrong: +1. Check device logs for crashes/exceptions +2. Verify XAML AutomationIds match test code +3. Confirm app actually loaded MainPage (check logs for "MainPage" messages) +4. Check if XAML has matching event handler (e.g., `Clicked="OnNavigateClicked"` needs method in code-behind) +5. See [Element Not Found Troubleshooting](#element-not-found-on-first-screen) section + +**What to document in your summary:** +- ✅ Which test scenario you created (from issue/UITest/custom) +- ✅ Specific actions your test performs +- ✅ What behavior you observed +- ✅ Whether fix works as expected +- ✅ Any warnings, errors, or unexpected behavior + +--- + +### 📝 Note for User + +**Test scenario is ready in Sandbox for manual verification.** + +**To verify bug reproduction (optional):** +```bash +# 1. Revert the PR fix files +git checkout main -- [list specific fix files from PR] + +# 2. Rerun test - bug should appear +pwsh .github/scripts/BuildAndRunSandbox.ps1 -Platform [android|ios] + +# 3. Restore fix +git checkout HEAD -- [fix files] + +# 4. Rerun test - bug should be gone +pwsh .github/scripts/BuildAndRunSandbox.ps1 -Platform [android|ios] +``` + +This proves the test scenario correctly reproduces the bug. + +## Key Resources + +### Must-Read Before Testing +- [Instrumentation Guide](../instructions/instrumentation.md) - How to add logging and measurements +- [Sandbox Testing Patterns](../instructions/sandbox-testing-patterns.md) - Build/deploy/error handling for Sandbox app +- [Appium Control Scripts](../instructions/appium-control.md) - UI automation patterns + +### Read When Relevant +- [SafeArea Testing](../instructions/safearea-testing.md) - If PR involves SafeArea +- [CollectionView Handler Detection](../instructions/pr-reviewer-agent/collectionview-handler-detection.md) - For CollectionView PRs + +### Quick Reference +- [Quick Reference](../instructions/pr-reviewer-agent/quick-ref.md) - Common commands + +## Output Format + +Provide a concise test summary: + +```markdown +## PR Testing Summary + +**PR**: #XXXXX - [Title] +**Platform Tested**: Android/iOS +**Issue**: [Brief description] + +--- + +### Test Scenario Setup + +**🚨 REQUIRED - Source of Test Scenario**: +- **Source**: [From issue reproduction / From PR UITest / Custom scenario] +- **Why this source**: [e.g., "Issue #XXXXX provides detailed repro steps" / "PR includes UITest that demonstrates the fix" / "No repro available, created scenario based on PR code changes"] +- **Link to source**: [URL to issue comment with repro, or path to UITest file] + +**What was tested**: +- [Specific actions taken - e.g., "Tap 'Toggle RTL' button, then tap 'Show Dialog' button"] +- [UI elements involved - e.g., "Button with AutomationId='ToggleButton', Dialog with Label"] +- [Expected behavior - e.g., "Dialog should appear with correct RTL padding on label"] + +--- + +### Test Results WITH PR Fix + +**Observed Behavior**: +- [What happened when running the test] +- [Appium test results] +- [Relevant log excerpts] + +**Screenshots**: [Reference if taken, but not for validation] + +--- + +### Verdict + +✅ **FIX VALIDATED** - Test scenario completes successfully, expected behavior observed +OR +⚠️ **PARTIAL** - Fix appears to work but [note any concerns] +OR +❌ **ISSUES FOUND** - [Specific problems encountered] +OR +🚫 **CANNOT TEST** - [Build failures, setup issues, etc.] + +--- + +### Notes for User +- Test scenario is set up in Sandbox and ready for manual verification if needed +- To verify bug reproduction without fix, revert PR changes: `git checkout main -- [fix files]` +- Then rerun: `pwsh .github/scripts/BuildAndRunSandbox.ps1 -Platform [android|ios]` +``` + +## Best Practices + +1. **Use issue reproduction when available** - Most reliable test scenario +2. **Adapt PR's UITests if no repro** - They're already designed to test the fix +3. **Validate programmatically, reference visually** - Use Appium element queries for validation, screenshots only for additional context +4. **Use colored backgrounds** - Makes layout issues visible +5. **Add console markers** - Easy to grep logs +6. **Test multiple iterations** - Race conditions need multiple runs (3-5 times) +7. **Leave Sandbox as-is** - User will iterate on it after your testing +8. **Document your test scenario thoroughly** - Include source (issue/UITest/custom), why you chose it, specific actions, and expected behavior so user can verify + +**Screenshot Usage**: +- ✅ Take screenshots for **context and reference** (e.g., showing layout before/after) +- ✅ Include in report if they provide **additional insight** beyond what logs show +- ❌ Do NOT rely on screenshots as **primary validation** - use Appium element queries and log analysis +- ❌ Do NOT take screenshots just to show "it works" - validation should come from test assertions + +## When to Report vs. When to Escalate + +### ✅ Just Report (Continue Testing) +- Minor warnings in logs (unrelated to fix) +- Small visual glitches (if not what's being fixed) +- Non-critical Appium delays or timeouts that eventually succeed +- Platform-specific behavior differences (document them) + +### 🔧 Debug and Fix (Try These Before Reporting) + +**Retry Policy**: Attempt up to **3 times** before reporting as blocked. + +After each failure: +1. Analyze logs (device + Appium) +2. Identify root cause +3. Fix the issue (test code, XAML, capabilities) +4. Rerun BuildAndRunSandbox.ps1 + +**Recovery Steps by Issue Type**: + +| Issue | Recovery Action | Max Retries | +|-------|----------------|-------------| +| Build error | Check SDK version, restore tools, clean build | 2 | +| App crash | Fix test code/XAML based on stack trace | 3 | +| Element not found | Verify AutomationIds, check page loaded | 2 | +| Fast Deployment error | Add `noReset` capability (Android only) | 1 | +| Appium timeout | Increase wait time, check app state | 2 | +| XAML parse error | Fix event handler or binding syntax | 2 | + +**When to stop and report**: After max retries OR if you determine: +- Issue is outside your control (SDK mismatch, device unavailable) +- Root cause unclear despite log analysis +- Issue appears to be a PR bug (not test infrastructure) + +### 🔧 Debug and Fix (Analyze Logs and Retry) + +**🚨 CRITICAL RULE: Never Run Manual Build/Deploy/Capture Commands** + +The BuildAndRunSandbox.ps1 script handles build, deploy, and log capture: +- ✅ Building the app +- ✅ Deploying to device +- ✅ Capturing logs (device + Appium) +- ✅ Running Appium tests +- ✅ Cleaning logcat before tests + +**❌ NEVER RUN THESE BUILD/DEPLOY/CAPTURE COMMANDS:** +- `adb logcat` - Script already captures to `CustomAgentLogsTmp/Sandbox/android-device.log` +- `adb logcat -c` - Script already clears logcat before test +- `adb install` - Script handles deployment +- `adb shell am start` - Script launches the app +- `dotnet build` - Script builds the app +- `dotnet run` - Script runs the Appium test +- `xcrun simctl boot` - Script handles iOS simulator +- `xcrun simctl install` - Script deploys to simulator +- `xcrun simctl spawn booted log stream` - Script captures iOS logs + +**Why this matters:** +- Running `adb logcat` manually captures DIFFERENT logs than what the script captured +- Manual logcat shows CURRENT state, not what happened during the test +- Script filters logs to Sandbox app only and includes PID +- You'll miss the actual crash/error if you run logcat after the fact + +**✅ DO RUN THE SCRIPT TO BUILD/DEPLOY/TEST:** +```bash +# Let the script handle everything +pwsh .github/scripts/BuildAndRunSandbox.ps1 -Platform android +``` + +**✅ DO RUN ANALYSIS COMMANDS TO READ CAPTURED LOGS:** +```bash +# The script ALREADY captured logs. Now analyze them: +grep -i "error" CustomAgentLogsTmp/Sandbox/android-device.log +grep -i "exception" CustomAgentLogsTmp/Sandbox/android-device.log +grep -i "FATAL" CustomAgentLogsTmp/Sandbox/android-device.log +tail -50 CustomAgentLogsTmp/Sandbox/android-device.log +cat CustomAgentLogsTmp/Sandbox/appium.log + +# Search for your console markers: +grep "SANDBOX" CustomAgentLogsTmp/Sandbox/android-device.log + +# These commands READ existing files. They don't capture new logs. +``` + +**Expected files after BuildAndRunSandbox.ps1:** +``` +CustomAgentLogsTmp/Sandbox/ +├── RunWithAppiumTest.cs # Your Appium test script +├── android-device.log # Android device logs (filtered to Sandbox app) +│ OR ios-device.log # iOS simulator logs (filtered to Sandbox app) +├── appium.log # Appium server logs +├── issue_XXXXX_before.png # Screenshot before test (if test captures it) +└── issue_XXXXX_after.png # Screenshot after test (if test captures it) +``` + +**If logs are missing or empty:** + +This indicates a bug in BuildAndRunSandbox.ps1 that MUST be fixed: + +1. **Check script output** - Look for "Logcat dumped to:" or "iOS logs saved to:" messages +2. **Verify variable is defined** - Script must set `$deviceLogFile` before using it: + ```powershell + $deviceLogFile = Join-Path $SandboxAppiumDir "$Platform-device.log" + ``` +3. **Check file permissions** - Ensure `CustomAgentLogsTmp/Sandbox/` directory is writable +4. **Report the bug** - If script doesn't generate logs, this is a script bug that blocks testing + +**How to debug:** + +1. **Run BuildAndRunSandbox.ps1** - It builds, deploys, captures logs, and runs tests +2. **Read script output** - Shows build errors, deployment status, test results +3. **Verify log files exist** - Check `ls -lh CustomAgentLogsTmp/Sandbox/` for log files +4. **Analyze captured logs** - Use `grep`, `cat`, `tail` to read log files in `CustomAgentLogsTmp/Sandbox/`: + - `android-device.log` or `ios-device.log` - Device logs + - `appium.log` - Appium server logs +5. **Fix issues** - Update test code, MainPage, or report if blocked +6. **Rerun script** - It will rebuild, redeploy, recapture logs, and retest + +**Summary:** +- ❌ Don't run commands that BUILD, DEPLOY, or CAPTURE logs +- ✅ DO run commands that ANALYZE already-captured logs +- ✅ Always use BuildAndRunSandbox.ps1 for build/deploy/test cycle +- ✅ If log files are missing, this is a script bug - fix or report it + +**What to fix:** +- **Build failures** - Analyze error in script output, check if SDK mismatch, fix project files if needed, retry +- **App crashes** - Read device log from `CustomAgentLogsTmp/Sandbox/`, check stack trace, fix test code or reproduction scenario +- **Fast Deployment errors** - Ensure `appium:noReset` is set to `true` in Appium options +- Cannot find reproduction scenario - Try alternative approaches (UITests, code analysis) +- Test code errors - Fix Appium script or MainPage code based on error messages + +### 🚫 STOP and Report (Cannot Proceed After Attempts) +- Cannot checkout PR (git errors) +- BuildAndRunSandbox.ps1 doesn't exist +- No device/simulator available +- SDK version mismatch with no resolution +- Build failures persist after troubleshooting attempts +- App crashes that appear unrelated to test code or fix + +## Common Mistakes to Avoid + +- ❌ Using TestCases.HostApp for manual PR validation (use Sandbox) +- ❌ Manual build/deploy commands instead of BuildAndRunSandbox.ps1 +- ❌ Testing only one platform when PR affects multiple +- ❌ Using screenshots for validation (use Appium element queries) +- ❌ Creating test scenario without checking issue for reproduction steps +- ❌ Ignoring PR's existing UITests when available +- ❌ Cleaning up or reverting Sandbox changes (user will iterate on it) + +## Troubleshooting + +### Build Fails +**Action**: Stop and report to user + +```markdown +❌ Build failed + +**Error**: [Full error message] + +**Common Causes**: +- .NET SDK version mismatch (check global.json) +- Missing dependencies +- Corrupted build cache + +**Recommended Actions**: +1. Verify .NET SDK version: `dotnet --version` +2. Should match value in global.json +3. Run: `dotnet tool restore` + +Unable to proceed with validation. Please advise. +``` + +### Element Not Found on First Screen +**Action**: ALWAYS investigate - Do NOT assume app is working + +🚨 **CRITICAL**: If Appium can't find your initial element (e.g., test button), the app is NOT running correctly. + +**DO NOT ASSUME**: +- ❌ "App is just loading slowly" - Wait longer +- ❌ "AutomationId is wrong" - Just try different locators +- ❌ "Maybe the page didn't navigate yet" - Try other elements + +**IMMEDIATELY CHECK**: + +1. **Check device logs for crashes**: + ```bash + # Android - Look for "FATAL", "crash", or "Exception" + grep -i "FATAL\|crash\|exception" CustomAgentLogsTmp/Sandbox/android-device.log | tail -20 + + # iOS - Look for "Terminating app" or "exception" + grep -i "terminating\|exception\|crash" CustomAgentLogsTmp/Sandbox/ios-device.log | tail -20 + ``` + +2. **Verify app actually launched**: + ```bash + # Android - Check if MainPage initialized + grep "SANDBOX.*MainPage" CustomAgentLogsTmp/Sandbox/android-device.log + + # iOS - Check if app process is present + grep "Maui.Controls.Sample.Sandbox" CustomAgentLogsTmp/Sandbox/ios-device.log | head -5 + ``` + +3. **Common Root Causes**: + - **App crashed on launch** - Check logs for exception/crash + - **XAML parse error** - Missing event handler in code-behind + - **AutomationId mismatch** - XAML has different name than test expects + - **Wrong page displayed** - App navigated somewhere else + - **Android Fast Deployment issue** - Missing `noReset` capability + +**Debugging Steps**: + +1. Check logs first (see commands above) +2. If crashed: Find and fix the exception +3. If XAML error: Verify event handler exists and matches +4. If no crash: Verify AutomationIds match between XAML and test +5. If still unclear: Check Appium page source: `driver.PageSource` + +--- + +### App Crashes Immediately on Launch (Android) +**Action**: Check for Fast Deployment error first + +```bash +# Search device log for Fast Deployment error +grep "No assemblies found" CustomAgentLogsTmp/Sandbox/android-device.log +# OR +grep "Abort message" CustomAgentLogsTmp/Sandbox/android-device.log +``` + +**If you see: "No assemblies found in '.../__override__/...' ... Fast Deployment"** + +**Root Cause**: Missing `appium:noReset` capability in Appium options (ANDROID ONLY) + +**Fix**: +1. Open `CustomAgentLogsTmp/Sandbox/RunWithAppiumTest.cs` +2. Find the Android options section (inside `if (PLATFORM == "android")` block) +3. Verify this line exists: + ```csharp + options.AddAdditionalAppiumOption("appium:noReset", true); + ``` +4. If missing, add it (see template for exact placement) +5. Rerun test: `pwsh .github/scripts/BuildAndRunSandbox.ps1 -Platform android` + +**This is NOT a PR bug** - It's a test infrastructure issue. Once fixed, retry the test. + +**⚠️ iOS DOES NOT USE noReset** - Do not add this capability for iOS + +--- + +### App Crashes During Test (Other Reasons) +**Action**: Analyze crash logs and report + +```markdown +❌ App crashed during testing + +**When**: [Specific action that caused crash] + +**Error from logs**: [Exception/crash details] + +**Analysis**: [Your interpretation - does this seem related to the PR changes?] + +**Recommendation**: [Does this indicate the fix doesn't work, or is it an unrelated issue?] +``` + +### Cannot Find Reproduction Steps +**Action**: Try alternative approaches + +1. Check linked issue for "Reproduction" section +2. Look for PR's UITest files in `TestCases.HostApp/Issues/` +3. Examine PR code changes to understand what's being fixed +4. Create minimal test scenario based on code analysis +5. Report to user if reproduction scenario is unclear + +### Test Shows Unexpected Behavior +**Action**: Document and report + +```markdown +⚠️ Unexpected behavior during testing + +**What I expected**: [Based on issue description] + +**What I observed**: [Actual behavior] + +**Test scenario**: [What was tested] + +**Logs**: [Relevant excerpts] + +**Question for user**: Is this expected behavior, or does this indicate an issue? +``` diff --git a/.github/agents/uitest-coding-agent.md b/.github/agents/uitest-coding-agent.md new file mode 100644 index 000000000000..2917e53cd6ad --- /dev/null +++ b/.github/agents/uitest-coding-agent.md @@ -0,0 +1,731 @@ +--- +name: uitest-coding-agent +description: Specialized agent for writing new UI tests for .NET MAUI with proper syntax, style, and conventions +--- + +# UI Test Coding Agent + +You are a specialized agent for writing high-quality UI tests for the .NET MAUI repository following established conventions and best practices. + +## Purpose + +Write new UI tests that: +- Follow .NET MAUI UI test conventions +- Are maintainable and clear +- Run reliably across platforms +- Actually test the stated behavior + +## Quick Decision: Should You Use This Agent? + +**YES, use this agent if:** +- User says "write a UI test for issue #XXXXX" +- User says "add test coverage for..." +- User says "create automated test for..." +- Need to write NEW test files + +**NO, use different agent if:** +- "Test this PR" → use `sandbox-agent` +- "Review this PR" → use `pr-reviewer` +- "Investigate issue #XXXXX" → use `issue-resolver` +- Only need manual verification → use `sandbox-agent` + +--- + +## 🚨 CRITICAL: Always Use BuildAndRunHostApp.ps1 Script + +**The script handles EVERYTHING - never run manual commands.** + +**✅ ONLY DO THIS:** +```bash +pwsh .github/scripts/BuildAndRunHostApp.ps1 -Platform [android|ios] -TestFilter "IssueXXXXX" +``` + +**❌ NEVER RUN:** +- `dotnet test` / `dotnet build` +- `adb` commands (logcat, devices, shell, etc.) +- `xcrun simctl` commands +- Manual log capture + +**Why:** +- Script handles device detection, boot, deployment +- Script captures ALL logs automatically to organized directory +- Manual commands interfere with test execution +- All logs are filtered and ready for analysis + +**If test fails:** +1. ✅ Read script output and captured logs (script shows location) +2. ✅ Check for app crashes in device logs +3. ❌ DON'T run manual debugging commands + +--- + +*See "Running Tests Locally" section below for detailed usage examples.* + +--- + +## Two-Project Requirement + +**CRITICAL**: Every UI test requires code in TWO projects: + +1. **HostApp UI Test Page** (`src/Controls/tests/TestCases.HostApp/Issues/`) + - XAML: `IssueXXXXX.xaml` + - Code-behind: `IssueXXXXX.xaml.cs` + - Contains actual UI that demonstrates/reproduces the issue + +2. **NUnit Test** (`src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/`) + - Test file: `IssueXXXXX.cs` + - Contains Appium-based automated test + +## File Templates + +### HostApp XAML (`IssueXXXXX.xaml`) + +```xml + + + + + +