Create agent tests

[sarah-inkeep]

No description provided.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 2f499da

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
agents-docs	Ready	Preview, Comment	Dec 15, 2025 10:01pm
agents-manage-api	Ready	Preview, Comment	Dec 15, 2025 10:01pm
agents-manage-ui	Ready	Preview, Comment	Dec 15, 2025 10:01pm
agents-run-api	Ready	Preview, Comment	Dec 15, 2025 10:01pm

[claude]

Claude finished @sarah-inkeep's task —— View job

---

PR Review: Create Agent Tests

I've conducted a comprehensive review of the new Cypress test file for creating agents. Here's my detailed analysis:

Todo List:

• Read and analyze the new Cypress test file
• Research existing test patterns and architecture in the codebase
• Examine the create agent functionality being tested
• Review for engineering best practices and potential issues
• Provide comprehensive feedback and recommendations

---

Executive Summary

This PR adds a solid foundation for testing agent creation functionality with good coverage of the happy path and key validation scenarios. However, there are several areas where the implementation could be more robust, maintainable, and aligned with existing patterns in the codebase.

Code Quality & Engineering Best Practices

✅ Strengths

• Good test coverage: Covers happy path, validation errors, and duplicate handling
• Proper cleanup: Uses after() hook to clean up created agents
• Realistic data: Uses timestamp-based unique identifiers
• Descriptive test names: Clear intent for each test case

⚠️ Areas for Improvement

1. Inconsistent ID Generation Pattern

Issue: The test uses Date.now() for ID generation, but the existing codebase has a randomId() utility in cypress/support/utils.ts:1-3 that's used elsewhere.

Impact: Inconsistency across the test suite and potential collision issues.

Recommendation:

// Instead of: const agentId = `test-agent-${Date.now()}`;
// Use: const agentId = `test-agent-${randomId()}`;

2. Brittle Element Selection Strategy

Issue: The openNewAgentDialog() function uses overly complex logic to find the "New Agent" button:

const openNewAgentDialog = () => {
  cy.get('body').then(($body) => {
    const newAgentButton = $body
      .find('button')
      .filter((_, el) => el.textContent?.includes('New Agent'));
      
    if (newAgentButton.length > 0) {
      cy.contains('button', 'New Agent').click();
    } else {
      cy.contains(/Create agent|New Agent/i).click();
    }
  });
};

Impact: This approach is fragile and doesn't follow Cypress best practices. The conditional logic suggests uncertainty about the UI state.

Recommendation: Use data-testid attributes for reliable element selection:

// In component: <Button>
// In test: cy.get('[data-testid="new-agent-button"]').click();

3. Inconsistent Wait Patterns

Issue: Mix of hard-coded waits (cy.wait(300), cy.wait(500), cy.wait(1000)) and timeout-based waits.

Impact: Tests are slower and less reliable than necessary.

Recommendation: Replace arbitrary waits with proper assertions:

// Instead of: cy.wait(300);
// Use: cy.get('[role=dialog]').should('be.visible');

4. Missing Data-Testid Strategy

Issue: Tests rely on generic selectors like [role=dialog], input[name="name"] which are fragile.

Impact: Tests break easily when UI structure changes.

Recommendation: Implement data-testid attributes in the form components for stable selection.

System Architecture & Design Considerations

5. Insufficient Error Handling Coverage

Issue: The duplicate agent test only checks for toast existence but doesn't validate specific error messages:

cy.get('[data-sonner-toast]', { timeout: 10000 }).should('exist');
cy.contains(/already exists|Failed to create agent/i, { timeout: 10000 }).should('exist');

Impact: Weak assertion that might pass for unrelated errors.

Recommendation: Test specific error scenarios more precisely and ensure error messages are user-friendly.

6. Hard-coded Test Data

Issue: Test uses hard-coded tenantId: 'default' and projectId: 'my-weather-project'.

Impact: Tests are tightly coupled to specific test data setup.

Recommendation: Consider using environment variables or test fixtures for more flexible test data management.

7. Limited Edge Case Coverage

Missing test scenarios:

• Network failure during agent creation
• Malformed API responses
• Extremely long agent names/IDs
• Special characters in agent names (beyond ID validation)
• Concurrent agent creation scenarios

Validation Logic Alignment

8. Validation Message Inconsistency

Issue: Test expects "Id must contain only alphanumeric characters, underscores, and dashes" but the actual validation schema message is "Id must contain only alphanumeric characters, underscores, and dashes. No spaces allowed." (line 13 in /src/lib/validation.ts).

Impact: Test might fail due to message mismatch.

Recommendation: Ensure exact message matching or use more flexible assertions.

Performance & Maintainability

9. Cleanup Strategy Limitations

Issue: The cleanup only happens in after() hook, not in case of test failures mid-execution.

Impact: Failed tests may leave orphaned test data.

Recommendation: Consider using afterEach() or more robust cleanup strategies.

10. Missing Abstraction Opportunities

Issue: Common operations like form filling are repeated across tests.

Recommendation: Create helper functions:

const fillAgentForm = (name: string, id: string) => {
  cy.get('[role=dialog]').within(() => {
    cy.get('input[name="name"]').clear().type(name);
    cy.get('input[name="id"]').clear().type(id);
  });
};

Security Considerations

11. Input Sanitization Testing

Missing: Tests for XSS prevention, SQL injection patterns, and other security edge cases in agent names/descriptions.

Recommendations Summary

High Priority

1. Align with existing patterns: Use randomId() utility for consistent ID generation
2. Implement data-testid strategy: Add stable selectors to UI components
3. Fix validation message alignment: Ensure test assertions match actual error messages
4. Remove arbitrary waits: Replace with proper assertions

Medium Priority

5. Add comprehensive error scenarios: Network failures, malformed responses
6. Improve cleanup robustness: Handle test failures better
7. Create test utilities: Abstract common operations like form filling

Low Priority

8. Add security edge case tests: XSS, injection patterns
9. Consider environment-based test data: Make tests less dependent on hard-coded values

Conclusion

This is a solid foundation for agent creation testing, but implementing the above recommendations would significantly improve test reliability, maintainability, and coverage. The most critical improvements focus on consistency with existing patterns and more robust element selection strategies.

---