Skip to content

Comments

fix: eval lint err for extra arguments and autofix failing test#1768

Merged
MrgSub merged 3 commits intostagingfrom
eval-lint
Jul 20, 2025
Merged

fix: eval lint err for extra arguments and autofix failing test#1768
MrgSub merged 3 commits intostagingfrom
eval-lint

Conversation

@retrogtx
Copy link
Contributor

@retrogtx retrogtx commented Jul 20, 2025

fixes eval check script error due to extra arguments from past code.


Summary by cubic

Fixed eval script errors by removing extra arguments from AiChatPrompt calls in all eval tasks.

Summary by CodeRabbit

  • Refactor
    • Simplified AI Chat evaluation tests by streamlining prompt inputs.
  • Chores
    • Disabled automatic draft generation in email thread workflows to adjust system behavior.

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jul 20, 2025

Walkthrough

The update removes the second and third arguments ("inbox" and an empty string) from all calls to the AiChatPrompt function within the system parameter of several evalite test tasks in the AI chat evaluation suite. Additionally, it disables the automatic draft generation logic by commenting out the entire draft creation block within the runThreadWorkflow function. No other logic or functionality is modified.

Changes

File(s) Change Summary
apps/server/evals/ai-chat-basic.eval.ts Removed extra arguments from AiChatPrompt calls in all evalite test cases, standardizing usage to a single argument.
apps/server/src/pipelines.effect.ts Commented out the entire automatic draft generation logic inside runThreadWorkflow, disabling draft creation while preserving other workflow steps.

Sequence Diagram(s)

sequenceDiagram
    participant TestCase
    participant AiChatPrompt

    TestCase->>AiChatPrompt: AiChatPrompt("test-thread-id")
    Note right of AiChatPrompt: (No longer receives "inbox" or "")
    AiChatPrompt-->>TestCase: Returns prompt result
Loading

Possibly related PRs

Poem

A single thread, so neat and bright,
No inbox shadows in the night.
AiChatPrompt now takes just one,
Simpler tests—oh what fun!
Drafts paused, the workflow’s light,
🐇 Hopping forward, code feels right!


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between beff219 and b489796.

📒 Files selected for processing (1)
  • apps/server/src/pipelines.effect.ts (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • apps/server/src/pipelines.effect.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: autofix
✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cubic analysis

No issues found across 1 file. Review in cubic

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (8)
apps/server/evals/ai-chat-basic.eval.ts (8)

127-129: Same duplication concern – see earlier comment
Nothing further; change is correct.


139-141: Same duplication concern – see earlier comment


151-153: Same duplication concern – see earlier comment


163-165: Same duplication concern – see earlier comment


175-177: Same duplication concern – see earlier comment


187-189: Same duplication concern – see earlier comment


199-201: Same duplication concern – see earlier comment


211-213: Same duplication concern – see earlier comment

🧹 Nitpick comments (1)
apps/server/evals/ai-chat-basic.eval.ts (1)

101-105: Removed extra args – good, but extract the hard-coded thread id

Eliminating the stale "inbox", "" parameters aligns with the updated AiChatPrompt signature – nice catch.
However, "test-thread-id" is now duplicated nine times below. Consider a small constant to avoid drift and make the intent explicit:

@@ line 12 @@
 const baseModel = openai("gpt-4o-mini");
 
+// Re-use across all AI-chat evals
+const TEST_THREAD_ID = "test-thread-id" as const;
+
@@
-      system: AiChatPrompt("test-thread-id"),
+      system: AiChatPrompt(TEST_THREAD_ID),
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7d9a6e4 and 15349e8.

📒 Files selected for processing (1)
  • apps/server/evals/ai-chat-basic.eval.ts (9 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: retrogtx
PR: Mail-0/Zero#1354
File: apps/mail/components/ui/prompts-dialog.tsx:148-159
Timestamp: 2025-06-20T05:34:41.297Z
Learning: In the prompt management system, users should be allowed to save empty prompts as this gives them the choice to disable certain AI functionality or start fresh. Validation should not prevent empty prompts from being saved.
apps/server/evals/ai-chat-basic.eval.ts (1)
Learnt from: retrogtx
PR: Mail-0/Zero#1354
File: apps/mail/components/ui/prompts-dialog.tsx:148-159
Timestamp: 2025-06-20T05:34:41.297Z
Learning: In the prompt management system, users should be allowed to save empty prompts as this gives them the choice to disable certain AI functionality or start fresh. Validation should not prevent empty prompts from being saved.
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Analyze (javascript-typescript)

@retrogtx retrogtx changed the title fix: eval lint err for extra arguments fix: eval lint err for extra arguments and autofix failing test Jul 20, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🔭 Outside diff range comments (1)
apps/server/src/pipelines.effect.ts (1)

21-24: Remove unused imports to fix pipeline failures.

The pipeline failures indicate that generateAutomaticDraft, shouldGenerateDraft, and analyzeEmailIntent are imported but never used, as they were part of the commented-out draft generation code.

Apply this diff to remove the unused imports:

-import {
-  generateAutomaticDraft,
-  shouldGenerateDraft,
-  analyzeEmailIntent,
-} from './thread-workflow-utils';

This will resolve the ESLint warnings and clean up the import statements.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 15349e8 and beff219.

📒 Files selected for processing (1)
  • apps/server/src/pipelines.effect.ts (1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📓 Common learnings
Learnt from: retrogtx
PR: Mail-0/Zero#1354
File: apps/mail/components/ui/prompts-dialog.tsx:148-159
Timestamp: 2025-06-20T05:34:41.297Z
Learning: In the prompt management system, users should be allowed to save empty prompts as this gives them the choice to disable certain AI functionality or start fresh. Validation should not prevent empty prompts from being saved.
🪛 GitHub Actions: autofix.ci
apps/server/src/pipelines.effect.ts

[warning] 21-21: ESLint (no-unused-vars): Identifier 'generateAutomaticDraft' is imported but never used. Consider removing this import.


[warning] 22-22: ESLint (no-unused-vars): Identifier 'shouldGenerateDraft' is imported but never used. Consider removing this import.


[warning] 23-23: ESLint (no-unused-vars): Identifier 'analyzeEmailIntent' is imported but never used. Consider removing this import.

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Analyze (javascript-typescript)

@nizzyabi nizzyabi requested a review from MrgSub July 20, 2025 18:24
@nizzyabi
Copy link
Collaborator

looks good, waiting for @MrgSub

@MrgSub MrgSub merged commit bf2c8f3 into staging Jul 20, 2025
7 checks passed
Copy link
Collaborator

MrgSub commented Jul 20, 2025

Merge activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants