Skip to content

fix: Prevent features from getting stuck in in_progress after server restart#699

Merged
Shironex merged 2 commits intov0.14.0rcfrom
feature/features-stuck-in-in-progress-after-server-restart-j0th
Jan 25, 2026
Merged

fix: Prevent features from getting stuck in in_progress after server restart#699
Shironex merged 2 commits intov0.14.0rcfrom
feature/features-stuck-in-in-progress-after-server-restart-j0th

Conversation

@Shironex
Copy link
Collaborator

@Shironex Shironex commented Jan 25, 2026

Summary

  • Fixes features getting stuck in in_progress status after server restart, making them unresumable
  • Implements graceful shutdown that marks running features as interrupted before exit
  • Adds orphan detection to identify features whose branches no longer exist
  • Adds idempotent checks to prevent race conditions during feature resumption

Changes

  • Graceful shutdown handler: Marks all running features as interrupted on SIGTERM/SIGINT with 30-second timeout
  • Orphan detection: Detects features with missing branches when project is loaded
  • Idempotent resumption: isFeatureRunning() check prevents duplicate resume attempts
  • Improved context handling: Features without saved context can now be resumed (starting fresh)
  • New status type: Added interrupted to FeatureStatusWithPipeline
  • Code quality: Replaced console.log with logger, added JSDoc documentation

Files Modified

  • apps/server/src/index.ts - Graceful shutdown with timeout
  • apps/server/src/services/auto-mode-service.ts - Core feature resumption and orphan detection logic
  • apps/server/src/routes/features/routes/list.ts - Orphan detection on project load
  • apps/server/src/routes/features/index.ts - Wire up autoModeService to list handler
  • apps/server/tests/unit/services/auto-mode-service.test.ts - 15 new unit tests
  • libs/types/src/pipeline.ts - Added interrupted status

Test plan

  • All 35 unit tests in auto-mode-service.test.ts pass
  • TypeScript compilation succeeds
  • Manual test: Start feature, restart server, verify feature shows as resumable
  • Manual test: Resume feature after server restart, verify it continues correctly

Fixes #696

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Graceful shutdown now properly interrupts and cleans up running features on server termination.
    • Added detection of orphaned features (features with missing branches).
    • Features are automatically recovered after server restart if interrupted.
    • New 'interrupted' status added for feature lifecycle tracking.
  • Bug Fixes

    • Added validation for required project path parameters.
  • Tests

    • Comprehensive unit tests added for new feature management capabilities.

✏️ Tip: You can customize this high-level summary in your review settings.

…restart

- Add graceful shutdown handler that marks running features as 'interrupted'
  before server exit (SIGTERM/SIGINT)
- Add 30-second shutdown timeout to prevent hanging on exit
- Add orphan detection to identify features with missing branches
- Add isFeatureRunning() for idempotent resume checks
- Improve resumeInterruptedFeatures() to handle features without saved context
- Add 'interrupted' status to FeatureStatusWithPipeline type
- Replace console.log with proper logger in auto-mode-service
- Add comprehensive unit tests for all new functionality (15 new tests)

Fixes #696

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @Shironex, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the robustness of feature management by addressing scenarios where features could become unresumable after unexpected server interruptions. It introduces a comprehensive graceful shutdown process to correctly mark active features, implements a system to identify and log features whose underlying Git branches have been removed, and refines the feature resumption logic to be more resilient and prevent concurrent operations from causing issues.

Highlights

  • Graceful Shutdown: Implemented a graceful shutdown mechanism that marks all currently running features as 'interrupted' upon receiving SIGTERM or SIGINT signals, preventing them from getting stuck in an 'in_progress' state after a server restart. A 30-second timeout is included to force exit if shutdown takes too long.
  • Orphaned Feature Detection: Added functionality to detect 'orphaned' features, which are features whose associated Git branches no longer exist. This detection runs in the background when a project is loaded, allowing for identification and potential cleanup of stale feature entries.
  • Idempotent Feature Resumption: Introduced idempotent checks to prevent race conditions during feature resumption. If a feature is already being resumed or is running, duplicate resume requests are now silently skipped.
  • Improved Context Handling for Resumption: Enhanced the feature resumption logic to handle cases where a feature was interrupted before any agent output (context) was saved. Such features can now be resumed, starting fresh, rather than remaining stuck.
  • New Feature Status: A new 'interrupted' status has been added to the FeatureStatusWithPipeline type, explicitly indicating features that were in progress but had their execution disrupted.
  • Code Quality Improvements: Replaced console.log and console.warn calls with structured logging using logger.info and logger.warn, and added JSDoc documentation to key methods for better code clarity and maintainability.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@coderabbitai
Copy link

coderabbitai bot commented Jan 25, 2026

Warning

Rate limit exceeded

@Shironex has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 9 minutes and 29 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

📝 Walkthrough

Walkthrough

The PR introduces graceful shutdown handling with feature interruption detection, adds orphan feature detection on list requests, implements reference counting for running features, and expands the feature status type to include 'interrupted'. Comprehensive tests for new AutoModeService functionality are included.

Changes

Cohort / File(s) Summary
Graceful Shutdown & Service Wiring
apps/server/src/index.ts
Introduces centralized gracefulShutdown flow (30-second timeout) with SIGTERM/SIGINT handlers. Calls autoModeService.markAllRunningFeaturesInterrupted() to mark features as interrupted before shutdown. Passes autoModeService to createFeaturesRoutes.
Routes Layer
apps/server/src/routes/features/index.ts, apps/server/src/routes/features/routes/list.ts
Threads autoModeService parameter through route factory and list handler. Adds orphan-detection logic in /list endpoint: calls autoModeService.detectOrphanedFeatures(projectPath) in background. Adds projectPath validation with 400 error response.
Feature Lifecycle & Orphan Detection
apps/server/src/services/auto-mode-service.ts
Major additions: per-feature reference counting (acquireRunningFeature, releaseRunningFeature), idempotent resume safeguards (isFeatureRunning), feature interruption control (markFeatureInterrupted, markAllRunningFeaturesInterrupted), orphan detection (detectOrphanedFeatures, getExistingBranches). Enhanced logging and UI event emission.
Type Definitions
libs/types/src/pipeline.ts
Added 'interrupted' literal to FeatureStatusWithPipeline union type.
Test Coverage
apps/server/tests/unit/services/auto-mode-service.test.ts
New comprehensive unit test suite covering detectOrphanedFeatures, markFeatureInterrupted, markAllRunningFeaturesInterrupted, and isFeatureRunning with edge cases, error handling, and internal state verification.

Sequence Diagrams

sequenceDiagram
    participant Server as Server (Restart)
    participant AMS as AutoModeService
    participant FL as FeatureLoader
    participant Git as Git Repository
    participant FS as File System

    Server->>AMS: Startup - discover running features
    Note over AMS: Iterate through persisted features<br/>with 'in_progress' status
    
    AMS->>FL: getAll(projectPath)
    FL-->>AMS: [Feature with status='in_progress']
    
    AMS->>Git: getExistingBranches(projectPath)
    Git-->>AMS: Set of branch names
    
    alt Branch exists
        AMS->>AMS: Feature is valid
    else Branch missing (orphaned)
        AMS->>AMS: Mark as orphaned
    end
    
    AMS->>AMS: markAllRunningFeaturesInterrupted()
    AMS->>FS: updateFeatureStatus(featureId, 'interrupted')
    FS-->>AMS: Saved
    
    AMS->>Server: Ready to serve (features resumable)
Loading
sequenceDiagram
    participant Client as Client
    participant API as /list endpoint
    participant AMS as AutoModeService
    participant FL as FeatureLoader
    participant Git as Git
    
    Client->>API: GET /list?projectPath=...
    
    API->>API: Validate projectPath (new)
    
    API->>FL: Load features
    FL-->>API: Feature list
    
    API->>AMS: detectOrphanedFeatures(projectPath)<br/>(background, non-blocking)
    
    par Background Detection
        AMS->>Git: getExistingBranches(projectPath)
        Git-->>AMS: Branch set
        AMS->>AMS: Find features with missing branches
        AMS->>AMS: Log orphaned features
    end
    
    API-->>Client: Return feature list
Loading

Estimated Code Review Effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly Related PRs

Suggested Labels

Bug, Tests

Poem

🐰 A graceful shutdown, features know to rest,
Interrupted kindly—not abandoned in the test.
When servers wake, orphaned branches found,
Resume they shall, on solid ground!
No more stuck in progress, no more dismay,
The warren's features bloom anew today! 🌱

🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately captures the main fix: preventing features from remaining stuck in 'in_progress' status after server restart, which is the primary objective of the PR.
Linked Issues check ✅ Passed The PR comprehensively addresses issue #696 by implementing graceful shutdown handling, orphan detection, idempotent resumption safeguards, new 'interrupted' status, and related improvements to prevent stuck features after restart.
Out of Scope Changes check ✅ Passed All changes are directly related to fixing the stuck features issue: graceful shutdown, orphan detection, resumption logic, status type updates, tests, and infrastructure changes are all scoped to addressing issue #696.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This is an excellent pull request that addresses a critical issue of features getting stuck after a server restart. The implementation of a graceful shutdown handler, orphan detection for feature branches, and idempotent feature resumption logic significantly improves the application's robustness and reliability. The code quality has also been enhanced with better JSDoc documentation and consistent use of the logger. The addition of comprehensive unit tests for the new functionality is also a great practice. I've identified a couple of minor areas where the error handling could be simplified due to the robustness of the new methods. Overall, this is a high-quality contribution.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@apps/server/src/services/auto-mode-service.ts`:
- Around line 3090-3152: markFeatureInterrupted currently unconditionally
overwrites a feature's status to 'interrupted', which clobbers pipeline_*
statuses and prevents resumePipelineFeature from picking up the correct step;
change markFeatureInterrupted (and the callers in
markAllRunningFeaturesInterrupted that use it) to detect pipeline statuses and
preserve them instead of overwriting — either skip calling updateFeatureStatus
for statuses that start with "pipeline_" or save the original status into a
metadata field (e.g., previousStatus or interruptedFlag) before marking
interrupted, using updateFeatureStatus only for non-pipeline statuses; refer to
markFeatureInterrupted, updateFeatureStatus, markAllRunningFeaturesInterrupted,
runningFeatures, and resumePipelineFeature when making the change.

… service

- Simplified the graceful shutdown process by removing redundant error handling for marking features as interrupted, as it is now managed internally.
- Updated orphan detection logging to streamline the process and enhance clarity.
- Added logic to preserve specific pipeline statuses when marking features as interrupted, ensuring correct resumption of features after a server restart.
- Enhanced unit tests to cover new behavior for preserving pipeline statuses and handling various feature states.
@Shironex Shironex merged commit 9c8265c into v0.14.0rc Jan 25, 2026
6 of 7 checks passed
@Shironex Shironex deleted the feature/features-stuck-in-in-progress-after-server-restart-j0th branch January 25, 2026 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments