Display Chain-of-Thought reasoning steps in UI

[manavgup]

Priority

🔴 High

Description

Show CoT reasoning steps, sub-questions, and intermediate answers in expandable accordion sections.

Current State

Backend CoT works but UI doesn't visualize reasoning steps

UI Evidence

Tested "CoT Reasoning Test" - no reasoning steps shown

Files to Modify

• frontend/src/components/search/CoTDisplay.tsx (new)
• frontend/src/components/search/MessageDisplay.tsx
• frontend/src/types/search.ts

Acceptance Criteria

• Display sub-questions generated during CoT
• Show intermediate reasoning steps
• Display final synthesized answer
• Use accordion/collapsible UI for each step
• Number steps sequentially
• Add "Show reasoning" toggle

Effort

1 week

Related Issues

#253, #136

[manavgup]

@claude please review and generate a detailed plan on how to address this.

[claude]

Claude finished @manavgup's task —— View job

---

Reviewing Issue #274: Display Chain-of-Thought Reasoning Steps in UI ✅

Analysis Complete

I've completed a thorough review of the codebase and created a comprehensive implementation plan for displaying CoT reasoning steps in the UI.

---

Analysis Summary

Current State ✅

Backend (Working)

• CoT service fully implemented in backend/rag_solution/services/chain_of_thought_service.py
• Returns structured ChainOfThoughtOutput with:
  • original_question: The user's question
  • reasoning_steps: List of ReasoningStep objects containing:
    • step_number: Sequential step numbering
    • question: Sub-question for this step
    • intermediate_answer: Answer for the sub-question
    • confidence_score: Confidence level (0-1)
    • reasoning_trace: Trace of the reasoning process
    • execution_time: Time taken for this step
    • token_usage: Tokens consumed
    • source_attributions: Sources used in this step
  • final_answer: Synthesized final answer
  • total_confidence: Overall confidence
  • reasoning_strategy: Strategy used (decomposition/iterative/etc.)
  • source_summary: Aggregated source information

Frontend (Partially Implemented)

• LightweightSearchInterface.tsx already has:
  • CoT accordion UI skeleton (lines 878-935)
  • Toggle state management for CoT display
  • Basic step rendering showing step number, question, and answer
  • Type definitions in apiClient.ts for ChainOfThoughtOutput and ChainOfThoughtStep

The Gap ⚠️
The frontend CoT display expects a simplified structure but backend provides richer data:

• Frontend expects: { enabled, total_steps, steps: [{step_number, question, answer, sources_used, reasoning}], final_synthesis }
• Backend provides: Full ChainOfThoughtOutput schema with ReasoningStep objects

The mapping happens in search_service.py:822-843 which transforms backend schema to frontend-friendly format, BUT:

1. Not all backend fields are being mapped (missing confidence scores, execution times, source attributions)
2. The UI needs enhancement to display ALL available CoT data
3. No separate CoTDisplay.tsx component exists yet
4. Type definitions need to be aligned between frontend and backend

---

Detailed Implementation Plan

Phase 1: Type System Alignment & Data Structure

Goal: Ensure frontend types match backend schema and add missing fields

Files to Modify:

1. frontend/src/services/apiClient.ts (lines 22-35)

   • Extend ChainOfThoughtStep interface to include:
     • confidence_score?: number
     • execution_time?: number
     • token_usage?: number
     • source_attributions?: SourceAttribution[]
     • reasoning_trace?: string
   • Add SourceAttribution interface:

     interface SourceAttribution {
       document_id: string;
       document_title?: string;
       relevance_score: number;
       excerpt?: string;
       chunk_index?: number;
       retrieval_rank?: number;
     }

   • Extend ChainOfThoughtOutput to include:
     • original_question: string
     • total_confidence: number
     • reasoning_strategy?: string
     • total_execution_time?: number
     • token_usage?: number

2. backend/rag_solution/services/search_service.py (lines 822-843)

   • Enhance the CoT output mapping to include ALL fields:

     cot_output = {
         "enabled": True,
         "original_question": cot_result.original_question,
         "total_steps": len(cot_result.reasoning_steps),
         "reasoning_strategy": cot_result.reasoning_strategy,
         "total_confidence": cot_result.total_confidence,
         "total_execution_time": cot_result.total_execution_time,
         "token_usage": cot_result.token_usage,
         "reasoning_steps": [
             {
                 "step_number": step.step_number,
                 "question": step.question,
                 "answer": step.intermediate_answer or "",
                 "confidence_score": step.confidence_score,
                 "reasoning_trace": step.reasoning_trace,
                 "execution_time": step.execution_time,
                 "token_usage": step.token_usage,
                 "source_attributions": [
                     {
                         "document_id": sa.document_id,
                         "document_title": sa.document_title,
                         "relevance_score": sa.relevance_score,
                         "excerpt": sa.excerpt,
                         "chunk_index": sa.chunk_index,
                         "retrieval_rank": sa.retrieval_rank,
                     }
                     for sa in step.source_attributions
                 ] if step.source_attributions else [],
             }
             for step in cot_result.reasoning_steps
         ],
         "final_synthesis": cot_result.final_answer,
     }

Phase 2: Create Dedicated CoT Display Component

Goal: Build a reusable, well-structured component for CoT visualization

New File: frontend/src/components/search/CoTDisplay.tsx

Component Structure:

interface CoTDisplayProps {
  cotOutput: ChainOfThoughtOutput;
  isExpanded: boolean;
  onToggle: () => void;
}

const CoTDisplay: React.FC<CoTDisplayProps> = ({ cotOutput, isExpanded, onToggle }) => {
  // Component implementation
}

Features to Implement:

1. Header Section

   • Title: "Chain of Thought ({total_steps} steps)"
   • Expand/collapse button with ChevronUp/Down icons
   • Summary badges showing:
     • Total steps
     • Overall confidence score (color-coded: green >0.7, yellow 0.4-0.7, red <0.4)
     • Reasoning strategy
     • Total execution time
     • Total tokens used

2. Steps Section (when expanded)

   • For each reasoning step:
     • Step Header:
       • Numbered badge (1, 2, 3, etc.) with color gradient
       • Step confidence indicator (progress bar or color)
       • Execution time badge
       • Token usage badge
     • Step Content:
       • Question section with distinct styling
       • Answer section with markdown support
       • Reasoning trace (collapsible if long)
       • Source attributions accordion:
         • Document name with relevance score
         • Excerpt preview
         • Chunk and retrieval rank info

3. Final Synthesis Section

   • Highlighted box showing the final synthesized answer
   • Visual connection to the main answer
   • Optional toggle to show/hide

4. Visual Design:

   • Use Carbon Design System components
   • Step progression visual (vertical timeline with connecting lines)
   • Color-coded confidence indicators
   • Smooth animations for expand/collapse
   • Responsive layout for mobile

Phase 3: Integrate Component into Main UI

Goal: Replace existing inline CoT rendering with new component

File to Modify: frontend/src/components/search/LightweightSearchInterface.tsx

Changes:

1. Import the new CoTDisplay component (line ~27)
2. Replace the existing CoT accordion (lines 878-935) with:

   {message.cot_output && message.cot_output.enabled && message.cot_output.steps.length > 0 && (
     <CoTDisplay
       cotOutput={message.cot_output}
       isExpanded={showCoT[message.id] || false}
       onToggle={() => toggleCoT(message.id)}
     />
   )}

Phase 4: Add "Show Reasoning" Toggle

Goal: Give users control over automatic CoT display

Implementation:

1. Add state in LightweightSearchInterface.tsx:

   const [autoShowCoT, setAutoShowCoT] = useState(true); // Auto-expand by default

2. Add toggle button in the header area (near connection status):

   <button
     onClick={() => setAutoShowCoT(!autoShowCoT)}
     className="flex items-center space-x-2 text-sm text-gray-70 hover:text-gray-100"
   >
     <LightBulbIcon className="w-4 h-4" />
     <span>{autoShowCoT ? 'Auto-show reasoning' : 'Manual reasoning'}</span>
   </button>

3. Update CoT toggle logic to respect auto-show preference:

   useEffect(() => {
     if (autoShowCoT && message.cot_output?.enabled) {
       setShowCoT(prev => ({ ...prev, [message.id]: true }));
     }
   }, [messages, autoShowCoT]);

Phase 5: Testing & Validation

Goal: Ensure CoT display works correctly

Test Cases:

1. Backend Response Validation

   • Verify all CoT fields are populated in search response
   • Test with different question types (simple, multi-part, comparison, causal)
   • Validate source attributions are correctly mapped

2. UI Rendering

   • Test expand/collapse functionality
   • Verify all step data is displayed correctly
   • Test responsive layout on different screen sizes
   • Validate confidence score visualizations
   • Check token usage and execution time display

3. Edge Cases

   • Handle missing optional fields gracefully
   • Test with 1 step vs multiple steps
   • Handle long reasoning traces (truncate/expand)
   • Test with no source attributions

4. Performance

   • Ensure smooth animations
   • Test with many messages containing CoT output
   • Validate memory usage with large conversations

---

Component Architecture Diagram

LightweightSearchInterface
├── Message List
│   └── For each message
│       ├── User Message
│       └── Assistant Message
│           ├── Answer Text
│           ├── Sources Accordion (existing)
│           ├── Token Usage Accordion (existing)
│           └── CoTDisplay (NEW)
│               ├── Header
│               │   ├── Title & Step Count
│               │   ├── Summary Badges
│               │   └── Expand/Collapse Button
│               ├── Steps List (when expanded)
│               │   └── For each step
│               │       ├── StepHeader
│               │       │   ├── Step Number Badge
│               │       │   ├── Confidence Indicator
│               │       │   └── Metrics (time, tokens)
│               │       ├── StepContent
│               │       │   ├── Question
│               │       │   ├── Answer
│               │       │   ├── Reasoning Trace
│               │       │   └── Source Attributions
│               └── Final Synthesis Section

---

UI/UX Design Approach

Visual Hierarchy

1. Primary: Final answer (most prominent)
2. Secondary: CoT reasoning steps (collapsible, available on-demand)
3. Tertiary: Step details (sources, reasoning traces)

Color Scheme (Carbon Design)

• High Confidence (>0.7): Green (green-50)
• Medium Confidence (0.4-0.7): Yellow (yellow-30)
• Low Confidence (<0.4): Red (red-50)
• Step Numbers: Blue gradient (blue-60 to blue-40)
• Final Synthesis: Highlighted blue background (blue-10)

Interaction Patterns

• Default State: CoT accordion visible but collapsed
• Auto-expand Option: User preference to auto-expand on new messages
• Smooth Transitions: 200ms ease-in-out for all animations
• Progressive Disclosure: Show summary first, details on demand
• Mobile-First: Touch-friendly targets, swipe gestures

---

Potential Challenges & Solutions

Challenge 1: Performance with Long Conversations

Problem: Many messages with CoT data could slow down rendering
Solution:

• Implement virtualized scrolling for message list
• Lazy-render CoT steps (only render when accordion is expanded)
• Memoize CoTDisplay component with React.memo
• Use React.lazy for code splitting

Challenge 2: Complex Data Mapping

Problem: Backend schema is rich, frontend needs simplified view
Solution:

• Create utility functions for data transformation
• Add TypeScript guards for type safety
• Implement fallback values for missing data
• Add comprehensive error boundaries

Challenge 3: Source Attribution Display

Problem: Multiple sources per step could clutter UI
Solution:

• Limit initial display to top 3 sources per step
• Add "Show all sources" expand option
• Use relevance score to prioritize display
• Provide visual indicators for source quality

Challenge 4: Mobile Responsiveness

Problem: Nested accordions can be hard to navigate on mobile
Solution:

• Use bottom sheet pattern for mobile
• Implement swipe gestures for navigation
• Larger touch targets (min 44px)
• Simplified view on small screens

Challenge 5: Accessibility

Problem: Nested interactive elements can confuse screen readers
Solution:

• Proper ARIA labels for all interactive elements
• Keyboard navigation support (Tab, Enter, Escape)
• Focus management for expand/collapse
• Semantic HTML structure

---

Acceptance Criteria Checklist

• Display sub-questions generated during CoT: Each reasoning step's question is shown in the UI
• Show intermediate reasoning steps: All steps with answers, confidence, and metrics are visible
• Display final synthesized answer: Final synthesis section clearly highlighted
• Use accordion/collapsible UI for each step: CoTDisplay component with expand/collapse functionality
• Number steps sequentially: Step numbers displayed prominently (1, 2, 3, etc.)
• Add "Show reasoning" toggle: Auto-show preference toggle in header

---

Estimated Implementation Breakdown

Phase 1: Type Alignment - 2 hours
Phase 2: CoTDisplay Component - 8 hours
Phase 3: Integration - 2 hours
Phase 4: Toggle Feature - 1 hour
Phase 5: Testing & Polish - 3 hours

Total: ~16 hours (~2 working days)

---

Recommended Implementation Order

1. ✅ Start with Phase 1 (type alignment) - ensures data flows correctly
2. ✅ Build Phase 2 (CoTDisplay component) - core functionality
3. ✅ Integrate Phase 3 - replace existing UI
4. ✅ Add Phase 4 (toggle) - user control
5. ✅ Complete Phase 5 (testing) - ensure quality

---

Next Steps

I've analyzed the requirements and created a comprehensive plan. The implementation is straightforward since:

1. Backend already provides all necessary data
2. Frontend has basic UI structure in place
3. Type definitions partially exist
4. Carbon Design components are available

Would you like me to proceed with implementing this plan? I can start with Phase 1 and work through each phase systematically, or I can implement specific phases based on your priority.

---

Job Run | Branch