[FEATURE]

[gyanLM10]

📋 Prerequisites

• I have searched the existing issues to avoid creating a duplicate
• By submitting this issue, you agree to follow our Code of Conduct

📝 Feature Summary

Introducing an LLM Guardian middleware that validates, repairs, and normalizes LLM-generated tool outputs to ensure reliable execution in kAgent, especially when using smaller local models.

❓ Problem Statement / Motivation

What problem does this feature solve?

When using smaller local models, the LLM may:

Return malformed or truncated JSON

Wrap responses in markdown blocks

Hallucinate incorrect tool names

Produce argument type mismatches (e.g., string instead of integer)

These issues cause tool execution failures and reduce the reliability of kAgent when running without OpenAI-grade models.

Why is it important?

This feature improves resilience and stability when running kAgent on local hardware. It enables more deterministic execution by validating and correcting structured outputs before they reach the tool engine.

Who would benefit?

Users running local LLMs (Ollama, vLLM, smaller models)

Developers integrating kAgent in cost-sensitive or offline environments

Teams seeking reliable tool execution without relying on external APIs

💡 Proposed Solution

Layer 1: Multi-Step JSON Salvaging

A tiered pipeline (Regex extraction + json-repair) to:

Extract JSON from markdown-wrapped responses

Repair truncated or malformed JSON

Ensure the engine always receives valid structured input

Layer 2: Fuzzy Tool Mapping

Use a similarity check (e.g., thefuzz) to:

Detect hallucinated tool names (e.g., get_k8s_pods)

Map them to the closest registered tool (e.g., kubectl_get_pods)

Apply mapping only when similarity confidence exceeds a defined threshold (e.g., >85%)

Layer 3: Type Coercion Middleware

Use Pydantic-based validation to:

Validate tool arguments

Coerce common mismatches (e.g., "80" → 80)

Prevent failures caused by minor type inconsistencies

Additionally, I plan to build a test suite of intentionally “broken” local model responses to benchmark and measure the Guardian’s effectiveness.

🔄 Alternatives Considered

Relying solely on prompt engineering improvements

Restricting usage to high-quality hosted LLMs

However, these approaches either do not fully eliminate structured-output errors or increase operational cost.

🎯 Affected Service(s)

Multiple services / System-wide

📚 Additional Context

This feature aims to lower the barrier to entry for users who want to run kAgent without expensive OpenAI API keys, improving accessibility and resilience for local deployments.

🙋 Are you willing to contribute?

• I am willing to submit a PR for this feature