Resource Content / LLM Context Placement Hint

[evalstate]

Pre-submission Checklist

• I have verified this would not be more appropriate as a feature request in a specific repository
• I have searched existing discussions to avoid duplicates

Your Idea

While MCP includes a Priority/Audience annotation, it would be nice to be able to have an explicit "Hint" to indicate whether a Resource is intended to be placed within LLM context or as an attachment.

Edit: The Audience annotation on Resources does this - instead I think some guidance on this topic would be appropriate.

I do think it would make sense to allow the return of a "Resource Pointer" Server the option of returning a Resource without necessarily embedding in the response. Of course, the Server could simply list Resources in it's textual response, but I think this would be cleaner.

We should also distinguish between the intended difference of returning an ImageContent as part of the CallToolResult vs returning it an EmbeddedResource.

A common case for this distinction would be:

1. Image Generation. User uses a tool to create an image from a prompt for viewing. In this case putting it in the LLM Context Window is unnecessary and consumes space. This is how HuggingFace chat-ui and ChatGPT behave.
2. Image Analysis. User references an image - e.g. has a tool retrieve it from a URL and requests analysis/OCR. In this case it is intended to be included in the Context Window for the next step. This is the current default behaviour for Claude Desktop.

Although ultimately a Client handling decision, I think in a lot of cases the Client and/or Server would be able to provide a hint for the expected handling of a Resource.

While the protocol provides mechanisms like Roots and Resource Templates for organizing and accessing resources, it doesn't currently offer a straightforward way to communicate these handling preferences between Client and Server.

Some ideas:

• Extend ResourceContents to have an optional contextPlacement indicator ("prefer-context")
• Include ResourceMetadata to have a preferred placement ("context" or "attachment").

Example Flows:

sequenceDiagram
    participant U as User
    participant C as Client
    participant L as LLM
    participant S as Server

    rect rgb(240, 240, 240)
        Note over U,S: Claude Desktop: Current Flow - All Images In Context
        U->>C: "Generate beach image"
        C->>L: Send prompt
        L->>C: Request tool 'generate_image'
        C->>S: tools/call Request
        S-->>C: Response with ImageContent
        C->>L: Include image in context
        C-->>U: Display image
    end

    rect rgb(230, 240, 250)
        Note over U,S: Desired Flow - Generated Images
        U->>C: "Generate beach image"
        C->>L: Send prompt
        L->>C: Request tool 'generate_image'
        C->>S: tools/call Request
        S-->>C: Response with ImageContent<br/>(hint: prefer-attachment)
        C-->>U: Display image
        Note over C: Image not added<br/>to LLM context
        C->>L: Continue conversation
    end

    rect rgb(230, 250, 240)
        Note over U,S: Desired Flow - Analysis Case (e.g. OCR)
        U->>C: "What's in this image?"
        C->>L: Send prompt
        L->>C: Request tool 'analyze_image'
        C->>S: tools/call Request
        S-->>C: Response with ImageContent<br/>(hint: prefer-context)
        C->>L: Include image in context
        L-->>C: Image analysis
        C-->>U: Display analysis
    end

Loading

In this example, the create_image tool has been hardcoded to generate beach scenes. Claude Desktop recognises the image is not as requested (and even retries!). This shows the Client incorporating the resource in to the Context Window.

Scope

• Protocol Specification
• SDK Features
• Documentation
• Developer Experience
• Other