.NET: Add ITextSearch<TRecord> with LINQ filtering and deprecate legacy ITextSearch (#10456) #13179

alzarei · 2025-09-25T09:03:42Z

.NET: Add LINQ-based ITextSearch interface and deprecate legacy ITextSearch (#10456)

Summary

This PR implements Option 3 from the architectural decision process for Issue #10456: introduces a new generic ITextSearch<TRecord> interface with type-safe LINQ filtering while maintaining the legacy ITextSearch interface marked as [Obsolete] for backward compatibility.

Zero breaking changes - existing code continues working unchanged.

What Changed

New Generic Interface (Recommended Path)

public interface ITextSearch<TRecord>
{
    Task<KernelSearchResults<string>> SearchAsync(
        string query, 
        TextSearchOptions<TRecord>? searchOptions = null, 
        CancellationToken cancellationToken = default);
    
    // + GetTextSearchResults and GetSearchResults methods
}

// Type-safe LINQ filtering with IntelliSense
var options = new TextSearchOptions<CorporateDocument>
{
    Filter = doc => doc.Department == "HR" && 
                   doc.IsActive && 
                   doc.CreatedDate > DateTime.Now.AddYears(-2)
};

Benefits:

✅ Compile-time type safety
✅ IntelliSense support for property names
✅ Full LINQ expression support
✅ No RequiresDynamicCode attributes
✅ AOT-compatible (simple equality/comparison patterns)

Legacy Interface (Deprecated)

[Obsolete("Use ITextSearch<TRecord> with LINQ-based filtering instead. This interface will be removed in a future version.")]
public interface ITextSearch
{
    Task<KernelSearchResults<string>> SearchAsync(
        string query, 
        TextSearchOptions? searchOptions = null, 
        CancellationToken cancellationToken = default);
}

// Legacy clause-based filtering (still works)
var options = new TextSearchOptions 
{ 
    Filter = new TextSearchFilter().Equality("Department", "HR") 
};

Migration Message: Users see deprecation warning directing them to modern ITextSearch<TRecord> with LINQ filtering.

Implementation Details

Dual-Path Architecture

VectorStoreTextSearch<TRecord> implements both interfaces with independent code paths:

Legacy Path (Non-Generic):

async IAsyncEnumerable<VectorSearchResult<TRecord>> ExecuteVectorSearchAsync(
    string query, TextSearchOptions options)
{
    var vectorOptions = new VectorSearchOptions<TRecord>
    {
        #pragma warning disable CS0618 // VectorSearchFilter is obsolete
        OldFilter = options.Filter?.FilterClauses != null 
            ? new VectorSearchFilter(options.Filter.FilterClauses) 
            : null
        #pragma warning restore CS0618
    };
    // ... execute search
}

Modern Path (Generic):

async IAsyncEnumerable<VectorSearchResult<TRecord>> ExecuteVectorSearchAsync(
    string query, TextSearchOptions<TRecord> options)
{
    var vectorOptions = new VectorSearchOptions<TRecord>
    {
        Filter = options.Filter  // Direct LINQ passthrough
    };
    // ... execute search
}

Key Characteristics:

Two independent methods (no translation layer, no conversion overhead)
Legacy path uses obsolete VectorSearchFilter with pragma suppressions (temporary during transition)
Modern path uses LINQ expressions directly (no obsolete APIs)
Both paths are AOT-compatible (no dynamic code generation)

Files Changed

Interfaces & Options

ITextSearch.cs: Added ITextSearch<TRecord> interface, marked legacy ITextSearch as [Obsolete]
TextSearchOptions.cs: Added generic TextSearchOptions<TRecord> class

Implementation

VectorStoreTextSearch.cs: Implemented dual interface pattern (~30 lines for both paths)

Backward Compatibility (Pragma Suppressions)

Added #pragma warning disable CS0618 to 27 files that use the obsolete interface:

Production (11 files):

Web search connectors (Bing, Google, Brave, Tavily)
Extension methods (WebServiceCollectionExtensions, TextSearchExtensions)
Core implementations (TextSearchProvider, TextSearchStore, VectorStoreTextSearch)

Tests/Samples (16 files):

Integration tests (Agents, AzureAISearch, InMemory, Qdrant, Web plugins)
Unit tests (Bing, Brave, Google, Tavily)
Sample tutorials (Step1_Web_Search, Step2_Search_For_RAG)
Mock implementations

Tests

Added 7 new tests for LINQ filtering scenarios
Maintained 10 existing legacy tests (unchanged)
Added DataModelWithTags to test base for collection filtering

Validation Results

✅ Build: 0 errors, 0 warnings with --warnaserror
✅ Tests: 1,581/1,581 passed (100%)
✅ Format: Clean
✅ AOT Compatibility: All checks passed
✅ CI/CD: Run #29857 succeeded

Breaking Changes

None. This is a non-breaking addition:

Legacy ITextSearch interface continues working (marked [Obsolete])
Existing implementations (Bing, Google, Azure AI Search) unchanged
Migration to ITextSearch<TRecord> is opt-in via deprecation warning

Multi-PR Context

This is PR 2 of 6 in the structured implementation for Issue #10456:

PR1 ✅: Generic interfaces foundation
PR2 ← YOU ARE HERE: Dual interface pattern + deprecation
PR3-PR6: Connector migrations (Bing, Google, Brave, Azure AI Search)

Architectural Decision

Option 3 Approved by Mark Wallace and Westey-m:

"We typically follow the pattern of obsoleting the old API when we introduce the new pattern. This avoids breaking changes which are very disruptive for projects that have a transient dependency." - Mark Wallace

"I prefer a clean separation between the old and new abstractions. Being able to obsolete the old ones and point users at the new ones is definitely valuable." - Westey-m

Options Considered:

Native LINQ Only: Replace TextSearchFilter entirely (breaking change)
Translation Layer: Convert TextSearchFilter to LINQ internally (RequiresDynamicCode cascade, AOT issues)
Dual Interface ✅: Add ITextSearch<TRecord> + deprecate legacy (no breaking changes, clean separation)

See ADR comments in conversation for detailed architectural analysis.

Migration Guide

Before (Legacy - Now Obsolete):

ITextSearch search = ...;
var options = new TextSearchOptions 
{ 
    Filter = new TextSearchFilter()
        .Equality("Department", "HR")
        .Equality("IsActive", "true")
};
var results = await search.SearchAsync("query", options);

After (Modern - Recommended):

ITextSearch<CorporateDocument> search = ...;
var options = new TextSearchOptions<CorporateDocument> 
{ 
    Filter = doc => doc.Department == "HR" && doc.IsActive
};
var results = await search.SearchAsync("query", options);

Next Steps

PR3-PR6 will migrate connector implementations (Bing, Google, Brave, Azure AI Search) to use ITextSearch<TRecord> with LINQ filtering, demonstrating the modern pattern while maintaining backward compatibility.

…obsolete VectorSearchFilter - Replace obsolete VectorSearchFilter conversion with direct LINQ filtering for simple equality filters - Add ConvertTextSearchFilterToLinq() method to handle TextSearchFilter.Equality() cases - Fall back to legacy approach only for complex filters that cannot be converted - Eliminates technical debt and performance overhead identified in Issue microsoft#10456 - Maintains 100% backward compatibility - all existing tests pass (1,574/1,574) - Reduces object allocations and removes obsolete API warnings for common filtering scenarios Addresses Issue microsoft#10456 - PR 2: VectorStoreTextSearch internal modernization

…pliance - Replace broad catch-all exception handling with specific exception types - Add comprehensive exception handling for reflection operations in CreateEqualityExpression: * ArgumentNullException for null parameters * ArgumentException for invalid property names or expression parameters * InvalidOperationException for invalid property access or operations * TargetParameterCountException for lambda expression parameter mismatches * MemberAccessException for property access permission issues * NotSupportedException for unsupported operations (e.g., byref-like parameters) - Maintain intentional catch-all Exception handler with #pragma warning disable CA1031 - Preserve backward compatibility by returning null for graceful fallback - Add clear documentation explaining exception handling rationale - Addresses CA1031 code analysis warning while maintaining robust error handling - All tests pass (1,574/1,574) and formatting compliance verified

- Add InvalidPropertyFilterThrowsExpectedExceptionAsync: Validates that new LINQ filtering creates expressions correctly and passes them to vector store connectors - Add ComplexFiltersUseLegacyBehaviorAsync: Tests graceful fallback for complex filter scenarios when LINQ conversion returns null - Add SimpleEqualityFilterUsesModernLinqPathAsync: Confirms end-to-end functionality of the new LINQ filtering optimization for simple equality filters Analysis: - All 15 VectorStoreTextSearch tests pass (3 new + 12 existing) - All 85 TextSearch tests pass, confirming no regressions - Tests prove the new ConvertTextSearchFilterToLinq() and CreateEqualityExpression() methods work correctly - Exception from InMemory connector in invalid property test confirms LINQ path is being used instead of fallback behavior - Improves edge case coverage for the filtering modernization introduced in previous commits

- Add NullFilterReturnsAllResultsAsync test to verify behavior when no filter is applied - Remove unnecessary Microsoft.Extensions.VectorData using statement - Enhance test coverage for VectorStoreTextSearch edge cases

…INQ filtering - Extend ConvertTextSearchFilterToLinq to handle AnyTagEqualToFilterClause - Add CreateAnyTagEqualToExpression for collection.Contains() operations - Add CreateMultipleClauseExpression for AND logic with Expression.AndAlso - Add 4 comprehensive tests for new filtering capabilities - Add RequiresDynamicCode attributes for AOT compatibility - Maintain backward compatibility with graceful fallback Fixes microsoft#10456

Fixes IL3051 compilation errors by adding RequiresDynamicCode attributes to: - SearchAsync(string, TextSearchOptions<TRecord>?, CancellationToken) - GetTextSearchResultsAsync(string, TextSearchOptions<TRecord>?, CancellationToken) - GetSearchResultsAsync(string, TextSearchOptions<TRecord>?, CancellationToken) The generic ITextSearch<TRecord> interface accepts LINQ expressions via TextSearchOptions<TRecord>.Filter, which requires dynamic code generation for expression tree processing. This change ensures interface methods match their implementations' RequiresDynamicCode attributes. Resolves: Issue microsoft#10456 IL3051 interface mismatch errors Cherry-pick-safe: Interface-only change, no implementation logic

- Fix CA1859: Use specific return types BinaryExpression? and MethodCallExpression? instead of generic Expression? for better performance - Improve test model: Use IReadOnlyList<string> instead of string[] for Tags property to follow .NET collection best practices These changes address code analyzer warnings and apply reviewer applicable feedback from other PRs in the Issue microsoft#10456 modernization series.

- Remove LINQ dependency from non-generic ITextSearch interface - Revert non-generic methods to direct VectorSearchFilter usage - Eliminates IL3051 warnings by avoiding RequiresDynamicCode on non-generic interface - Preserves backward compatibility with legacy TextSearchFilter path - Maintains modern LINQ expressions for generic ITextSearch<TRecord> interface Architectural separation: - Non-generic: TextSearchOptions → VectorSearchFilter (legacy path) - Generic: TextSearchOptions<TRecord> → Expression<Func<TRecord, bool>> (LINQ path) Resolves remaining IL3051 compilation errors while maintaining Issue microsoft#10456 objectives.

alzarei · 2025-10-17T13:54:58Z

ADR1 (PR2)

status: superseded-by-option-3 (ADR2)
contact: @alzarei
date: 2025-10-17
deciders: architecture-team
consulted: @westey-m, @roji
informed: @markwallace-microsoft

RequiresDynamicCode on ITextSearch Interface

Context and Problem Statement

VectorStoreTextSearch implementation of ITextSearch<TRecord> processes LINQ expressions directly, requiring RequiresDynamicCode attributes on its methods. However, the interface definition lacks these attributes, causing 31 IL3051 compilation errors:

IL3051: Member 'VectorStoreTextSearch.SearchAsync(...)' with 'RequiresDynamicCodeAttribute' 
implements interface member 'ITextSearch<TRecord>.SearchAsync(...)' without 'RequiresDynamicCodeAttribute'. 
Annotations must match across all interface implementations or overrides.

Issue #10456 aims to eliminate technical debt by modernizing from legacy TextSearchFilter to direct LINQ processing, but this creates an architectural mismatch between interface contracts and implementation requirements.

Decision Drivers

IL3051 Compliance: Resolve compilation errors preventing Issue .Net MEVD: Modify ITextSearch to react to the new LINQ-based vector search filtering #10456 completion
Interface Contract Accuracy: TextSearchOptions<TRecord>.Filter accepts Expression<Func<TRecord, bool>>
Performance: Direct LINQ processing eliminates conversion overhead in VectorStoreTextSearch
Architecture Consistency: Align with Microsoft.Extensions.VectorData patterns

Considered Options

Option A: Add RequiresDynamicCode to Interface (Recommended)

public interface ITextSearch<TRecord>
{
    [RequiresDynamicCode("LINQ filtering requires dynamic code generation for expression trees.")]
    Task<KernelSearchResults<string>> SearchAsync(string query, TextSearchOptions<TRecord>? searchOptions = null, CancellationToken cancellationToken = default);
    // ... other methods
}

Advantages:

Resolves IL3051 compilation errors
Interface contract matches implementation requirements
Enables direct LINQ processing in VectorStoreTextSearch
Documents AOT constraints accurately

Disadvantages:

Interface metadata change (runtime behavior unchanged)
Restricts AOT compilation scenarios

Option B: Convert VectorStoreTextSearch to Adapter Pattern

Force VectorStoreTextSearch to use LINQ → Legacy conversion like other implementations.

Advantages:

No interface changes required
Maintains AOT compatibility

Disadvantages:

Contradicts Issue .Net MEVD: Modify ITextSearch to react to the new LINQ-based vector search filtering #10456 objective of eliminating technical debt
Introduces unnecessary conversion overhead
Creates architectural inconsistency with MEVD foundation

Decision

Option A: Add RequiresDynamicCode to ITextSearch interface methods

Rationale

Interface Contract: TextSearchOptions<TRecord>.Filter is Expression<Func<TRecord, bool>>? - dynamic code generation is inherent to the contract
Performance: VectorStoreTextSearch is the primary implementation and benefits from direct LINQ processing
Precedent: Entity Framework uses same pattern - RequiresDynamicCode on interfaces with LINQ contracts
Compatibility: Adapter implementations can continue existing patterns with minimal changes

Implementation Impact

Immediate:

Resolves 31 IL3051 compilation errors
Enables direct LINQ processing in VectorStoreTextSearch
No runtime breaking changes for consumers

Implementation Requirements:

Add RequiresDynamicCode attribute to interface methods
Existing adapter implementations add attribute (reflects current behavior)
Update XML documentation for AOT implications

References

Issue #10456: Modify ITextSearch to use LINQ-based filtering
PR #10273: Foundation LINQ filtering implementation
.NET RequiresDynamicCodeAttribute

alzarei · 2025-10-20T09:31:01Z

ADR2 (PR2)

status: "accepted", decision makers: @markwallace-microsoft , @westey-m, date: 2025-10-23
contact: @alzarei
date: 2025-10-17
deciders: architecture-team
consulted: @westey-m @roji @markwallace-microsoft
informed:

Dual Interface Pattern for Text Search Filtering

Context and Problem Statement

Issue #10456 aims to modernize text search filtering by adding LINQ-based type safety. However, analysis revealed that forcing both generic and non-generic interfaces through LINQ conversion creates unnecessary technical debt and architectural violations.

Decision

Implement Dual Interface Pattern with Clear Separation:

Generic Interface: Modern LINQ Path

ITextSearch<TRecord> → TextSearchOptions<TRecord> → Expression<Func<TRecord, bool>> → Direct LINQ

Uses RequiresDynamicCode (inherent to LINQ expressions)
Provides compile-time type safety and IntelliSense
Targets modern applications requiring type safety

Non-Generic Interface: Legacy Compatibility Path

ITextSearch → TextSearchOptions → TextSearchFilter → VectorSearchFilter → Legacy Processing

No RequiresDynamicCode needed (no LINQ conversion)
Maintains backward compatibility for existing applications
Direct path to obsolete API for compatibility

Rationale

Technical Debt Clarity: We preserve legacy path for compatibility while providing modern alternative
Performance: Each interface uses its optimal processing path without conversion overhead
Architecture Separation: Clear distinction between legacy and modern approaches
IL3051 Resolution: Non-generic interface doesn't need LINQ, eliminating warnings
Developer Choice: Teams can migrate to generic interface when ready

Implementation Impact

Non-generic methods: Revert to direct VectorSearchFilter usage (original baseline approach)
Generic methods: Continue using direct LINQ expressions
No breaking changes: Existing code continues working unchanged
Clear migration path: Developers can upgrade interface when adopting type safety

This approach eliminates IL3051 warnings correctly while maintaining the intended architectural benefits of Issue #10456.

…arning suppressions around reflection operations in LINQ expression building - IL2075 (GetMethod) and IL2060 (MakeGenericMethod) warnings are acceptable for dynamic property access - Fixes GitHub Actions CI/CD pipeline failures where --warnaserror treats warnings as errors - Targeted suppressions with explanatory comments for maintainability

- Add UnconditionalSuppressMessage attributes for IL2075/IL2060 warnings during AOT analysis - Expand DynamicallyAccessedMembers to include PublicMethods for GetMethod reflection calls - Maintains RequiresDynamicCode attribute to properly indicate AOT incompatibility - Addresses AOT test failures where --warnaserror treats warnings as compilation errors The reflection-based LINQ expression building is inherently incompatible with AOT compilation, but these attributes allow the build system to handle this known limitation gracefully instead of failing with cryptic errors. Regular and AOT compilation phases require different suppression mechanisms - pragma directives for regular builds, UnconditionalSuppressMessage for AOT analysis.

dotnet/src/SemanticKernel.Core/Data/TextSearch/VectorStoreTextSearch.cs

alzarei · 2025-10-23T09:18:43Z

@roji @markwallace-microsoft @westey-m while waiting for the architecture decision input from you, I am entertaining and validating option 3.

Add ITextSearch<TRecord> generic interface with LINQ filtering while maintaining existing ITextSearch non-generic interface for backward compatibility. ## Background: Architectural Decision (Issue microsoft#10456) Three options considered: Option 1 (Native LINQ): Replace TextSearchFilter with Expression<Func<T, bool>> - Breaking change: requires user migration - Best long-term architecture Option 2 (Translation Layer): Convert TextSearchFilter to LINQ internally - Breaking change: RequiresDynamicCode propagates through API surface - Reflection overhead, AOT incompatible Option 3 (Dual Interface): Add ITextSearch<TRecord> alongside ITextSearch - No breaking changes - Maintains AOT compatibility - Uses obsolete VectorSearchFilter in legacy path (temporary during transition) ## Implementation ### Generic Interface - ITextSearch<TRecord> with 3 methods accepting TextSearchOptions<TRecord> - TextSearchOptions<TRecord> with Expression<Func<TRecord, bool>>? Filter - Explicit interface implementation in VectorStoreTextSearch<TRecord> ### Dual-Path Architecture Two independent code paths, no translation layer: Legacy path (non-generic): - ITextSearch with TextSearchOptions and TextSearchFilter (clause-based) - Uses VectorSearchOptions.OldFilter (obsolete) with pragma warning suppression - No dynamic code, AOT compatible - 10 existing tests unchanged Modern path (generic): - ITextSearch<TRecord> with TextSearchOptions<TRecord> and Expression filter - Uses VectorSearchOptions.Filter (LINQ native, not obsolete) - No dynamic code, AOT compatible - 7 new tests ## Changes - Added ITextSearch<TRecord> interface and TextSearchOptions<TRecord> class - Implemented dual interface in VectorStoreTextSearch<TRecord> - Deleted ~150 lines of Option 2 translation layer code - Removed all RequiresDynamicCode attributes - Removed DynamicallyAccessedMemberTypes.PublicMethods from 5 locations: - VectorStoreTextSearch.cs - TextSearchServiceCollectionExtensions.cs (3 methods) - TextSearchKernelBuilderExtensions.cs (1 method) - Deleted 7 Option 2 translation tests - Added 7 LINQ filtering tests - Added DataModelWithTags to test base - Reverted Program.cs to original state ## Files Changed 8 files, +144 insertions, -395 deletions - ITextSearch.cs - TextSearchOptions.cs (added generic class) - VectorStoreTextSearch.cs (removed translation layer, added dual interface) - TextSearchServiceCollectionExtensions.cs (removed PublicMethods annotation) - TextSearchKernelBuilderExtensions.cs (removed PublicMethods annotation) - VectorStoreTextSearchTestBase.cs (added DataModelWithTags) - VectorStoreTextSearchTests.cs (removed 7 tests, added 7 tests) - Program.cs in AotTests (removed suppression, restored tests)

markwallace-microsoft · 2025-10-24T08:00:49Z

@roji @markwallace-microsoft @westey-m while waiting for the architecture decision input from you, I am entertaining and validating option 3.

We typically follow the pattern of obsoleting the old API when we introduce the new pattern. This avoids breaking changes which are very disruptive for projects that have a transient dependency.

westey-m · 2025-10-24T09:30:34Z

@alzarei, I prefer a clean separation between the old and new abstractions. Being able to obsolete the old ones and point users at the new ones is definitely valuable. Then we can remove the obsoleted code after a period of time.

- Add [Obsolete] attribute to ITextSearch interface - Add pragma suppressions in production classes for backward compatibility - Add pragma suppressions in test/sample files - Follows Microsoft pattern: introduce new API, deprecate old - ITextSearch<TRecord> is the replacement with LINQ filtering Build: 0 errors, 0 warnings Tests: 1,581 passed

alzarei · 2025-10-25T08:14:17Z

@roji @westey-m @markwallace-microsoft

Thank you all for the excellent feedback! After your review and architectural guidance, we pivoted from Option 2 (translation layer, commits c2c783b - a802eaa) to Option 3 (dual interface pattern) in commit d1f2733.

This removed the ~150 lines of translation layer code that had the issues you identified. Current implementation:

Legacy path: Direct OldFilter usage with pragma (lines 284-289)
Modern path: Direct LINQ passthrough (line 305)
No dynamic code generation, no exception handling, no RequiresDynamicCode
The architectural discussion led to the current solution - Option 3 with [Obsolete] marking (commit e333850). Thank you for the input and guidance!

roji

@alzarei can you please push your changes into a single PR? I'm getting a bit confused as to which branch corresponds to which PRs (there are quite a few), and this PR represents incremental work on top of something else. This is all making reviewing quite complex.

Just to raise the obvious: this PR adds the new LINQ-based filter support to VectorStoreTextSearch, but all the other implementations are left implementing the now-obsolete non-generic ITextSearch; it's problematic for these implementations to not have a non-obsolete way of being used. I can see that ultimately a feature-text-search-linq is being targeted - it's indeed a good idea to do this work in a feature branch so that everything can be done across all providers and samples before releasing anything.

dotnet/src/SemanticKernel.Abstractions/Data/TextSearch/ITextSearch.cs

alzarei · 2025-10-26T20:24:30Z

@roji @westey-m @markwallace-microsoft

All review feedback has been addressed. PR is ready for final review.

Validation Results

All validation steps completed successfully:

Format Check - PASSED

dotnet format SK-dotnet.slnx --verify-no-changes

Status: PASSED (no formatting changes needed)

Build with Warnings as Errors - PASSED

dotnet build SK-dotnet.slnx --configuration Release --warnaserror

Errors: 0
Warnings: 0
Matches GitHub Actions CI/CD requirements

Core Unit Tests - PASSED

dotnet test SemanticKernel.UnitTests.csproj --configuration Release --no-build

Failed: 0
Passed: 1,581
Skipped: 0
Total: 1,581
Duration: 2 seconds

AOT Compatibility - PASSED

dotnet publish --configuration Release (SemanticKernel.AotTests)

Errors: 0
Warnings: 0
Successfully published to win-x64

@roji

- Add using System.Threading and System.Threading.Tasks directives - Replace fully-qualified type names with short names (Task, CancellationToken) - Remove repetitive documentation line about LINQ filtering Addresses inline code feedback from @roji in PR microsoft#13179

alzarei · 2025-10-28T08:09:27Z

@alzarei can you please push your changes into a single PR? ...

@roji Thanks for the feedback! Let me clarify the multi-PR strategy:

This is Part 2 of 6 in Issue #10456

PR Chain (all target feature-text-search-linq):

PR	Status	Scope
PR #13175	Merged (Oct 7)	Add `ITextSearch<TRecord>` interface
PR #13179	This PR	VectorStoreTextSearch
PR #13188	Ready	Migrate BingTextSearch
PR #13190	Ready	Migrate GoogleTextSearch
PR #13191	Validating changes	Migrate Tavily & Brave
PR #13194	Depends on PRs above	Update samples & docs

Your Concern About Obsolete Implementations

"all the other implementations are left implementing the now-obsolete non-generic ITextSearch"

You're absolutely right to raise this! PR3-PR6 address exactly this issue.

After all 6 PRs merge, every implementation will have both interfaces:

VectorStoreTextSearch: ITextSearch<TRecord> (modern) + ITextSearch (legacy)
BingTextSearch: ITextSearch<BingWebPage> (modern) + ITextSearch (legacy)
GoogleTextSearch: ITextSearch<GoogleWebPage> (modern) + ITextSearch (legacy)
TavilyTextSearch: ITextSearch<TavilyWebPage> (modern) + ITextSearch (legacy)
BraveTextSearch: ITextSearch<BraveWebPage> (modern) + ITextSearch (legacy)

Pattern (Option 3 - applied consistently in all PRs):

#pragma warning disable CS0618 // ITextSearch is obsolete - backward compatibility
public sealed class BingTextSearch : ITextSearch, ITextSearch<BingWebPage>
#pragma warning restore CS0618
{
    // Modern LINQ path
    Task<KernelSearchResults<string>> ITextSearch<BingWebPage>.SearchAsync(...) { }
    
    // Legacy path (backward compatible)
    Task<KernelSearchResults<string>> ITextSearch.SearchAsync(...) { }
}

Why Separate PRs?

Each connector has unique patterns: Bing REST URLs, Google JSON API, Tavily/Brave web search - distinct conversion logic
Easier review: 5 focused PRs (~15-30 files) vs 1 massive PR (~93 files)
Parallel development: Independent CI/CD validation per connector
Clear separation: Architecture (PR2) vs implementations (PR3-6)

Review Approach

Suggested: Review PR2 for the deprecation pattern architecture, then PR3-6 will apply it consistently across connectors.

Alternative: I can consolidate if you prefer to see everything in one PR.

Does this clarify the strategy? Happy to adjust if you have an approach in mind that makes it easier to reviewe!

Thanks!

alzarei · 2025-10-28T09:50:54Z

@westey-m @roji @markwallace-microsoft

The integration deployment went through successfully 🎉 and all feedback has been addressed. Feel free to take a look, and if everything looks good, let's move forward. I'm wrapping up the next PRs in this batch to stay aligned with the ADRs we made here. Thank you!

…arch.GetSearchResultsAsync (#13318) This PR enhances the type safety of the `ITextSearch<TRecord>` interface by changing the `GetSearchResultsAsync` method to return `KernelSearchResults<TRecord>` instead of `KernelSearchResults<object>`. This improvement eliminates the need for manual casting and provides better IntelliSense support for consumers. ## Motivation and Context The current implementation of `ITextSearch<TRecord>.GetSearchResultsAsync` returns `KernelSearchResults<object>`, which requires consumers to manually cast results to the expected type. This reduces type safety and degrades the developer experience by losing compile-time type checking and IntelliSense support. This change aligns the return type with the generic type parameter `TRecord`, providing the expected strongly-typed results that users of a generic interface would anticipate. ## Changes Made ### Interface (ITextSearch.cs) - Changed `ITextSearch<TRecord>.GetSearchResultsAsync` return type from `KernelSearchResults<object>` to `KernelSearchResults<TRecord>` - Updated XML documentation to reflect strongly-typed return value - Legacy `ITextSearch` interface (non-generic) remains unchanged, continuing to return `KernelSearchResults<object>` for backward compatibility ### Implementation (VectorStoreTextSearch.cs) - Added new `GetResultsAsTRecordAsync` helper method returning `IAsyncEnumerable<TRecord>` - Updated generic interface implementation to use the new strongly-typed helper - Retained `GetResultsAsRecordAsync` method for the legacy non-generic interface ### Tests (VectorStoreTextSearchTests.cs) - Updated 3 unit tests to use strongly-typed `DataModel` or `DataModelWithRawEmbedding` instead of `object` - Improved test assertions to leverage direct property access without casting - All 19 tests pass successfully ## Breaking Changes **Interface Change (Experimental API):** - `ITextSearch<TRecord>.GetSearchResultsAsync` now returns `KernelSearchResults<TRecord>` instead of `KernelSearchResults<object>` - This interface is marked with `[Experimental("SKEXP0001")]`, indicating that breaking changes are expected during the preview period - Legacy `ITextSearch` interface (non-generic) is unaffected and maintains full backward compatibility ## Benefits - **Improved Type Safety**: Eliminates runtime casting errors by providing compile-time type checking - **Enhanced Developer Experience**: Full IntelliSense support for TRecord properties and methods - **Cleaner Code**: Consumers no longer need to cast results from object to the expected type - **Consistent API Design**: Generic interface now behaves as expected, returning strongly-typed results - **Zero Impact on Legacy Code**: Non-generic ITextSearch interface remains unchanged ## Testing - All 19 existing unit tests pass - Updated tests demonstrate improved type safety with direct property access - Verified both generic and legacy interfaces work correctly - Confirmed zero breaking changes to non-generic ITextSearch consumers ## Related Work This PR is part of the Issue #10456 multi-PR chain for modernizing ITextSearch with LINQ-based filtering: - PR #13175: Foundation (ITextSearch<TRecord> interface) - Merged - PR #13179: VectorStoreTextSearch + deprecation pattern - In Review - **This PR (2.1)**: API refinement for improved type safety - PR #13188-13191: Connector migrations (Bing, Google, Tavily, Brave) - Pending - PR #13194: Samples and documentation - Pending All PRs target the `feature-text-search-linq` branch for coordinated release. ## Migration Guide for Consumers ### Before (Previous API) ```csharp ITextSearch<DataModel> search = ...; KernelSearchResults<object> results = await search.GetSearchResultsAsync("query", options); foreach (var obj in results.Results) { var record = (DataModel)obj; // Manual cast required Console.WriteLine(record.Name); } ``` ### After (Improved API) ```csharp ITextSearch<DataModel> search = ...; KernelSearchResults<DataModel> results = await search.GetSearchResultsAsync("query", options); foreach (var record in results.Results) // Strongly typed! { Console.WriteLine(record.Name); // Direct property access with IntelliSense } ``` ## Checklist - [x] Changes build successfully - [x] All unit tests pass (19/19) - [x] XML documentation updated - [x] Breaking change documented (experimental API only) - [x] Legacy interface backward compatibility maintained - [x] Code follows project coding standards Co-authored-by: Alexander Zarei <alzarei@users.noreply.github.com>

moonbox3 added the .NET Issue or Pull requests regarding .NET code label Sep 25, 2025

alzarei marked this pull request as ready for review September 25, 2025 09:06

alzarei requested a review from a team as a code owner September 25, 2025 09:06

alzarei force-pushed the feature-text-search-linq-pr2 branch from 0e78309 to 3c9fc7b Compare September 26, 2025 05:44

alzarei closed this Sep 26, 2025

alzarei deleted the feature-text-search-linq-pr2 branch September 26, 2025 05:46

alzarei restored the feature-text-search-linq-pr2 branch September 26, 2025 05:49

alzarei deleted the feature-text-search-linq-pr2 branch September 26, 2025 05:52

alzarei restored the feature-text-search-linq-pr2 branch September 26, 2025 05:56

alzarei reopened this Sep 26, 2025

alzarei mentioned this pull request Sep 27, 2025

.Net: feat: Modernize BingTextSearch connector with ITextSearch<TRecord> interface (microsoft#10456) #13188

Merged

moonbox3 added the kernel Issues or pull requests impacting the core kernel label Sep 28, 2025

alzarei mentioned this pull request Sep 29, 2025

.Net MEVD: Modify ITextSearch to react to the new LINQ-based vector search filtering #10456

Open

alzarei added 2 commits September 30, 2025 23:30

test: Add null filter test case and cleanup unused using statement

0ba75b2

- Add NullFilterReturnsAllResultsAsync test to verify behavior when no filter is applied - Remove unnecessary Microsoft.Extensions.VectorData using statement - Enhance test coverage for VectorStoreTextSearch edge cases

alzarei had a problem deploying to integration October 10, 2025 09:40 — with GitHub Actions Failure

alzarei added 2 commits October 16, 2025 01:44

markwallace-microsoft added the kernel.core label Oct 16, 2025

alzarei had a problem deploying to integration October 21, 2025 06:26 — with GitHub Actions Failure

alzarei had a problem deploying to integration October 21, 2025 08:13 — with GitHub Actions Failure

westey-m reviewed Oct 21, 2025

View reviewed changes

dotnet/src/SemanticKernel.Core/Data/TextSearch/VectorStoreTextSearch.cs Outdated Show resolved Hide resolved

alzarei force-pushed the feature-text-search-linq-pr2 branch from 837cf0e to a802eaa Compare October 22, 2025 12:24

alzarei requested review from roji and westey-m October 22, 2025 12:25

alzarei had a problem deploying to integration October 24, 2025 08:01 — with GitHub Actions Failure

alzarei temporarily deployed to integration October 24, 2025 09:15 — with GitHub Actions Inactive

alzarei force-pushed the feature-text-search-linq-pr2 branch from 88244e4 to e333850 Compare October 25, 2025 06:40

alzarei changed the title ~~.Net: feat: Eliminate obsolete VectorSearchFilter technical debt in VectorStoreTextSearch (microsoft#10456)~~ .Net: feat: Add ITextSearch<TRecord> with LINQ filtering and deprecate legacy ITextSearch (microsoft#10456) Oct 25, 2025

alzarei changed the title ~~.Net: feat: Add ITextSearch<TRecord> with LINQ filtering and deprecate legacy ITextSearch (microsoft#10456)~~ .NET: feat: Add ITextSearch<TRecord> with LINQ filtering and deprecate legacy ITextSearch (microsoft#10456) Oct 25, 2025

alzarei changed the title ~~.NET: feat: Add ITextSearch<TRecord> with LINQ filtering and deprecate legacy ITextSearch (microsoft#10456)~~ .NET: Add ITextSearch<TRecord> with LINQ filtering and deprecate legacy ITextSearch (#10456) Oct 25, 2025

roji reviewed Oct 26, 2025

View reviewed changes

dotnet/src/SemanticKernel.Abstractions/Data/TextSearch/ITextSearch.cs Outdated Show resolved Hide resolved

dotnet/src/SemanticKernel.Abstractions/Data/TextSearch/ITextSearch.cs Outdated Show resolved Hide resolved

alzarei requested a review from roji October 28, 2025 08:11

alzarei temporarily deployed to integration October 28, 2025 09:24 — with GitHub Actions Inactive

westey-m approved these changes Oct 28, 2025

View reviewed changes

markwallace-microsoft approved these changes Oct 29, 2025

View reviewed changes

markwallace-microsoft merged commit ebd579d into microsoft:feature-text-search-linq Oct 29, 2025
10 checks passed

alzarei mentioned this pull request Oct 31, 2025

.Net Improve type safety: Return TRecord instead of object in ITextSearch.GetSearchResultsAsync #13318

Merged

6 tasks

alzarei mentioned this pull request Nov 5, 2025

.Net: docs: Add ADR-0065 for LINQ-based ITextSearch filtering migration strategy #13335

Open

7 tasks

This was referenced Nov 23, 2025

.Net: feat: Add LINQ-based filtering for ITextSearch with type safety and migration path (#10456) #13381

Closed

.Net: Feature text search linq #13384

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

.NET: Add ITextSearch<TRecord> with LINQ filtering and deprecate legacy ITextSearch (#10456) #13179

.NET: Add ITextSearch<TRecord> with LINQ filtering and deprecate legacy ITextSearch (#10456) #13179

Uh oh!

alzarei commented Sep 25, 2025 •

edited

Loading

Uh oh!

alzarei commented Oct 17, 2025 •

edited

Loading

Uh oh!

alzarei commented Oct 20, 2025 •

edited

Loading

Uh oh!

Uh oh!

alzarei commented Oct 23, 2025

Uh oh!

markwallace-microsoft commented Oct 24, 2025

Uh oh!

westey-m commented Oct 24, 2025

Uh oh!

alzarei commented Oct 25, 2025

Uh oh!

roji left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

alzarei commented Oct 26, 2025

Uh oh!

alzarei commented Oct 28, 2025 •

edited

Loading

Uh oh!

alzarei commented Oct 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

.NET: Add ITextSearch<TRecord> with LINQ filtering and deprecate legacy ITextSearch (#10456) #13179

.NET: Add ITextSearch<TRecord> with LINQ filtering and deprecate legacy ITextSearch (#10456) #13179

Uh oh!

Conversation

alzarei commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

.NET: Add LINQ-based ITextSearch interface and deprecate legacy ITextSearch (#10456)

Summary

What Changed

New Generic Interface (Recommended Path)

Legacy Interface (Deprecated)

Implementation Details

Dual-Path Architecture

Files Changed

Interfaces & Options

Implementation

Backward Compatibility (Pragma Suppressions)

Tests

Validation Results

Breaking Changes

Multi-PR Context

Architectural Decision

Options Considered:

Migration Guide

Next Steps

Uh oh!

alzarei commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ADR1 (PR2)

status: superseded-by-option-3 (ADR2) contact: @alzarei date: 2025-10-17 deciders: architecture-team consulted: @westey-m, @roji informed: @markwallace-microsoft

RequiresDynamicCode on ITextSearch Interface

Context and Problem Statement

Decision Drivers

Considered Options

Option A: Add RequiresDynamicCode to Interface (Recommended)

Option B: Convert VectorStoreTextSearch to Adapter Pattern

Decision

Rationale

Implementation Impact

References

Uh oh!

alzarei commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ADR2 (PR2)

status: "accepted", decision makers: @markwallace-microsoft , @westey-m, date: 2025-10-23 contact: @alzarei date: 2025-10-17 deciders: architecture-team consulted: @westey-m @roji @markwallace-microsoft informed:

Dual Interface Pattern for Text Search Filtering

Context and Problem Statement

Decision

Generic Interface: Modern LINQ Path

Non-Generic Interface: Legacy Compatibility Path

Rationale

Implementation Impact

Uh oh!

Uh oh!

alzarei commented Oct 23, 2025

Uh oh!

markwallace-microsoft commented Oct 24, 2025

Uh oh!

westey-m commented Oct 24, 2025

Uh oh!

alzarei commented Oct 25, 2025

Uh oh!

roji left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

alzarei commented Oct 26, 2025

Validation Results

Uh oh!

alzarei commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

This is Part 2 of 6 in Issue #10456

Your Concern About Obsolete Implementations

Why Separate PRs?

Review Approach

Uh oh!

alzarei commented Oct 28, 2025

Uh oh!

alzarei commented Sep 25, 2025 •

edited

Loading

alzarei commented Oct 17, 2025 •

edited

Loading

status: superseded-by-option-3 (ADR2)
contact: @alzarei
date: 2025-10-17
deciders: architecture-team
consulted: @westey-m, @roji
informed: @markwallace-microsoft

alzarei commented Oct 20, 2025 •

edited

Loading

status: "accepted", decision makers: @markwallace-microsoft , @westey-m, date: 2025-10-23
contact: @alzarei
date: 2025-10-17
deciders: architecture-team
consulted: @westey-m @roji @markwallace-microsoft
informed:

roji left a comment •

edited

Loading

alzarei commented Oct 28, 2025 •

edited

Loading