Skip to content

Commit 2a5f88e

Browse files
maumarmaumar
committed
moving fts docs to providers and only mentioning it briefly in whats new, updating vector search page, addressing cr feedback
1 parent 595bc5f commit 2a5f88e

File tree

3 files changed

+142
-149
lines changed

3 files changed

+142
-149
lines changed

‎entity-framework/core/providers/cosmos/full-text-search.md

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
---
2+
title: Full Text Search - Azure Cosmos DB Provider - EF Core
3+
description: Full text search with the Azure Cosmos DB EF Core Provider
4+
author: maumar
5+
ms.date: 04/19/2025
6+
uid: core/providers/cosmos/full-text-search
7+
---
8+
# Full text search
9+
10+
Azure Cosmos DB now offers support for [full-text search](/azure/cosmos-db/gen-ai/full-text-search). It enables efficient and effective text searches using advanced techniques like stemming, as well as evaluating the relevance of documents to a given search query. It can be used in combination with vector search (i.e. hybrid search) to improve the accuracy of responses in some AI scenarios.
11+
EF Core allows for modeling the database with full-text search enabled properties and using full-text search functions inside queries targeting Azure Cosmos DB.
12+
13+
## Model configuration
14+
15+
A property can be configured inside `OnModelCreating` to use full-text search by enabling it for the property and defining a full-text index:
16+
17+
```c#
18+
public class Blog
19+
{
20+
...
21+
22+
public string Contents { get; set; }
23+
}
24+
25+
public class BloggingContext
26+
{
27+
...
28+
29+
protected override void OnModelCreating(ModelBuilder modelBuilder)
30+
{
31+
modelBuilder.Entity<Blog>(b =>
32+
{
33+
b.Property(x => x.Contents).EnableFullTextSearch();
34+
b.HasIndex(x => x.Contents).IsFullTextIndex();
35+
});
36+
}
37+
}
38+
```
39+
40+
> [!NOTE]
41+
> Configuring the index is not mandatory, but it is recommended as it greatly improves performance of full-text search queries.
42+
43+
Full-text search operations are language specific, using American English (`en-US`) by default. You can customize the language for individual properties as part of `EnableFullTextSearch` call:
44+
45+
```c#
46+
protected override void OnModelCreating(ModelBuilder modelBuilder)
47+
{
48+
modelBuilder.Entity<Blog>(b =>
49+
{
50+
b.Property(x => x.Contents).EnableFullTextSearch();
51+
b.HasIndex(x => x.Contents).IsFullTextIndex();
52+
b.Property(x => x.ContentsGerman).EnableFullTextSearch("de-DE");
53+
b.HasIndex(x => x.ContentsGerman).IsFullTextIndex();
54+
});
55+
}
56+
```
57+
58+
You can also set a default language for the container - unless overridden in the `EnableFullTextSearch` method, all full-text properties inside the container will use that language.
59+
60+
```c#
61+
protected override void OnModelCreating(ModelBuilder modelBuilder)
62+
{
63+
modelBuilder.Entity<Blog>(b =>
64+
{
65+
b.HasDefaultFullTextLanguage("de-DE");
66+
b.Property(x => x.ContentsEnglish).EnableFullTextSearch("en-US");
67+
b.HasIndex(x => x.ContentsEnglish).IsFullTextIndex();
68+
b.Property(x => x.ContentsGerman).EnableFullTextSearch();
69+
b.HasIndex(x => x.ContentsGerman).IsFullTextIndex();
70+
b.Property(x => x.TagsGerman).EnableFullTextSearch();
71+
b.HasIndex(x => x.TagsGerman).IsFullTextIndex();
72+
});
73+
}
74+
```
75+
76+
> [!NOTE]
77+
> Configuring the index is not mandatory, but it is recommended as it greatly improves performance of full-text search queries.
78+
79+
## Querying
80+
81+
As part of the full-text search feature, Azure Cosmos DB introduced several built-in functions which allow for efficient querying of content inside the full-text search enabled properties. These functions are: [`FullTextContains`](/azure/cosmos-db/nosql/query/fulltextcontains), [`FullTextContainsAll`](/azure/cosmos-db/nosql/query/fulltextcontainsall), [`FullTextContainsAny`](/azure/cosmos-db/nosql/query/fulltextcontainsany), which look for specific keyword or keywords and [`FullTextScore`](/azure/cosmos-db/nosql/query/fulltextscore), which returns [BM25 score](https://en.wikipedia.org/wiki/Okapi_BM25) based on provided keywords.
82+
83+
> [!NOTE]
84+
> `FullTextScore` can only be used inside `OrderBy` to rank the documents based on the score.
85+
86+
EF Core exposes these functions as part of `EF.Functions` so they can be used in queries:
87+
88+
```c#
89+
var cosmosBlogs = await context.Blogs.Where(x => EF.Functions.FullTextContainsAll(x.Contents, "database", "cosmos")).ToListAsync();
90+
91+
var keywords = new string[] { "AI", "agent", "breakthrough" };
92+
var mostInteresting = await context.Blogs.OrderBy(x => EF.Functions.FullTextScore(x.Contents, keywords)).Take(5).ToListAsync();
93+
```
94+
95+
## Hybrid search
96+
97+
Full-text search can be used with vector search in the same query (i.e. hybrid search), by combining results of `FullTextScore` and `VectorDistance` functions. It can be done using the [`RRF`](/azure/cosmos-db/nosql/query/rrf) (Reciprocal Rank Fusion) function, which EF Core also provides inside `EF.Functions`:
98+
99+
```c#
100+
public class Blog
101+
{
102+
...
103+
104+
public float[] Vector { get; set; }
105+
public string Contents { get; set; }
106+
}
107+
108+
public class BloggingContext
109+
{
110+
...
111+
112+
protected override void OnModelCreating(ModelBuilder modelBuilder)
113+
{
114+
modelBuilder.Entity<Blog>(b =>
115+
{
116+
b.Property(x => x.Contents).EnableFullTextSearch();
117+
b.HasIndex(x => x.Contents).IsFullTextIndex();
118+
119+
b.Property(x => x.Vector).IsVectorProperty(DistanceFunction.Cosine, dimensions: 1536);
120+
b.HasIndex(x => x.Vector).IsVectorIndex(VectorIndexType.Flat);
121+
});
122+
}
123+
}
124+
125+
float[] myVector = /* generate vector data from text, image, etc. */
126+
var hybrid = await context.Blogs.OrderBy(x => EF.Functions.Rrf(
127+
EF.Functions.FullTextScore(x.Contents, "database"),
128+
EF.Functions.VectorDistance(x.Vector, myVector)))
129+
.Take(10)
130+
.ToListAsync();
131+
```
132+
133+
> [!TIP]
134+
> You can combine more than two scoring functions inside `Rrf` call, as well as using only `FullTextScore`, or only `VectorDistance`.

‎entity-framework/core/providers/cosmos/vector-search.md

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -7,14 +7,9 @@ uid: core/providers/cosmos/vector-search
77
---
88
# Vector search
99

10-
> [!WARNING]
11-
> Azure Cosmos DB vector search is currently in preview. As a result, using EF's vector search APIs will generate an "experimental API" warning (`EF9103`) which must be suppressed. The APIs and capabilities may change in breaking ways in the future.
10+
Azure Cosmos DB now offers support for vector similarity search. Vector search is a fundamental part of some application types, including AI, semantic search and others. Azure Cosmos DB allows you to store vectors directly in your documents alongside the rest of your data, meaning you can perform all of your queries against a single database. This can considerably simplify your architecture and remove the need for an additional, dedicated vector database solution in your stack. To learn more about Azure Cosmos DB vector search, [see the documentation](/azure/cosmos-db/nosql/vector-search).
1211

13-
Azure Cosmos DB now offers preview support for vector similarity search. Vector search is a fundamental part of some application types, including AI, semantic search and others. Azure Cosmos DB allows you to store vectors directly in your documents alongside the rest of your data, meaning you can perform all of your queries against a single database. This can considerably simplify your architecture and remove the need for an additional, dedicated vector database solution in your stack. To learn more about Azure Cosmos DB vector search, [see the documentation](/azure/cosmos-db/nosql/vector-search).
14-
15-
To use vector search, you must first [enroll in the preview feature](/azure/cosmos-db/nosql/vector-search#enroll-in-the-vector-search-preview-feature). Then, [define vector policies on your container](/azure/cosmos-db/nosql/vector-search#container-vector-policies) to identify which JSON properties in your documents contain vectors and vector-related information for those properties (dimensions, data type, distance function).
16-
17-
Once your container is properly set up, add a vector property to your model in the path you defined in the container policy, and configure it with EF as a vector:
12+
Vector property can be configured inside `OnModelCreating`:
1813

1914
```c#
2015
public class Blog
@@ -30,9 +25,11 @@ public class BloggingContext
3025

3126
protected override void OnModelCreating(ModelBuilder modelBuilder)
3227
{
33-
modelBuilder.Entity<Blog>()
34-
.Property(b => b.Embeddings)
35-
.IsVector(DistanceFunction.Cosine, dimensions: 1536);
28+
modelBuilder.Entity<Blog>(b =>
29+
{
30+
b.Property(b => b.Vector).IsVectorProperty(DistanceFunction.Cosine, dimensions: 1536);
31+
b.HasIndex(x => x.Vector).IsVectorIndex(VectorIndexType.Flat);
32+
});
3633
}
3734
}
3835
```

‎entity-framework/core/what-is-new/ef-core-10.0/whatsnew.md

Lines changed: 1 addition & 139 deletions
Original file line numberDiff line numberDiff line change
@@ -26,145 +26,7 @@ EF10 requires the .NET 10 SDK to build and requires the .NET 10 runtime to run.
2626
### Full-text search support
2727

2828
Azure Cosmos DB now offers support for [full-text search](/azure/cosmos-db/gen-ai/full-text-search). It enables efficient and effective text searches, as well as evaluating the relevance of documents to a given search query. It can be used in combination with vector search to improve the accuracy of responses in some AI scenarios.
29-
EF Core 10 is adding support for this feature allowing for modeling the database with full-text search enabled properties and using full-text search functions inside queries targeting Azure Cosmos DB.
30-
31-
<a name="full-text-search-model-configuration"></a>
32-
33-
#### Model configuration
34-
35-
A property can be configured inside `OnModelCreating` to use full-text search by enabling it for the property and defining a full-text index:
36-
37-
```c#
38-
public class Blog
39-
{
40-
...
41-
42-
public string Contents { get; set; }
43-
}
44-
45-
public class BloggingContext
46-
{
47-
...
48-
49-
protected override void OnModelCreating(ModelBuilder modelBuilder)
50-
{
51-
modelBuilder.Entity<Blog>(b =>
52-
{
53-
b.Property(x => x.Contents).EnableFullTextSearch();
54-
b.HasIndex(x => x.Contents).IsFullTextIndex();
55-
});
56-
}
57-
}
58-
```
59-
60-
> [!NOTE]
61-
> Configuring the index is not mandatory, but it is recommended as it greatly improves performance of full-text search queries.
62-
63-
Full-text search operations are language specific, using American English (`en-US`) by default. You can customize the language for individual properties as part of `EnableFullTextSearch` call:
64-
65-
```c#
66-
protected override void OnModelCreating(ModelBuilder modelBuilder)
67-
{
68-
modelBuilder.Entity<Blog>(b =>
69-
{
70-
b.Property(x => x.Contents).EnableFullTextSearch();
71-
b.HasIndex(x => x.Contents).IsFullTextIndex();
72-
b.Property(x => x.ContentsGerman).EnableFullTextSearch("de-DE");
73-
b.HasIndex(x => x.ContentsGerman).IsFullTextIndex();
74-
});
75-
}
76-
```
77-
78-
You can also set a default language for the container - unless overridden in the `EnableFullTextSearch` method, all full-text properties inside the container will use that language.
79-
80-
```c#
81-
protected override void OnModelCreating(ModelBuilder modelBuilder)
82-
{
83-
modelBuilder.Entity<Blog>(b =>
84-
{
85-
b.HasDefaultFullTextLanguage("de-DE");
86-
b.Property(x => x.ContentsEnglish).EnableFullTextSearch("en-US");
87-
b.HasIndex(x => x.ContentsEnglish).IsFullTextIndex();
88-
b.Property(x => x.ContentsGerman).EnableFullTextSearch();
89-
b.HasIndex(x => x.ContentsGerman).IsFullTextIndex();
90-
b.Property(x => x.TagsGerman).EnableFullTextSearch();
91-
b.HasIndex(x => x.TagsGerman).IsFullTextIndex();
92-
});
93-
}
94-
```
95-
96-
<a name="full-text-search-querying"></a>
97-
98-
#### Querying
99-
100-
As part of the full-text search feature, Azure Cosmos DB introduced several built-in functions which allow for efficient querying of content inside the full-text search enabled properties. These functions are: `FullTextContains`, `FullTextContainsAll`, `FullTextContainsAny`, which look for specific keyword or keywords and `FullTextScore`, which returns [BM25 score](https://en.wikipedia.org/wiki/Okapi_BM25) based on provided keywords.
101-
102-
> [!NOTE]
103-
> `FullTextScore` can only be used inside `OrderBy` to rank the documents based on the score.
104-
105-
EF Core exposes these functions as part of `EF.Functions` so they can be used in queries:
106-
107-
```c#
108-
var cosmosBlogs = await context.Blogs.Where(x => EF.Functions.FullTextContainsAll(x.Contents, "database", "cosmos")).ToListAsync();
109-
110-
var keywords = new string[] { "AI", "agent", "breakthrough" };
111-
var mostInteresting = await context.Blogs.OrderBy(x => EF.Functions.FullTextScore(x.Contents, keywords)).Take(5).ToListAsync();
112-
```
113-
114-
<a name="full-text-search-hybrid-search"></a>
115-
116-
#### Hybrid search
117-
118-
Full-text search can be used with vector search in the same query (i.e. hybrid search), by combining results of `FullTextScore` and `VectorDistance` functions. It can be done using the `RRF` function (Reciprocal Rank Fusion), which EF Core also provides inside `EF.Functions`:
119-
120-
```c#
121-
public class Blog
122-
{
123-
...
124-
125-
public float[] Vector { get; set; }
126-
public string Contents { get; set; }
127-
}
128-
129-
public class BloggingContext
130-
{
131-
...
132-
133-
protected override void OnModelCreating(ModelBuilder modelBuilder)
134-
{
135-
modelBuilder.Entity<Blog>(b =>
136-
{
137-
b.Property(x => x.Contents).EnableFullTextSearch();
138-
b.HasIndex(x => x.Contents).IsFullTextIndex();
139-
140-
b.Property(x => x.Vector).IsVectorProperty(DistanceFunction.Cosine, dimensions: 1536);
141-
b.HasIndex(x => x.Vector).IsVectorIndex(VectorIndexType.Flat);
142-
});
143-
}
144-
}
145-
146-
float[] myVector = ...
147-
var hybrid = await context.Blogs.OrderBy(x => EF.Functions.Rrf(
148-
EF.Functions.FullTextScore(x.Contents, "database"),
149-
EF.Functions.VectorDistance(x.Vector, myVector)))
150-
.Take(10)
151-
.ToListAsync();
152-
```
153-
154-
> [!TIP]
155-
> You can combine more than two scoring functions inside `Rrf` call, as well as using only `FullTextScore`, or only `VectorDistance`.
156-
157-
<a name="full-text-search-limitations"></a>
158-
159-
#### Limitations
160-
161-
Here are some current limitations of the full-text search using EF Core:
162-
163-
- `FullTextScore` and `RRF` functions can only be used inside `OrderBy` and are mutually exclusive with other forms of ordering.
164-
- EF Core can't create a container with full-text search or vector properties defined inside collection navigation - you can use those properties in queries, but the database has to be generated by other means (e.g. Microsoft.Azure.Cosmos SDK or using UI on Azure Portal).
165-
- Keyword arguments for `FullTextContainsAll`, `FullTextContainsAny` and `FullTextScore` have to be either constants or parameters. `FullTextContains` allows more complex expressions.
166-
167-
<a name="vector-search-exits-preview"></a>
29+
EF Core 10 is adding support for this feature allowing for modeling the database with full-text search enabled properties and using full-text search functions inside queries targeting Azure Cosmos DB. See [documentation](xref:core/providers/cosmos/full-text-search) to learn how to take advantage of full-text search using EF Core.
16830

16931
### Vector similarity search exits preview
17032

0 commit comments

Comments
 (0)