From d97bf28629c21b2ad439fc6f921b01dd52e5067c Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Wed, 10 Jan 2024 16:48:44 -0500 Subject: [PATCH 01/59] [AV-68976] Adjust all UI-based search index creation topics to match Figma designs. --- ...ld-field.adoc => .create-child-field.adoc} | 0 ...apping.adoc => .create-child-mapping.adoc} | 0 .../search/pages/create-custom-analyzer.adoc | 25 +- .../pages/create-custom-character-filter.adoc | 19 +- .../pages/create-custom-date-time-parser.adoc | 19 +- .../pages/create-custom-token-filter.adoc | 39 +-- .../search/pages/create-custom-tokenizer.adoc | 19 +- .../search/pages/create-custom-wordlist.adoc | 19 +- modules/search/pages/create-quick-index.adoc | 56 ++++ .../search/pages/create-search-index-ui.adoc | 52 ++-- .../search/pages/create-search-indexes.adoc | 11 +- modules/search/pages/create-type-mapping.adoc | 289 +++++++++++++++--- modules/search/pages/customize-index.adoc | 70 +++-- .../pages/field-data-types-reference.adoc | 6 +- modules/search/pages/geo-search-ui.adoc | 47 ++- .../quick-index-supported-languages.adoc | 62 ++++ .../search-query-auto-complete-code.adoc | 8 +- .../pages/search-query-auto-complete-ui.adoc | 48 +-- .../search/pages/set-advanced-settings.adoc | 23 +- modules/search/pages/set-type-identifier.adoc | 76 ++--- modules/search/partials/nav.adoc | 44 +-- 21 files changed, 663 insertions(+), 269 deletions(-) rename modules/search/pages/{create-child-field.adoc => .create-child-field.adoc} (100%) rename modules/search/pages/{create-child-mapping.adoc => .create-child-mapping.adoc} (100%) create mode 100644 modules/search/pages/create-quick-index.adoc create mode 100644 modules/search/pages/quick-index-supported-languages.adoc diff --git a/modules/search/pages/create-child-field.adoc b/modules/search/pages/.create-child-field.adoc similarity index 100% rename from modules/search/pages/create-child-field.adoc rename to modules/search/pages/.create-child-field.adoc diff --git a/modules/search/pages/create-child-mapping.adoc b/modules/search/pages/.create-child-mapping.adoc similarity index 100% rename from modules/search/pages/create-child-mapping.adoc rename to modules/search/pages/.create-child-mapping.adoc diff --git a/modules/search/pages/create-custom-analyzer.adoc b/modules/search/pages/create-custom-analyzer.adoc index 7b3fc0af1..68acabab1 100644 --- a/modules/search/pages/create-custom-analyzer.adoc +++ b/modules/search/pages/create-custom-analyzer.adoc @@ -5,23 +5,33 @@ [abstract] {description} -For more information, see xref:customize-index.adoc#analyzers[]. +NOTE: You must use Advanced Mode to add a custom analyzer to your Search index. +For more information, see xref:create-search-index-ui.adoc[]. + +For more information, see xref:customize-index.adoc#analyzers[Search Index Features]. == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. + +* You have created an index with xref:create-search-index-ui.adoc[Advanced Mode]. + +* You have logged in to the Couchbase Capella UI. == Procedure -To create a custom analyzer with the Capella UI: +To create a custom analyzer with the Capella UI in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index where you want to create a custom analyzer. -. Expand *Analyzers*. +. Under *Advanced Settings*, expand *Analyzers*. ++ +NOTE: Make sure you use *Advanced Mode*. . Click btn:[Add Analyzer]. . In the *Name* field, enter a name for the new custom analyzer. . To remove specific characters from search input before tokenizing, in the *Character Filters* list, do one of the following: @@ -46,12 +56,13 @@ For more information, see xref:default-token-filters-reference.adoc[] or xref:cr After you create a custom analyzer, you can xref:set-advanced-settings.adoc#default-analyzer[set it as the default analyzer] for your Search index. -You can also use the custom analyzer when you xref:create-type-mapping.adoc[], xref:create-child-mapping.adoc[], and xref:create-child-field.adoc[]. +You can also use the custom analyzer when you xref:create-type-mapping.adoc[create a type mapping with Advanced Mode]. To continue customizing your Search index, you can also: * xref:set-advanced-settings.adoc[] * xref:set-type-identifier.adoc[] +* xref:create-type-mapping.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-tokenizer.adoc[] * xref:create-custom-token-filter.adoc[] diff --git a/modules/search/pages/create-custom-character-filter.adoc b/modules/search/pages/create-custom-character-filter.adoc index 3277676ff..f992fdccf 100644 --- a/modules/search/pages/create-custom-character-filter.adoc +++ b/modules/search/pages/create-custom-character-filter.adoc @@ -7,19 +7,26 @@ == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. + +* You have logged in to the Couchbase Capella UI. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a custom character filter with the Capella UI: +To create a custom character filter with the Capella UI in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index where you want to create a custom character filter. -. Expand *Custom Filters*. +. Under *Advanced Settings*, expand *Custom Filters*. ++ +NOTE: Make sure you use *Advanced Mode*. . Click btn:[Add Character Filter]. . In the *Name* field, enter a name for the character filter. . In the *Regular Expression* field, enter the regular expression for the character filter. @@ -35,8 +42,6 @@ To continue customizing your Search index, you can also: * xref:set-advanced-settings.adoc[] * xref:set-type-identifier.adoc[] * xref:create-type-mapping.adoc[] -* xref:create-child-field.adoc[] -* xref:create-child-mapping.adoc[] * xref:create-custom-tokenizer.adoc[] * xref:create-custom-token-filter.adoc[] * xref:create-custom-wordlist.adoc[] diff --git a/modules/search/pages/create-custom-date-time-parser.adoc b/modules/search/pages/create-custom-date-time-parser.adoc index 7317adf67..a16ced107 100644 --- a/modules/search/pages/create-custom-date-time-parser.adoc +++ b/modules/search/pages/create-custom-date-time-parser.adoc @@ -9,19 +9,26 @@ If you store date data in a format other than RFC-3339 (ISO-8601), then you need == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. + +* You have logged in to the Couchbase Capella UI. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a custom date/time parser with the Capella UI: +To create a custom date/time parser with the Capella UI in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index where you want to create a custom date/time parser. -. Expand *Date/Time Parsers*. +. Under *Advanced Settings*, expand *Date Time Parsers*. ++ +NOTE: Make sure you use *Advanced Mode*. . Click btn:[Add Date/Time Parser] . In the *Name* field, enter a name for the date/time parser. . In the *New Word* field, enter a date/time layout with Go syntax. @@ -40,8 +47,6 @@ To continue customizing your Search index, you can also: * xref:set-advanced-settings.adoc[] * xref:set-type-identifier.adoc[] * xref:create-type-mapping.adoc[] -* xref:create-child-field.adoc[] -* xref:create-child-mapping.adoc[] * xref:create-custom-analyzer.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-tokenizer.adoc[] diff --git a/modules/search/pages/create-custom-token-filter.adoc b/modules/search/pages/create-custom-token-filter.adoc index 2857c889f..67d12b37f 100644 --- a/modules/search/pages/create-custom-token-filter.adoc +++ b/modules/search/pages/create-custom-token-filter.adoc @@ -10,19 +10,26 @@ xref:customize-index.adoc#token-filters[Token filters] can improve your search r == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. + +* You have logged in to the Couchbase Capella UI. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a custom token filter with the Capella UI: +To create a custom token filter with the Capella UI in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index where you want to create a custom token filter. -. Expand *Custom Filters*. +. Under *Advanced Settings*, expand *Custom Filters*. ++ +NOTE: Make sure you use *Advanced Mode*. . Click btn:[Add Token Filter]. . In the *Name* field, enter a name for the token filter. @@ -68,7 +75,7 @@ You can create any of the following custom token filters: include::partial$custom-token-filters-descriptions.adoc[tags=dict;!dict_example] -To create a new `dict_compound` token filter with the Capella UI: +To create a new `dict_compound` token filter with the Capella UI in Advanced Mode: . In the *Type* list, select *dict_compound*. . In the *Sub Words* list, select the wordlist to use to find subwords in input tokens. @@ -80,7 +87,7 @@ You can choose your own xref:create-custom-wordlist.adoc[custom wordlist] or a x === Create a Custom `edge_ngram` Token Filter include::partial$custom-token-filters-descriptions.adoc[tags=edge;!edge_example] -To create a new `edge_ngram` token filter with the Capella UI: +To create a new `edge_ngram` token filter with the Capella UI in Advanced Mode: . In the *Type* list, select *edge_ngram*. . Do one of the following: @@ -95,7 +102,7 @@ To create a new `edge_ngram` token filter with the Capella UI: include::partial$custom-token-filters-descriptions.adoc[tags=elision;!elision_example] -To create a new `elision` token filter with the Capella UI: +To create a new `elision` token filter with the Capella UI in Advanced Mode: . In the *Type* list, select *elision*. . In the *Articles* list, select a wordlist to use to find elisions in input tokens. @@ -108,7 +115,7 @@ You can choose your own xref:create-custom-wordlist.adoc[custom wordlist] or a x include::partial$custom-token-filters-descriptions.adoc[tags=keyword;!keyword_example] -To create a new `keyword_marker` token filter with the Capella UI: +To create a new `keyword_marker` token filter with the Capella UI in Advanced Mode: . In the *Type* list, select *keyword_marker*. . In the *Articles* list, select a wordlist to use to find keywords to create tokens. @@ -121,7 +128,7 @@ You can choose your own xref:create-custom-wordlist.adoc[custom wordlist] or a x include::partial$custom-token-filters-descriptions.adoc[tags=length;!length_example] -To create a new `length` token filter with the Capella UI: +To create a new `length` token filter with the Capella UI in Advanced Mode: . In the *Type* list, select *length*. . In the *Min* box, enter the minimum character length for a new token. @@ -133,7 +140,7 @@ To create a new `length` token filter with the Capella UI: include::partial$custom-token-filters-descriptions.adoc[tags=ngram;!ngram_example] -To create a new `ngram` token filter with the Capella UI: +To create a new `ngram` token filter with the Capella UI in Advanced Mode: . In the *Type* list, select *ngram*. . In the *Min* box, enter the minimum character length for a new token. @@ -145,7 +152,7 @@ To create a new `ngram` token filter with the Capella UI: include::partial$custom-token-filters-descriptions.adoc[tags=normalize;!normalize_example] -To create a new `normalize_unicode` token filter with the Capella UI: +To create a new `normalize_unicode` token filter with the Capella UI in Advanced Mode: . In the *Type* list, select *normalize_unicode*. . In the *Form* list, select the type of Unicode normalization to apply: @@ -161,7 +168,7 @@ To create a new `normalize_unicode` token filter with the Capella UI: include::partial$custom-token-filters-descriptions.adoc[tags=shingle;!shingle_example] -To create a new `shingle` token filter with the Capella UI: +To create a new `shingle` token filter with the Capella UI in Advanced Mode: . In the *Type* list, select *shingle*. . In the *Min* box, enter the minimum character length for a new token before concatenation. @@ -178,7 +185,7 @@ To create a new `shingle` token filter with the Capella UI: include::partial$custom-token-filters-descriptions.adoc[tags=stop;!stop_example] -To create a new `stop_tokens` token filter with the Capella UI: +To create a new `stop_tokens` token filter with the Capella UI in Advanced Mode: . In the *Type* list, select *stop_tokens*. . In the *Stop Words* list, select a wordlist to use to remove tokens. @@ -191,7 +198,7 @@ You can choose your own xref:create-custom-wordlist.adoc[custom wordlist] or a x include::partial$custom-token-filters-descriptions.adoc[tags=truncate;!truncate_example] -To create a new `truncate_token` token filter with the Capella UI: +To create a new `truncate_token` token filter with the Capella UI in Advanced Mode: . In the *Type* list, select *truncate_token*. . In the *Length* box, enter the maximum character length for an output token. @@ -206,8 +213,6 @@ To continue customizing your Search index, you can also: * xref:set-advanced-settings.adoc[] * xref:set-type-identifier.adoc[] * xref:create-type-mapping.adoc[] -* xref:create-child-field.adoc[] -* xref:create-child-mapping.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-tokenizer.adoc[] * xref:create-custom-wordlist.adoc[] diff --git a/modules/search/pages/create-custom-tokenizer.adoc b/modules/search/pages/create-custom-tokenizer.adoc index 0a02fe663..267dda30a 100644 --- a/modules/search/pages/create-custom-tokenizer.adoc +++ b/modules/search/pages/create-custom-tokenizer.adoc @@ -8,10 +8,15 @@ == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. + +* You have logged in to the Couchbase Capella UI. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure @@ -34,7 +39,9 @@ To create a regular expression tokenizer with the Capella UI: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index where you want to create a custom tokenizer. -. Expand *Custom Filters*. +. Under *Advanced Settings*, expand *Custom Filters*. ++ +NOTE: Make sure you use *Advanced Mode*. . Click btn:[Add Tokenizer]. . In the *Name* field, enter a name for the custom tokenizer. . In the *Type* list, select *regexp*. @@ -44,7 +51,7 @@ To create a regular expression tokenizer with the Capella UI: [#excep] === Create an Exception Custom Tokenizer -To create an exception custom tokenizer with the Capella UI: +To create an exception custom tokenizer with the Capella UI in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. @@ -70,8 +77,6 @@ To continue customizing your Search index, you can also: * xref:set-advanced-settings.adoc[] * xref:set-type-identifier.adoc[] * xref:create-type-mapping.adoc[] -* xref:create-child-field.adoc[] -* xref:create-child-mapping.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-token-filter.adoc[] * xref:create-custom-wordlist.adoc[] diff --git a/modules/search/pages/create-custom-wordlist.adoc b/modules/search/pages/create-custom-wordlist.adoc index 9310468e6..619c3ed38 100644 --- a/modules/search/pages/create-custom-wordlist.adoc +++ b/modules/search/pages/create-custom-wordlist.adoc @@ -11,19 +11,26 @@ For more information about how to create a custom token filter, see xref:create- == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. + +* You have logged in to the Couchbase Capella UI. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a custom wordlist with the Capella UI: +To create a custom wordlist with the Capella UI in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index where you want to create a custom wordlist. -. Expand *Custom Filters*. +. Under *Advanced Settings*, expand *Custom Filters*. ++ +NOTE: Make sure you use *Advanced Mode*. . Click btn:[Add Wordlist]. . In the *Name* field, enter a name for the wordlist. . In the *New Word* field, enter a word you want to add to the wordlist. @@ -40,8 +47,6 @@ To continue customizing your Search index, you can also: * xref:set-advanced-settings.adoc[] * xref:set-type-identifier.adoc[] * xref:create-type-mapping.adoc[] -* xref:create-child-field.adoc[] -* xref:create-child-mapping.adoc[] * xref:create-custom-analyzer.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-token-filter.adoc[] diff --git a/modules/search/pages/create-quick-index.adoc b/modules/search/pages/create-quick-index.adoc new file mode 100644 index 000000000..61e50484a --- /dev/null +++ b/modules/search/pages/create-quick-index.adoc @@ -0,0 +1,56 @@ += Create a Search Index in Quick Mode +:page-topic-type: guide +:description: Use Quick Mode to create a Search index in Couchbase Capella with streamlined options. + +[abstract] +{description} + +Use xref:create-search-index-ui.adoc[Advanced Mode] to add additional settings and more advanced options for your Search index and xref:customize-index.adoc#type-mappings[type mappings]. + +You must create a Search index before you can xref:simple-search-ui.adoc[run a search] with the Search Service. + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. + +* You have logged in to the Couchbase Capella UI. + +== Procedure + +To use Quick Mode in Couchbase Capella to create a Search index: + +. On the *Databases* page, select the database where you want to create a Search index. +. Go to menu:Data Tools[Search]. +. Click btn:[Create Search Index]. ++ +By default, *Quick Mode* should be on. +. In the *Index Name* field, enter a name for the index. ++ +[NOTE] +==== +Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). + +For version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +==== +. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. +. In the *Scope* list, select the scope where you want to create the index. +. Choose a type mapping to add to your index: +.. To add an entire collection as a type mapping, select a collection name. +.. To only add a single field as a type mapping, expand a collection, then select a field from your document schema. +. Configure the options for your new type mapping. +For more information about the available options, see xref:create-type-mapping.adoc[]. +. Click btn:[Add]. +. (Optional) Add additional collections or field type mappings to your index. +. Click btn:[Create Index]. + +== Next Steps + +If you want to add more advanced settings and customization to your Search index, you can xref:create-search-index-ui.adoc[use Advanced Mode]. + +CAUTION: If you edit your Search index in Advanced Mode, you cannot return to Quick Mode and keep any Advanced Mode settings. + +To run a search and test the contents of your Search index, see xref:simple-search-ui.adoc[] or xref:simple-search-rest-api.adoc[]. \ No newline at end of file diff --git a/modules/search/pages/create-search-index-ui.adoc b/modules/search/pages/create-search-index-ui.adoc index 198c4eaf7..a57658f4e 100644 --- a/modules/search/pages/create-search-index-ui.adoc +++ b/modules/search/pages/create-search-index-ui.adoc @@ -1,45 +1,61 @@ -= Create a Basic Search Index with the Capella UI += Create a Search Index in Advanced Mode :page-topic-type: guide -:description: You can create a Search index with the Couchbase Capella UI. +:description: Use Advanced Mode to create a Search index in Couchbase Capella with full customization and advanced features. [abstract] {description} +Use xref:create-quick-index.adoc[Quick Mode] if you do not need full customization or advanced settings to quickly create a Search index and xref:customize-index.adoc#type-mappings[type mappings]. + You must create a Search index before you can xref:simple-search-ui.adoc[run a search] with the Search Service. == Prerequisites -* You've deployed the Search Service on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have a bucket with scopes and collections in your database. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase Capella UI. == Procedure -To create a Search index with the Capella UI: +To use Advanced Mode in Couchbase Capella to create a Search index: . On the *Databases* page, select the database where you want to create a Search index. . Go to menu:Data Tools[Search]. . Click btn:[Create Search Index]. +. Click the btn:[Advanced Mode] toggle. . In the *Index Name* field, enter a name for the Search index. + -NOTE: Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). +[NOTE] +==== +Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). + +For version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +==== -. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create a type mapping. -. In the *Scope* list, select the scope where you want to create a type mapping. -. Click btn:[Add Mapping]. -. In the *Collection* list, select the collection for the type mapping. -. Click btn:[Submit]. +. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. +. In the *Scope* list, select the scope where you want to create the index. +. (Optional) In the *Document Filter* list, configure an optional filter to add or remove documents from your Search index. ++ +For more information about how to configure a document filter, see xref:set-type-identifier.adoc[]. +. Add at least one mapping to your Search index: +.. In your list of collections, click a collection name. +.. (Optional) To index all fields from the document schema in the collection as a xref:customize-index.adoc#type-mappings[dynamic type mapping], click *Index everything from collection*. +. Click btn:[Add]. +. (Optional) If you did not configure your collection mapping as a xref:customize-index.adoc#type-mappings[dynamic type mapping]: +.. Expand a collection, then select a field from your document schema. +.. Configure a JSON object or child field type mapping. +.. Click btn:[Add]. +. (Optional) Add additional collection mappings or type mappings to your Search index. +. (Optional) Configure *Advanced Settings* for your Search index. +For more information, see xref:set-advanced-settings.adoc[]. . Click btn:[Create Index]. -TIP: To view and copy the JSON definition for your Search index to use xref:create-search-index-rest-api.adoc[with the REST API], click btn:[Index Definition]. - == Next Steps - -This basic index includes all documents from the collection you selected in the Data Insights area. -You can run a search against this index, but it's recommended that you customize your index to improve performance and reduce the index size. -For more information about how to customize an index, see xref:customize-index.adoc[]. +For more information about the different features you can add to your Search index to improve performance and search results, see xref:customize-index.adoc[]. For more information about how to run a search, see xref:simple-search-ui.adoc[] or xref:simple-search-rest-api.adoc[]. diff --git a/modules/search/pages/create-search-indexes.adoc b/modules/search/pages/create-search-indexes.adoc index 3cf984b69..881bff1ea 100644 --- a/modules/search/pages/create-search-indexes.adoc +++ b/modules/search/pages/create-search-indexes.adoc @@ -11,17 +11,16 @@ You can create a Search index with: // * The <>. * The <>. -TIP: If you're new to developing with the Search Service, xref:create-search-index-ui.adoc[create a Search index with the UI]. -You can copy the Search index definition JSON payload from the UI to create your index xref:create-search-index-rest-api.adoc[with the REST API]. +TIP: If you're new to developing with the Search Service, <>. +You can export the Search index definition JSON payload from the UI to create your index xref:create-search-index-rest-api.adoc[with the REST API]. [#ui] == Creating a Search Index with the Capella UI -To xref:create-search-index-ui.adoc[create a basic Search index], provide the following information: +Couchbase Capella supports two editing modes for the Search Service: -* The name of the index. -* The bucket, scope and collection or collections where you want to create the index. -* A xref:customize-index.adoc#type-mappings[type mapping] for the documents that you want to add to the index. +* xref:create-quick-index.adoc[Quick Mode] +* xref:create-search-index-ui.adoc[Advanced Mode] After you create a Search index, the Search Service streams data from your chosen collection or collections, and any document mutations, into the index builder. Before your index finishes building, you can run a search and return partial results. diff --git a/modules/search/pages/create-type-mapping.adoc b/modules/search/pages/create-type-mapping.adoc index 4c3db04a5..dbd0bb126 100644 --- a/modules/search/pages/create-type-mapping.adoc +++ b/modules/search/pages/create-type-mapping.adoc @@ -1,63 +1,264 @@ -= Create a Type Mapping -:page-topic-type: guide -:description: Create a type mapping with the Couchbase Capella UI to control what documents are included or excluded from a Search index. += Type Mapping and Mapping Settings +:page-topic-type: reference +:description: When you create a Search index in Couchbase Capella, you need to configure each type mapping or mapping you add to your index. +:page-toclevels: 3 +:page-aliases: create-child-field.adoc, create-child-mapping.adoc [abstract] {description} -For more information, see xref:customize-index.adoc#type-mappings[Customize a Search Index with the Capella UI]. +The configuration changes for <> and <>. -== Prerequisites +[#quick-mode] +== Quick Mode Type Mappings -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +You can create the following types of type mappings in Quick Mode: + +* <> +* <> + +TIP: Indexing an entire collection in Quick Mode creates a dynamic type mapping. +Indexing a field creates a static type mapping. +For more information about static and dynamic type mappings, see xref:customize-index.adoc#type-mappings[Search Index Features]. + +For more information about how to create a Search index with Quick Mode, see xref:create-quick-index.adoc[]. + +[#quick-collection] +=== Quick Collection Type Mapping Options + +Configure an entire collection in Quick Mode to add or remove all documents in that collection from your Search index. +For more information about how to create an index in Quick Mode, see xref:create-quick-index.adoc[]. + +You can configure the following settings in Quick Mode for a type mapping that uses an entire collection: + +|==== +|Option |Description + +| Index everything from collection: "`$COLLECTION_NAME`" a| + +You must select *Index everything from collection* to add a collection type mapping in Quick Mode. + +| Index text fields as identifiers a| + +To use a *Text* field to set the document's type in the index, select *Index text fields as identifiers*. + +To not use a *Text* field as the document's type, clear *Index text fields as identifiers*. + +|Analyzer/Language a| + +Select the language for the content inside any *Text* fields. + +The Search Service automatically applies an xref:customize-index.adoc#analyzers[analyzer] to the field's contents based on the selected language. + +For more information about the available language options, see xref:quick-index-supported-languages.adoc[]. + +|==== + +[#quick-field] +=== Quick Field Type Mapping Options + +Configure a single field in Quick Mode to add or remove it from your Search index. +For more information about how to create an index in Quick Mode, see xref:create-quick-index.adoc[]. + +You can configure the following settings in Quick Mode for a type mapping that uses a single field: + +|==== +|Option |Description + +//|Dimension (Vector Only) a| + +//|Similarity Metric (Vector Only) a| + +//* *dotproduct*: +//* *euclidean*: -* You've logged in to the Couchbase Capella UI. +|Type a| + +Choose the type of data stored in this field inside your documents. + +For more information about the available field data types, see xref:field-data-types-reference.adoc[]. + +|Index this field as an identifier (Text Only) a| + +To use a *Text* field to set the document's type in the index, select *Index this field as an identifier*. + +To not use a *Text* field as the document's type, clear *Index this field as an identifier*. + +|Analyzer/Language (Text Only) a| + +Select the language for the content inside a *Text* field. + +The Search Service automatically applies an xref:customize-index.adoc#analyzers[analyzer] to the field's contents based on the selected language. + +For more information about the available language options, see xref:quick-index-supported-languages.adoc[]. + +|Include in search results a| + +To include content from the field in search results, select *Include in search results*. + +To exclude the field's content from search results, clear *Include in search results*. + +|Support highlighting a| + +The Search Service can highlight matching search terms in search results from an index. + +To enable highlighting in search results, select *Support highlighting*. + +To turn off highlighting in search results, clear *Support highlighting*. + +NOTE: To enable *Support highlighting*, you must also enable *Include in search results*. + +|Support phrase matching (Text Only) a| + +To support searches for whole phrases in *Text* fields, select *Support phrase matching*. + +To turn off phrase matching for *Text* fields, clear *Support phrase matching*. + +|Support field agnostic search a| + +To search the field's contents without specifying the field name in a search query, select *Support field agnostic search*. + +To turn off field agnostic search, clear *Support field agnostic search*. + +|Support sorting and faceting a| + +To sort search results and use xref:search-request-params.adoc#facets[facets] with the field's contents, select *Support sorting and faceting*. + +To turn off sorting and facets, clear *Support sorting and faceting*. + +|Searchable As a| + +Set a different name that you can use to search the field's contents in a query. + +The default value is the field's name. + +|==== + +[#adv-mode] +== Advanced Mode Mappings + +You can create the following types of mappings in Advanced Mode: + +* <> +* <> +* <> + +TIP: In Advanced Mode, you can index an entire collection or a field as a static or dynamic type mapping. +For more information about static and dynamic type mappings, see xref:customize-index.adoc#type-mappings[Search Index Features]. + +For more information about how to create a Search index with Advanced Mode, see xref:create-search-index-ui.adoc[]. + +[#adv-collection] +=== Advanced Collection Mapping Options + +Map a collection to add or remove documents from that collection from your Search index. + +NOTE: Every index in Advanced Mode must have at least one collection mapping. + +For more information about how to create a Search index with Advanced Mode, see xref:create-search-index-ui.adoc[]. + +You can choose to set your collection mappings as xref:customize-index.adoc#type-mappings[dynamic type mappings]: + +|==== +|Option |Description + +|Index everything from collection: "`$COLLECTION_NAME`" a| + +To set the selected collection as a xref:customize-index.adoc#type-mappings[dynamic type mapping] and index all fields from its document schema, select *Index everything from collection*. + +TIP: You can change this behavior in your xref:set-advanced-settings.adoc[general settings]. + +To only index specific fields from the document schema, clear *Index everything from collection*. + +|==== + +[#adv-object] +=== Advanced JSON Object Field Mapping Options + +You can use the child fields of a JSON object field from your document schema in your Search index. + +In Advanced Mode, index a JSON object as an *Object* mapping. +You can then choose whether to use the JSON object as a xref:customize-index.adoc#type-mappings[dynamic type mapping]. + +For more information about how to create an index in Advanced Mode, see xref:create-search-index-ui.adoc[]. + +You can configure the following settings in Advanced Mode for a mapping that uses a JSON object field: + +|==== +|Option |Description + +|Analyzer/Language a| + +Select the analyzer to use for any text content inside the JSON object. + +You can use a xref:default-analyzers-reference.adoc[default analyzer], inherit the xref:set-advanced-settings.adoc[analyzer set for the Search index], or xref:create-custom-analyzer.adoc[create a custom analyzer]. + +|Index only specified fields a| + +To only index the fields you select and configure from the document schema under the JSON object, select *Index everything from collection*. + +To set the selected JSON object as a xref:customize-index.adoc#type-mappings[dynamic type mapping] and index all child fields from the document schema, clear *Index only specified fields*. + +TIP: You can change this behavior in your xref:set-advanced-settings.adoc[general settings]. + +|==== + + +[#adv-child] +=== Advanced Child Field Type Mapping Options + +To add or remove a single child field from your Search index, index the field as a *Child* type mapping. + +For more information about how to create an index in Advanced Mode, see xref:create-search-index-ui.adoc[]. + +You can configure the following settings in Advanced Mode for a type mapping that uses a single field: + +|==== +|Option |Description + +|Type a| + +Choose the type of data stored in this field inside your documents. + +For more information about the available field data types, see xref:field-data-types-reference.adoc[]. + +|Analyzer/Language a| + +If the child field's Type is *Text*, select the analyzer to use. + +You can use a xref:default-analyzers-reference.adoc[default analyzer], inherit the xref:set-advanced-settings.adoc[analyzer set for the Search index], or xref:create-custom-analyzer.adoc[create a custom analyzer]. + +|Index this field a| + +To add this field to your Search index, select *Index this field*. + +To exclude this field from your Search index and any search results, clear *Index this field*. + +|Store field values a| -== Procedure +To store the values from the field in the index and return them in search results, select *Store*. +This increases your index's size and indexing time. -To create a type mapping with the Capella UI: +To remove the field's values from the index, clear *Store*. -. On the *Databases* page, select the database that has the Search index you want to edit. -. Go to menu:Data Tools[Search]. -. Click the index where you want to create a type mapping. -. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create a type mapping. -. In the *Scope* list, select the scope where you want to create a type mapping. -. Click btn:[Add Mapping]. -. In the *Collection* list, select the collection for the type mapping. -. (Optional) Do any of the following: -.. To use a specific analyzer for documents in the type mapping, in the *Analyzer* list, select an analyzer. -+ -By default, the mapping inherits the default analyzer you set for the Search index. -For more information about analyzers, see xref:customize-index.adoc#analyzers[Customize a Search Index with the Capella UI]. -.. To switch from a xref:customize-index.adoc#type-mappings[dynamic type mapping to a static type mapping], select *Only index specified fields*. -. Click btn:[Submit]. +|Include in _all field a| -[TIP] -==== -You can also create type mappings with the Data Insights area: +The `_all` field is a composite field that has the content from multiple fields in an index. +It allows searches to query the content of a field without specifying the field's name. -. Expand the scope where you want to create a type mapping. -. Next to the collection where you want to create the type mapping, go to menu:More Options (⋮)[Map Collection to Index]. -. (Optional) To edit the analyzer or switch to a static type mapping, next to the type mapping, go to menu:More Options (⋮)[Edit]. -==== +To include this field in the `_all` field, select *Include in _all field*. -== Next Steps +To exclude this field from the `_all` field, clear *Include in _all field*. -After you create a static type mapping, you can xref:create-child-field.adoc[] to add or remove specific document fields from your Search index. +TIP: To change the name of the `_all` field, see xref:set-advanced-settings.adoc#all-field[Default Field]. -To add an extra filter to the documents selected by your type mapping, you can also xref:set-type-identifier.adoc[]. +|Store docvalues for facets a| -If you have a document field that contains a JSON object, xref:create-child-mapping.adoc[] for that field. +Doc values are the value for each instance of the field in an index. +Use doc values for Search xref:search-request-params.adoc#facets[Facets] and sorting search results. -To continue customizing your Search index, you can: +To store doc values, select *Doc Values*. -* xref:set-advanced-settings.adoc[] -* xref:set-type-identifier.adoc[] -* xref:create-custom-analyzer.adoc[] -* xref:create-custom-character-filter.adoc[] -* xref:create-custom-token-filter.adoc[] -* xref:create-custom-tokenizer.adoc[] -* xref:create-custom-wordlist.adoc[] +To exclude doc values from the index, clear *Doc Values*. -To run a search and test the contents of your Search index, see xref:simple-search-ui.adoc[] or xref:simple-search-rest-api.adoc[]. \ No newline at end of file +|==== \ No newline at end of file diff --git a/modules/search/pages/customize-index.adoc b/modules/search/pages/customize-index.adoc index 5f923df81..0feaed0de 100644 --- a/modules/search/pages/customize-index.adoc +++ b/modules/search/pages/customize-index.adoc @@ -1,51 +1,61 @@ -= Customize a Search Index with the Capella UI += Search Index Features :page-topic-type: concept -:description: Configure additional options for a Search index to improve performance and fine tune your search results. +:description: Search indexes in Couchbase Capella have multiple features that you can configure to improve performance and fine tune your search results. [abstract] {description} -You can add the following components and configure the following options for a Search index: +You can use the following components and configure the following options for a Search index: -[cols="1,2"] +[cols="2,1,1,4"] |==== -|Option |Description +|Option |Quick Mode |Advanced Mode |Description -|[[type-mappings]]Mappings a| +|[[type-mappings]]Type Mappings and Mappings +|✓ +|✓ +a|Use a type mapping to include or exclude specific documents in a collection from an index. -Use a type mapping to include or exclude specific documents in a scope or collection from an index, based on their type. +Type mappings also set a field's data type, how the Search Service should tokenize or analyze the field's data, and other settings. + +Mappings contain type mappings, and usually correspond to whole collections or parent fields. You can create two types of type mappings: -* *Dynamic type mappings*: Add all available fields from a matching document type to an index. -* *Static type mappings*: Add only specific fields from a matching document type to an index. +* *Dynamic type mappings*: When you do not know the structure of your data fields ahead of time, use a dynamic type mapping to add all available fields from a matching document type to an index. +For example, you could create a dynamic type mapping to include all documents from the `hotel` collection in your Search index. +* *Static type mappings*: When your data fields are stable and unlikely to change, use a static type mapping to add and define only specific fields from a matching document type to an index. +For example, you could create a static type mapping to only include the contents of the `city` field from the `hotel` collection in your Search index, as a text field with an `en` analyzer. -By default, all indexes have a dynamic type mapping that includes all documents from the *_default* scope and *_default* collection in a bucket. +//By default, all indexes have a dynamic type mapping that includes all documents from the *_default* scope and *_default* collection in a bucket. -Add xref:create-child-field.adoc[child fields] to a type mapping to create a static type mapping. +Add child fields to a mapping to create a static type mapping. Child fields set the specific fields from a document that you want to include or exclude from an index. -For more information about how to add a type mapping to an index, see xref:create-type-mapping.adoc[]. - -|General Settings a| +For more information about how to configure settings for mappings and type mappings, see xref:create-type-mapping.adoc[]. -Set general settings to change your index's default analyzer, replication, and more. +|Advanced Settings +| +|✓ +a|Set advanced settings with xref:create-search-index-ui.adoc[Advanced Mode] to change your index's default analyzer, replication, and more. For more information about how to change general settings, see xref:set-advanced-settings.adoc[]. -|[[type-identifiers]]Type Identifier a| - -Set a type identifier to add a filter to the documents added to your Search index: +|[[type-identifiers]]Document Filter +| +|✓ +a|In xref:create-search-indexes-ui.adoc[Advanced Mode], configure a document filter to add or remove documents in your Search index that meet certain conditions: * JSON Type Field: Selects only documents that contain a specific field with a specified string value. * Doc ID up to Separator: Selects only documents with an ID or key up to a specific substring. * Doc ID with Regex: Selects only documents with an ID or key that matches a regular expression. -For more information about how to configure a type identifier, see xref:set-type-identifier.adoc[]. +For more information about how to configure a document filter, see xref:set-type-identifier.adoc[]. -|[[analyzers]]Analyzers a| - -Use analyzers to improve and customize the search results in your index. +|[[analyzers]]Analyzers +| +|✓ +a|In xref:create-search-indexes-ui.adoc[Advanced Mode], use analyzers to improve and customize the search results in your index. Analyzers transform input text into tokens, which give you greater control over your index's text matching. @@ -57,15 +67,17 @@ When you create a custom analyzer, you can choose these components. Both custom and default analyzers can contain custom filters. -|[[custom-filters-table]]Custom Filters a| - -Use custom filters to add more customization to a custom analyzer. +|[[custom-filters-table]]Custom Filters +| +|✓ +a|In xref:create-search-indexes-ui.adoc[Advanced Mode], use custom filters to add more customization to a custom analyzer. For more information about these filters, see the <> section. -|[[date-time]]Date/Time Parsers a| - -If the documents in your index contain date and time data in a format other than RFC-3339 (ISO-8601), then you need to create a date/time parser. +|[[date-time]]Date/Time Parsers +| +|✓ +a|In xref:create-search-indexes-ui.adoc[Advanced Mode], if the documents in your index contain date and time data in a format other than RFC-3339 (ISO-8601), then you need to create a date/time parser. A custom date/time parser tells the Search index how to interpret date data from your documents. @@ -78,7 +90,7 @@ For more information about how to add a custom date/time parser, see xref:create Custom filters are components of a Search index <>. -Create and add these components to a custom analyzer to improve search results and performance for an index. +Create and add these components to a custom analyzer to improve search results and performance for an index in xref:create-search-indexes-ui.adoc[Advanced Mode]. You can create the following custom filters: diff --git a/modules/search/pages/field-data-types-reference.adoc b/modules/search/pages/field-data-types-reference.adoc index bb72c1e38..0dcbb64c0 100644 --- a/modules/search/pages/field-data-types-reference.adoc +++ b/modules/search/pages/field-data-types-reference.adoc @@ -5,9 +5,9 @@ [abstract] {description} -When you xref:create-child-field.adoc[create a child field] on a xref:customize-index.adoc#type-mappings[type mapping], you need to set a field's data type. +When you xref:create-type-mapping.adoc[create a type mapping for a child field], you need to set its data type. -When you create a Search index and don't set a data type for a field, the Search Service automatically assigns a field data type. +If you create a Search index and do not set a data type for a field, the Search Service automatically assigns a field data type. The following field data types are available: @@ -16,7 +16,7 @@ The following field data types are available: |text |The field contains a string. The string can contain numbers and special characters. -|number |The field contains a number. It doesn't contain any alphabetic characters. +|number |The field contains a number. It does not contain any alphabetic characters. |datetime |The field contains a date/time value that matches the format of a xref:customize-index.adoc#date-time[Date/Time Parser] in the index. diff --git a/modules/search/pages/geo-search-ui.adoc b/modules/search/pages/geo-search-ui.adoc index 05163c2df..4d068c0a3 100644 --- a/modules/search/pages/geo-search-ui.adoc +++ b/modules/search/pages/geo-search-ui.adoc @@ -7,37 +7,56 @@ == Prerequisites -* You've deployed the Search Service on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. * You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have documents in your database that contain geospatial data. + +* You have logged in to the Couchbase Capella UI. == Procedure -To run a geospatial Search query, <>. +To run a geospatial Search query, <>. Then, <>. [#geospatial-index] === Create a Search Index with a Geospatial Type Mapping -To create the Search index from the Capella UI: +To create the Search index in the Capella UI with Advanced Mode: -. On the *Databases* page, select the database where you want to create the Search index. +. On the *Databases* page, select the database where you want to create a Search index. . Go to menu:Data Tools[Search]. . Click btn:[Create Search Index]. +. Click the btn:[Advanced Mode] toggle. . In the *Index Name* field, enter a name for the Search index. + -NOTE: Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). -. xref:create-type-mapping.adoc[] on the scope and collection in your database that you want to search. -. xref:create-child-field.adoc[] on the new type mapping with the following settings: -.. In the *Field* field, enter the name of the field in your documents that contains the geospatial data you want to search. -.. In the *Type* list, select *geopoint*. -.. Select *Index*. +[NOTE] +==== +Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). + +For version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +==== + +. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. +. In the *Scope* list, select the scope where you want to create the index. +. In the *Filter by collection or field* field, enter the name of the collection you want to search. +.. When the collection appears, click the collection name. +.. Click btn:[Add]. +. In the *Filter by collection or field* field, enter the name of the field that contains the geospatial data you want to search. ++ +This field should contain child fields with `lat` and `long` data. ++ +.. When the field appears, click the field name. +.. Click the btn:[Child] toggle. +.. In the *Type* list, select *Geopoint*. +.. Select *Index this field*. .. Select *Include in _all field*. -. Click btn:[Submit]. -. Click btn:[Create Index]. +.. Click btn:[Add]. +. Click btn:[Create Index] [#geospatial-query] === Run a Geospatial Search Query @@ -56,6 +75,6 @@ include::example$geospatial-search-query.jsonc[] == Next Steps -You can xref:customize-index.adoc[customize your Search index] to improve search results and performance. +For more information about the different features you can add to your Search index to improve performance and search results, see xref:customize-index.adoc[]. If you want to add autocomplete to your database's search, see xref:search-query-auto-complete.adoc[]. diff --git a/modules/search/pages/quick-index-supported-languages.adoc b/modules/search/pages/quick-index-supported-languages.adoc new file mode 100644 index 000000000..944493062 --- /dev/null +++ b/modules/search/pages/quick-index-supported-languages.adoc @@ -0,0 +1,62 @@ += Quick Mode Editor Supported Languages +:page-topic-type: reference +:description: When you create a Search index with Quick Mode in Couchbase Capella, you need to specify a language for your text fields. + +[abstract] +{description} + +For more information about how to create a Search index with Quick Mode, see xref:create-quick-index.adoc[]. + +The following language options are available for Quick Index text fields: + +|==== +|Language Option |Description + +|Unknown/Various |Select this option if the text in the field or fields contains multiple languages, or doesn't match another language option. + +|English |For text in English. + +|Arabic |For text in Arabic. + +|Chinese, Japanese, and Korean |For text in Chinese, Japanese, or Korean. + +|Croatian |For text in Croatian. + +|Danish |For text in Danish. + +|Dutch |For text in Dutch. + +|Finnish |For text in Finnish. + +|French |For text in French. + +|German |For text in German. + +|Hebrew |For text in Hebrew. + +|Hindi |For text in Hindi. + +|Hungarian |For text in Hungarian. + +|Italian |For text in Italian. + +|Norwegian |For text in Norwegian. + +|Persian |For text in Persian. + +|Portuguese |For text in Portuguese. + +|Romanian |For text in Romanian. + +|Russian |For text in Russian. + +|Sorani Kurdish |For text in Sorani. + +|Spanish |For text in Spanish. + +|Swedish |For text in Swedish. + +|Turkish |For text in Turkish. + +|Web |Select this option if the text in the field contains content like URLs, email addresses, Twitter usernames, or hashtags. +|==== \ No newline at end of file diff --git a/modules/search/pages/search-query-auto-complete-code.adoc b/modules/search/pages/search-query-auto-complete-code.adoc index bd9f89761..39572edf7 100644 --- a/modules/search/pages/search-query-auto-complete-code.adoc +++ b/modules/search/pages/search-query-auto-complete-code.adoc @@ -10,11 +10,13 @@ After you xref:search-query-auto-complete-ui.adoc[create and configure a Search == Prerequisites -* You've deployed the Search Service on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. * You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've created a compatible Search index. For more information, see xref:search-query-auto-complete-ui.adoc[]. +* You have created a compatible Search index. For more information, see xref:search-query-auto-complete-ui.adoc[]. == Procedure @@ -72,5 +74,5 @@ include::example$autocomplete-sample.py[] After you add autocomplete to your application, to improve your search results, you can: -* xref:customize-index.adoc[Customize your Search index with the Capella UI]. +* xref:customize-index.adoc[Add additional features to your Search index from the UI]. * Change the xref:search-index-params.adoc[JSON payload] for your Search index. diff --git a/modules/search/pages/search-query-auto-complete-ui.adoc b/modules/search/pages/search-query-auto-complete-ui.adoc index 567177c2f..b1471f3b6 100644 --- a/modules/search/pages/search-query-auto-complete-ui.adoc +++ b/modules/search/pages/search-query-auto-complete-ui.adoc @@ -7,34 +7,42 @@ == Prerequisites -* You've deployed the Search Service on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. * You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase Capella UI. == Procedure -You can create a compatible Search index with the <> or the <>. +You can create a compatible Search index with the <> or the <>. [#ui] === Create an Autocomplete Search Index with the Capella UI -To create the Search index with the Capella UI: +To create the Search index in the Capella UI with Advanced Mode: . On the *Databases* page, select the database where you want to create the Search index. . Go to menu:Data Tools[Search]. . Click btn:[Create Search Index]. +. Click the btn:[Advanced Mode] toggle. . In the *Index Name* field, enter a name for the Search index. + -NOTE: Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). -. Expand *Analyzers*. +[NOTE] +==== +Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). + +For version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +==== +. Under *Advanced Settings*, expand *Analyzers*. . xref:create-custom-analyzer.adoc[] with the following settings: .. In the *Name* field, enter `keyword_to_lower`. .. In the *Tokenizer* list, select *single*. .. In the *Token Filters* list, select and add the *to_lower* token filter. -. Expand *Custom Filters*. +. Under *Advanced Settings*, expand *Custom Filters*. . xref:create-custom-token-filter.adoc#edge-ngram[Create a custom token filter] with the following settings: .. In the *Name* field, enter `edge_ngram_2_8`. .. In the *Type* list, select *edge_ngram*. @@ -44,21 +52,21 @@ NOTE: Your index name must start with an alphabetic character (a-z or A-Z). It c .. In the *Name* field, enter `edge_ngram`. .. In the *Tokenizer* list, select *unicode*. .. In the *Token Filters* list, click both the *to_lower* and your custom *edge_ngram_2_8* token filter. -. xref:create-type-mapping.adoc[] with the following settings: -.. Clear *index all contained fields*. -. (Optional) If the field you want to search with autocomplete is nested inside a JSON object in your documents, xref:create-child-mapping.adoc[] for the JSON object with the following settings: -.. Clear *index all contained fields*. -. Under the type mapping or child mapping, xref:create-child-field.adoc[] with the following settings: -.. In the *Field* field, enter the name of a text field in a document from your selected scope and collection. -.. In the *Analyzer* list, select your *edge_ngram* analyzer. -.. Select *Index*. -.. Select *Store*. -.. Select *Include in _all field*. +. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. +. In the *Scope* list, select the scope where you want to create the index. +. In the *Filter by collection or field* field, enter the name of the collection you want to search from your Search index. +.. When the collection appears, click the collection name. +.. Clear *Index everything from collection: "$COLLECTION_NAME"*. +.. Click btn:[Add]. +. In the *Filter by collection or field* field, enter the name of a field in your collection's document schema that contains the data you want to search. +. Configure the field: +.. In the *Analyzer/Language* list, select your *edge_ngram* analyzer. +.. Select *Index this field*. +.. Select *Store field values*. +.. Select *Include in _all field*. . Expand *General Settings*. . In the *Default Analyzer* list, select your *keyword_to_lower* analyzer. -. Click btn:[Create Index] - -TIP: You can use the generated index definition JSON payload from the UI to create a Search index with the REST API. To view the generated JSON, click btn:[Index Definition]. +. Click btn:[Create Index]. [#api] === Create an Autocomplete Search Index with the REST API diff --git a/modules/search/pages/set-advanced-settings.adoc b/modules/search/pages/set-advanced-settings.adoc index 4c971da0c..3af2a2e38 100644 --- a/modules/search/pages/set-advanced-settings.adoc +++ b/modules/search/pages/set-advanced-settings.adoc @@ -7,14 +7,15 @@ == Prerequisites -* You've deployed the Search Service on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have a bucket with scopes and collections in your database. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. - -* You've logged in to the Couchbase Capella UI. +* You have created an index with xref:create-search-index-ui.adoc[Advanced Mode]. + +* You have logged in to the Couchbase Capella UI. == Procedure @@ -23,14 +24,12 @@ To set general settings for a Search index with the Capella UI: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index you want to edit. -. Expand *General Settings*. +. Under *Advanced Settings*, expand *General Settings*. . Configure any of the following general settings for your index: + |==== |Option |Description -|Type Identifier |Change how the Search Service sets a document's type. For more information, see xref:set-type-identifier.adoc[]. - |Default Type |Change the default type assigned to documents in the index. The default value is `_default`. |[[default-analyzer]]Default Analyzer a| @@ -45,7 +44,7 @@ For more information about how to create your own custom analyzer, see xref:crea |[[all-field]]Default Field a| -When you xref:create-child-field.adoc[create a child field] in a type mapping, you can choose to include that field in an `_all` field. +When you xref:create-type-mapping.adoc[create a type mapping for a child field], you can choose to include that field in an `_all` field. You can add fields to the `_all` field to search their contents without specifying their field name in your search query. @@ -55,7 +54,7 @@ Enter a value in the *Default Field* field to change the name of this default fi |Index Dynamic Fields |Select *Index Dynamic Fields* to include fields from a xref:customize-index.adoc#type-mappings[dynamic type mapping] in the index. -|DocValues for Dynamic Fields |Select *DocValues for Dynamic Fields* to include the values of each field from a xref:customize-index.adoc#type-mappings[dynamic type mapping] in the index. +|DocValues for Dynamic Fields |Select *DocValues for Dynamic Fields* to include the values of each field from a xref:customize-index.adoc#type-mappings[dynamic type mapping] in the index for xref:search-request-params.adoc#facets[Facets] and sorting search results. |Number of Replicas a| @@ -75,8 +74,6 @@ After you change the settings for your Search index, you can continue to customi * xref:set-type-identifier.adoc[] * xref:create-type-mapping.adoc[] -* xref:create-child-field.adoc[] -* xref:create-child-mapping.adoc[] * xref:create-custom-analyzer.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-token-filter.adoc[] diff --git a/modules/search/pages/set-type-identifier.adoc b/modules/search/pages/set-type-identifier.adoc index aff500fb4..9c5c67926 100644 --- a/modules/search/pages/set-type-identifier.adoc +++ b/modules/search/pages/set-type-identifier.adoc @@ -1,98 +1,82 @@ -= Set a Search Index Type Identifier += Set a Search Index Document Filter :page-topic-type: guide -:description: Use a type identifier with a type mapping to add an extra filter to the documents you want to include in a Search index. +:description: Use a document filter with a type mapping to add an extra filter to the documents you want to include in a Search index. :page-toclevels: 3 [abstract] {description} -For more information about type identifiers and type mappings, see xref:customize-index.adoc#type-identifiers[Customize a Search Index with the Capella UI]. +For more information about document filters and type mappings, see xref:customize-index.adoc#type-identifiers[Customize a Search Index with the Capella UI]. == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You've created at least one type mapping in your Search index. -For more information, see xref:create-type-mapping.adoc[]. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase Capella UI. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To set a type identifier for a Search index with the Capella UI: +To set a document filter for a Search index with the Capella UI: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. -. Click the index where you want to change the type identifier. -. Expand *General Settings*. -. Do one of the following: +. Click the index where you want to create a document filter +. Under *Type Mappings*, in the *Document Filter* list, choose the document filter you want to add to your Search index: .. <> .. <> .. <> [#json-type] -=== Create a JSON Type Field Type Identifier +=== Create a JSON Type Field Document Filter To only add documents to your Search index that contain a specific field with a specified string value: -. Click the *JSON type field* tab. -. In the *JSON Type Field* field, enter the name of the field in your documents that you want to use to filter documents in your Search index. +. In the *Field* field, enter the name of the field in your documents that you want to use to filter documents in your Search index. + -For example, if your documents include a `type` field, you could enter `type` in the *JSON Type Field* field. +For example, if your documents include a `type` field, you could enter `type` in the *Field* field. + -NOTE: You can't use a field as a type identifier if the field name contains a period (.). -. Under *Type Mappings*, next to the type mapping where you want to add the type identifier, go to menu:More Options (⋮)[Edit]. -. In the *Name* field, add a period (.) to the end of the current type mapping name. -. After the period, add the exact string from the document field that you want to use as a filter. +NOTE: You cannot use a field as a document filter if the field name contains a period (.). +. In the *docvalue* field, enter the exact string from the document field that you want to use as a filter. + -For example, if you wanted your Search index to only return documents that had a `type` value of `hotel`, you could enter `scope.collection.hotel` in the type mapping *Name* field. -. Click btn:[Submit]. -. Click btn:[Update Index]. +For example, if you wanted your Search index to only return documents that had a `type` value of `hotel`, you could enter `hotel`. [#doc-id-sep] -=== Create a Doc ID Up To Separator Type Identifier +=== Create a Doc ID Up To Separator Document Filter To only add documents to your Search index that have IDs that match a specified prefix: - -. Click the *Doc ID up to Separator* tab. -. In the *Doc ID up to Separator* field, enter the ID prefix, up to the separator character, that you want to use to filter documents in your Search index. + +. In the *Separator* field, enter the separator character from the ID prefix in your document ID values. + -For example, if you know all of your document ID values are prefixed by a string and an underscore, enter `_`. -. Under *Type Mappings*, next to the type mapping where you want to add the type identifier, go to menu:More Options (⋮)[Edit]. -. In the *Name* field, add a period (.) to the end of the current type mapping name. -. After the period, add the exact prefix from the document's ID value that you want to use as a filter. +For example, if you know all of your document ID values are prefixed by a string and an underscore (_), enter `_`. +. In the *ID prefix* field, enter the exact prefix from the document's ID value that you want to use as a filter. + -For example, if you wanted your Search index to only return documents that have an prefix of `landmark_`, you could enter `scope.collection.landmark` in the type mapping *Name* field. -. Click btn:[Submit]. -. Click btn:[Update Index]. +For example, if you wanted your Search index to only return documents that have an prefix of `landmark_`, you could enter `landmark`. [#doc-id-regex] -=== Create a Doc ID with Regex Type Identifier +=== Create a Doc ID with Regex document filter To only add documents to your Search index that have IDs that match a specified https://github.com/google/re2/wiki/Syntax[RE2] regular expression: -. Click the *Doc ID with Regex* tab. -. In the *Doc ID with Regex* field, enter the regular expression that you want to use to filter documents in your Search index. +. In the *regex* field, enter the regular expression that you want to use to filter documents in your Search index. + For example, if you wanted only documents with ID values that contained `_40`, you could enter `_[3-5]0` as your regular expression. -. Under *Type Mappings*, next to the type mapping where you want to add the type identifier, go to menu:More Options (⋮)[Edit]. -. In the *Name* field, add a period (.) to the end of the current type mapping name. -. After the period, add a match for the regular expression from the document's ID value that you want to use as a filter. +. In the *ID prefix* field, add a match for the regular expression from the document's ID value that you want to use as a filter. + -For example, if you wanted your Search index to only return documents with ID values that contained `_40`, you could enter `scope.collection._40` in the type mapping *Name* field. -. Click btn:[Submit]. -. Click btn:[Update Index]. +For example, if you wanted your Search index to only return documents with ID values that contained `_40`, you could enter `_40`. == Next Steps -After you set the type identifier for your Search index, you can continue to customize your Search index: +After you set the document filter for your Search index, you can continue to customize your Search index: * xref:set-advanced-settings.adoc[] * xref:create-type-mapping.adoc[] -* xref:create-child-field.adoc[] -* xref:create-child-mapping.adoc[] * xref:create-custom-analyzer.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-token-filter.adoc[] diff --git a/modules/search/partials/nav.adoc b/modules/search/partials/nav.adoc index ed0d8e9c1..4ad218774 100644 --- a/modules/search/partials/nav.adoc +++ b/modules/search/partials/nav.adoc @@ -1,8 +1,28 @@ * xref:cloud:search:search.adoc[] ** xref:cloud:search:create-search-indexes.adoc[] - *** xref:cloud:search:create-search-index-ui.adoc[] - *** xref:cloud:search:create-search-index-rest-api.adoc[] - **** xref:cloud:search:search-index-params.adoc[] + *** xref:cloud:search:customize-index.adoc[] + *** xref:cloud:search:create-quick-index.adoc[] + **** xref:cloud:search:create-type-mapping.adoc[] + **** xref:cloud:search:quick-index-supported-languages.adoc[] + **** xref:cloud:search:field-data-types-reference.adoc[] + *** xref:cloud:search:create-search-index-ui.adoc[] + **** xref:cloud:search:set-type-identifier.adoc[] + **** xref:cloud:search:create-type-mapping.adoc[] + **** xref:cloud:search:set-advanced-settings.adoc[] + **** xref:cloud:search:create-custom-analyzer.adoc[] + **** xref:cloud:search:create-custom-character-filter.adoc[] + **** xref:cloud:search:create-custom-tokenizer.adoc[] + **** xref:cloud:search:create-custom-token-filter.adoc[] + **** xref:cloud:search:create-custom-wordlist.adoc[] + **** xref:cloud:search:create-custom-date-time-parser.adoc[] + **** xref:cloud:search:default-analyzers-reference.adoc[] + **** xref:cloud:search:default-character-filters-reference.adoc[] + **** xref:cloud:search:default-token-filters-reference.adoc[] + **** xref:cloud:search:default-tokenizers-reference.adoc[] + **** xref:cloud:search:default-wordlists-reference.adoc[] + **** xref:cloud:search:field-data-types-reference.adoc[] + *** xref:cloud:search:create-search-index-rest-api.adoc[] + **** xref:cloud:search:search-index-params.adoc[] ** xref:cloud:search:run-searches.adoc[] *** xref:cloud:search:simple-search-ui.adoc[] *** xref:cloud:search:geo-search-ui.adoc[] @@ -14,21 +34,3 @@ *** xref:cloud:search:search-query-auto-complete-code.adoc[] ** xref:cloud:search:index-aliases.adoc[] *** xref:cloud:search:create-search-index-alias.adoc[] - ** xref:cloud:search:customize-index.adoc[] - *** xref:cloud:search:set-advanced-settings.adoc[] - *** xref:cloud:search:set-type-identifier.adoc[] - *** xref:cloud:search:create-type-mapping.adoc[] - *** xref:cloud:search:create-child-field.adoc[] - *** xref:cloud:search:create-child-mapping.adoc[] - *** xref:cloud:search:create-custom-analyzer.adoc[] - *** xref:cloud:search:create-custom-character-filter.adoc[] - *** xref:cloud:search:create-custom-tokenizer.adoc[] - *** xref:cloud:search:create-custom-token-filter.adoc[] - *** xref:cloud:search:create-custom-wordlist.adoc[] - *** xref:cloud:search:create-custom-date-time-parser.adoc[] - *** xref:cloud:search:default-analyzers-reference.adoc[] - *** xref:cloud:search:default-character-filters-reference.adoc[] - *** xref:cloud:search:default-token-filters-reference.adoc[] - *** xref:cloud:search:default-tokenizers-reference.adoc[] - *** xref:cloud:search:default-wordlists-reference.adoc[] - *** xref:cloud:search:field-data-types-reference.adoc[] \ No newline at end of file From ff2e8d3c1c23f01974559c0c98f3786204136b8a Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 11 Jan 2024 12:46:35 -0500 Subject: [PATCH 02/59] [AV-68976] Tweak explanation of mapping vs. type mapping --- modules/search/pages/customize-index.adoc | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/modules/search/pages/customize-index.adoc b/modules/search/pages/customize-index.adoc index 0feaed0de..7c07b9963 100644 --- a/modules/search/pages/customize-index.adoc +++ b/modules/search/pages/customize-index.adoc @@ -18,8 +18,6 @@ a|Use a type mapping to include or exclude specific documents in a collection fr Type mappings also set a field's data type, how the Search Service should tokenize or analyze the field's data, and other settings. -Mappings contain type mappings, and usually correspond to whole collections or parent fields. - You can create two types of type mappings: * *Dynamic type mappings*: When you do not know the structure of your data fields ahead of time, use a dynamic type mapping to add all available fields from a matching document type to an index. @@ -29,8 +27,8 @@ For example, you could create a static type mapping to only include the contents //By default, all indexes have a dynamic type mapping that includes all documents from the *_default* scope and *_default* collection in a bucket. -Add child fields to a mapping to create a static type mapping. -Child fields set the specific fields from a document that you want to include or exclude from an index. +Add child field mappings to the type mapping for a collection or JSON object to create a static type mapping. +Child fields mappings set the specific fields from a document that you want to include or exclude from an index. For more information about how to configure settings for mappings and type mappings, see xref:create-type-mapping.adoc[]. From 1e353485156b8ddfe84ba1ce9a76b046edcc0699 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 11 Jan 2024 14:42:32 -0500 Subject: [PATCH 03/59] [AV-68976] More edits to description of mappings/type mappings. Adding missing context/property to search-index-params.adoc --- modules/search/pages/customize-index.adoc | 16 +++++++------ modules/search/pages/search-index-params.adoc | 23 +++++++++++-------- 2 files changed, 23 insertions(+), 16 deletions(-) diff --git a/modules/search/pages/customize-index.adoc b/modules/search/pages/customize-index.adoc index 7c07b9963..24af22f0b 100644 --- a/modules/search/pages/customize-index.adoc +++ b/modules/search/pages/customize-index.adoc @@ -16,19 +16,21 @@ You can use the following components and configure the following options for a S |✓ a|Use a type mapping to include or exclude specific documents in a collection from an index. -Type mappings also set a field's data type, how the Search Service should tokenize or analyze the field's data, and other settings. +Type mappings can also set a field's data type and other settings. -You can create two types of type mappings: +Add mappings for JSON objects or child fields to type mappings to improve your Search index's performance, instead of indexing entire collections. + +You can create two types of type mappings with the Search Service: * *Dynamic type mappings*: When you do not know the structure of your data fields ahead of time, use a dynamic type mapping to add all available fields from a matching document type to an index. -For example, you could create a dynamic type mapping to include all documents from the `hotel` collection in your Search index. +For example, you could create a dynamic type mapping to include all documents from the `hotel` collection in your Search index, or include all fields under a JSON object from your document schema. +Configure this type of mapping by selecting a collection or use the *Object* option in xref:create-search-index-ui.adoc[Advanced Mode]. * *Static type mappings*: When your data fields are stable and unlikely to change, use a static type mapping to add and define only specific fields from a matching document type to an index. For example, you could create a static type mapping to only include the contents of the `city` field from the `hotel` collection in your Search index, as a text field with an `en` analyzer. +Configure this type of mapping by selecting a field and using the *Child* option in xref:create-search-index-ui.adoc[Advanced Mode]. -//By default, all indexes have a dynamic type mapping that includes all documents from the *_default* scope and *_default* collection in a bucket. - -Add child field mappings to the type mapping for a collection or JSON object to create a static type mapping. -Child fields mappings set the specific fields from a document that you want to include or exclude from an index. +Type mappings start at the collection level. +Create additional mappings under a collection-level type mapping to restrict the documents added to your index. For more information about how to configure settings for mappings and type mappings, see xref:create-type-mapping.adoc[]. diff --git a/modules/search/pages/search-index-params.adoc b/modules/search/pages/search-index-params.adoc index 90245f4ec..3a8665308 100644 --- a/modules/search/pages/search-index-params.adoc +++ b/modules/search/pages/search-index-params.adoc @@ -58,13 +58,13 @@ The UUID of the bucket where you want to create the Search index. The Search Service automatically finds the UUID for the bucket. -Don't include the `sourceUUID` property when you want to copy an index to a different cluster, or create a new index. +Do not include the `sourceUUID` property when you want to copy an index to a different cluster, or create a new index. |sourceParams |Object |No a| This object contains advanced settings for index behavior. -Don't add content into this object unless instructed by Couchbase Support. +Do not add content into this object unless instructed by Couchbase Support. |planParams |Object |Yes |An object that sets the Search index's partitions and replications. For more information, see <>. @@ -229,7 +229,7 @@ For more information about the properties inside the `default_mapping` object, s |default_type |String |No |This setting is included for compatibility with earlier indexes only. |docvalues_dynamic |Boolean |Yes a| -To include the values for an indexed field in the Search index, set `docvalues_dynamic` to `true`. +To include the value for each instance of an indexed field in the Search index to support xref:search-request-params.adoc#facet-name[facets] and sorting search results, set `docvalues_dynamic` to `true`. To exclude the values for an indexed field in the index, set `docvalues_dynamic` to `false`. @@ -833,7 +833,7 @@ To only index the fields you specify in the type mapping, set `dynamic` to `fals |enabled |Boolean |Yes a| To enable the Search Service's default type mapping, set `enabled` to `true`. -The default type mapping includes all documents in the bucket in the Search index, even if they don't match another configured type mapping. +The default type mapping includes all documents in the bucket in the Search index, even if they do not match another configured type mapping. This can increase index size and indexing time. To disable the default type mapping, set `enabled` to `false`. @@ -871,9 +871,9 @@ TIP: To add a type identifier as an additional filter to your type mapping, add |==== [#scope-collection] -=== \{Scope\}.\{collection\} Object +=== \{Scope\}.\{collection\} Object and JSON Object Field Objects -The `\{scope\}.\{collection\}` object defines a custom type mapping for a Search index: +The `\{scope\}.\{collection\}` object defines a custom type mapping for a Search index, on a specific scope and collection in the database: [source,json] ---- @@ -881,7 +881,12 @@ include::example$simple-search-index-payload.jsonc[tag=scope_collection] ---- A `\{scope\}.\{collection\}` object is a child object of the <>. -It contains the following properties: + +A JSON object field object is a child object of the `\{scope\}.\{collection\}` object. +It defines a mapping for a field that contains a JSON object in your document schema. +It can contain additional mappings as <> under its <> object. + +Both objects can contain the following properties: [cols="1,1,1,2"] |==== @@ -905,7 +910,7 @@ The `properties` object is only enabled if `dynamic` is set to `false`. Specifies properties for the fields to index in the type mapping. Contains any number of `\{field_name\}` objects. -For more information, see <> +For more information, see <>. |==== @@ -1021,6 +1026,6 @@ For more information about the available field data types, see xref:field-data-t If the child field's `type` is `text`, set the analyzer to use for the child field. -NOTE: If you want to use the default analyzer for the content of this child field, you don't need to include an `analyzer` property. +NOTE: If you want to use the default analyzer for the content of this child field, you do not need to include an `analyzer` property. |==== From b24d41084f2b38b999ff06f8e383b44d73832362 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 11 Jan 2024 15:06:29 -0500 Subject: [PATCH 04/59] [AV-69144] Updating code samples and Search Indexes overview to reflect scoping changes --- .../examples/create-search-index-header.sh | 2 +- .../examples/create-search-index-payload.sh | 2 +- modules/search/examples/run-search-header.sh | 2 +- modules/search/examples/run-search-payload.sh | 2 +- modules/search/pages/create-search-indexes.adoc | 17 +++++++++++++++++ 5 files changed, 21 insertions(+), 4 deletions(-) diff --git a/modules/search/examples/create-search-index-header.sh b/modules/search/examples/create-search-index-header.sh index 7b32f5f28..570df8f45 100644 --- a/modules/search/examples/create-search-index-header.sh +++ b/modules/search/examples/create-search-index-header.sh @@ -1,3 +1,3 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/index/landmark-content-index + -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index -d \ \ No newline at end of file diff --git a/modules/search/examples/create-search-index-payload.sh b/modules/search/examples/create-search-index-payload.sh index be736d834..15ee4868f 100644 --- a/modules/search/examples/create-search-index-payload.sh +++ b/modules/search/examples/create-search-index-payload.sh @@ -1,5 +1,5 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/index/landmark-content-index + -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index -d \ '{ "type": "fulltext-index", diff --git a/modules/search/examples/run-search-header.sh b/modules/search/examples/run-search-header.sh index d014a4b57..770a66e39 100644 --- a/modules/search/examples/run-search-header.sh +++ b/modules/search/examples/run-search-header.sh @@ -1,2 +1,2 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/index/$INDEX-NAME/query -d \ \ No newline at end of file + -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/$BUCKET-NAME/scope/$SCOPE-NAME/index/$INDEX-NAME/query -d \ \ No newline at end of file diff --git a/modules/search/examples/run-search-payload.sh b/modules/search/examples/run-search-payload.sh index 5101e2fa0..0eb9dce95 100644 --- a/modules/search/examples/run-search-payload.sh +++ b/modules/search/examples/run-search-payload.sh @@ -1,5 +1,5 @@ curl -XPOST -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/index/landmark-content-index/query \ + -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index/query \ -d '{ "explain": true, "fields": [ diff --git a/modules/search/pages/create-search-indexes.adoc b/modules/search/pages/create-search-indexes.adoc index 881bff1ea..a6aab0d6e 100644 --- a/modules/search/pages/create-search-indexes.adoc +++ b/modules/search/pages/create-search-indexes.adoc @@ -22,6 +22,14 @@ Couchbase Capella supports two editing modes for the Search Service: * xref:create-quick-index.adoc[Quick Mode] * xref:create-search-index-ui.adoc[Advanced Mode] +[NOTE] +==== +Your index name must be unique inside its bucket and scope. +You cannot have 2 indexes with the same name inside the same bucket and scope. + +The Capella UI marks indexes as scoped or not scoped to a specific bucket and scope. +==== + After you create a Search index, the Search Service streams data from your chosen collection or collections, and any document mutations, into the index builder. Before your index finishes building, you can run a search and return partial results. @@ -45,6 +53,15 @@ You can create a Search index with the REST API through a JSON payload. Most properties in the JSON payload correspond to settings in the Capella UI. You can also copy the Search index definition JSON payload from a Search index in the Capella UI to use in a REST API call. +[NOTE] +==== +All index names must be unique inside a bucket and scope. +You cannot have 2 indexes with the same name inside the same bucket and scope. + +Use the scoped name for an index with the xref:rest-api:rest-fts.adoc[Search Service REST API] for any endpoints that do not include the bucket and scope in their path. +For example, you must use `bucket.scope.index_name` as the format for your index name with the `analyzeDoc` endpoint, but not with the new 7.6 `query` endpoint. +==== + For more information about how to use the REST API to create a Search index, see xref:create-search-index-rest-api.adoc[]. For more information about the available properties for a Search index, see xref:search-index-params.adoc[Search Index JSON Properties]. From 2dab97225193fd599b7b8c63cb69b06165329bd5 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 11 Jan 2024 15:24:23 -0500 Subject: [PATCH 05/59] [AV-69145] Adding topic import-search-index.adoc Adding link from index-aliases.adoc and nav.adoc --- modules/search/pages/import-search-index.adoc | 39 +++++++++++++++++++ modules/search/pages/index-aliases.adoc | 1 + modules/search/partials/nav.adoc | 1 + 3 files changed, 41 insertions(+) create mode 100644 modules/search/pages/import-search-index.adoc diff --git a/modules/search/pages/import-search-index.adoc b/modules/search/pages/import-search-index.adoc new file mode 100644 index 000000000..94cfa148f --- /dev/null +++ b/modules/search/pages/import-search-index.adoc @@ -0,0 +1,39 @@ += Import a Search Index Definition with the Capella UI +:page-topic-type: guide +:description: Use the Couchbase Capella UI to import a JSON Search index definition or Search index alias. + +[abstract] +{description} + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. + +* You have logged in to the Couchbase Capella UI. + +== Procedure + +To import a xref:create-search-indexes.adoc[Search index definition] or xref:index-aliases.adoc[Search alias] with the Capella UI: + +. On the *Databases* page, select the database where you want to import a Search index or index alias. +. Go to menu:Data Tools[Search]. +. Do one of the following: +.. To import a Search index definition, click btn:[Import Search Index]. +.. To import a Search index alias definition, click btn:[Import Search Alias]. +. In the *Index Name* or *Alias Name* field, enter a name for your new Search index or alias. +. Upload a JSON file smaller than 40 MB that contains your Search index or alias definition. +. Click btn:[Import Search Index] or btn:[Import Search Alias]. +//. (Optional) Make any changes to your Search index or index alias settings. +//+ +//For more information, see xref:customize-index.adoc[] or xref:create-search-index-alias.adoc[]. +//. Click btn:[Create Index] or btn:[Create Index Alias]. + +== Next Steps + +To change the settings for your imported Search index, see xref:create-quick-index.adoc[] or xref:create-search-index-ui.adoc. + +To run a search with your Search index or index alias, see xref:simple-search-ui.adoc[] or xref:simple-search-rest-api.adoc[]. \ No newline at end of file diff --git a/modules/search/pages/index-aliases.adoc b/modules/search/pages/index-aliases.adoc index e0b204a4f..b92005842 100644 --- a/modules/search/pages/index-aliases.adoc +++ b/modules/search/pages/index-aliases.adoc @@ -22,4 +22,5 @@ For more information about how to create a Search index alias, see xref:create-s == See Also * xref:create-search-index-alias.adoc[] +* xref:import-search-index.adoc[] diff --git a/modules/search/partials/nav.adoc b/modules/search/partials/nav.adoc index 4ad218774..c26bd06da 100644 --- a/modules/search/partials/nav.adoc +++ b/modules/search/partials/nav.adoc @@ -21,6 +21,7 @@ **** xref:cloud:search:default-tokenizers-reference.adoc[] **** xref:cloud:search:default-wordlists-reference.adoc[] **** xref:cloud:search:field-data-types-reference.adoc[] + *** xref:cloud:search:import-search-index.adoc[] *** xref:cloud:search:create-search-index-rest-api.adoc[] **** xref:cloud:search:search-index-params.adoc[] ** xref:cloud:search:run-searches.adoc[] From 5cd1b8595895d1445e3465672e411d724a3901f5 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 12 Jan 2024 10:36:39 -0500 Subject: [PATCH 06/59] [AV-68976] Changing location of topics in nav.adoc Changing title of quick-index-supported-languages.adoc --- modules/search/pages/quick-index-supported-languages.adoc | 2 +- modules/search/partials/nav.adoc | 6 ++---- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/modules/search/pages/quick-index-supported-languages.adoc b/modules/search/pages/quick-index-supported-languages.adoc index 944493062..a108e7a34 100644 --- a/modules/search/pages/quick-index-supported-languages.adoc +++ b/modules/search/pages/quick-index-supported-languages.adoc @@ -1,4 +1,4 @@ -= Quick Mode Editor Supported Languages += Quick Mode Supported Languages :page-topic-type: reference :description: When you create a Search index with Quick Mode in Couchbase Capella, you need to specify a language for your text fields. diff --git a/modules/search/partials/nav.adoc b/modules/search/partials/nav.adoc index 4ad218774..33f9b02ba 100644 --- a/modules/search/partials/nav.adoc +++ b/modules/search/partials/nav.adoc @@ -1,13 +1,12 @@ * xref:cloud:search:search.adoc[] ** xref:cloud:search:create-search-indexes.adoc[] *** xref:cloud:search:customize-index.adoc[] + *** xref:cloud:search:create-type-mapping.adoc[] + *** xref:cloud:search:field-data-types-reference.adoc[] *** xref:cloud:search:create-quick-index.adoc[] - **** xref:cloud:search:create-type-mapping.adoc[] **** xref:cloud:search:quick-index-supported-languages.adoc[] - **** xref:cloud:search:field-data-types-reference.adoc[] *** xref:cloud:search:create-search-index-ui.adoc[] **** xref:cloud:search:set-type-identifier.adoc[] - **** xref:cloud:search:create-type-mapping.adoc[] **** xref:cloud:search:set-advanced-settings.adoc[] **** xref:cloud:search:create-custom-analyzer.adoc[] **** xref:cloud:search:create-custom-character-filter.adoc[] @@ -20,7 +19,6 @@ **** xref:cloud:search:default-token-filters-reference.adoc[] **** xref:cloud:search:default-tokenizers-reference.adoc[] **** xref:cloud:search:default-wordlists-reference.adoc[] - **** xref:cloud:search:field-data-types-reference.adoc[] *** xref:cloud:search:create-search-index-rest-api.adoc[] **** xref:cloud:search:search-index-params.adoc[] ** xref:cloud:search:run-searches.adoc[] From 69bf4e6bfe8fecef9b048c8258671d13551d7a5b Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Mon, 15 Jan 2024 13:48:02 -0500 Subject: [PATCH 07/59] [AV-68962] First draft of vector search overview Created nav.adoc partial for new vector search module. Updated search.adoc to provide brief overview of Vector Search and link. --- modules/search/pages/search.adoc | 10 +++++ .../vector-search/pages/vector-search.adoc | 41 +++++++++++++++++++ modules/vector-search/partials/nav.adoc | 1 + 3 files changed, 52 insertions(+) create mode 100644 modules/vector-search/pages/vector-search.adoc create mode 100644 modules/vector-search/partials/nav.adoc diff --git a/modules/search/pages/search.adoc b/modules/search/pages/search.adoc index 036fb0dc5..1c2c44010 100644 --- a/modules/search/pages/search.adoc +++ b/modules/search/pages/search.adoc @@ -52,9 +52,19 @@ You can run a Search query: //* With the Couchbase SDKs * xref:simple-search-rest-api.adoc[From the REST API] +== Vector Search for AI Applications + +Vector Search builds on Capella's Search Service to provide vector index support for Retrieval Augmented Generation (RAG) with an existing Large Language Model (LLM). + +Vector Search adds a new index type to the Search Service to support AI application development, known as a Vector Search index. +Using Vector Search and Couchbase Capella, you can develop applications with an existing LLM while giving context and up-to-date information from your own data. + +For more information about Vector Search, see xref:vector-search:vector-search.adoc[]. + == See Also * xref:create-search-indexes.adoc[] * xref:customize-index.adoc[] * xref:index-aliases.adoc[] * xref:run-searches.adoc[] +* xref:vector-search:vector-search.adoc[]. diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc new file mode 100644 index 000000000..80884457d --- /dev/null +++ b/modules/vector-search/pages/vector-search.adoc @@ -0,0 +1,41 @@ += Use Vector Search for AI Applications +:page-topic-type: concept +:description: Use Couchbase Capella's Vector Search features to add fast and accurate semantic search to your applications. + +[abstract] +{description} +// Use the description you provided as a quick introduction to the page. + +== About Vector Search + +Vector Search builds on Capella's xref:search.adoc[Search Service] to provide vector index support for Retrieval Augmented Generation (RAG) with an existing Large Language Model (LLM). + +Vector Search adds a new index type to the Search Service to support AI application development, known as a Vector Search index. +Using Vector Search and Couchbase Capella, you can develop applications with an existing LLM while giving context and up-to-date information from your own data. + +You can use an LLM and Vector Search to develop applications that include: + +* *Similarity search:* Search for documents, products, images, and more that are similar to a given query. + +* *Generative AI:* Create new text, images, and more based on a prompt given to the LLM. + +// More use cases worth calling out? + +Vector Search supports integrations with frameworks like https://python.langchain.com/docs/get_started/introduction[LangChain^] to support AI application development. +For more information about all frameworks and integrations supported by Vector Search, see xref:[]. +//Don't forget to fill in this link! + +== Using Vector Search Indexes + +Use a Couchbase Capella database to store the data you want to use to provide context to a LLM. +Use a Vector Search index to store the necessary vectors for your data, created using a LLM. + +Give a search query to the LLM to create an embedding and vector, then use the generated embeddings from the data in your database to return a response to the LLM. + +For more information about how to create a Vector Search index, see xref:[]. +For information about how to run a Vector Search query, see xref:[]. +// Don't forget to provide these links! And add to See Also + +== See Also + +* xref:search:search.adoc[] \ No newline at end of file diff --git a/modules/vector-search/partials/nav.adoc b/modules/vector-search/partials/nav.adoc new file mode 100644 index 000000000..291008ecd --- /dev/null +++ b/modules/vector-search/partials/nav.adoc @@ -0,0 +1 @@ +* xref:cloud:vector-search:vector-search.adoc[] \ No newline at end of file From 12d9510bd36222f756eb042bc3a42a4f896c049e Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Wed, 17 Jan 2024 12:25:13 -0500 Subject: [PATCH 08/59] [AV-68962] Adding more notes about vector search. --- modules/vector-search/pages/vector-search.adoc | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc index 80884457d..05e791a11 100644 --- a/modules/vector-search/pages/vector-search.adoc +++ b/modules/vector-search/pages/vector-search.adoc @@ -15,7 +15,8 @@ Using Vector Search and Couchbase Capella, you can develop applications with an You can use an LLM and Vector Search to develop applications that include: -* *Similarity search:* Search for documents, products, images, and more that are similar to a given query. +* *Similarity search:* Search for documents, products, images, and more that are similar to a given query. +Using vector embeddings to represent data, your end users can search based on descriptions, rather than using specific keywords. * *Generative AI:* Create new text, images, and more based on a prompt given to the LLM. @@ -28,9 +29,11 @@ For more information about all frameworks and integrations supported by Vector S == Using Vector Search Indexes Use a Couchbase Capella database to store the data you want to use to provide context to a LLM. -Use a Vector Search index to store the necessary vectors for your data, created using a LLM. +Generate vector embeddings using your chosen LLM, then store the embeddings in an array inside your documents. -Give a search query to the LLM to create an embedding and vector, then use the generated embeddings from the data in your database to return a response to the LLM. +Create a Vector Search index to use the vector embeddings from your data and find matches from a search query's generated vectors. + +NOTE: When creating a Vector Search index, the dimension of your data vector embeddings must match the dimension for any search query vectors. For more information about how to create a Vector Search index, see xref:[]. For information about how to run a Vector Search query, see xref:[]. From 6b38713e0b21d55548923dbcfc0a60fe4399d933 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Wed, 17 Jan 2024 15:12:13 -0500 Subject: [PATCH 09/59] [AV-68962] Adding link to OpenAI documentation --- modules/vector-search/pages/vector-search.adoc | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc index 05e791a11..497262403 100644 --- a/modules/vector-search/pages/vector-search.adoc +++ b/modules/vector-search/pages/vector-search.adoc @@ -33,6 +33,10 @@ Generate vector embeddings using your chosen LLM, then store the embeddings in a Create a Vector Search index to use the vector embeddings from your data and find matches from a search query's generated vectors. +You can use the API for an existing LLM to generate these embeddings for your data. +For example, you could use the OpenAI `embeddings` endpoint to generate embeddings for your text. +For more information about how to generate and obtain embeddings for text strings using the OpenAI API, see the https://platform.openai.com/docs/guides/embeddings/what-are-embeddings[Embeddings documentation]. + NOTE: When creating a Vector Search index, the dimension of your data vector embeddings must match the dimension for any search query vectors. For more information about how to create a Vector Search index, see xref:[]. From 8226a0777eaa72d005acecabfcd6c0b4534f02d6 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 18 Jan 2024 11:47:46 -0500 Subject: [PATCH 10/59] [AV-68962] More language tweaks on overview --- modules/vector-search/pages/vector-search.adoc | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc index 497262403..a3bac9c5e 100644 --- a/modules/vector-search/pages/vector-search.adoc +++ b/modules/vector-search/pages/vector-search.adoc @@ -8,12 +8,12 @@ == About Vector Search -Vector Search builds on Capella's xref:search.adoc[Search Service] to provide vector index support for Retrieval Augmented Generation (RAG) with an existing Large Language Model (LLM). +Vector Search builds on Capella's xref:search.adoc[Search Service] to provide vector index support. +You can use these new Vector Search indexes for Retrieval Augmented Generation (RAG) with an existing Large Language Model (LLM). -Vector Search adds a new index type to the Search Service to support AI application development, known as a Vector Search index. -Using Vector Search and Couchbase Capella, you can develop applications with an existing LLM while giving context and up-to-date information from your own data. +Using Vector Search, Capella, and your chosen LLM, you can develop AI applications while giving context and up-to-date information from your own data. -You can use an LLM and Vector Search to develop applications that include: +You can develop applications that include: * *Similarity search:* Search for documents, products, images, and more that are similar to a given query. Using vector embeddings to represent data, your end users can search based on descriptions, rather than using specific keywords. @@ -23,7 +23,7 @@ Using vector embeddings to represent data, your end users can search based on de // More use cases worth calling out? Vector Search supports integrations with frameworks like https://python.langchain.com/docs/get_started/introduction[LangChain^] to support AI application development. -For more information about all frameworks and integrations supported by Vector Search, see xref:[]. +For more information about all frameworks and integrations supported by Vector Search and Capella, see xref:[]. //Don't forget to fill in this link! == Using Vector Search Indexes From 54c89b731ec3ea2042b6e080557d1961d44aa2ad Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 18 Jan 2024 14:33:46 -0500 Subject: [PATCH 11/59] [AV-68962] Another language tweak --- modules/vector-search/pages/vector-search.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc index a3bac9c5e..648dab7ff 100644 --- a/modules/vector-search/pages/vector-search.adoc +++ b/modules/vector-search/pages/vector-search.adoc @@ -33,7 +33,7 @@ Generate vector embeddings using your chosen LLM, then store the embeddings in a Create a Vector Search index to use the vector embeddings from your data and find matches from a search query's generated vectors. -You can use the API for an existing LLM to generate these embeddings for your data. +If you do not want to use an integration like LangChain, you can also use the API for an existing LLM to generate these embeddings for your data. For example, you could use the OpenAI `embeddings` endpoint to generate embeddings for your text. For more information about how to generate and obtain embeddings for text strings using the OpenAI API, see the https://platform.openai.com/docs/guides/embeddings/what-are-embeddings[Embeddings documentation]. From 993846088e4e09c4768fb71d353db237ae27e4c3 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 19 Jan 2024 15:18:44 -0500 Subject: [PATCH 12/59] [AV-68966] Initial draft of create and run vector search index topics for REST API --- .../create-vector-search-index-header.sh | 3 + .../create-vector-search-index-payload.sh | 76 +++++++++++++++++++ .../examples/run-vector-search-header.sh | 2 + .../examples/run-vector-search-payload.sh | 20 +++++ .../vector-search-index-payload.jsonc | 74 ++++++++++++++++++ .../create-vector-search-index-rest-api.adoc | 59 ++++++++++++++ .../pages/run-vector-search-rest-api.adoc | 58 ++++++++++++++ modules/vector-search/partials/nav.adoc | 5 +- 8 files changed, 296 insertions(+), 1 deletion(-) create mode 100644 modules/vector-search/examples/create-vector-search-index-header.sh create mode 100644 modules/vector-search/examples/create-vector-search-index-payload.sh create mode 100644 modules/vector-search/examples/run-vector-search-header.sh create mode 100644 modules/vector-search/examples/run-vector-search-payload.sh create mode 100644 modules/vector-search/examples/vector-search-index-payload.jsonc create mode 100644 modules/vector-search/pages/create-vector-search-index-rest-api.adoc create mode 100644 modules/vector-search/pages/run-vector-search-rest-api.adoc diff --git a/modules/vector-search/examples/create-vector-search-index-header.sh b/modules/vector-search/examples/create-vector-search-index-header.sh new file mode 100644 index 000000000..a3d0546d8 --- /dev/null +++ b/modules/vector-search/examples/create-vector-search-index-header.sh @@ -0,0 +1,3 @@ +curl -s -XPUT -H "Content-Type: application/json" \ + -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/$BUCKET_NAME/scope/$SCOPE_NAME/index/$INDEX_NAME + -d \ \ No newline at end of file diff --git a/modules/vector-search/examples/create-vector-search-index-payload.sh b/modules/vector-search/examples/create-vector-search-index-payload.sh new file mode 100644 index 000000000..05a768a8f --- /dev/null +++ b/modules/vector-search/examples/create-vector-search-index-payload.sh @@ -0,0 +1,76 @@ +curl -s -XPUT -H "Content-Type: application/json" \ + -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/vector-sample/scope/color/index/color-index + -d \ + '{ + "type": "fulltext-index", + "name": "vector-sample.color.color-index", + "sourceType": "gocbcore", + "sourceName": "vector-sample", + "sourceUUID": "789365cccdf940ee2814a5dd2752040a", + "planParams": { + "maxPartitionsPerPIndex": 1024, + "indexPartitions": 1 + }, + "params": { + "doc_config": { + "docid_prefix_delim": "", + "docid_regexp": "", + "mode": "scope.collection.type_field", + "type_field": "type" + }, + "mapping": { + "analysis": {}, + "default_analyzer": "standard", + "default_datetime_parser": "dateTimeOptional", + "default_field": "_all", + "default_mapping": { + "dynamic": false, + "enabled": false + }, + "default_type": "_default", + "docvalues_dynamic": false, + "index_dynamic": false, + "store_dynamic": false, + "type_field": "_type", + "types": { + "color.rgb": { + "dynamic": false, + "enabled": true, + "properties": { + "colorvect_dot": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 3, + "index": true, + "name": "colorvect_dot", + "similarity": "dot_product", + "type": "vector" + } + ] + }, + "colorvect_l2": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 3, + "index": true, + "name": "colorvect_l2", + "similarity": "l2_norm", + "type": "vector" + } + ] + } + } + } + } + }, + "store": { + "indexType": "scorch", + "segmentVersion": 16 + } + }, + "sourceParams": {} + } \ No newline at end of file diff --git a/modules/vector-search/examples/run-vector-search-header.sh b/modules/vector-search/examples/run-vector-search-header.sh new file mode 100644 index 000000000..45d695f17 --- /dev/null +++ b/modules/vector-search/examples/run-vector-search-header.sh @@ -0,0 +1,2 @@ +curl -s -XPUT -H "Content-Type: application/json" \ +-u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/$BUCKET_NAME/scope/$SCOPE_NAME/index/$INDEX_NAME/query -d \ \ No newline at end of file diff --git a/modules/vector-search/examples/run-vector-search-payload.sh b/modules/vector-search/examples/run-vector-search-payload.sh new file mode 100644 index 000000000..d5f884ff3 --- /dev/null +++ b/modules/vector-search/examples/run-vector-search-payload.sh @@ -0,0 +1,20 @@ +curl -XPOST -H "Content-Type: application/json" \ + -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/vector-sample/scope/color/index/color-index/query \ +-d '{ + "fields": ["*"], + "query": { + "min": 70, + "max": 80, + "inclusive_min": false, + "inclusive_max": true, + "field": "brightness" + }, + "knn": [ + { + "k": 10, + "field": "colorvect_dot", + "vector": [ 0.707106781186548, 0, 0.707106781186548 ] + } + ], + "size": 10 + }' \ No newline at end of file diff --git a/modules/vector-search/examples/vector-search-index-payload.jsonc b/modules/vector-search/examples/vector-search-index-payload.jsonc new file mode 100644 index 000000000..96133d9d5 --- /dev/null +++ b/modules/vector-search/examples/vector-search-index-payload.jsonc @@ -0,0 +1,74 @@ +{ + "type": "fulltext-index", + "name": "vector-sample.color.color-test", + "uuid": "5b23225c54c55b64", + "sourceType": "gocbcore", + "sourceName": "vector-sample", + "sourceUUID": "789365cccdf940ee2814a5dd2752040a", + "planParams": { + "maxPartitionsPerPIndex": 1024, + "indexPartitions": 1 + }, + "params": { + "doc_config": { + "docid_prefix_delim": "", + "docid_regexp": "", + "mode": "scope.collection.type_field", + "type_field": "type" + }, + "mapping": { + "analysis": {}, + "default_analyzer": "standard", + "default_datetime_parser": "dateTimeOptional", + "default_field": "_all", + "default_mapping": { + "dynamic": false, + "enabled": false + }, + "default_type": "_default", + "docvalues_dynamic": false, + "index_dynamic": false, + "store_dynamic": false, + "type_field": "_type", + "types": { + "color.rgb": { + "dynamic": false, + "enabled": true, + "properties": { + "colorvect_dot": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 3, + "index": true, + "name": "colorvect_dot", + "similarity": "dot_product", + "type": "vector" + } + ] + }, + "colorvect_l2": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 3, + "index": true, + "name": "colorvect_l2", + "similarity": "l2_norm", + "type": "vector" + } + ] + } + } + } + } + }, + "store": { + "indexType": "scorch", + "segmentVersion": 16 + } + }, + "sourceParams": {} + } \ No newline at end of file diff --git a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc new file mode 100644 index 000000000..74db5c0af --- /dev/null +++ b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc @@ -0,0 +1,59 @@ += Create a Vector Search Index with the REST API and curl/HTTP +:page-topic-type: guide +:description: You can create a Vector Search index with the Search Service API and Couchbase Capella. + +[abstract] +{description} + +You must create a Search index before you can xref:run-vector-search-rest-api.adoc[run a search] with the Search Service. +// Add link + +== Prerequisites + +* You have the Search Service enabled on a node in your database. + +* You have a bucket with scopes and collections in your database. + +* You have documents in a keyspace inside that bucket that contain vector embeddings. + +* You have installed the Couchbase command-line tool (CLI). + +* You have the hostname for your database. ++ +For more information about how to find the hostname for your Capella database, see xref:clouds:get-capella-host-name.adoc[]. + +== Procedure + +To create a Search index with the REST API and Couchbase Capella: + +. In your command-line tool, enter a `curl` command with the `XPUT` verb. +. Set your header content to include `Content-Type: application/json`. +. Enter your username, password, and the Search Service endpoint on port `8094` with the name of the index you want to create: ++ +[source,console] +---- +include::example$create-vector-search-index-header.sh[] +---- +. Enter the JSON payload for the settings you want in your index. ++ +Do not include the xref:search:search-index-params.adoc#uuid[uuid] or xref:search:search-index-params.adoc#sourceuuid[sourceUUID] parameters. ++ +TIP: If you remove the xref:search:search-index-params.adoc#uuid[uuid] and xref:search:search-index-params.adoc#sourceuuid[sourceUUID] parameters, you can copy the Search index definition JSON payload from the Capella UI to use in a REST API call. +For more information about how to create a Vector Search index with the UI, see xref:create-vector-search-index-ui.adoc[]. ++ +In the following example, the JSON payload creates an index named `color-index` on the `vector-sample` bucket. It creates two child field mappings, `colorvect_dot` and `colorvect_l2` on two different vector fields in the documents under the `color.rgb` keyspace: ++ +[source,console] +---- +include::example$create-vector-search-index-payload.sh[] +---- ++ +For more information about all the available JSON properties for a Search index, see xref:search:search-index-params.adoc[]. + +== Next Steps + +After you create a Search index, you can xref:run-vector-search-rest-api.adoc[] to test your Search index. + +If you want to edit your index with another REST API call, include the xref:search:search-index-params.adoc#uuid[uuid] parameter with the UUID the Search Service assigned to your Vector Search index. + +You can also xref:create-vector-search-index-ui.adoc[]. \ No newline at end of file diff --git a/modules/vector-search/pages/run-vector-search-rest-api.adoc b/modules/vector-search/pages/run-vector-search-rest-api.adoc new file mode 100644 index 000000000..8e2d6f551 --- /dev/null +++ b/modules/vector-search/pages/run-vector-search-rest-api.adoc @@ -0,0 +1,58 @@ += Run a Vector Search with the REST API and curl/HTTP +:page-topic-type: guide +:description: You can use the REST API and a curl command to run a search against a Vector Search index and return similar vectors from a Couchbase Capella database. + +[abstract] +{description} + +== Prerequisites + +* You have the Search Service enabled on a node in your database. + +* Your user account has the *Search Admin* or *Search Reader* role. + +* You installed the Couchbase command-line tool (CLI). + +* You have the hostname for your database. ++ +For more information about how to find the hostname for your Capella database, see xref:clouds:get-capella-host-name.adoc[]. + +* You have created a Vector Search index. ++ +For more information about how to create a Vector Search index, see xref:create-vector-search-index-ui.adoc[] or xref:create-vector-search-index-rest-api.adoc[]. + +== Procedure + +To run a Vector search with the REST API: + +. In your command-line tool, enter a `curl` command with the `XPOST` verb. +. Set your header content to include `Content-Type: application/json`. +. Enter your username, password, and the Search Service endpoint on port `8094` with the name of the Vector Search index you want to query: ++ +[source,console] +---- +include::example$run-vector-search-header.sh[] +---- ++ +To use SSL, use the `https` protocol in the Search Service endpoint URL and port `18094`. +. Enter the JSON payload for your query. ++ +TIP: You can copy the JSON for a Query Request from the Capella UI to use in your REST API call. +For more information about how to perform a search with the UI, see xref:search:simple-search-ui.adoc[]. + ++ +In the following example, the JSON payload uses a hybrid Vector Search and Search query on an index named `color-index`. +The query searches for a specified normalized color vector, but uses the regular query parameters to limit the brightness of the returned color to the range of `70-80`: ++ +[source,console] +---- +include::example$run-vectorsearch-payload.sh[] +---- ++ +For more information about the available properties for a Search query JSON payload, see xref:search:search-request-params.adoc[]. + +== Next Steps + +If you do not get the search results you were expecting, you can change the JSON definition xref:search:search-index-params.adoc[for your Search index] or change the parameters xref:search:search-request-params.adoc[for your Search query]. + +You can also xref:search:customize-index.adoc[]. \ No newline at end of file diff --git a/modules/vector-search/partials/nav.adoc b/modules/vector-search/partials/nav.adoc index 291008ecd..dd2a947d2 100644 --- a/modules/vector-search/partials/nav.adoc +++ b/modules/vector-search/partials/nav.adoc @@ -1 +1,4 @@ -* xref:cloud:vector-search:vector-search.adoc[] \ No newline at end of file +* xref:cloud:vector-search:vector-search.adoc[] +** xref:cloud:vector-search:create-vector-search-index-ui.adoc[] +** xref:cloud:vector-search:create-vector-search-index-rest-api.adoc[] +** xref:cloud:vector-search:run-vector-search-rest-api.adoc[] From eebd12b5b8dd4a6fe681d5fd5be4926a593cd99a Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 19 Jan 2024 15:47:30 -0500 Subject: [PATCH 13/59] [AV-68966] Fixing typo in include --- modules/vector-search/pages/run-vector-search-rest-api.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/vector-search/pages/run-vector-search-rest-api.adoc b/modules/vector-search/pages/run-vector-search-rest-api.adoc index 8e2d6f551..0e50d95e5 100644 --- a/modules/vector-search/pages/run-vector-search-rest-api.adoc +++ b/modules/vector-search/pages/run-vector-search-rest-api.adoc @@ -46,7 +46,7 @@ The query searches for a specified normalized color vector, but uses the regular + [source,console] ---- -include::example$run-vectorsearch-payload.sh[] +include::example$run-vector-search-payload.sh[] ---- + For more information about the available properties for a Search query JSON payload, see xref:search:search-request-params.adoc[]. From 12aeac419eab9fa38da057f5c6e25f8e423c9f60 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 19 Jan 2024 15:53:22 -0500 Subject: [PATCH 14/59] [AV-68571] Commenting out non-existent entry for now --- modules/vector-search/partials/nav.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/vector-search/partials/nav.adoc b/modules/vector-search/partials/nav.adoc index dd2a947d2..8e7e4a18c 100644 --- a/modules/vector-search/partials/nav.adoc +++ b/modules/vector-search/partials/nav.adoc @@ -1,4 +1,4 @@ * xref:cloud:vector-search:vector-search.adoc[] -** xref:cloud:vector-search:create-vector-search-index-ui.adoc[] +//** xref:cloud:vector-search:create-vector-search-index-ui.adoc[] ** xref:cloud:vector-search:create-vector-search-index-rest-api.adoc[] ** xref:cloud:vector-search:run-vector-search-rest-api.adoc[] From 584c3aea1ff592dbe018df61c88fbf7107323b40 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Wed, 31 Jan 2024 16:37:16 -0500 Subject: [PATCH 15/59] [AV-68962] Language tweak after discussion with SME --- modules/vector-search/pages/vector-search.adoc | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc index 648dab7ff..ca32961ba 100644 --- a/modules/vector-search/pages/vector-search.adoc +++ b/modules/vector-search/pages/vector-search.adoc @@ -4,14 +4,13 @@ [abstract] {description} -// Use the description you provided as a quick introduction to the page. == About Vector Search Vector Search builds on Capella's xref:search.adoc[Search Service] to provide vector index support. You can use these new Vector Search indexes for Retrieval Augmented Generation (RAG) with an existing Large Language Model (LLM). -Using Vector Search, Capella, and your chosen LLM, you can develop AI applications while giving context and up-to-date information from your own data. +Using Vector Search, Couchbase Server, an embedding model, and your chosen LLM, you can develop AI applications while giving context and up-to-date information from your own data. You can develop applications that include: @@ -29,12 +28,12 @@ For more information about all frameworks and integrations supported by Vector S == Using Vector Search Indexes Use a Couchbase Capella database to store the data you want to use to provide context to a LLM. -Generate vector embeddings using your chosen LLM, then store the embeddings in an array inside your documents. +Generate vector embeddings using your preferred embedding model, then store the embeddings in an array inside your documents. Create a Vector Search index to use the vector embeddings from your data and find matches from a search query's generated vectors. -If you do not want to use an integration like LangChain, you can also use the API for an existing LLM to generate these embeddings for your data. -For example, you could use the OpenAI `embeddings` endpoint to generate embeddings for your text. +If you do not want to use an integration like LangChain, you can also use the API for an existing LLM and one of their embedding models to generate these embeddings for your data. +For example, the OpenAI `embeddings` endpoint can generate embeddings for a text string using a specified embedding model. For more information about how to generate and obtain embeddings for text strings using the OpenAI API, see the https://platform.openai.com/docs/guides/embeddings/what-are-embeddings[Embeddings documentation]. NOTE: When creating a Vector Search index, the dimension of your data vector embeddings must match the dimension for any search query vectors. From 45b29258fce1bdd1cbb8e44d6bec7ff35e8003f5 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 8 Feb 2024 12:06:59 -0500 Subject: [PATCH 16/59] [AV-68976] Making changes to reflect current UI, instead of Figma designs. Updating descriptions of type mappings and GeoJSON objects. Fitting everything into nav.adoc Updating links in some topics and changing anchors to support better understanding of type mappings and other changes. --- modules/search/pages/.create-child-field.adoc | 119 -------- .../pages/child-field-options-reference.adoc | 88 ++++++ modules/search/pages/create-child-field.adoc | 58 ++++ ...mapping.adoc => create-child-mapping.adoc} | 12 +- .../search/pages/create-custom-analyzer.adoc | 2 +- .../search/pages/create-search-index-ui.adoc | 33 +-- .../search/pages/create-search-indexes.adoc | 3 + modules/search/pages/create-type-mapping.adoc | 279 +++--------------- modules/search/pages/customize-index.adoc | 26 +- .../pages/default-analyzers-reference.adoc | 2 +- .../pages/field-data-types-reference.adoc | 29 +- .../pages/quick-index-field-options.adoc | 117 ++++++++ .../search/pages/set-advanced-settings.adoc | 2 +- modules/search/pages/set-type-identifier.adoc | 57 ++-- modules/search/partials/nav.adoc | 6 +- 15 files changed, 410 insertions(+), 423 deletions(-) delete mode 100644 modules/search/pages/.create-child-field.adoc create mode 100644 modules/search/pages/child-field-options-reference.adoc create mode 100644 modules/search/pages/create-child-field.adoc rename modules/search/pages/{.create-child-mapping.adoc => create-child-mapping.adoc} (84%) create mode 100644 modules/search/pages/quick-index-field-options.adoc diff --git a/modules/search/pages/.create-child-field.adoc b/modules/search/pages/.create-child-field.adoc deleted file mode 100644 index 0dac7c824..000000000 --- a/modules/search/pages/.create-child-field.adoc +++ /dev/null @@ -1,119 +0,0 @@ -= Create a Child Field -:page-topic-type: guide -:description: Create a child field with the Couchbase Capella UI to add or remove a specific field's content from a Search index. - -[abstract] -{description} - -== Prerequisites - -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. - -* You've created a type mapping. -For more information about how to create a type mapping on an index, see xref:create-type-mapping.adoc[]. - -* You've logged in to the Couchbase Capella UI. - -== Procedure - -To add or remove a child field from a Search index with the Capella UI: - -. On the *Databases* page, select the database that has the Search index you want to edit. -. Go to menu:Data Tools[Search]. -. Click the index where you want to create a child field. -. Under *Type Mappings*, next to an existing type mapping, go to menu:More Options (⋮)[Insert Child Field]. -. In the *Field* field, enter the name of a field in your documents that you want to add or remove from the index. -+ -NOTE: The field must contain either a single value or an array. If the field contains a JSON object, xref:create-child-mapping.adoc[Create a Child Mapping], instead. -. In the *Type* list, select the data type for the field: -+ -For more information about the available data types, see xref:field-data-types-reference.adoc[]. - -. Configure optional settings for the child field: -+ -|==== -|Option |Description - -|Searchable As a| - -Set a different name that you can use to search the field's contents in a query. - -The default value is the value set in *Field*. - -|Analyzer a| - -Choose an analyzer for the content in the child field. - -You can choose a default analyzer or a custom analyzer. - -For more information about the available default analyzers, see xref:default-analyzers-reference.adoc[]. - -For more information about how to create a custom analyzer, see xref:create-custom-analyzer.adoc[]. - -|Index a| - -To include the child field in the index, select *Index*. - -To remove the child field from the index, clear *Index*. - -|[[store]]Store a| - -To store the values from the field in the index and return them in search results, select *Store*. -This increases your index's size and indexing time. - -To remove the field's values from the index, clear *Store*. - -|Include in _all field a| - -The `_all` field is a composite field that has the content from multiple fields in an index. -It allows searches to query the content of a field without specifying the field's name. - -To include this field in the `_all` field, select *Include in _all field*. - -To exclude this field from the `_all` field, clear *Include in _all field*. - -TIP: To change the name of the `_all` field, see xref:set-advanced-settings.adoc#all-field[Default Field]. - -|[[term-vectors]]Include Term Vectors a| - -Term vectors store the location of terms in a field for an index. -You can use term vectors to highlighting matching search terms in search results, and perform phrase searches. -Term vectors increase your index's size and indexing time. - -To enable term vectors for this field, select *Include Term Vectors*. - -To turn off term vectors, clear *Include Term Vectors*. - -NOTE: To enable term vectors, you must also enable <>. - -|Doc Values a| - -Doc values are the value for each instance of the field in an index. -Use doc values for Search xref:search-request-params.adoc#facets[Facets] and sorting search results. - -To store doc values, select *Doc Values*. - -To exclude doc values from the index, clear *Doc Values*. -|==== -. Click btn:[Submit]. -. Click btn:[Update Index]. - -== Next Steps - -You can continue to create child fields to add or remove the contents of a document from your Search index. - -If a field in your documents contains a JSON object, xref:create-child-mapping.adoc[], instead. - -To continue customizing your Search index, you can also: - -* xref:set-advanced-settings.adoc[] -* xref:set-type-identifier.adoc[] -* xref:create-type-mapping.adoc[] -* xref:create-custom-analyzer.adoc[] -* xref:create-custom-character-filter.adoc[] -* xref:create-custom-tokenizer.adoc[] -* xref:create-custom-token-filter.adoc[] -* xref:create-custom-wordlist.adoc[] - -To run a search and test the contents of your Search index, see xref:simple-search-ui.adoc[] or xref:simple-search-rest-api.adoc[]. \ No newline at end of file diff --git a/modules/search/pages/child-field-options-reference.adoc b/modules/search/pages/child-field-options-reference.adoc new file mode 100644 index 000000000..242dfa22b --- /dev/null +++ b/modules/search/pages/child-field-options-reference.adoc @@ -0,0 +1,88 @@ += Child Field Options +:page-topic-type: reference +:description: You can configure a child field from the Couchbase Capella UI's Advanced Mode to control how its data is added to your Search index. + +[abstract] +{description} + +For more information about how to create a child field using Advanced Mode, see xref:create-child-field.adoc[]. + +For more information about creating an index and child fields in Quick Mode, see xref:create-quick-index.adoc[]. + +You can configure the following options for a child field in Advanced Mode, based on its type: + + +|==== +|Option |Description + +|Searchable As a| + +Set a different name that you can use to search the field's contents in a query. + +The default value is the value set in the *Field* field when you xref:create-child-field.adoc#field[create a child field]. + +|Analyzer a| + +Choose an analyzer for the content in the child field. + +You can choose a default analyzer or a custom analyzer. + +For more information about the available default analyzers, see xref:default-analyzers-reference.adoc[]. + +For more information about how to create a custom analyzer, see xref:create-custom-analyzer.adoc[]. + +|Index a| + +To include the child field in the index, select *Index*. + +To remove the child field from the index, clear *Index*. + +If you remove the child field from the index, it cannot appear in search results. + +|[[store]]Store a| + +To store the values from the field in the index and return them in search results, select *Store*. +This increases your index's size and indexing time. + +To remove the field's values from the index, clear *Store*. + +|Include in _all field a| + +The `_all` field is a composite field that has the content from multiple fields in an index. +It allows searches to query the content of a field without specifying the field's name. + +To include this field in the `_all` field, select *Include in _all field*. + +To exclude this field from the `_all` field, clear *Include in _all field*. + +TIP: To change the name of the `_all` field, see xref:set-advanced-settings.adoc#all-field[Default Field]. + +|[[term-vectors]]Include Term Vectors a| + +Term vectors store the location of terms in a field for an index. +You can use term vectors to xref:search-request-params.adoc#highlight[highlight matching search terms in search results], and xref:search-request-params.adoc#match_phrase[perform phrase searches]. +Term vectors increase your index's size and indexing time. + +To enable term vectors for this field, select *Include Term Vectors*. + +To turn off term vectors, clear *Include Term Vectors*. + +NOTE: To enable term vectors, you must also enable <>. + +|Doc Values a| + +Use doc values for a field for Search xref:search-request-params.adoc#facets[Facets] and sorting your search results. + +To store doc values and enable faceting and sorting for this field, select *Doc Values*. + +To exclude doc values from the index for this field, clear *Doc Values*. +|==== + +== See Also + +* xref:create-child-field.adoc[] +* xref:create-search-index-ui.adoc[] +* xref:vector-search:create-vector-search-index-ui.adoc[] +* xref:set-type-identifier.adoc[] +* xref:create-type-mapping.adoc[] +* xref:set-advanced-settings.adoc[] \ No newline at end of file diff --git a/modules/search/pages/create-child-field.adoc b/modules/search/pages/create-child-field.adoc new file mode 100644 index 000000000..a6315b407 --- /dev/null +++ b/modules/search/pages/create-child-field.adoc @@ -0,0 +1,58 @@ += Create a Child Field in Advanced Mode +:page-topic-type: guide +:description: Create a child field with the Couchbase Capella UI's Advanced Mode to add or remove a specific field's content from a Search index. + +[abstract] +{description} + +== Prerequisites + +* You have created an index. +For more information, see xref:create-search-index-ui.adoc[]. + +* You have created a type mapping. +For more information about how to create a type mapping on an index, see xref:create-type-mapping.adoc[]. + +* You have logged in to the Couchbase Capella UI. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. + +== Procedure + +To add or remove a child field from a Search index with the Capella UI: + +. On the *Databases* page, select the database that has the Search index you want to edit. +. Go to menu:Data Tools[Search]. +. Click the index where you want to create a child field. +. Under *Type Mappings*, next to an existing type mapping, go to menu:More Options (⋮)[Insert Child Field]. +. In the *Field* field, enter the name of a field in your documents that you want to add or remove from the index. ++ +NOTE: The field must contain either a single value or an array. If the field contains a JSON object, xref:create-child-mapping.adoc[Create a Child Mapping], instead. +. In the *Type* list, select the data type for the field: ++ +For more information about the available data types, see xref:field-data-types-reference.adoc[]. + +. Configure optional settings for the child field. ++ +For more information about the available settings for child fields, see xref:child-field-options-reference.adoc[]. +. Click btn:[Submit]. +. Click btn:[Update Index]. + +== Next Steps + +You can continue to create child fields to add or remove the contents of a document from your Search index. + +If a field in your documents contains a JSON object, xref:create-child-mapping.adoc[], instead. + +To continue customizing your Search index, you can also: + +* xref:set-advanced-settings.adoc[] +* xref:set-type-identifier.adoc[] +* xref:create-type-mapping.adoc[] +* xref:create-custom-analyzer.adoc[] +* xref:create-custom-character-filter.adoc[] +* xref:create-custom-tokenizer.adoc[] +* xref:create-custom-token-filter.adoc[] +* xref:create-custom-wordlist.adoc[] + +To run a search and test the contents of your Search index, see xref:simple-search-ui.adoc[] or xref:simple-search-rest-api.adoc[]. \ No newline at end of file diff --git a/modules/search/pages/.create-child-mapping.adoc b/modules/search/pages/create-child-mapping.adoc similarity index 84% rename from modules/search/pages/.create-child-mapping.adoc rename to modules/search/pages/create-child-mapping.adoc index faa9e2f66..dd5f6fc87 100644 --- a/modules/search/pages/.create-child-mapping.adoc +++ b/modules/search/pages/create-child-mapping.adoc @@ -1,19 +1,21 @@ -= Create a Child Mapping += Create a Child Mapping in Advanced Mode :page-topic-type: guide -:description: Create a child mapping with the Couchbase Capella UI to add or remove a field that contains a JSON object from a Search index. +:description: Create a child mapping with the Couchbase Capella UI's Advanced Mode to add or remove a field that contains a JSON object from a Search index. [abstract] {description} == Prerequisites -* You've created an index. +* You have created an index. For more information, see xref:create-search-index-ui.adoc[]. -* You've created a type mapping. +* You have created a type mapping. For more information about how to create a type mapping on an index, see xref:create-type-mapping.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase Capella UI. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure diff --git a/modules/search/pages/create-custom-analyzer.adoc b/modules/search/pages/create-custom-analyzer.adoc index 68acabab1..af132d1bf 100644 --- a/modules/search/pages/create-custom-analyzer.adoc +++ b/modules/search/pages/create-custom-analyzer.adoc @@ -1,6 +1,6 @@ = Create a Custom Analyzer :page-topic-type: guide -:description: Create a custom analyzer with the Couchbase Capella UI to modify the input text from a Search query or Search index and improve search results. +:description: Create a custom analyzer with the Couchbase Capella UI's Advanced Mode to modify the input text from a Search query or Search index and improve search results. [abstract] {description} diff --git a/modules/search/pages/create-search-index-ui.adoc b/modules/search/pages/create-search-index-ui.adoc index a57658f4e..fa4aac882 100644 --- a/modules/search/pages/create-search-index-ui.adoc +++ b/modules/search/pages/create-search-index-ui.adoc @@ -1,4 +1,4 @@ -= Create a Search Index in Advanced Mode += Create a Basic Search Index in Advanced Mode :page-topic-type: guide :description: Use Advanced Mode to create a Search index in Couchbase Capella with full customization and advanced features. @@ -26,7 +26,7 @@ To use Advanced Mode in Couchbase Capella to create a Search index: . On the *Databases* page, select the database where you want to create a Search index. . Go to menu:Data Tools[Search]. . Click btn:[Create Search Index]. -. Click the btn:[Advanced Mode] toggle. +. Click btn:[Advanced Mode]. . In the *Index Name* field, enter a name for the Search index. + [NOTE] @@ -36,25 +36,22 @@ Your index name must start with an alphabetic character (a-z or A-Z). It can onl For version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. ==== -. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. -. In the *Scope* list, select the scope where you want to create the index. -. (Optional) In the *Document Filter* list, configure an optional filter to add or remove documents from your Search index. -+ -For more information about how to configure a document filter, see xref:set-type-identifier.adoc[]. -. Add at least one mapping to your Search index: -.. In your list of collections, click a collection name. -.. (Optional) To index all fields from the document schema in the collection as a xref:customize-index.adoc#type-mappings[dynamic type mapping], click *Index everything from collection*. -. Click btn:[Add]. -. (Optional) If you did not configure your collection mapping as a xref:customize-index.adoc#type-mappings[dynamic type mapping]: -.. Expand a collection, then select a field from your document schema. -.. Configure a JSON object or child field type mapping. -.. Click btn:[Add]. -. (Optional) Add additional collection mappings or type mappings to your Search index. -. (Optional) Configure *Advanced Settings* for your Search index. -For more information, see xref:set-advanced-settings.adoc[]. +. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. +. In the *Scope* list, select the scope where you want to create your Search index. +. Click btn:[Add Mapping]. +. In the *Collection* list, select a collection for the type mapping. +. (Optional) To switch from a xref:customize-index.adoc#type-mappings[dynamic to a static type mapping], click *Index only specified fields*. +. Click btn:[Submit]. +. (Optional) Do any of the following: +.. If you created a xref:customize-index.adoc#type-mappings[static type mapping], then you should xref:create-child-field.adoc[add a child field] or xref:create-child-mapping.adoc[add a child mapping] to your type mapping to specify the documents you want to include in your index. +.. Add additional features to your Search index to improve performance and search results. +For more information, see xref:customize-index.adoc[]. . Click btn:[Create Index]. == Next Steps + +If you did not create a static type mapping, this basic index includes all documents from the collection you selected. +You can run a search against this index, but it's recommended that you customize your index to improve performance and reduce the index size. For more information about the different features you can add to your Search index to improve performance and search results, see xref:customize-index.adoc[]. diff --git a/modules/search/pages/create-search-indexes.adoc b/modules/search/pages/create-search-indexes.adoc index 881bff1ea..f20626dde 100644 --- a/modules/search/pages/create-search-indexes.adoc +++ b/modules/search/pages/create-search-indexes.adoc @@ -22,6 +22,9 @@ Couchbase Capella supports two editing modes for the Search Service: * xref:create-quick-index.adoc[Quick Mode] * xref:create-search-index-ui.adoc[Advanced Mode] +Use Quick Mode for a streamlined editing experience. +Use Advanced Mode to fully customize your Search Index with advanced settings and features. + After you create a Search index, the Search Service streams data from your chosen collection or collections, and any document mutations, into the index builder. Before your index finishes building, you can run a search and return partial results. diff --git a/modules/search/pages/create-type-mapping.adoc b/modules/search/pages/create-type-mapping.adoc index dbd0bb126..0d5fe3632 100644 --- a/modules/search/pages/create-type-mapping.adoc +++ b/modules/search/pages/create-type-mapping.adoc @@ -1,264 +1,53 @@ -= Type Mapping and Mapping Settings += Create a Type Mapping in Advanced Mode :page-topic-type: reference -:description: When you create a Search index in Couchbase Capella, you need to configure each type mapping or mapping you add to your index. -:page-toclevels: 3 -:page-aliases: create-child-field.adoc, create-child-mapping.adoc +:description: Add a type mapping to your Search index using the Couchbase Capella UI's Advanced Mode to control what documents are added to your index. [abstract] {description} -The configuration changes for <> and <>. +For more information about type mappings, see xref:customize-index.adoc#type-mappings[Search Index Features]. -[#quick-mode] -== Quick Mode Type Mappings +== Prerequisites -You can create the following types of type mappings in Quick Mode: +* You have started to create or already created a Search index in xref:create-search-index-ui.adoc[Advanced Mode]. -* <> -* <> +* You have logged in to the Couchbase Capella UI. -TIP: Indexing an entire collection in Quick Mode creates a dynamic type mapping. -Indexing a field creates a static type mapping. -For more information about static and dynamic type mappings, see xref:customize-index.adoc#type-mappings[Search Index Features]. +== Procedure -For more information about how to create a Search index with Quick Mode, see xref:create-quick-index.adoc[]. +To create a type mapping with the Capella UI: -[#quick-collection] -=== Quick Collection Type Mapping Options +. On the *Databases* page, select the database that has the Search index you want to edit. +. Go to menu:Data Tools[Search]. +. Click the index where you want to create a type mapping. +. Under *Type Mappings*, in the Bucket list, select the bucket where you want to create a type mapping. +. In the *Scope* list, select the scope where you want to create a type mapping. +. Click btn:[Add Mapping]. +. In the *Collection* list, select the collection to use for the type mapping. +. (Optional) Do any of the following: +.. To use a specific analyzer for documents added from this type mapping, in the *Analyzer* list, select an analyzer. ++ +By default, the mapping inherits the default analyzer you set for the Search index. For more information about analyzers, see xref:customize-index.adoc#analyzers[Search Index Features]. +.. To switch from a xref:customize-index.adoc#type-mappings[dynamic type mapping to a static type mapping], select *Only index specified fields*. +. Click btn:[Submit]. -Configure an entire collection in Quick Mode to add or remove all documents in that collection from your Search index. -For more information about how to create an index in Quick Mode, see xref:create-quick-index.adoc[]. +== Next Steps -You can configure the following settings in Quick Mode for a type mapping that uses an entire collection: +After you create a static type mapping, you can xref:create-child-field.adoc[] to add or remove specific document fields from your Search index. -|==== -|Option |Description +To add an extra filter to the documents selected by your type mapping, you can also xref:set-type-identifier.adoc[]. -| Index everything from collection: "`$COLLECTION_NAME`" a| +If you have a document field that contains a JSON object, xref:create-child-mapping.adoc[] for that field. -You must select *Index everything from collection* to add a collection type mapping in Quick Mode. +To continue customizing your Search index, you can: -| Index text fields as identifiers a| +* xref:set-advanced-settings.adoc[] +* xref:set-type-identifier.adoc[] +* xref:create-custom-analyzer.adoc[] +* xref:create-custom-character-filter.adoc[] +* xref:create-custom-token-filter.adoc[] +* xref:create-custom-tokenizer.adoc[] +* xref:create-custom-wordlist.adoc[] -To use a *Text* field to set the document's type in the index, select *Index text fields as identifiers*. -To not use a *Text* field as the document's type, clear *Index text fields as identifiers*. - -|Analyzer/Language a| - -Select the language for the content inside any *Text* fields. - -The Search Service automatically applies an xref:customize-index.adoc#analyzers[analyzer] to the field's contents based on the selected language. - -For more information about the available language options, see xref:quick-index-supported-languages.adoc[]. - -|==== - -[#quick-field] -=== Quick Field Type Mapping Options - -Configure a single field in Quick Mode to add or remove it from your Search index. -For more information about how to create an index in Quick Mode, see xref:create-quick-index.adoc[]. - -You can configure the following settings in Quick Mode for a type mapping that uses a single field: - -|==== -|Option |Description - -//|Dimension (Vector Only) a| - -//|Similarity Metric (Vector Only) a| - -//* *dotproduct*: -//* *euclidean*: - -|Type a| - -Choose the type of data stored in this field inside your documents. - -For more information about the available field data types, see xref:field-data-types-reference.adoc[]. - -|Index this field as an identifier (Text Only) a| - -To use a *Text* field to set the document's type in the index, select *Index this field as an identifier*. - -To not use a *Text* field as the document's type, clear *Index this field as an identifier*. - -|Analyzer/Language (Text Only) a| - -Select the language for the content inside a *Text* field. - -The Search Service automatically applies an xref:customize-index.adoc#analyzers[analyzer] to the field's contents based on the selected language. - -For more information about the available language options, see xref:quick-index-supported-languages.adoc[]. - -|Include in search results a| - -To include content from the field in search results, select *Include in search results*. - -To exclude the field's content from search results, clear *Include in search results*. - -|Support highlighting a| - -The Search Service can highlight matching search terms in search results from an index. - -To enable highlighting in search results, select *Support highlighting*. - -To turn off highlighting in search results, clear *Support highlighting*. - -NOTE: To enable *Support highlighting*, you must also enable *Include in search results*. - -|Support phrase matching (Text Only) a| - -To support searches for whole phrases in *Text* fields, select *Support phrase matching*. - -To turn off phrase matching for *Text* fields, clear *Support phrase matching*. - -|Support field agnostic search a| - -To search the field's contents without specifying the field name in a search query, select *Support field agnostic search*. - -To turn off field agnostic search, clear *Support field agnostic search*. - -|Support sorting and faceting a| - -To sort search results and use xref:search-request-params.adoc#facets[facets] with the field's contents, select *Support sorting and faceting*. - -To turn off sorting and facets, clear *Support sorting and faceting*. - -|Searchable As a| - -Set a different name that you can use to search the field's contents in a query. - -The default value is the field's name. - -|==== - -[#adv-mode] -== Advanced Mode Mappings - -You can create the following types of mappings in Advanced Mode: - -* <> -* <> -* <> - -TIP: In Advanced Mode, you can index an entire collection or a field as a static or dynamic type mapping. -For more information about static and dynamic type mappings, see xref:customize-index.adoc#type-mappings[Search Index Features]. - -For more information about how to create a Search index with Advanced Mode, see xref:create-search-index-ui.adoc[]. - -[#adv-collection] -=== Advanced Collection Mapping Options - -Map a collection to add or remove documents from that collection from your Search index. - -NOTE: Every index in Advanced Mode must have at least one collection mapping. - -For more information about how to create a Search index with Advanced Mode, see xref:create-search-index-ui.adoc[]. - -You can choose to set your collection mappings as xref:customize-index.adoc#type-mappings[dynamic type mappings]: - -|==== -|Option |Description - -|Index everything from collection: "`$COLLECTION_NAME`" a| - -To set the selected collection as a xref:customize-index.adoc#type-mappings[dynamic type mapping] and index all fields from its document schema, select *Index everything from collection*. - -TIP: You can change this behavior in your xref:set-advanced-settings.adoc[general settings]. - -To only index specific fields from the document schema, clear *Index everything from collection*. - -|==== - -[#adv-object] -=== Advanced JSON Object Field Mapping Options - -You can use the child fields of a JSON object field from your document schema in your Search index. - -In Advanced Mode, index a JSON object as an *Object* mapping. -You can then choose whether to use the JSON object as a xref:customize-index.adoc#type-mappings[dynamic type mapping]. - -For more information about how to create an index in Advanced Mode, see xref:create-search-index-ui.adoc[]. - -You can configure the following settings in Advanced Mode for a mapping that uses a JSON object field: - -|==== -|Option |Description - -|Analyzer/Language a| - -Select the analyzer to use for any text content inside the JSON object. - -You can use a xref:default-analyzers-reference.adoc[default analyzer], inherit the xref:set-advanced-settings.adoc[analyzer set for the Search index], or xref:create-custom-analyzer.adoc[create a custom analyzer]. - -|Index only specified fields a| - -To only index the fields you select and configure from the document schema under the JSON object, select *Index everything from collection*. - -To set the selected JSON object as a xref:customize-index.adoc#type-mappings[dynamic type mapping] and index all child fields from the document schema, clear *Index only specified fields*. - -TIP: You can change this behavior in your xref:set-advanced-settings.adoc[general settings]. - -|==== - - -[#adv-child] -=== Advanced Child Field Type Mapping Options - -To add or remove a single child field from your Search index, index the field as a *Child* type mapping. - -For more information about how to create an index in Advanced Mode, see xref:create-search-index-ui.adoc[]. - -You can configure the following settings in Advanced Mode for a type mapping that uses a single field: - -|==== -|Option |Description - -|Type a| - -Choose the type of data stored in this field inside your documents. - -For more information about the available field data types, see xref:field-data-types-reference.adoc[]. - -|Analyzer/Language a| - -If the child field's Type is *Text*, select the analyzer to use. - -You can use a xref:default-analyzers-reference.adoc[default analyzer], inherit the xref:set-advanced-settings.adoc[analyzer set for the Search index], or xref:create-custom-analyzer.adoc[create a custom analyzer]. - -|Index this field a| - -To add this field to your Search index, select *Index this field*. - -To exclude this field from your Search index and any search results, clear *Index this field*. - -|Store field values a| - -To store the values from the field in the index and return them in search results, select *Store*. -This increases your index's size and indexing time. - -To remove the field's values from the index, clear *Store*. - -|Include in _all field a| - -The `_all` field is a composite field that has the content from multiple fields in an index. -It allows searches to query the content of a field without specifying the field's name. - -To include this field in the `_all` field, select *Include in _all field*. - -To exclude this field from the `_all` field, clear *Include in _all field*. - -TIP: To change the name of the `_all` field, see xref:set-advanced-settings.adoc#all-field[Default Field]. - -|Store docvalues for facets a| - -Doc values are the value for each instance of the field in an index. -Use doc values for Search xref:search-request-params.adoc#facets[Facets] and sorting search results. - -To store doc values, select *Doc Values*. - -To exclude doc values from the index, clear *Doc Values*. - -|==== \ No newline at end of file +To run a search and test the contents of your Search index, see xref:simple-search-ui.adoc[] or xref:simple-search-rest-api.adoc[]. diff --git a/modules/search/pages/customize-index.adoc b/modules/search/pages/customize-index.adoc index 24af22f0b..6699ecc26 100644 --- a/modules/search/pages/customize-index.adoc +++ b/modules/search/pages/customize-index.adoc @@ -5,6 +5,8 @@ [abstract] {description} +Some features are only available in xref:create-search-index-ui.adoc[Advanced Mode]. + You can use the following components and configure the following options for a Search index: [cols="2,1,1,4"] @@ -18,21 +20,27 @@ a|Use a type mapping to include or exclude specific documents in a collection fr Type mappings can also set a field's data type and other settings. -Add mappings for JSON objects or child fields to type mappings to improve your Search index's performance, instead of indexing entire collections. - You can create two types of type mappings with the Search Service: * *Dynamic type mappings*: When you do not know the structure of your data fields ahead of time, use a dynamic type mapping to add all available fields from a matching document type to an index. -For example, you could create a dynamic type mapping to include all documents from the `hotel` collection in your Search index, or include all fields under a JSON object from your document schema. -Configure this type of mapping by selecting a collection or use the *Object* option in xref:create-search-index-ui.adoc[Advanced Mode]. +For example, you could create a dynamic type mapping to include all documents from the `hotel` collection in your Search index, or include all fields under a JSON object from your document schema. ++ +Configure this type of mapping by selecting a collection in xref:create-quick-index.adoc[Quick Mode] or by clearing *Only index specified fields* when you xref:create-type-mapping.adoc[]. + * *Static type mappings*: When your data fields are stable and unlikely to change, use a static type mapping to add and define only specific fields from a matching document type to an index. -For example, you could create a static type mapping to only include the contents of the `city` field from the `hotel` collection in your Search index, as a text field with an `en` analyzer. -Configure this type of mapping by selecting a field and using the *Child* option in xref:create-search-index-ui.adoc[Advanced Mode]. +For example, you could create a static type mapping to only include the contents of the `city` field from the `hotel` collection in your Search index, as a text field with an `en` analyzer. ++ +Configure this type of mapping by selecting a field in your document schema in xref:create-quick-index.adoc[Quick Mode]. ++ +In Advanced Mode, xref:create-type-mapping.adoc[create a type mapping] and xref:create-child-field.adoc[add a child field mapping] to create a static type mapping. Type mappings start at the collection level. -Create additional mappings under a collection-level type mapping to restrict the documents added to your index. +Create additional mappings for child fields or JSON objects under a collection's type mapping to restrict the documents added to your index. +This can improve Search index performance over indexing entire collections. + +For more information about how to configure settings for mappings and type mappings in Quick Mode, see xref:quick-index-field-options.adoc[]. -For more information about how to configure settings for mappings and type mappings, see xref:create-type-mapping.adoc[]. +For more information about how to configure a type mapping in Advanced Mode, see xref:create-type-mapping.adoc[]. |Advanced Settings | @@ -88,7 +96,7 @@ For more information about how to add a custom date/time parser, see xref:create [#custom-filters] == Custom Filters -Custom filters are components of a Search index <>. +Custom filters are components of a Search index <>. Create and add these components to a custom analyzer to improve search results and performance for an index in xref:create-search-indexes-ui.adoc[Advanced Mode]. diff --git a/modules/search/pages/default-analyzers-reference.adoc b/modules/search/pages/default-analyzers-reference.adoc index f42c335ba..97ad3b56c 100644 --- a/modules/search/pages/default-analyzers-reference.adoc +++ b/modules/search/pages/default-analyzers-reference.adoc @@ -52,7 +52,7 @@ The following default analyzer options are available: |it |The `it` analyzer uses character filters, tokenizers, and token filters designed for Italian language searches. -|keyword a| +|[[keyword]]keyword a| The `keyword` analyzer turns input into a single token. It forces exact matches and preserves whitespace characters like spaces. diff --git a/modules/search/pages/field-data-types-reference.adoc b/modules/search/pages/field-data-types-reference.adoc index 0dcbb64c0..1f495a6f0 100644 --- a/modules/search/pages/field-data-types-reference.adoc +++ b/modules/search/pages/field-data-types-reference.adoc @@ -5,9 +5,10 @@ [abstract] {description} -When you xref:create-type-mapping.adoc[create a type mapping for a child field], you need to set its data type. +When you create a mapping for a field in xref:create-quick-index.adoc[Quick Mode] or xref:xref:create-child-field.adoc[Advanced Mode], you need to set the field's data type. If you create a Search index and do not set a data type for a field, the Search Service automatically assigns a field data type. +For example, if you created a xref:customize-index.adoc#type-mappings[dynamic type mapping], the Search Service automatically assigns data types to all fields in the type mapping. The following field data types are available: @@ -33,5 +34,29 @@ The field contains geopoint (latitude and longitude) data, represented as either * An array, as two floating point integers. * A JSON object, with the properties `lon`/`lng` and `lat`. -|geoshape |The field contains a GeoJSON object. For more information about GeoJSON objects, see xref:server:fts:fts-supported-queries-geojson-spatial.adoc[]. +|geoshape a| + +The field contains a GeoJSON object. + +A GeoJSON object describes a shape made of floating point coordinates with the following JSON properties: + +* A `type` string, for the type of GeoJSON object. +For example, a `point` or `MultiLineString`. + +* A `coordinates` array of floating point numbers, for each latitude and longitude coordinate point in the GeoJSON shape. + +For example: +---- +{ + "type": "linestring", + "coordinates": [ + + [-2.753735609842721, 53.94860827535115], + [-2.599898256093695,53.65007434185782] + ] +} +---- +This JSON object describes a `LineString` GeoJSON object with 2 latitude and longitude coordinates. + +For more information about GeoJSON queries, see xref:search-request-params.adoc#query-object[the Query object]. |==== \ No newline at end of file diff --git a/modules/search/pages/quick-index-field-options.adoc b/modules/search/pages/quick-index-field-options.adoc new file mode 100644 index 000000000..bad9c82b0 --- /dev/null +++ b/modules/search/pages/quick-index-field-options.adoc @@ -0,0 +1,117 @@ += Quick Index Mapping and Field Options +:page-topic-type: reference +:description: When you create a Search index with Quick Mode in the Capella UI, you must set options for each field or type mapping you add to the index. + +[abstract] +{description} + +You can create the following types of type mappings in Quick Mode: + +* <> +* <> + +TIP: Indexing an entire collection in Quick Mode creates a dynamic type mapping for that collection. +Indexing a field creates a static type mapping for the parent collection. +For more information about static and dynamic type mappings, see xref:customize-index.adoc#type-mappings[Search Index Features]. + +For more information about how to create a Search index with Quick Mode, see xref:create-quick-index.adoc[]. + +[#quick-collection] +== Quick Collection Type Mapping Options + +Configure an entire collection in Quick Mode to add or remove all documents in that collection from your Search index. + +You can configure the following settings in Quick Mode for a type mapping that uses an entire collection: + +|==== +|Option |Description + +| Index everything from collection: "`$COLLECTION_NAME`" a| + +You must select *Index everything from collection* to add a collection type mapping in Quick Mode. + +| Index text fields as identifiers a| + +To index any *Text* type fields with the xref:default-analyzers-reference.adoc#keyword[keyword analyzer], select *Index text fields as identifiers*. + +To use other analyzers or settings on text fields, clear *Index text fields as identifiers*. + +For more information about analyzers, see xref:customize-index.adoc#analyzers[Search Index Features]. + +|Analyzer/Language a| + +Select the language for the content inside any *Text* fields. + +The Search Service automatically applies an xref:customize-index.adoc#analyzers[analyzer] to the field's contents based on the selected language. + +For more information about the available language options, see xref:quick-index-supported-languages.adoc[]. + +|==== + +[#quick-field] +== Quick Field Type Mapping Options + +Configure a single field in Quick Mode to add or remove it from your Search index. + +You can configure the following settings in Quick Mode for a type mapping that uses a single field: + +|==== +|Option |Description + +|Index this field as an identifier (Text Only) a| + +To index this field with the xref:default-analyzers-reference.adoc#keyword[keyword analyzer], select *Index this field as an identifier*. + +To use other analyzers or settings on this text field, clear *Index this field as an identifier*. + +For more information about analyzers, see xref:customize-index.adoc#analyzers.adoc[Search Index Features]. + +|Language (Text Only) a| + +Select the language for the content inside a text field. + +The Search Service automatically applies an xref:customize-index.adoc#analyzers[analyzer] to the field's contents based on the selected language. + +For more information about the available language options, see xref:quick-index-supported-languages.adoc[]. + +|Include in search results a| + +To include content from the field in search results, select *Include in search results*. + +To exclude the field's content from search results, clear *Include in search results*. + +|Support highlighting a| + +The Search Service can highlight matching search terms in search results from an index. + +To enable highlighting in search results, select *Support highlighting*. + +To turn off highlighting in search results, clear *Support highlighting*. + +NOTE: To enable *Support highlighting*, you must also enable *Include in search results*. + +|Support phrase matching a| + +To support searches for whole phrases, select *Support phrase matching*. + +To turn off phrase matching, clear *Support phrase matching*. + +|Support field agnostic search a| + +To search the field's contents without specifying the field name in a search query, select *Support field agnostic search*. + +To turn off field agnostic search, clear *Support field agnostic search*. + +|Support sorting and faceting a| + +To sort search results and use xref:search-request-params.adoc#facets[facets] with the field's contents, select *Support sorting and faceting*. + +To turn off sorting and facets, clear *Support sorting and faceting*. + +|Searchable As a| + +Set a different name that you can use to search the field's contents in a query. + +The default value is the field's name. + +|==== \ No newline at end of file diff --git a/modules/search/pages/set-advanced-settings.adoc b/modules/search/pages/set-advanced-settings.adoc index 3af2a2e38..048dd1bbc 100644 --- a/modules/search/pages/set-advanced-settings.adoc +++ b/modules/search/pages/set-advanced-settings.adoc @@ -44,7 +44,7 @@ For more information about how to create your own custom analyzer, see xref:crea |[[all-field]]Default Field a| -When you xref:create-type-mapping.adoc[create a type mapping for a child field], you can choose to include that field in an `_all` field. +When you xref:create-child-field.adoc[create a mapping for a child field], you can choose to include that field in an `_all` field. You can add fields to the `_all` field to search their contents without specifying their field name in your search query. diff --git a/modules/search/pages/set-type-identifier.adoc b/modules/search/pages/set-type-identifier.adoc index 9c5c67926..9ab1cfd1c 100644 --- a/modules/search/pages/set-type-identifier.adoc +++ b/modules/search/pages/set-type-identifier.adoc @@ -1,12 +1,12 @@ -= Set a Search Index Document Filter += Set a Search Index Type Identifier :page-topic-type: guide -:description: Use a document filter with a type mapping to add an extra filter to the documents you want to include in a Search index. +:description: Use a type identifier with a type mapping to add an extra filter to the documents you want to include in a Search index. :page-toclevels: 3 [abstract] {description} -For more information about document filters and type mappings, see xref:customize-index.adoc#type-identifiers[Customize a Search Index with the Capella UI]. +For more information about type identifiers and type mappings, see xref:customize-index.adoc#type-identifiers[Search Index Features]. == Prerequisites @@ -22,58 +22,73 @@ For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[] == Procedure -To set a document filter for a Search index with the Capella UI: +To set a type identifier for a Search index with the Capella UI: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. -. Click the index where you want to create a document filter -. Under *Type Mappings*, in the *Document Filter* list, choose the document filter you want to add to your Search index: +. Click the index where you want to create a type identifier. +. Expand *General Settings*. +. Do one of the following: .. <> .. <> .. <> [#json-type] -=== Create a JSON Type Field Document Filter +=== Create a JSON Type Field Type Identifier To only add documents to your Search index that contain a specific field with a specified string value: -. In the *Field* field, enter the name of the field in your documents that you want to use to filter documents in your Search index. +. Click the *JSON Type Field* tab. +. In the *JSON Type Field* field, enter the name of the field in your documents that you want to use to filter documents in your Search index. + -For example, if your documents include a `type` field, you could enter `type` in the *Field* field. +For example, if your documents include a `type` field, you could enter `type` in the *JSON Type Field* field. + -NOTE: You cannot use a field as a document filter if the field name contains a period (.). -. In the *docvalue* field, enter the exact string from the document field that you want to use as a filter. +NOTE: You cannot use a field as a type identifier if the field name contains a period (.). +. Under *Type Mappings*, next to the type mapping where you want to add the type identifier, go to menu:More Options (⋮)[Edit]. +. In the *Name* field, add a period (.) to the end of the current type mapping name. +. After the period, add the exact string from the document field that you want to use as a filter. + For example, if you wanted your Search index to only return documents that had a `type` value of `hotel`, you could enter `hotel`. +. Click btn:[Submit]. +. Click btn:[Update Index]. [#doc-id-sep] -=== Create a Doc ID Up To Separator Document Filter +=== Create a Doc ID Up To Separator Type Identifier To only add documents to your Search index that have IDs that match a specified prefix: - -. In the *Separator* field, enter the separator character from the ID prefix in your document ID values. + +. Click the *Doc ID up to Separator* tab. +. In the *Doc ID up to Separator* field, enter the separator character from the ID prefix in your document ID values. + For example, if you know all of your document ID values are prefixed by a string and an underscore (_), enter `_`. -. In the *ID prefix* field, enter the exact prefix from the document's ID value that you want to use as a filter. +. Under *Type Mappings*, next to the type mapping where you want to add the type identifier, go to menu:More Options (⋮)[Edit]. +. In the *Name* field, add a period (.) to the end of the current type mapping name. +. After the period, add the exact prefix from the document's ID value that you want to use as a filter. + For example, if you wanted your Search index to only return documents that have an prefix of `landmark_`, you could enter `landmark`. +. Click btn:[Submit]. +. Click btn:[Update Index]. [#doc-id-regex] -=== Create a Doc ID with Regex document filter +=== Create a Doc ID with Regex Type Identifier To only add documents to your Search index that have IDs that match a specified https://github.com/google/re2/wiki/Syntax[RE2] regular expression: -. In the *regex* field, enter the regular expression that you want to use to filter documents in your Search index. +. Click the *Doc ID with Regex* tab. +. In the *Doc ID with Regex* field, enter the regular expression that you want to use to filter documents in your Search index. + -For example, if you wanted only documents with ID values that contained `_40`, you could enter `_[3-5]0` as your regular expression. -. In the *ID prefix* field, add a match for the regular expression from the document's ID value that you want to use as a filter. +For example, if you wanted documents with ID values that contained `_40`, you could enter `_[3-5]0` as your regular expression. +. Under *Type Mappings*, next to the type mapping where you want to add the type identifier, go to menu:More Options (⋮)[Edit]. +. In the *Name* field, add a period (.) to the end of the current type mapping name. +. After the period, add a match for the regular expression from the document's ID value that you want to use as a filter. + For example, if you wanted your Search index to only return documents with ID values that contained `_40`, you could enter `_40`. - +. Click btn:[Submit]. +. Click btn:[Update Index]. == Next Steps -After you set the document filter for your Search index, you can continue to customize your Search index: +After you set the type identifier for your Search index, you can continue to customize your Search index: * xref:set-advanced-settings.adoc[] * xref:create-type-mapping.adoc[] diff --git a/modules/search/partials/nav.adoc b/modules/search/partials/nav.adoc index 33f9b02ba..e2aace02c 100644 --- a/modules/search/partials/nav.adoc +++ b/modules/search/partials/nav.adoc @@ -1,11 +1,15 @@ * xref:cloud:search:search.adoc[] ** xref:cloud:search:create-search-indexes.adoc[] *** xref:cloud:search:customize-index.adoc[] - *** xref:cloud:search:create-type-mapping.adoc[] *** xref:cloud:search:field-data-types-reference.adoc[] *** xref:cloud:search:create-quick-index.adoc[] + **** xref:cloud:search:quick-index-field-options.adoc[] **** xref:cloud:search:quick-index-supported-languages.adoc[] *** xref:cloud:search:create-search-index-ui.adoc[] + **** xref:cloud:search:create-type-mapping.adoc[] + **** xref:cloud:search:create-child-field.adoc[] + ***** xref:cloud:search:child-field-options-reference.adoc[] + **** xref:cloud:search:create-child-mapping.adoc[] **** xref:cloud:search:set-type-identifier.adoc[] **** xref:cloud:search:set-advanced-settings.adoc[] **** xref:cloud:search:create-custom-analyzer.adoc[] From f4989efaf9d01c3ba0e6c5c0a1dc2117580c25c0 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 8 Feb 2024 12:14:11 -0500 Subject: [PATCH 17/59] [AV-68962] Copying over changes from Server version. --- .../vector-search/pages/vector-search.adoc | 20 +++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc index ca32961ba..60e1e5f53 100644 --- a/modules/vector-search/pages/vector-search.adoc +++ b/modules/vector-search/pages/vector-search.adoc @@ -7,10 +7,10 @@ == About Vector Search -Vector Search builds on Capella's xref:search.adoc[Search Service] to provide vector index support. +Vector Search builds on Couchbase Capella's xref:search.adoc[Search Service] to provide vector index support. You can use these new Vector Search indexes for Retrieval Augmented Generation (RAG) with an existing Large Language Model (LLM). -Using Vector Search, Couchbase Server, an embedding model, and your chosen LLM, you can develop AI applications while giving context and up-to-date information from your own data. +Using Vector Search, Capella, an embedding model, and your chosen LLM, you can develop AI applications while giving context and up-to-date information from your own data. You can develop applications that include: @@ -27,21 +27,25 @@ For more information about all frameworks and integrations supported by Vector S == Using Vector Search Indexes -Use a Couchbase Capella database to store the data you want to use to provide context to a LLM. +Use a Capella database to store the data you want to use to provide context to a LLM. Generate vector embeddings using your preferred embedding model, then store the embeddings in an array inside your documents. Create a Vector Search index to use the vector embeddings from your data and find matches from a search query's generated vectors. If you do not want to use an integration like LangChain, you can also use the API for an existing LLM and one of their embedding models to generate these embeddings for your data. -For example, the OpenAI `embeddings` endpoint can generate embeddings for a text string using a specified embedding model. +For example, the OpenAI `embeddings` endpoint can generate embeddings for a text string using a specified embedding model. For more information about how to generate and obtain embeddings for text strings using the OpenAI API, see the https://platform.openai.com/docs/guides/embeddings/what-are-embeddings[Embeddings documentation]. NOTE: When creating a Vector Search index, the dimension of your data vector embeddings must match the dimension for any search query vectors. -For more information about how to create a Vector Search index, see xref:[]. -For information about how to run a Vector Search query, see xref:[]. -// Don't forget to provide these links! And add to See Also +For more information about how to create a Vector Search index, see xref:create-vector-search-index-ui.adoc[] or xref:create-vector-search-index-rest-api.adoc[]. + +For information about how to run a Vector Search query, see xref:run-vector-search-ui.adoc[] or xref:run-vector-search-rest-api.adoc[]. == See Also -* xref:search:search.adoc[] \ No newline at end of file +* xref:search:search.adoc[] +* xref:create-vector-search-index-ui.adoc[] +* xref:create-vector-search-index-rest-api.adoc[] +* xref:run-vector-search-ui.adoc[] +* xref:run-vector-search-rest-api.adoc[] \ No newline at end of file From 469b5ee1963e272fd2f3095f30f625bbeb868a13 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 8 Feb 2024 14:36:32 -0500 Subject: [PATCH 18/59] [AV-68963] Adding new vector settings to child-field-options-reference Adding ip and vector type to field-data-types-reference Making consistency changes and adding vector fields to quick-index-field-options Adding page-toclevels, vector properties, and ip type to search-index-params. Correcting page links in search-request-params. Adding partial for vector search field descriptions Adding topic for creating a vector search index with the UI Adding UI topic back into the nav for vector search --- .../pages/child-field-options-reference.adoc | 12 +++ .../pages/field-data-types-reference.adoc | 20 +++++ .../pages/quick-index-field-options.adoc | 16 +++- modules/search/pages/search-index-params.adoc | 23 +++++- .../search/pages/search-request-params.adoc | 10 +-- .../vector-search-field-descriptions.adoc | 37 ++++++++++ .../pages/create-vector-search-index-ui.adoc | 74 +++++++++++++++++++ modules/vector-search/partials/nav.adoc | 2 +- 8 files changed, 182 insertions(+), 12 deletions(-) create mode 100644 modules/search/partials/vector-search-field-descriptions.adoc create mode 100644 modules/vector-search/pages/create-vector-search-index-ui.adoc diff --git a/modules/search/pages/child-field-options-reference.adoc b/modules/search/pages/child-field-options-reference.adoc index 242dfa22b..0c2d3be94 100644 --- a/modules/search/pages/child-field-options-reference.adoc +++ b/modules/search/pages/child-field-options-reference.adoc @@ -15,6 +15,18 @@ You can configure the following options for a child field in Advanced Mode, base |==== |Option |Description +|Dimension (Vector Fields Only) a| + +include::partial$vector-search-field-descriptions.adoc[tag=dimension] + +|Similarity Metric (Vector Fields Only) a| + +include::partial$vector-search-field-descriptions.adoc[tag=similarity_metric] + +|Optimized For (Vector Fields Only) a| + +include::partial$vector-search-field-descriptions.adoc[tag=optimized_for] + |Searchable As a| Set a different name that you can use to search the field's contents in a query. diff --git a/modules/search/pages/field-data-types-reference.adoc b/modules/search/pages/field-data-types-reference.adoc index 1f495a6f0..100285421 100644 --- a/modules/search/pages/field-data-types-reference.adoc +++ b/modules/search/pages/field-data-types-reference.adoc @@ -59,4 +59,24 @@ For example: This JSON object describes a `LineString` GeoJSON object with 2 latitude and longitude coordinates. For more information about GeoJSON queries, see xref:search-request-params.adoc#query-object[the Query object]. + +|ip a| + +The field contains an IP address, formatted in IPv4 or IPv6 CIDR syntax. + +For example: +---- +{ + "ipv4": "4.7.44.162", + "ipv6": "2001:4800:0000:0000:0000:0000:0000:0000" +} +---- + +|vector a| + +The field contains an array of floating point numbers. + +Use the `vector` type to perform vector similarity searches with Vector Search. + +For more information about Vector Search, see xref:vector-search:vector-search.adoc[]. |==== \ No newline at end of file diff --git a/modules/search/pages/quick-index-field-options.adoc b/modules/search/pages/quick-index-field-options.adoc index bad9c82b0..5ef063fb9 100644 --- a/modules/search/pages/quick-index-field-options.adoc +++ b/modules/search/pages/quick-index-field-options.adoc @@ -58,7 +58,7 @@ You can configure the following settings in Quick Mode for a type mapping that u |==== |Option |Description -|Index this field as an identifier (Text Only) a| +|Index this field as an identifier (Text Fields Only) a| To index this field with the xref:default-analyzers-reference.adoc#keyword[keyword analyzer], select *Index this field as an identifier*. @@ -66,7 +66,7 @@ To use other analyzers or settings on this text field, clear *Index this field a For more information about analyzers, see xref:customize-index.adoc#analyzers.adoc[Search Index Features]. -|Language (Text Only) a| +|Language (Text Fields Only) a| Select the language for the content inside a text field. @@ -74,6 +74,18 @@ The Search Service automatically applies an xref:customize-index.adoc#analyzers[ For more information about the available language options, see xref:quick-index-supported-languages.adoc[]. +|Dimension (Vector Fields Only) a| + +include::partial$vector-search-field-descriptions.adoc[tag=dimension] + +|Similarity Metric (Vector Fields Only) a| + +include::partial$vector-search-field-descriptions.adoc[tag=similarity_metric] + +|Optimized For (Vector Fields Only) a| + +include::partial$vector-search-field-descriptions.adoc[tag=optimized_for] + |Include in search results a| To include content from the field in search results, select *Include in search results*. diff --git a/modules/search/pages/search-index-params.adoc b/modules/search/pages/search-index-params.adoc index 3a8665308..21fb75899 100644 --- a/modules/search/pages/search-index-params.adoc +++ b/modules/search/pages/search-index-params.adoc @@ -1,6 +1,7 @@ = Search Index JSON Properties :page-topic-type: reference :description: Use a JSON payload to control the settings for a Search index. +:page-toclevels: 2 [abstract] {description} @@ -970,6 +971,16 @@ It contains the following properties: |==== |Property |Type |Required? |Description +|analyzer |String |Text Only a| + +If the child field's `type` is `text`, set the analyzer to use for the child field. + +NOTE: If you want to use the default analyzer for the content of this child field, you don't need to include an `analyzer` property. + +|[[dims]]dims |Number |Vector Only a| + +include::partial$vector-search-field-descriptions.adoc[tag=dimension] + |docvalues |Boolean |Yes a| To include the value for each instance of the field in the Search index to support xref:search-request-params.adoc#facet-name[facets] and sorting search results, set `docvalues` to `true`. @@ -1002,6 +1013,10 @@ To exclude the child field from the index, set `index` to `false`. |name |String |Yes |The child field's name. +|[[similarity]]similarity |String |Vector Only a| + +include::partial$vector-search-field-descriptions.adoc[tag=similarity_metric] + |store |Boolean |Yes a| To include the content of the child field in the Search index and allow its content to be viewed in search results, set `store` to `true`. @@ -1019,13 +1034,13 @@ The child field's type. Can be one of: * `geopoint` * `geoshape` * `disabled` +* `ip` +* `vector` For more information about the available field data types, see xref:field-data-types-reference.adoc[]. -|analyzer |String |No a| - -If the child field's `type` is `text`, set the analyzer to use for the child field. +|vector_index_optimized_for |String |Vector Only a| -NOTE: If you want to use the default analyzer for the content of this child field, you do not need to include an `analyzer` property. +include::partial$vector-search-field-descriptions.adoc[tag=optimized_for] |==== diff --git a/modules/search/pages/search-request-params.adoc b/modules/search/pages/search-request-params.adoc index 4cc2f2138..2b05038b4 100644 --- a/modules/search/pages/search-request-params.adoc +++ b/modules/search/pages/search-request-params.adoc @@ -92,7 +92,7 @@ For more information about the properties for a `sort` JSON object, see < To return the position of each occurrence of a search term inside a document, set `includeLocations` to `true`. NOTE: You must have *Include Term Vectors* enabled or the `include_term_vectors` property set to `true` on a field to use `includeLocations`. -For more information about how to enable term vectors, see xref:create-child-field.adoc#term-vectors[Create a Child Field] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. +For more information about how to enable term vectors, see xref:child-field-options-reference.adoc#term-vectors[Child Field Options] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. To turn off term locations, set `includeLocations` to `false`. @@ -430,7 +430,7 @@ To use a match phrase query: * You must specify a field to search with the <> or the <> syntax in your search query. * You must have *Include Term Vectors* enabled or the `include_term_vectors` property set to `true` on the field you want to search. -For more information about how to enable term vectors, see xref:create-child-field.adoc#term-vectors[Create a Child Field] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. +For more information about how to enable term vectors, see xref:child-field-options-reference.adoc#term-vectors[Child Field Options] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. The Search Service searches for exact matches to the phrase you specify. @@ -471,7 +471,7 @@ a| include::example$query-analytic.jsonc[tag=match] ---- -| match_phrase +| [[match_phrase]]match_phrase | String a| Use the `match_phrase` property to run a match phrase query. The Search Service searches for exact matches to the phrase you specify. @@ -483,7 +483,7 @@ To use a match phrase query: * You must specify a field to search with the <> or the <> syntax in your search query. * You must have *Include Term Vectors* enabled or the `include_term_vectors` property set to `true` on the field you want to search. -For more information about how to enable term vectors, see xref:create-child-field.adoc#term-vectors[Create a Child Field] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. +For more information about how to enable term vectors, see xref:child-field-options-reference.adoc#term-vectors[Child Field Options] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. For more information about the additional properties you can specify with a match phrase query, see <>. a| @@ -589,7 +589,7 @@ To use a phrase query: * You must set the <> in your search query. * You must have *Include Term Vectors* enabled or the `include_term_vectors` property set to `true` on the field you want to search. -For more information about how to enable term vectors, see xref:create-child-field.adoc#term-vectors[Create a Child Field] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. +For more information about how to enable term vectors, see xref:child-field-options-reference.adoc#term-vectors[Child Field Options] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. a| .Search for the phrase "nice view" in the reviews.content field [source,json] diff --git a/modules/search/partials/vector-search-field-descriptions.adoc b/modules/search/partials/vector-search-field-descriptions.adoc new file mode 100644 index 000000000..307acb27a --- /dev/null +++ b/modules/search/partials/vector-search-field-descriptions.adoc @@ -0,0 +1,37 @@ +// tag::optimized_for[] +For a `vector` child field, choose whether the Search Service should prioritize recall or latency when returning similar vectors in search results: + +* *recall*: The Search Service prioritizes returning the most accurate result. +This may increase resource usage for Search queries. + +* *latency*: The Search Service prioritizes returning results with lower latency. +This may reduce the accuracy of results. + +For more information about Vector Search indexes, see xref:vector-search:vector-search.adoc[] or xref:vector-search:create-vector-search-index-ui.adoc[]. +// end::optimized_for[] +// tag::similarity_metric[] +For a `vector` child field, choose the method to calculate the similarity between the vector embedding in a Vector Search index and the vector embedding in a Vector Search query. + +NOTE: It's recommended to choose the same similarity metric for your Search index as the one used in your embedding model. + +* *dot_product*: Calculated by adding the result of multiplying a vector's components, or the product of the magnitudes of the vectors and the cosine of the angle between them. +The dot product of 2 vectors is affected by the length and direction of each of the vectors, rather than just taking a straight-line distance. ++ +Dot product similarity is commonly used by Large Language Models (LLMs). +Use *dot_product* to get the best results with an embedding model that uses dot product similarity. + +* *l2_norm*: Also known as euclidean distance. +Uses the straight-line distance between 2 vectors to calculate similarity. +Smaller euclidean distances mean that the values of each coordinate in the vectors are closer together. ++ +It's best to use *l2_norm* similarity when your embeddings contain information about the count or measure of specific things, and your embedding model uses the same similarity metric. + +For more information about Vector Search indexes, see xref:vector-search:vector-search.adoc[] or xref:vector-search:create-vector-search-index-ui.adoc[]. +// end::similarity_metric[] +// tag::dimension[] +For a `vector` child field, enter the total number of elements in the vector embedding array. + +Vector Search indexes can support arrays with up to 2048 elements. + +For more information about Vector Search indexes, see xref:vector-search:vector-search.adoc[] or xref:vector-search:create-vector-search-index-ui.adoc[]. +// end::dimension[] \ No newline at end of file diff --git a/modules/vector-search/pages/create-vector-search-index-ui.adoc b/modules/vector-search/pages/create-vector-search-index-ui.adoc new file mode 100644 index 000000000..bea0c9a04 --- /dev/null +++ b/modules/vector-search/pages/create-vector-search-index-ui.adoc @@ -0,0 +1,74 @@ += Create a Vector Search Index in Quick Mode +:page-topic-type: guide +:description: Use Quick Mode to create a Vector Search index in Couchbase Capella. + +[abstract] +{description} + +Use xref:create-search-index-ui.adoc[Advanced Mode] to add additional settings and more advanced options for your Vector Search index and xref:customize-index.adoc#type-mappings[type mappings]. + +You must create a Vector Search index before you can xref:run-vector-search-ui.adoc[run a search] that supports vector comparisons. + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. + +* You have documents in a keyspace inside that bucket that contain vector embeddings. + +* You have logged in to the Couchbase Capella UI. + +== Procedure + +To create a Vector Search index with Quick Mode in Capella: + +. On the *Databases* page, select the database where you want to create a Search index. +. Go to menu:Data Tools[Search]. +. Click btn:[Create Search Index]. ++ +By default, *Quick Mode* should be on. +. In the *Index Name* field, enter a name for the Vector Search index. ++ +[NOTE] +==== +Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). + +For version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +==== +. Under *Type Mappings*, in the *Bucket* list, select the bucket that contains the documents you want to include in your index. +. In the *Scope* list, select the scope that contains these documents. +. Expand the collection that contains these documents. +. In your document schema, select a field that contains your vector embeddings. ++ +Vectors should be an array of floating point numbers. +. Configure the options for your new child field as follows: +.. In the *Type* list, select *vector*. +.. In the *Dimension* field, check that the value matches the total number of elements in your vector field's array. ++ +The Search Service supports arrays up to 2048 elements. +Capella should automatically fill in the dimension value for your selected array field. +.. In the *Similarity metric* list, select the method to use to calculate the similarity between search term and Search index vectors. ++ +For more information, see xref:search:quick-index-field-options.adoc#quick-field[Quick Field Type Mapping Options]. +.. In the *Optimized for* list, select whether the Search Service should optimize Search queries for accuracy (*recall*) or speed (*latency*). ++ +For more information, see xref:xref:search:quick-index-field-options.adoc#quick-field[Quick Field Type Mapping Options]. +.. Select *Index*. +. Click btn:[Submit]. +. (Optional) Add additional collections or field type mappings to your index. +. Click btn:[Create Index]. + +== Next Steps + +This basic Vector Search index includes the vector embeddings from the field you specified in your type mapping. +If you chose to add additional child fields, the Search Service can also return data from those fields when you run a Vector Search query. + +You can customize your Vector Search index like any other Search index to add additional data and improve search results. +For more information about how to customize an index, see xref:search:customize-index.adoc[]. + +CAUTION: Some Search index features are only available in Advanced Mode. If you edit your Search index in Advanced Mode, you cannot return to Quick Mode and keep any Advanced Mode settings. + +For more information about how to run a search against a Vector Search index, see xref:run-vector-search-ui.adoc[] or xref:run-vector-search-rest-api.adoc[]. \ No newline at end of file diff --git a/modules/vector-search/partials/nav.adoc b/modules/vector-search/partials/nav.adoc index 8e7e4a18c..dd2a947d2 100644 --- a/modules/vector-search/partials/nav.adoc +++ b/modules/vector-search/partials/nav.adoc @@ -1,4 +1,4 @@ * xref:cloud:vector-search:vector-search.adoc[] -//** xref:cloud:vector-search:create-vector-search-index-ui.adoc[] +** xref:cloud:vector-search:create-vector-search-index-ui.adoc[] ** xref:cloud:vector-search:create-vector-search-index-rest-api.adoc[] ** xref:cloud:vector-search:run-vector-search-rest-api.adoc[] From 832a275f4aa16efbadc2b3046e61446a6b1a608a Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 8 Feb 2024 15:08:51 -0500 Subject: [PATCH 19/59] [AV-68964] Adding changes for running a vector search with the REST API, based on Server work --- .../examples/run-search-full-request.jsonc | 10 +++- .../search/pages/search-request-params.adoc | 55 ++++++++++++++++++- .../run-vector-search-payload-ui.jsonc | 13 +++++ .../pages/run-vector-search-ui.adoc | 43 +++++++++++++++ modules/vector-search/partials/nav.adoc | 1 + 5 files changed, 119 insertions(+), 3 deletions(-) create mode 100644 modules/vector-search/examples/run-vector-search-payload-ui.jsonc create mode 100644 modules/vector-search/pages/run-vector-search-ui.adoc diff --git a/modules/search/examples/run-search-full-request.jsonc b/modules/search/examples/run-search-full-request.jsonc index e689da111..4a2ab1bdd 100644 --- a/modules/search/examples/run-search-full-request.jsonc +++ b/modules/search/examples/run-search-full-request.jsonc @@ -41,7 +41,15 @@ } }, // end::query[] - + // tag::knn[] + "knn": [ + { + "k": 10, + "field": "vector_field", + "vector": [ 0.707106781186548, 0, 0.707106781186548 ] + } + ], + // end::knn[] // tag::ctl[] "ctl": { "timeout": 10000, diff --git a/modules/search/pages/search-request-params.adoc b/modules/search/pages/search-request-params.adoc index 2b05038b4..ef65a3027 100644 --- a/modules/search/pages/search-request-params.adoc +++ b/modules/search/pages/search-request-params.adoc @@ -25,13 +25,19 @@ An object that contains the properties for one of the supported query types. For more information about how to configure the `query` object, see <>. +|[[knn]]knn |Array of Objects |Vector Only a| + +An array that contains objects that describe a xref:vector-search:vector-search.adoc[Vector Search] query. + +For more information about how to configure the objects inside a `knn` array, see <>. + |ctl |Object |No a| An object that contains properties for query consistency. For more information about how to configure Search query consistency inside the `ctl` object, see <>. -|size/limit |Integer |No a| +|[[size-limit]]size/limit |Integer |No a| Set the total number of results to return for a single page of search results. @@ -132,6 +138,51 @@ For example, if you had a set of 10 documents to sort based on `_id` values of 1 |==== +[#knn-object] +== Knn Objects + +Each object inside the `knn` array in a Search query describes a xref:vector-search:vector-search.adoc[Vector Search] query. + +Add the `knn` array with at least one object to run a Vector Search query: + +[source,json] +---- +include::example$run-search-full-request.jsonc[tag=knn] +---- + +NOTE: To run a Vector Search query, you must still include a `query` object with your Search request. +To return only results from your Vector Search query, you can set the `query` object to a <>. +To run a hybrid query that uses regular Search Service parameters together with Vector Search to return results, see the <> section. + +An object in the `knn` array can contain the following properties: + +[cols="1,1,4,4"] +|==== +|Property |Type |Required? |Description + +|k |Integer |Yes a| + +Enter the total number of results that you want to return from your Vector Search query. + +The Search Service returns the `k` closest vectors to the vector given in `vector`. + +NOTE: The value of `k` overrides any value set in the <> if you use a <> query. + +|field |String |Yes a| + +The name of the field that contains the vector data you want to search. + +|vector |Array of Floats |Yes a| + +Enter the vector that you want to compare to the vector data in `field`. + +The Search Service uses the similarity metric defined in the xref:search-index-params.adoc#similarity[Search index definition] to return the `k` closest vectors from the Search index. + +NOTE: The vector in your Search query must match the dimension of the vectors stored in your Search index. +For more information about the dimension value, see the xref:search-index-params.adoc#dims[dims property] or the xref:child-field-options-reference.adoc#dimension[Dimension option] in the UI. + +|==== + [#query-object] == Query Object @@ -1612,7 +1663,7 @@ a| include::example$query-special.jsonc[tag=match_all] ---- -| match_none +| [[match_none]]match_none | Object | Use the `match_none` object as the only property in your `query` object to return no documents from the Search index in search results. a| diff --git a/modules/vector-search/examples/run-vector-search-payload-ui.jsonc b/modules/vector-search/examples/run-vector-search-payload-ui.jsonc new file mode 100644 index 000000000..1eb6e9f25 --- /dev/null +++ b/modules/vector-search/examples/run-vector-search-payload-ui.jsonc @@ -0,0 +1,13 @@ +{ + "fields": ["*"], + "query": { + "match_none": "" + }, + "knn": [ + { + "k": 2, + "field": "colorvect_dot", + "vector": [ 0.707106781186548, 0, 0.707106781186548 ] + } + ] +} \ No newline at end of file diff --git a/modules/vector-search/pages/run-vector-search-ui.adoc b/modules/vector-search/pages/run-vector-search-ui.adoc new file mode 100644 index 000000000..936af27e2 --- /dev/null +++ b/modules/vector-search/pages/run-vector-search-ui.adoc @@ -0,0 +1,43 @@ += Run A Vector Search with the Capella UI +:page-topic-type: guide +:description: Run a Vector Search query from the Couchbase Capella UI to preview the search results from a Vector Search index. + +[abstract] +{description} + +== Prerequisites + +* You have the Search Service enabled on a node in your database. + +* You have created a Vector Search index. ++ +For more information about how to create a Vector Search index, see xref:create-vector-search-index-ui.adoc[]. + +* You have logged in to the Couchbase Capella UI. + +== Procedure + +To run a Vector Search with the Capella UI: + +. On the *Databases* page, select the database where you created your Search index. +. Go to menu:Data Tools[Search]. +. Next to your Vector Search index, click btn:[Search]. +. In the *Search* field, enter a search query. ++ +For example, the following query searches for the top 2 vectors similar to the vector `[ 0.707106781186548, 0, 0.707106781186548 ]` in the `colorvect_dot` field: ++ +[source,json] +---- +include::example$run-vector-search-payload-ui.jsonc[] +---- ++ +By using the special `match_none` query in the `query` field, the Search query only returns the `k` number of similar vectors. + +. Press kbd:[Enter]. +. (Optional) To view a document and its source collection, click a document name in the search results list. + +== Next Steps + +If you do not get the search results you were expecting, you can change the xref:search-request-params.adoc[JSON payload for your Search query]. + +You can also xref:search:customize-index.adoc[add additional features to your Search index]. \ No newline at end of file diff --git a/modules/vector-search/partials/nav.adoc b/modules/vector-search/partials/nav.adoc index dd2a947d2..6fcd98a68 100644 --- a/modules/vector-search/partials/nav.adoc +++ b/modules/vector-search/partials/nav.adoc @@ -1,4 +1,5 @@ * xref:cloud:vector-search:vector-search.adoc[] ** xref:cloud:vector-search:create-vector-search-index-ui.adoc[] ** xref:cloud:vector-search:create-vector-search-index-rest-api.adoc[] +** xref:cloud:vector-search:run-vector-search-ui.adoc[] ** xref:cloud:vector-search:run-vector-search-rest-api.adoc[] From 2f7c6523e406e36adc31e645a7a8f12bf884b015 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 8 Feb 2024 15:09:21 -0500 Subject: [PATCH 20/59] [AV-68964] Adding changes for running a vector search with the UI, based on Server work --- .../examples/run-search-full-request.jsonc | 10 +++- .../search/pages/search-request-params.adoc | 55 ++++++++++++++++++- .../run-vector-search-payload-ui.jsonc | 13 +++++ .../pages/run-vector-search-ui.adoc | 43 +++++++++++++++ modules/vector-search/partials/nav.adoc | 1 + 5 files changed, 119 insertions(+), 3 deletions(-) create mode 100644 modules/vector-search/examples/run-vector-search-payload-ui.jsonc create mode 100644 modules/vector-search/pages/run-vector-search-ui.adoc diff --git a/modules/search/examples/run-search-full-request.jsonc b/modules/search/examples/run-search-full-request.jsonc index e689da111..4a2ab1bdd 100644 --- a/modules/search/examples/run-search-full-request.jsonc +++ b/modules/search/examples/run-search-full-request.jsonc @@ -41,7 +41,15 @@ } }, // end::query[] - + // tag::knn[] + "knn": [ + { + "k": 10, + "field": "vector_field", + "vector": [ 0.707106781186548, 0, 0.707106781186548 ] + } + ], + // end::knn[] // tag::ctl[] "ctl": { "timeout": 10000, diff --git a/modules/search/pages/search-request-params.adoc b/modules/search/pages/search-request-params.adoc index 2b05038b4..ef65a3027 100644 --- a/modules/search/pages/search-request-params.adoc +++ b/modules/search/pages/search-request-params.adoc @@ -25,13 +25,19 @@ An object that contains the properties for one of the supported query types. For more information about how to configure the `query` object, see <>. +|[[knn]]knn |Array of Objects |Vector Only a| + +An array that contains objects that describe a xref:vector-search:vector-search.adoc[Vector Search] query. + +For more information about how to configure the objects inside a `knn` array, see <>. + |ctl |Object |No a| An object that contains properties for query consistency. For more information about how to configure Search query consistency inside the `ctl` object, see <>. -|size/limit |Integer |No a| +|[[size-limit]]size/limit |Integer |No a| Set the total number of results to return for a single page of search results. @@ -132,6 +138,51 @@ For example, if you had a set of 10 documents to sort based on `_id` values of 1 |==== +[#knn-object] +== Knn Objects + +Each object inside the `knn` array in a Search query describes a xref:vector-search:vector-search.adoc[Vector Search] query. + +Add the `knn` array with at least one object to run a Vector Search query: + +[source,json] +---- +include::example$run-search-full-request.jsonc[tag=knn] +---- + +NOTE: To run a Vector Search query, you must still include a `query` object with your Search request. +To return only results from your Vector Search query, you can set the `query` object to a <>. +To run a hybrid query that uses regular Search Service parameters together with Vector Search to return results, see the <> section. + +An object in the `knn` array can contain the following properties: + +[cols="1,1,4,4"] +|==== +|Property |Type |Required? |Description + +|k |Integer |Yes a| + +Enter the total number of results that you want to return from your Vector Search query. + +The Search Service returns the `k` closest vectors to the vector given in `vector`. + +NOTE: The value of `k` overrides any value set in the <> if you use a <> query. + +|field |String |Yes a| + +The name of the field that contains the vector data you want to search. + +|vector |Array of Floats |Yes a| + +Enter the vector that you want to compare to the vector data in `field`. + +The Search Service uses the similarity metric defined in the xref:search-index-params.adoc#similarity[Search index definition] to return the `k` closest vectors from the Search index. + +NOTE: The vector in your Search query must match the dimension of the vectors stored in your Search index. +For more information about the dimension value, see the xref:search-index-params.adoc#dims[dims property] or the xref:child-field-options-reference.adoc#dimension[Dimension option] in the UI. + +|==== + [#query-object] == Query Object @@ -1612,7 +1663,7 @@ a| include::example$query-special.jsonc[tag=match_all] ---- -| match_none +| [[match_none]]match_none | Object | Use the `match_none` object as the only property in your `query` object to return no documents from the Search index in search results. a| diff --git a/modules/vector-search/examples/run-vector-search-payload-ui.jsonc b/modules/vector-search/examples/run-vector-search-payload-ui.jsonc new file mode 100644 index 000000000..1eb6e9f25 --- /dev/null +++ b/modules/vector-search/examples/run-vector-search-payload-ui.jsonc @@ -0,0 +1,13 @@ +{ + "fields": ["*"], + "query": { + "match_none": "" + }, + "knn": [ + { + "k": 2, + "field": "colorvect_dot", + "vector": [ 0.707106781186548, 0, 0.707106781186548 ] + } + ] +} \ No newline at end of file diff --git a/modules/vector-search/pages/run-vector-search-ui.adoc b/modules/vector-search/pages/run-vector-search-ui.adoc new file mode 100644 index 000000000..936af27e2 --- /dev/null +++ b/modules/vector-search/pages/run-vector-search-ui.adoc @@ -0,0 +1,43 @@ += Run A Vector Search with the Capella UI +:page-topic-type: guide +:description: Run a Vector Search query from the Couchbase Capella UI to preview the search results from a Vector Search index. + +[abstract] +{description} + +== Prerequisites + +* You have the Search Service enabled on a node in your database. + +* You have created a Vector Search index. ++ +For more information about how to create a Vector Search index, see xref:create-vector-search-index-ui.adoc[]. + +* You have logged in to the Couchbase Capella UI. + +== Procedure + +To run a Vector Search with the Capella UI: + +. On the *Databases* page, select the database where you created your Search index. +. Go to menu:Data Tools[Search]. +. Next to your Vector Search index, click btn:[Search]. +. In the *Search* field, enter a search query. ++ +For example, the following query searches for the top 2 vectors similar to the vector `[ 0.707106781186548, 0, 0.707106781186548 ]` in the `colorvect_dot` field: ++ +[source,json] +---- +include::example$run-vector-search-payload-ui.jsonc[] +---- ++ +By using the special `match_none` query in the `query` field, the Search query only returns the `k` number of similar vectors. + +. Press kbd:[Enter]. +. (Optional) To view a document and its source collection, click a document name in the search results list. + +== Next Steps + +If you do not get the search results you were expecting, you can change the xref:search-request-params.adoc[JSON payload for your Search query]. + +You can also xref:search:customize-index.adoc[add additional features to your Search index]. \ No newline at end of file diff --git a/modules/vector-search/partials/nav.adoc b/modules/vector-search/partials/nav.adoc index dd2a947d2..6fcd98a68 100644 --- a/modules/vector-search/partials/nav.adoc +++ b/modules/vector-search/partials/nav.adoc @@ -1,4 +1,5 @@ * xref:cloud:vector-search:vector-search.adoc[] ** xref:cloud:vector-search:create-vector-search-index-ui.adoc[] ** xref:cloud:vector-search:create-vector-search-index-rest-api.adoc[] +** xref:cloud:vector-search:run-vector-search-ui.adoc[] ** xref:cloud:vector-search:run-vector-search-rest-api.adoc[] From ddfcb32269a92ceca140d71351d580368738548a Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 8 Feb 2024 15:20:03 -0500 Subject: [PATCH 21/59] [AV-68967] Language and link fixes to newer Search docs, plus adding a Vector Search specific example --- .../n1ql-language-reference/searchfun.adoc | 65 +++++++++++++++---- 1 file changed, 53 insertions(+), 12 deletions(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc index 7013fbf4a..1ec23641c 100644 --- a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc @@ -5,14 +5,14 @@ :imagesdir: ../../assets/images :underscore: _ -Search functions enable you to use full text search (xref:server:fts:full-text-intro.adoc[FTS]) queries directly within a {sqlpp} query. +Search functions enable you to use xref:search:search.adoc[full text search (FTS)] queries directly within a {sqlpp} query. [float] === Prerequisites To use any of the search functions, the Search service must be available on the cluster. It is also recommended, but not required, that you should create suitable full text indexes for the searches that you need to perform. -For more information, refer to xref:server:fts:fts-quickstart-guide.adoc#preparing-for-full-text-searches[Preparing for Full Text Searches]. +For more information, refer to xref:search:create-search-indexes.adoc[]. [NOTE] -- @@ -22,7 +22,7 @@ The examples in this page all assume that demonstration full text indexes have b [float] === Authorization -You do not need credentials for the FTS service to be able to use the search functions in a query. +You do not need credentials for the Search Service to be able to use the search functions in a query. The role *Data Admin* must be assigned to those who intend to create indexes; and the role *Data Reader* to those who intend to perform searches. For information on creating users and assigning roles, see xref:server:learn:security/authorization-overview.adoc[Authorization]. @@ -31,7 +31,7 @@ For information on creating users and assigning roles, see xref:server:learn:sec The search functions are useful when you need to combine a full text search with the power of a {sqlpp} query; for example, combining joins and natural-language search in the same query. -If you only need to use the capabilities of a full text search without any {sqlpp} features, consider making use of the Search service directly, through the user interface, the REST API, or an SDK. +If you only need to use the capabilities of a full text search without any {sqlpp} features, consider making use of the Search Service directly, through the user interface, the REST API, or an SDK. [[search,SEARCH()]] == SEARCH(`identifier`, `query`[, `options`]) @@ -47,7 +47,7 @@ The Query engine then passes the search specification over to the Search engine [TIP] -- If no suitable full text index can be selected, or no full text index exists, the Query engine falls back on a Primary index or qualified GSI index to produce document keys, and then fetches the documents. -The Search service then creates a temporary index in memory to perform the search. +The Search Service then creates a temporary index in memory to perform the search. This process may be slower than using a suitable full text index. -- @@ -85,21 +85,21 @@ This may be one of the following: | string | A query string. -For more details, refer to xref:server:fts:fts-query-types.adoc#query-string-query-syntax[Query String Query]. +For more information about how to format a query string, refer to xref:search:search-request-params.adoc#query-string-query-syntax[Query String Query]. | object -| The query object within a full text search request. -For more details, refer to xref:fts-supported-queries.adoc[Supported Queries]. +| The `query` object within a full text search request. +For more information about how to format the `query` object, see xref:search:search-request-params.adoc#query-object[Query object]. | object | A complete full text search request, including sort and pagination options, and so on. -For more details, refer to xref:server:fts:fts-sorting.adoc[Sorting Query Results]. +For more information about how to format a full search request object, refer to xref:search:search-request-params.adoc[]. [NOTE] ==== When specifying a complete full text search request with the {sqlpp} SEARCH() function, if the value of the `size` parameter is greater than the xref:server:fts:fts-response-object-schema.adoc#request[maximum number of full text search results], the query ignores the `size` parameter and returns all matching results. -This is different to the behavior of a complete full text search request in the Search service, where the query returns an error if the value of the `size` parameter is greater than the maximum number of full text search results. +This is different to the behavior of a complete full text search request in the Search Service, where the query returns an error if the value of the `size` parameter is greater than the maximum number of full text search results. ==== |=== + @@ -368,11 +368,51 @@ WHERE t1.type = "hotel" AND SEARCH(t1.description, "amazing"); If the full text search index being queried has its default mapping disabled and has a custom type mapping defined, the query needs to specify the type explicitly. The above query uses the demonstration index xref:server:fts:fts-demonstration-indexes.adoc#travel-sample-index-hotel-description[travel-sample-index-hotel-description], which has the custom type mapping "hotel". -For more information on defining custom type mappings within the full text search index, refer to xref:server:fts:fts-creating-indexes.adoc#specifying-type-mappings[Specifying Type Mappings]. +For more information on defining custom type mappings within the full text search index, refer to xref:search:create-type-mapping.adoc[]. Note that for {sqlpp} queries, only full text search indexes with one type mapping are searchable. Also the supported type identifiers at the moment are "type_field" and "docid_prefix"; "docid_regexp" isn't supported yet for SEARCH queries via {sqlpp}. ==== +.Search against a Vector Search index for the closest 2 vectors +==== +.Query +[source,sqlpp] +---- +SELECT t1.color +FROM rgb AS t1 +WHERE SEARCH(t1, + { + "fields": ["*"], + "query": { + "match_none": "" + }, + "knn": [ + { + "k": 2, + "field": "colorvect_dot", + "vector": [ 0.707106781186548, 0, 0.707106781186548 ] + } + ] + } +) +---- + +.Results +---- +[ + { + "color": "magenta / fuchsia" + }, + { + "color": "dark lavender" + } +] +---- +Using a Search index that contains vectors that describe colors, the query returns the color names of the closest `k` vectors to the vector in the `knn` object. + +For more information about Vector Search and the Search Services, see xref:vector-search:vector-search.adoc[]. +==== + [[search_meta,SEARCH_META()]] == SEARCH_META([`identifier`]) @@ -572,4 +612,5 @@ LIMIT 3; == Related Links -xref:server:fts:full-text-intro.adoc[Full Text Search] +xref:search:search.adoc[] +xref:rest-api:rest-fts.adoc[] From 92d3f36ec7d01d2480956af07554ef5fa7ccd017 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 8 Feb 2024 16:37:10 -0500 Subject: [PATCH 22/59] [AV-68967] Some more link and text cleanup - some things were too Server specific --- .../n1ql/pages/n1ql-language-reference/searchfun.adoc | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc index 1ec23641c..a7b928794 100644 --- a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc @@ -24,7 +24,7 @@ The examples in this page all assume that demonstration full text indexes have b You do not need credentials for the Search Service to be able to use the search functions in a query. The role *Data Admin* must be assigned to those who intend to create indexes; and the role *Data Reader* to those who intend to perform searches. -For information on creating users and assigning roles, see xref:server:learn:security/authorization-overview.adoc[Authorization]. +For information on creating users and roles in Capella, see xref:organizations:organization-projects-overview.adoc[]. [float] === When to Use Search Functions @@ -136,7 +136,7 @@ This is treated as an input to the index mapping. It overrides the default mapping and is used during index selection and filtering. The object must either have a default mapping with no type mapping, or a single type mapping with the default mapping disabled. -For more information, refer to xref:server:fts:fts-creating-indexes.adoc[Creating Indexes]. +For more information, refer to xref:search:customize-index.adoc[]. | `indexUUID` [Optional] @@ -150,7 +150,7 @@ The `indexUUID` field and the `index` field must both identify the same full tex If they identify different full text indexes, or if either of them does not identify a full text index, an error occurs. You can find the UUID of a full text index by viewing the index definition. -You can do this using the xref:server:fts:fts-creating-indexes.adoc#using-the-index-definition-preview[Index Definition Preview] in the Query Workbench, or the xref:server:rest-api:rest-fts-indexing.adoc[Index Definition] endpoints provided by the Full Text Search REST API. +Click a Search index name in the Capella UI to open the editor. Click btn:[Index Definition] to view the index definition. | `out` [Optional] @@ -375,6 +375,8 @@ Also the supported type identifiers at the moment are "type_field" and "docid_pr .Search against a Vector Search index for the closest 2 vectors ==== +NOTE: This example does not use the travel sample data or query context mentioned before, as it requires documents that contain vector data. + .Query [source,sqlpp] ---- @@ -613,4 +615,5 @@ LIMIT 3; == Related Links xref:search:search.adoc[] -xref:rest-api:rest-fts.adoc[] +xref:vector-search:vector-search.adoc + From d41265eb4c390ca05e98a0ff27703f2344d11a9b Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 9 Feb 2024 10:11:34 -0500 Subject: [PATCH 23/59] [AV-68962] Adjusting note to link to relevant documentation and explain what happens --- modules/vector-search/pages/vector-search.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc index 60e1e5f53..cd8626cf4 100644 --- a/modules/vector-search/pages/vector-search.adoc +++ b/modules/vector-search/pages/vector-search.adoc @@ -36,7 +36,8 @@ If you do not want to use an integration like LangChain, you can also use the AP For example, the OpenAI `embeddings` endpoint can generate embeddings for a text string using a specified embedding model. For more information about how to generate and obtain embeddings for text strings using the OpenAI API, see the https://platform.openai.com/docs/guides/embeddings/what-are-embeddings[Embeddings documentation]. -NOTE: When creating a Vector Search index, the dimension of your data vector embeddings must match the dimension for any search query vectors. +NOTE: When you create a Vector Search index, the xref:search:child-field-options-reference.adoc#dimension[dimension] of your data vector embeddings must match the dimension for any search query vectors. +Otherwise, a Vector Search query fails to return any results. For more information about how to create a Vector Search index, see xref:create-vector-search-index-ui.adoc[] or xref:create-vector-search-index-rest-api.adoc[]. From eadf2ed87382e2d85211416dbcae2922a2e6d3e3 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 9 Feb 2024 10:40:16 -0500 Subject: [PATCH 24/59] [AV-68976] Building out See Also section in customize-index --- modules/search/pages/customize-index.adoc | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/modules/search/pages/customize-index.adoc b/modules/search/pages/customize-index.adoc index 6699ecc26..4f719a969 100644 --- a/modules/search/pages/customize-index.adoc +++ b/modules/search/pages/customize-index.adoc @@ -161,6 +161,15 @@ For more information about how to create your own wordlist, see xref:create-cust == See Also -* xref:create-search-indexes.adoc[] -* xref:index-aliases.adoc[] -* xref:run-searches.adoc[] \ No newline at end of file +* xref:set-type-identifier.adoc[] +* xref:create-type-mapping.adoc[] +* xref:create-child-field.adoc[] +* xref:create-child-mapping.adoc[] +* xref:create-custom-analyzer.adoc[] +* xref:create-custom-character-filter.adoc[] +* xref:create-custom-tokenizer.adoc[] +* xref:create-custom-token-filter.adoc[] +* xref:create-custom-wordlist.adoc[] +* xref:set-advanced-settings.adoc[] +* xref:run-searches.adoc[] +* xref:index-aliases.adoc[] \ No newline at end of file From db6b17098e5d0aaeed6b884f1904052bc6d5acb8 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 9 Feb 2024 14:52:43 -0500 Subject: [PATCH 25/59] [AV-68966] Remove unnecessary "size" parameter --- modules/vector-search/examples/run-vector-search-payload.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/modules/vector-search/examples/run-vector-search-payload.sh b/modules/vector-search/examples/run-vector-search-payload.sh index d5f884ff3..463f263b3 100644 --- a/modules/vector-search/examples/run-vector-search-payload.sh +++ b/modules/vector-search/examples/run-vector-search-payload.sh @@ -15,6 +15,5 @@ curl -XPOST -H "Content-Type: application/json" \ "field": "colorvect_dot", "vector": [ 0.707106781186548, 0, 0.707106781186548 ] } - ], - "size": 10 + ] }' \ No newline at end of file From b078b2e6689b840ca6f9cb2d8f36d8ec5751f373 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 9 Feb 2024 14:59:10 -0500 Subject: [PATCH 26/59] [AV-68571] Correcting incorrect table parameter --- modules/search/pages/search-request-params.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/search/pages/search-request-params.adoc b/modules/search/pages/search-request-params.adoc index ef65a3027..cc5ced544 100644 --- a/modules/search/pages/search-request-params.adoc +++ b/modules/search/pages/search-request-params.adoc @@ -156,7 +156,7 @@ To run a hybrid query that uses regular Search Service parameters together with An object in the `knn` array can contain the following properties: -[cols="1,1,4,4"] +[cols="1,1,1,4"] |==== |Property |Type |Required? |Description From 18b9a0aa0a6c5e68214b0d174f6a1f372364adc0 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 9 Feb 2024 15:25:20 -0500 Subject: [PATCH 27/59] [AV-68571] Removing unnecessary period in search.adoc Fixing broken links in field-data-types-reference.adoc and customize-index.adoc --- modules/search/pages/customize-index.adoc | 10 +++++----- modules/search/pages/field-data-types-reference.adoc | 2 +- modules/search/pages/search.adoc | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/modules/search/pages/customize-index.adoc b/modules/search/pages/customize-index.adoc index 4f719a969..91f62ba82 100644 --- a/modules/search/pages/customize-index.adoc +++ b/modules/search/pages/customize-index.adoc @@ -52,7 +52,7 @@ For more information about how to change general settings, see xref:set-advanced |[[type-identifiers]]Document Filter | |✓ -a|In xref:create-search-indexes-ui.adoc[Advanced Mode], configure a document filter to add or remove documents in your Search index that meet certain conditions: +a|In xref:create-search-index-ui.adoc[Advanced Mode], configure a document filter to add or remove documents in your Search index that meet certain conditions: * JSON Type Field: Selects only documents that contain a specific field with a specified string value. * Doc ID up to Separator: Selects only documents with an ID or key up to a specific substring. @@ -63,7 +63,7 @@ For more information about how to configure a document filter, see xref:set-type |[[analyzers]]Analyzers | |✓ -a|In xref:create-search-indexes-ui.adoc[Advanced Mode], use analyzers to improve and customize the search results in your index. +a|In xref:create-search-index-ui.adoc[Advanced Mode], use analyzers to improve and customize the search results in your index. Analyzers transform input text into tokens, which give you greater control over your index's text matching. @@ -78,14 +78,14 @@ Both custom and default analyzers can contain custom filters. |[[custom-filters-table]]Custom Filters | |✓ -a|In xref:create-search-indexes-ui.adoc[Advanced Mode], use custom filters to add more customization to a custom analyzer. +a|In xref:create-search-index-ui.adoc[Advanced Mode], use custom filters to add more customization to a custom analyzer. For more information about these filters, see the <> section. |[[date-time]]Date/Time Parsers | |✓ -a|In xref:create-search-indexes-ui.adoc[Advanced Mode], if the documents in your index contain date and time data in a format other than RFC-3339 (ISO-8601), then you need to create a date/time parser. +a|In xref:create-search-index-ui.adoc[Advanced Mode], if the documents in your index contain date and time data in a format other than RFC-3339 (ISO-8601), then you need to create a date/time parser. A custom date/time parser tells the Search index how to interpret date data from your documents. @@ -98,7 +98,7 @@ For more information about how to add a custom date/time parser, see xref:create Custom filters are components of a Search index <>. -Create and add these components to a custom analyzer to improve search results and performance for an index in xref:create-search-indexes-ui.adoc[Advanced Mode]. +Create and add these components to a custom analyzer to improve search results and performance for an index in xref:create-search-index-ui.adoc[Advanced Mode]. You can create the following custom filters: diff --git a/modules/search/pages/field-data-types-reference.adoc b/modules/search/pages/field-data-types-reference.adoc index 100285421..b9a3ced68 100644 --- a/modules/search/pages/field-data-types-reference.adoc +++ b/modules/search/pages/field-data-types-reference.adoc @@ -5,7 +5,7 @@ [abstract] {description} -When you create a mapping for a field in xref:create-quick-index.adoc[Quick Mode] or xref:xref:create-child-field.adoc[Advanced Mode], you need to set the field's data type. +When you create a mapping for a field in xref:create-quick-index.adoc[Quick Mode] or xref:create-child-field.adoc[Advanced Mode], you need to set the field's data type. If you create a Search index and do not set a data type for a field, the Search Service automatically assigns a field data type. For example, if you created a xref:customize-index.adoc#type-mappings[dynamic type mapping], the Search Service automatically assigns data types to all fields in the type mapping. diff --git a/modules/search/pages/search.adoc b/modules/search/pages/search.adoc index 1c2c44010..00f1f5c60 100644 --- a/modules/search/pages/search.adoc +++ b/modules/search/pages/search.adoc @@ -67,4 +67,4 @@ For more information about Vector Search, see xref:vector-search:vector-search.a * xref:customize-index.adoc[] * xref:index-aliases.adoc[] * xref:run-searches.adoc[] -* xref:vector-search:vector-search.adoc[]. +* xref:vector-search:vector-search.adoc[] From 5fd1e85121f32bfa9d0b1a0f6b22c1128dcbf131 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 15 Feb 2024 13:48:15 -0500 Subject: [PATCH 28/59] [AV-68962] Adding link to integrations page and removing comments --- modules/vector-search/pages/vector-search.adoc | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc index cd8626cf4..63c2f8e35 100644 --- a/modules/vector-search/pages/vector-search.adoc +++ b/modules/vector-search/pages/vector-search.adoc @@ -19,11 +19,8 @@ Using vector embeddings to represent data, your end users can search based on de * *Generative AI:* Create new text, images, and more based on a prompt given to the LLM. -// More use cases worth calling out? - Vector Search supports integrations with frameworks like https://python.langchain.com/docs/get_started/introduction[LangChain^] to support AI application development. -For more information about all frameworks and integrations supported by Vector Search and Capella, see xref:[]. -//Don't forget to fill in this link! +For more information about all frameworks and integrations supported by Vector Search and Capella, see xref:third-party:integrations.adoc[]. == Using Vector Search Indexes From 4597a38f663497e6d5e1a00fe666acc8a3cea703 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 1 Mar 2024 12:34:24 -0500 Subject: [PATCH 29/59] [AV-68571] Consistency edits based on peer review --- .../n1ql-language-reference/searchfun.adoc | 76 +++++++++---------- modules/search/examples/query-geojson.jsonc | 12 +-- .../pages/field-data-types-reference.adoc | 2 +- .../search/pages/search-request-params.adoc | 26 +++---- 4 files changed, 58 insertions(+), 58 deletions(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc index a7b928794..9487de6e9 100644 --- a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc @@ -1,22 +1,22 @@ = Search Functions -:description: Search functions enable you to use full text search (FTS) queries directly within a {sqlpp} query. +:description: Search functions enable you to use Full Text Search (FTS) queries directly within a {sqlpp} query. :page-topic-type: reference :example-caption!: :imagesdir: ../../assets/images :underscore: _ -Search functions enable you to use xref:search:search.adoc[full text search (FTS)] queries directly within a {sqlpp} query. +Search functions enable you to use xref:search:search.adoc[Full Text Search (FTS)] queries directly within a {sqlpp} query. [float] === Prerequisites To use any of the search functions, the Search service must be available on the cluster. -It is also recommended, but not required, that you should create suitable full text indexes for the searches that you need to perform. +It is also recommended, but not required, that you should create suitable Search indexes for the searches that you need to perform. For more information, refer to xref:search:create-search-indexes.adoc[]. [NOTE] -- -The examples in this page all assume that demonstration full text indexes have been created, as described in xref:server:fts:fts-demonstration-indexes.adoc[Demonstration Indexes]. +The examples in this page all assume that demonstration Search indexes have been created, as described in xref:server:fts:fts-demonstration-indexes.adoc[Demonstration Indexes]. -- [float] @@ -29,16 +29,16 @@ For information on creating users and roles in Capella, see xref:organizations:o [float] === When to Use Search Functions -The search functions are useful when you need to combine a full text search with the power of a {sqlpp} query; for example, combining joins and natural-language search in the same query. +The search functions are useful when you need to combine a Full Text Search with the power of a {sqlpp} query; for example, combining joins and natural-language search in the same query. -If you only need to use the capabilities of a full text search without any {sqlpp} features, consider making use of the Search Service directly, through the user interface, the REST API, or an SDK. +If you only need to use the capabilities of a Full Text Search without any {sqlpp} features, consider making use of the Search Service directly, through the user interface, the REST API, or an SDK. [[search,SEARCH()]] == SEARCH(`identifier`, `query`[, `options`]) === Description -This function enables you to use a full text search to filter a result set, or as a join predicate. +This function enables you to use a Full Text Search to filter a result set, or as a join predicate. It is only allowed in the xref:n1ql-language-reference/where.adoc[WHERE] clause or the xref:n1ql-language-reference/join.adoc[ON] clause. If a query contains a SEARCH function, the Query engine analyzes the entire query, including the search specification, to select the best index to use with this search, taking any index hints into account. @@ -46,9 +46,9 @@ The Query engine then passes the search specification over to the Search engine [TIP] -- -If no suitable full text index can be selected, or no full text index exists, the Query engine falls back on a Primary index or qualified GSI index to produce document keys, and then fetches the documents. +If no suitable Search index can be selected, or no Search index exists, the Query engine falls back on a Primary index or qualified GSI index to produce document keys, and then fetches the documents. The Search Service then creates a temporary index in memory to perform the search. -This process may be slower than using a suitable full text index. +This process may be slower than using a suitable Search index. -- === Arguments @@ -75,7 +75,7 @@ Similarly, if the _query_ argument is a query object, the path is ignored. The _identifier_ argument cannot be replaced by a {sqlpp} query parameter. query:: -[Required] The full text search query. +[Required] The Full Text Search query. This may be one of the following: + [cols="1a,4a", options="header"] @@ -88,18 +88,18 @@ This may be one of the following: For more information about how to format a query string, refer to xref:search:search-request-params.adoc#query-string-query-syntax[Query String Query]. | object -| The `query` object within a full text search request. +| The `query` object within a Full Text Search request. For more information about how to format the `query` object, see xref:search:search-request-params.adoc#query-object[Query object]. | object -| A complete full text search request, including sort and pagination options, and so on. +| A complete Full Text Search request, including sort and pagination options, and so on. For more information about how to format a full search request object, refer to xref:search:search-request-params.adoc[]. [NOTE] ==== -When specifying a complete full text search request with the {sqlpp} SEARCH() function, if the value of the `size` parameter is greater than the xref:server:fts:fts-response-object-schema.adoc#request[maximum number of full text search results], the query ignores the `size` parameter and returns all matching results. +When specifying a complete Full Text Search request with the {sqlpp} SEARCH() function, if the value of the `size` parameter is greater than the xref:server:fts:fts-response-object-schema.adoc#request[maximum number of full text search results], the query ignores the `size` parameter and returns all matching results. -This is different to the behavior of a complete full text search request in the Search Service, where the query returns an error if the value of the `size` parameter is greater than the maximum number of full text search results. +This is different to the behavior of a complete Full Text Search request in the Search Service, where the query returns an error if the value of the `size` parameter is greater than the maximum number of Full Text Search results. ==== |=== + @@ -118,10 +118,10 @@ The object may contain the following fields: | `index` [Optional] | string, object -| The `index` field may be a string, containing the name of a full text index in the keyspace. -(This may be a full text index alias, but only if the full text index is in the same keyspace.) +| The `index` field may be a string, containing the name of a Search index in the keyspace. +(This may be a Search index alias, but only if the Search index is in the same keyspace.) This provides an index hint to the Query engine. -If the full text index does not exist, an error occurs. +If the Search index does not exist, an error occurs. [TIP] -- @@ -131,7 +131,7 @@ This takes precedence over a hint provided by the `index` field. ''' -The `index` field may also be an object, containing an example of a full text index mapping. +The `index` field may also be an object, containing an example of a Search index mapping. This is treated as an input to the index mapping. It overrides the default mapping and is used during index selection and filtering. @@ -141,23 +141,23 @@ For more information, refer to xref:search:customize-index.adoc[]. | `indexUUID` [Optional] | string -| A string, containing the UUID of a full text index in the keyspace. +| A string, containing the UUID of a Search index in the keyspace. This provides an index hint to the Query engine. -If the full text index cannot be identified, an error occurs. +If the Search index cannot be identified, an error occurs. -You can use the `indexUUID` field alongside the `index` field to help identify a full text index. -The `indexUUID` field and the `index` field must both identify the same full text index. -If they identify different full text indexes, or if either of them does not identify a full text index, an error occurs. +You can use the `indexUUID` field alongside the `index` field to help identify a Search index. +The `indexUUID` field and the `index` field must both identify the same Search index. +If they identify different Search indexes, or if either of them does not identify a Search index, an error occurs. -You can find the UUID of a full text index by viewing the index definition. +You can find the UUID of a Search index by viewing the index definition. Click a Search index name in the Capella UI to open the editor. Click btn:[Index Definition] to view the index definition. | `out` [Optional] | string -| A name given to this full text search operation in this keyspace. +| A name given to this Full Text Search operation in this keyspace. You can use this name to refer to this operation using the <> and <> functions. -If this field is omitted, the name of this full text search operation defaults to `"out"`. +If this field is omitted, the name of this Full Text Search operation defaults to `"out"`. | (other) [Optional] @@ -177,7 +177,7 @@ This returns `true` if the search query is found within the input path, or `fals === Limitations -The Query service can select a full text index for efficient search in the following cases: +The Query service can select a Search index for efficient search in the following cases: * If the SEARCH() function is used in a WHERE clause or in an ANSI JOIN. The SEARCH() function must be on the leftmost (first) JOIN. @@ -186,7 +186,7 @@ RIGHT OUTER JOINs are rewritten as LEFT OUTER JOINs. * If the SEARCH() function is evaluated on the `true` condition in positive cases: for example, `SEARCH(_field_, _query_, _options_)`, `SEARCH(_field_, _query_, _options_) = true`, `SEARCH(_field_, _query_, _options_) IN [true, true, true]`, or a condition including one of these with `AND` or `OR`. -The Query service cannot select a full text index for efficient search in the following cases: +The Query service cannot select a Search index for efficient search in the following cases: * If a USE KEYS hint is present; or if the SEARCH() function is used on the inner side of a nested-loop JOIN, a lookup JOIN or lookup NEST, an index JOIN or index NEST, an UNNEST clause, a subquery expression, a subquery result, or a correlated query. @@ -285,7 +285,7 @@ WHERE SEARCH(t1, { The results are unordered, so they may be returned in a different order each time. ==== -.Search using a complete full text search request +.Search using a complete Full Text Search request ==== .Query [source,sqlpp] @@ -336,7 +336,7 @@ This query returns 5 results, and the results are ordered, as specified by the s As an alternative, you could limit the number of results and order them using the {sqlpp} xref:n1ql-language-reference/limit.adoc[LIMIT] and xref:n1ql-language-reference/orderby.adoc[ORDER BY] clauses. ==== -.Search against a full text search index that carries a custom type mapping +.Search against a Full Text Search index that carries a custom type mapping ==== .Query [source,sqlpp] @@ -365,11 +365,11 @@ WHERE t1.type = "hotel" AND SEARCH(t1.description, "amazing"); ] ---- -If the full text search index being queried has its default mapping disabled and has a custom type mapping defined, the query needs to specify the type explicitly. +If the Full Text Search index being queried has its default mapping disabled and has a custom type mapping defined, the query needs to specify the type explicitly. The above query uses the demonstration index xref:server:fts:fts-demonstration-indexes.adoc#travel-sample-index-hotel-description[travel-sample-index-hotel-description], which has the custom type mapping "hotel". -For more information on defining custom type mappings within the full text search index, refer to xref:search:create-type-mapping.adoc[]. -Note that for {sqlpp} queries, only full text search indexes with one type mapping are searchable. +For more information on defining custom type mappings within the Full Text Search index, refer to xref:search:create-type-mapping.adoc[]. +Note that for {sqlpp} queries, only Full Text Search indexes with one type mapping are searchable. Also the supported type identifiers at the moment are "type_field" and "docid_prefix"; "docid_regexp" isn't supported yet for SEARCH queries via {sqlpp}. ==== @@ -422,12 +422,12 @@ For more information about Vector Search and the Search Services, see xref:vecto This function is intended to be used in a query which contains a <> function. It returns the metadata given by the Search engine for each document found by the <> function. -If there is no <> function in the query, or if a full text index was not used to evaluate the search, the function returns MISSING. +If there is no <> function in the query, or if a Search index was not used to evaluate the search, the function returns MISSING. === Arguments identifier:: -[Optional] An expression in the form `{startsb}__keyspaceAlias__.{endsb}__outname__`, consisting of the keyspace or keyspace alias in which the full text search operation was performed, followed by the outname of the full text search operation, using dot notation. +[Optional] An expression in the form `{startsb}__keyspaceAlias__.{endsb}__outname__`, consisting of the keyspace or keyspace alias in which the Full Text Search operation was performed, followed by the outname of the Full Text Search operation, using dot notation. [NOTE] -- @@ -472,7 +472,7 @@ LIMIT 3; ---- <1> There is only one <> function in this query, so the SEARCH_META() function does not need to specify the outname. -<2> The full text search specifies that locations should be included in the search result metadata. +<2> The Full Text Search specifies that locations should be included in the search result metadata. .Result [source,json] @@ -547,14 +547,14 @@ This query only uses one data source, so there is no need to specify the keyspac This function is intended to be used in a query which contains a <> function. It returns the score given by the Search engine for each document found by the <> function. -If there is no <> function in the query, or if a full text index was not used to evaluate the search, the function returns MISSING. +If there is no <> function in the query, or if a Search index was not used to evaluate the search, the function returns MISSING. This function is the same as <>. === Arguments identifier:: -[Optional] An expression in the form `{startsb}__keyspaceAlias__.{endsb}__outname__`, consisting of the keyspace or keyspace alias in which the full text search operation was performed, followed by the outname of the full text search operation, using dot notation. +[Optional] An expression in the form `{startsb}__keyspaceAlias__.{endsb}__outname__`, consisting of the keyspace or keyspace alias in which the Full Text Search operation was performed, followed by the outname of the Full Text Search operation, using dot notation. [NOTE] -- diff --git a/modules/search/examples/query-geojson.jsonc b/modules/search/examples/query-geojson.jsonc index 7d79b8d63..fbc32802c 100644 --- a/modules/search/examples/query-geojson.jsonc +++ b/modules/search/examples/query-geojson.jsonc @@ -4,7 +4,7 @@ "field": "geojson", "geometry": { "shape": { - "type": "point", + "type": "Point", "coordinates": [0.47482593026924746, 51.31232878073189] }, "relation": "intersects" @@ -18,7 +18,7 @@ "field": "geojson", "geometry": { "shape": { - "type": "linestring", + "type": "LineString", "coordinates": [ [-2.753735609842721, 53.94860827535115], [-2.599898256093695,53.65007434185782] @@ -138,17 +138,17 @@ "field": "geojson", "geometry": { "shape": { - "type": "geometrycollection", + "type": "GeometryCollection", "geometries": [ { - "type": "linestring", + "type": "LineString", "coordinates": [ [-2.753735609842721, 53.94860827535115], [-2.599898256093695, 53.65007434185782] ] }, { - "type": "multipolygon", + "type": "MultiPolygon", "coordinates": [ [[ [-1.8167959002718135, 53.8626654046235], @@ -183,7 +183,7 @@ "geometry": { "shape": { "coordinates": [-2.2450285424707204, 53.48503270828377], - "type": "circle", + "type": "Circle", "radius": "100mi" }, "relation": "within" diff --git a/modules/search/pages/field-data-types-reference.adoc b/modules/search/pages/field-data-types-reference.adoc index b9a3ced68..68607e872 100644 --- a/modules/search/pages/field-data-types-reference.adoc +++ b/modules/search/pages/field-data-types-reference.adoc @@ -48,7 +48,7 @@ For example, a `point` or `MultiLineString`. For example: ---- { - "type": "linestring", + "type": "LineString", "coordinates": [ [-2.753735609842721, 53.94860827535115], diff --git a/modules/search/pages/search-request-params.adoc b/modules/search/pages/search-request-params.adoc index cc5ced544..81e02cca4 100644 --- a/modules/search/pages/search-request-params.adoc +++ b/modules/search/pages/search-request-params.adoc @@ -1006,7 +1006,7 @@ Defines the specific GeoJSON shape to use in the query. |[[point-type]]type |String |Yes -a| Set `type` to `point`. +a| Set `type` to `Point`. |[[point-coordinates]]coordinates |Array @@ -1074,7 +1074,7 @@ Defines the specific GeoJSON shape to use in the query. |[[linestring-type]]type |String |Yes -a| Set `type` to `linestring`. +a| Set `type` to `LineString`. |[[linestring-coordinates]]coordinates |Array of Arrays @@ -1142,7 +1142,7 @@ Defines the specific GeoJSON shape to use in the query. |[[polygon-type]]type |String |Yes -a| Set `type` to `polygon`. +a| Set `type` to `Polygon`. |[[polygon-coordinates]]coordinates |Array of Arrays @@ -1213,7 +1213,7 @@ Defines the specific GeoJSON shape to use in the query. |[[multipoint-type]]type |String |Yes -a| Set `type` to `multipoint`. +a| Set `type` to `MultiPoint`. |[[multipoint-coordinates]]coordinates |Array of Arrays @@ -1283,7 +1283,7 @@ Defines the specific GeoJSON shape to use in the query. |[[multilinestring-type]]type |String |Yes -a| Set `type` to `multilinestring`. +a| Set `type` to `MultiLineString`. |[[multilinestring-coordinates]]coordinates |Array of Arrays @@ -1364,7 +1364,7 @@ Defines the specific GeoJSON shape to use in the query. |[[multipolygon-type]]type |String |Yes -a| Set `type` to `multipolygon`. +a| Set `type` to `MultiPolygon`. |[[multipolygon-coordinates]]coordinates |Array of Arrays @@ -1435,7 +1435,7 @@ a| Contains the <> and <> -* <> +* <> * <> -* <> -* <> -* <> +* <> +* <> +* <> |[[geometrycollection-coordinates]]coordinates |Array or Array of Arrays @@ -1523,7 +1523,7 @@ Defines the specific GeoJSON shape to use in the query. |[[circle-type]]type |String |Yes -a| Set `type` to `circle`. +a| Set `type` to `Circle`. |[[circle-coordinates]]coordinates |Array @@ -1610,7 +1610,7 @@ Defines the specific GeoJSON shape to use in the query. |[[envelope-type]]type |String |Yes -a| Set `type` to `envelope`. +a| Set `type` to `Envelope`. |[[envelope-coordinates]]coordinates |Array of 2 Arrays From 7d7784b216bb1fe5f030cd7ec4ac40ccc7db5b9e Mon Sep 17 00:00:00 2001 From: sarahlwelton <110928505+sarahlwelton@users.noreply.github.com> Date: Fri, 1 Mar 2024 13:01:20 -0500 Subject: [PATCH 30/59] Update modules/n1ql/pages/n1ql-language-reference/searchfun.adoc Co-authored-by: Daniel Nicholson --- modules/n1ql/pages/n1ql-language-reference/searchfun.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc index 9487de6e9..dd9c835c0 100644 --- a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc @@ -614,6 +614,6 @@ LIMIT 3; == Related Links -xref:search:search.adoc[] -xref:vector-search:vector-search.adoc +* xref:search:search.adoc[] +* xref:vector-search:vector-search.adoc[] From fd709fa36eb7301ad8fd88c35fb114a032ed372e Mon Sep 17 00:00:00 2001 From: sarahlwelton <110928505+sarahlwelton@users.noreply.github.com> Date: Fri, 1 Mar 2024 13:23:11 -0500 Subject: [PATCH 31/59] Update modules/vector-search/pages/vector-search.adoc Co-authored-by: Daniel Nicholson --- modules/vector-search/pages/vector-search.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc index 63c2f8e35..52eb7bcd7 100644 --- a/modules/vector-search/pages/vector-search.adoc +++ b/modules/vector-search/pages/vector-search.adoc @@ -7,7 +7,7 @@ == About Vector Search -Vector Search builds on Couchbase Capella's xref:search.adoc[Search Service] to provide vector index support. +Vector Search builds on Couchbase Capella's xref:search:search.adoc[Search Service] to provide vector index support. You can use these new Vector Search indexes for Retrieval Augmented Generation (RAG) with an existing Large Language Model (LLM). Using Vector Search, Capella, an embedding model, and your chosen LLM, you can develop AI applications while giving context and up-to-date information from your own data. From 17d63271f30225b03773c0cd1b6429a86b2a28d0 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 1 Mar 2024 13:30:33 -0500 Subject: [PATCH 32/59] [AV-68571] Cleaning up some comments from peer review --- modules/search/pages/create-quick-index.adoc | 2 ++ .../pages/create-search-index-rest-api.adoc | 4 +++- .../vector-search-field-descriptions.adoc | 2 +- .../create-vector-search-index-rest-api.adoc | 12 ++++++++---- .../pages/create-vector-search-index-ui.adoc | 19 ++++++++++--------- .../pages/run-vector-search-rest-api.adoc | 5 +++-- .../pages/run-vector-search-ui.adoc | 8 ++++---- .../vector-search/pages/vector-search.adoc | 11 ++++++----- 8 files changed, 37 insertions(+), 26 deletions(-) diff --git a/modules/search/pages/create-quick-index.adoc b/modules/search/pages/create-quick-index.adoc index 61e50484a..64e0f3ba1 100644 --- a/modules/search/pages/create-quick-index.adoc +++ b/modules/search/pages/create-quick-index.adoc @@ -17,6 +17,8 @@ For more information about how to change Services on your database, see xref:clo * You have a bucket with scopes and collections in your database. For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. +* You have documents in a keyspace inside your bucket that contain vector embeddings. + * You have logged in to the Couchbase Capella UI. == Procedure diff --git a/modules/search/pages/create-search-index-rest-api.adoc b/modules/search/pages/create-search-index-rest-api.adoc index 5e85c1094..70c21a9bd 100644 --- a/modules/search/pages/create-search-index-rest-api.adoc +++ b/modules/search/pages/create-search-index-rest-api.adoc @@ -35,7 +35,9 @@ Don't include the xref:search-index-params.adoc#uuid[uuid] or xref:search-index- + TIP: If you remove the xref:search-index-params.adoc#uuid[uuid] and xref:search-index-params.adoc#sourceuuid[sourceUUID] parameters, you can copy the Search index definition JSON payload from the Couchbase Capella UI to use in your REST API call. For more information about how to create an index with the UI, see xref:create-search-index-ui.adoc[]. -+ + +=== Example + In the following example, the JSON payload creates a simple index named `landmark-content-index` on the `travel-sample` bucket. It creates a type mapping for the `inventory.landmark` collection, with a child field, `content`: + [source,console] diff --git a/modules/search/partials/vector-search-field-descriptions.adoc b/modules/search/partials/vector-search-field-descriptions.adoc index 307acb27a..fd7079a25 100644 --- a/modules/search/partials/vector-search-field-descriptions.adoc +++ b/modules/search/partials/vector-search-field-descriptions.adoc @@ -20,7 +20,7 @@ The dot product of 2 vectors is affected by the length and direction of each of Dot product similarity is commonly used by Large Language Models (LLMs). Use *dot_product* to get the best results with an embedding model that uses dot product similarity. -* *l2_norm*: Also known as euclidean distance. +* *l2_norm*: Also known as Euclidean distance. Uses the straight-line distance between 2 vectors to calculate similarity. Smaller euclidean distances mean that the values of each coordinate in the vectors are closer together. + diff --git a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc index 74db5c0af..de2f6ec0c 100644 --- a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc +++ b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc @@ -10,11 +10,13 @@ You must create a Search index before you can xref:run-vector-search-rest-api.ad == Prerequisites -* You have the Search Service enabled on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have a bucket with scopes and collections in your database. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You have documents in a keyspace inside that bucket that contain vector embeddings. +* You have documents in a keyspace inside your bucket that contain vector embeddings. * You have installed the Couchbase command-line tool (CLI). @@ -40,7 +42,9 @@ Do not include the xref:search:search-index-params.adoc#uuid[uuid] or xref:searc + TIP: If you remove the xref:search:search-index-params.adoc#uuid[uuid] and xref:search:search-index-params.adoc#sourceuuid[sourceUUID] parameters, you can copy the Search index definition JSON payload from the Capella UI to use in a REST API call. For more information about how to create a Vector Search index with the UI, see xref:create-vector-search-index-ui.adoc[]. -+ + +=== Example + In the following example, the JSON payload creates an index named `color-index` on the `vector-sample` bucket. It creates two child field mappings, `colorvect_dot` and `colorvect_l2` on two different vector fields in the documents under the `color.rgb` keyspace: + [source,console] diff --git a/modules/vector-search/pages/create-vector-search-index-ui.adoc b/modules/vector-search/pages/create-vector-search-index-ui.adoc index bea0c9a04..31e0814b9 100644 --- a/modules/vector-search/pages/create-vector-search-index-ui.adoc +++ b/modules/vector-search/pages/create-vector-search-index-ui.adoc @@ -5,9 +5,9 @@ [abstract] {description} -Use xref:create-search-index-ui.adoc[Advanced Mode] to add additional settings and more advanced options for your Vector Search index and xref:customize-index.adoc#type-mappings[type mappings]. +Use xref:search:create-search-index-ui.adoc[Advanced Mode] to add additional settings and more advanced options for your Vector Search index and xref:search:customize-index.adoc#type-mappings[type mappings]. -You must create a Vector Search index before you can xref:run-vector-search-ui.adoc[run a search] that supports vector comparisons. +You must create a Vector Search index before you can xref:search:run-vector-search-ui.adoc[run a search] that supports vector comparisons. == Prerequisites @@ -17,7 +17,7 @@ For more information about how to change Services on your database, see xref:clo * You have a bucket with scopes and collections in your database. For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You have documents in a keyspace inside that bucket that contain vector embeddings. +* You have documents in a keyspace inside your bucket that contain vector embeddings. * You have logged in to the Couchbase Capella UI. @@ -41,7 +41,7 @@ For version 7.6 and later, your index name must be unique inside your selected b . Under *Type Mappings*, in the *Bucket* list, select the bucket that contains the documents you want to include in your index. . In the *Scope* list, select the scope that contains these documents. . Expand the collection that contains these documents. -. In your document schema, select a field that contains your vector embeddings. +. In your document schema, select the child field that contains your vector embeddings. + Vectors should be an array of floating point numbers. . Configure the options for your new child field as follows: @@ -49,26 +49,27 @@ Vectors should be an array of floating point numbers. .. In the *Dimension* field, check that the value matches the total number of elements in your vector field's array. + The Search Service supports arrays up to 2048 elements. -Capella should automatically fill in the dimension value for your selected array field. +Capella automatically fills in the dimension value for your selected array field. .. In the *Similarity metric* list, select the method to use to calculate the similarity between search term and Search index vectors. + For more information, see xref:search:quick-index-field-options.adoc#quick-field[Quick Field Type Mapping Options]. .. In the *Optimized for* list, select whether the Search Service should optimize Search queries for accuracy (*recall*) or speed (*latency*). + -For more information, see xref:xref:search:quick-index-field-options.adoc#quick-field[Quick Field Type Mapping Options]. +For more information, see xref:search:quick-index-field-options.adoc#quick-field[Quick Field Type Mapping Options]. .. Select *Index*. . Click btn:[Submit]. -. (Optional) Add additional collections or field type mappings to your index. +. (Optional) Add additional collections or child field type mappings to your index. . Click btn:[Create Index]. == Next Steps -This basic Vector Search index includes the vector embeddings from the field you specified in your type mapping. +This basic Vector Search index includes the vector embeddings from the child field you specified in your type mapping. If you chose to add additional child fields, the Search Service can also return data from those fields when you run a Vector Search query. You can customize your Vector Search index like any other Search index to add additional data and improve search results. For more information about how to customize an index, see xref:search:customize-index.adoc[]. -CAUTION: Some Search index features are only available in Advanced Mode. If you edit your Search index in Advanced Mode, you cannot return to Quick Mode and keep any Advanced Mode settings. +CAUTION: Some Search index features are only available in Advanced Mode. +If you edit your Search index in Advanced Mode, you cannot make any additional edits in Quick Mode without losing all Advanced Mode settings. For more information about how to run a search against a Vector Search index, see xref:run-vector-search-ui.adoc[] or xref:run-vector-search-rest-api.adoc[]. \ No newline at end of file diff --git a/modules/vector-search/pages/run-vector-search-rest-api.adoc b/modules/vector-search/pages/run-vector-search-rest-api.adoc index 0e50d95e5..fdb33afc7 100644 --- a/modules/vector-search/pages/run-vector-search-rest-api.adoc +++ b/modules/vector-search/pages/run-vector-search-rest-api.adoc @@ -7,11 +7,12 @@ == Prerequisites -* You have the Search Service enabled on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. * Your user account has the *Search Admin* or *Search Reader* role. -* You installed the Couchbase command-line tool (CLI). +* You have installed the Couchbase command-line tool (CLI). * You have the hostname for your database. + diff --git a/modules/vector-search/pages/run-vector-search-ui.adoc b/modules/vector-search/pages/run-vector-search-ui.adoc index 936af27e2..bd8d7aa5b 100644 --- a/modules/vector-search/pages/run-vector-search-ui.adoc +++ b/modules/vector-search/pages/run-vector-search-ui.adoc @@ -1,6 +1,6 @@ -= Run A Vector Search with the Capella UI += Run a Vector Search with the Capella UI :page-topic-type: guide -:description: Run a Vector Search query from the Couchbase Capella UI to preview the search results from a Vector Search index. +:description: Run a Vector Search query from the Couchbase Capella UI to preview and test the search results from a Vector Search index. [abstract] {description} @@ -38,6 +38,6 @@ By using the special `match_none` query in the `query` field, the Search query o == Next Steps -If you do not get the search results you were expecting, you can change the xref:search-request-params.adoc[JSON payload for your Search query]. +If you do not get the search results you were expecting, you can change the xref:search:search-request-params.adoc[JSON payload for your Search query]. -You can also xref:search:customize-index.adoc[add additional features to your Search index]. \ No newline at end of file +You can also xref:search:search:customize-index.adoc[add additional features to your Search index]. \ No newline at end of file diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc index 63c2f8e35..ea6b74474 100644 --- a/modules/vector-search/pages/vector-search.adoc +++ b/modules/vector-search/pages/vector-search.adoc @@ -7,7 +7,7 @@ == About Vector Search -Vector Search builds on Couchbase Capella's xref:search.adoc[Search Service] to provide vector index support. +Vector Search builds on Couchbase Capella's xref:search:search.adoc[Search Service] to provide vector index support. You can use these new Vector Search indexes for Retrieval Augmented Generation (RAG) with an existing Large Language Model (LLM). Using Vector Search, Capella, an embedding model, and your chosen LLM, you can develop AI applications while giving context and up-to-date information from your own data. @@ -24,12 +24,13 @@ For more information about all frameworks and integrations supported by Vector S == Using Vector Search Indexes -Use a Capella database to store the data you want to use to provide context to a LLM. -Generate vector embeddings using your preferred embedding model, then store the embeddings in an array inside your documents. +You can get started using Vector Search in Capella by: -Create a Vector Search index to use the vector embeddings from your data and find matches from a search query's generated vectors. +. Using a Capella database to store the data you want to use to provide context to a LLM. +. Generating vector embeddings using your preferred embedding model, then storing the embeddings in an array inside your documents. +. Creating a Vector Search index to use the vector embeddings from your data and find matches from a search query's generated vectors. -If you do not want to use an integration like LangChain, you can also use the API for an existing LLM and one of their embedding models to generate these embeddings for your data. +In addition to supporting integrations with frameworks like LangChain, you can also use the API for an existing LLM and one of their embedding models to generate these embeddings for your data. For example, the OpenAI `embeddings` endpoint can generate embeddings for a text string using a specified embedding model. For more information about how to generate and obtain embeddings for text strings using the OpenAI API, see the https://platform.openai.com/docs/guides/embeddings/what-are-embeddings[Embeddings documentation]. From 6a15a892cbf27097a14882b1d15331f767a1ec84 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 1 Mar 2024 13:33:09 -0500 Subject: [PATCH 33/59] [AV-68571] Undoing a mistaken change. --- modules/search/pages/create-quick-index.adoc | 2 -- 1 file changed, 2 deletions(-) diff --git a/modules/search/pages/create-quick-index.adoc b/modules/search/pages/create-quick-index.adoc index 64e0f3ba1..61e50484a 100644 --- a/modules/search/pages/create-quick-index.adoc +++ b/modules/search/pages/create-quick-index.adoc @@ -17,8 +17,6 @@ For more information about how to change Services on your database, see xref:clo * You have a bucket with scopes and collections in your database. For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You have documents in a keyspace inside your bucket that contain vector embeddings. - * You have logged in to the Couchbase Capella UI. == Procedure From 6527900bc84b3807e5082c1c4c17dcd22a86f79f Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 1 Mar 2024 13:45:51 -0500 Subject: [PATCH 34/59] [AV-68571] Applying some changes that came in through peer review. --- modules/n1ql/pages/n1ql-language-reference/searchfun.adoc | 2 +- modules/search/pages/create-type-mapping.adoc | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc index dd9c835c0..d165bb621 100644 --- a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc @@ -119,7 +119,7 @@ The object may contain the following fields: [Optional] | string, object | The `index` field may be a string, containing the name of a Search index in the keyspace. -(This may be a Search index alias, but only if the Search index is in the same keyspace.) +(This might be a Search index alias, but only if the Search index is in the same keyspace.) This provides an index hint to the Query engine. If the Search index does not exist, an error occurs. diff --git a/modules/search/pages/create-type-mapping.adoc b/modules/search/pages/create-type-mapping.adoc index 0d5fe3632..adcf1d194 100644 --- a/modules/search/pages/create-type-mapping.adoc +++ b/modules/search/pages/create-type-mapping.adoc @@ -29,6 +29,10 @@ To create a type mapping with the Capella UI: + By default, the mapping inherits the default analyzer you set for the Search index. For more information about analyzers, see xref:customize-index.adoc#analyzers[Search Index Features]. .. To switch from a xref:customize-index.adoc#type-mappings[dynamic type mapping to a static type mapping], select *Only index specified fields*. +.. To only include documents of a specific type from a collection, in the *Name* field, add the document type to the end of the collection. ++ +For example, `collection.document_type`. +For more information, see xref:set-type-identifier.adoc[]. . Click btn:[Submit]. == Next Steps From c8d6de5f261a4b045935d478210ed79fb8f05131 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 1 Mar 2024 14:38:35 -0500 Subject: [PATCH 35/59] [AV-68571] Another correction from peer review comments --- modules/vector-search/pages/create-vector-search-index-ui.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/vector-search/pages/create-vector-search-index-ui.adoc b/modules/vector-search/pages/create-vector-search-index-ui.adoc index 31e0814b9..53644315a 100644 --- a/modules/vector-search/pages/create-vector-search-index-ui.adoc +++ b/modules/vector-search/pages/create-vector-search-index-ui.adoc @@ -43,7 +43,7 @@ For version 7.6 and later, your index name must be unique inside your selected b . Expand the collection that contains these documents. . In your document schema, select the child field that contains your vector embeddings. + -Vectors should be an array of floating point numbers. +Vectors must be represented as an array of floating point numbers. . Configure the options for your new child field as follows: .. In the *Type* list, select *vector*. .. In the *Dimension* field, check that the value matches the total number of elements in your vector field's array. From ca923427dd7782857234d3aab8e49fd90e68fbbe Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 1 Mar 2024 14:49:35 -0500 Subject: [PATCH 36/59] [AV-68571] Adding a tip to make the connection between Vector Search and Search clear. --- .../pages/create-vector-search-index-rest-api.adoc | 4 +++- .../vector-search/pages/create-vector-search-index-ui.adoc | 3 +++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc index de2f6ec0c..edcaade69 100644 --- a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc +++ b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc @@ -6,7 +6,9 @@ {description} You must create a Search index before you can xref:run-vector-search-rest-api.adoc[run a search] with the Search Service. -// Add link + +TIP: Vector Search indexes can include all the same features and settings as a Search index. +For more information about Search indexes, see the xref:search:search.adoc[Search documentation]. == Prerequisites diff --git a/modules/vector-search/pages/create-vector-search-index-ui.adoc b/modules/vector-search/pages/create-vector-search-index-ui.adoc index 53644315a..b6e492dad 100644 --- a/modules/vector-search/pages/create-vector-search-index-ui.adoc +++ b/modules/vector-search/pages/create-vector-search-index-ui.adoc @@ -7,6 +7,9 @@ Use xref:search:create-search-index-ui.adoc[Advanced Mode] to add additional settings and more advanced options for your Vector Search index and xref:search:customize-index.adoc#type-mappings[type mappings]. +TIP: Vector Search indexes can include all the same features and settings as a Search index. +For more information about Search indexes, see the xref:search:search.adoc[Search documentation]. + You must create a Vector Search index before you can xref:search:run-vector-search-ui.adoc[run a search] that supports vector comparisons. == Prerequisites From e0d6c4724675c971bde90926f13235dc9d2b6acf Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Tue, 5 Mar 2024 16:10:12 -0500 Subject: [PATCH 37/59] [AV-68571] Applying more suggestions from peer review --- modules/n1ql/pages/n1ql-language-reference/searchfun.adoc | 4 ++-- modules/search/pages/create-quick-index.adoc | 7 +++++-- modules/search/pages/create-search-index-ui.adoc | 2 +- modules/search/pages/customize-index.adoc | 3 +-- modules/search/pages/geo-search-ui.adoc | 2 +- modules/search/pages/search-query-auto-complete-ui.adoc | 2 +- .../vector-search/pages/create-vector-search-index-ui.adoc | 7 +++++-- 7 files changed, 16 insertions(+), 11 deletions(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc index d165bb621..87c6b4e40 100644 --- a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc @@ -10,8 +10,8 @@ Search functions enable you to use xref:search:search.adoc[Full Text Search (FTS [float] === Prerequisites -To use any of the search functions, the Search service must be available on the cluster. -It is also recommended, but not required, that you should create suitable Search indexes for the searches that you need to perform. +To use any of the search functions, the Search Service must be available on the cluster. +It's recommended that you create a suitable Search index for the searches that you want to run. For more information, refer to xref:search:create-search-indexes.adoc[]. [NOTE] diff --git a/modules/search/pages/create-quick-index.adoc b/modules/search/pages/create-quick-index.adoc index 61e50484a..8fa0c038d 100644 --- a/modules/search/pages/create-quick-index.adoc +++ b/modules/search/pages/create-quick-index.adoc @@ -5,7 +5,10 @@ [abstract] {description} -Use xref:create-search-index-ui.adoc[Advanced Mode] to add additional settings and more advanced options for your Search index and xref:customize-index.adoc#type-mappings[type mappings]. +Quick Mode works best when you need to create a basic Search index to start testing and prototyping with the Search Service. +You must use Advanced Mode to have greater control over how the Search Service returns such results, such as changing your xref:customize-index.adoc#analyzers[analyzer]. + +For more information about how to create a Search index in Advanced Mode, see xref:create-search-index-ui.adoc[]. You must create a Search index before you can xref:simple-search-ui.adoc[run a search] with the Search Service. @@ -34,7 +37,7 @@ By default, *Quick Mode* should be on. ==== Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). -For version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. ==== . Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. . In the *Scope* list, select the scope where you want to create the index. diff --git a/modules/search/pages/create-search-index-ui.adoc b/modules/search/pages/create-search-index-ui.adoc index fa4aac882..d41cda3b9 100644 --- a/modules/search/pages/create-search-index-ui.adoc +++ b/modules/search/pages/create-search-index-ui.adoc @@ -33,7 +33,7 @@ To use Advanced Mode in Couchbase Capella to create a Search index: ==== Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). -For version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. ==== . Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. diff --git a/modules/search/pages/customize-index.adoc b/modules/search/pages/customize-index.adoc index 91f62ba82..532274e67 100644 --- a/modules/search/pages/customize-index.adoc +++ b/modules/search/pages/customize-index.adoc @@ -72,8 +72,7 @@ For more information about how to create a custom analyzer, see xref:create-cust Analyzers have different components that control how text is transformed for search. When you create a custom analyzer, you can choose these components. - -Both custom and default analyzers can contain custom filters. +For more information, see <>. |[[custom-filters-table]]Custom Filters | diff --git a/modules/search/pages/geo-search-ui.adoc b/modules/search/pages/geo-search-ui.adoc index 4d068c0a3..c944692fc 100644 --- a/modules/search/pages/geo-search-ui.adoc +++ b/modules/search/pages/geo-search-ui.adoc @@ -38,7 +38,7 @@ To create the Search index in the Capella UI with Advanced Mode: ==== Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). -For version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. ==== . Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. diff --git a/modules/search/pages/search-query-auto-complete-ui.adoc b/modules/search/pages/search-query-auto-complete-ui.adoc index b1471f3b6..6d1563126 100644 --- a/modules/search/pages/search-query-auto-complete-ui.adoc +++ b/modules/search/pages/search-query-auto-complete-ui.adoc @@ -35,7 +35,7 @@ To create the Search index in the Capella UI with Advanced Mode: ==== Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). -For version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. ==== . Under *Advanced Settings*, expand *Analyzers*. . xref:create-custom-analyzer.adoc[] with the following settings: diff --git a/modules/vector-search/pages/create-vector-search-index-ui.adoc b/modules/vector-search/pages/create-vector-search-index-ui.adoc index b6e492dad..f45c5bcae 100644 --- a/modules/vector-search/pages/create-vector-search-index-ui.adoc +++ b/modules/vector-search/pages/create-vector-search-index-ui.adoc @@ -5,7 +5,10 @@ [abstract] {description} -Use xref:search:create-search-index-ui.adoc[Advanced Mode] to add additional settings and more advanced options for your Vector Search index and xref:search:customize-index.adoc#type-mappings[type mappings]. +Quick Mode works best when you need to create a basic Search index to start testing and prototyping with the Search Service. +You must use Advanced Mode to have greater control over how the Search Service returns such results, such as changing your xref:customize-index.adoc#analyzers[analyzer]. + +For more information about how to create a Search index in Advanced Mode, see xref:create-search-index-ui.adoc[]. TIP: Vector Search indexes can include all the same features and settings as a Search index. For more information about Search indexes, see the xref:search:search.adoc[Search documentation]. @@ -39,7 +42,7 @@ By default, *Quick Mode* should be on. ==== Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). -For version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. ==== . Under *Type Mappings*, in the *Bucket* list, select the bucket that contains the documents you want to include in your index. . In the *Scope* list, select the scope that contains these documents. From c99b4f2603d844a88cb3cdb19e2041cf8e1cbfa1 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Fri, 8 Mar 2024 11:12:16 -0500 Subject: [PATCH 38/59] [AV-68571] Feedback from SME review --- .../examples/create-search-index-header.sh | 2 +- .../examples/create-search-index-payload.sh | 2 +- modules/search/examples/run-search-header.sh | 2 +- .../pages/create-search-index-rest-api.adoc | 20 +- modules/search/pages/geo-search-rest-api.adoc | 10 +- .../search/pages/simple-search-rest-api.adoc | 16 +- .../create-vector-search-index-header.sh | 2 +- .../create-vector-search-index-payload.sh | 127 +++++++++---- .../examples/run-vector-search-header.sh | 2 +- .../run-vector-search-payload-ui.jsonc | 4 +- .../examples/run-vector-search-payload.sh | 6 +- .../vector-search-index-payload.jsonc | 179 +++++++++++------- .../create-vector-search-index-rest-api.adoc | 19 +- .../pages/create-vector-search-index-ui.adoc | 12 +- .../pages/run-vector-search-rest-api.adoc | 24 +-- .../pages/run-vector-search-ui.adoc | 21 +- .../vector-search/pages/vector-search.adoc | 23 ++- 17 files changed, 296 insertions(+), 175 deletions(-) diff --git a/modules/search/examples/create-search-index-header.sh b/modules/search/examples/create-search-index-header.sh index 570df8f45..97a651624 100644 --- a/modules/search/examples/create-search-index-header.sh +++ b/modules/search/examples/create-search-index-header.sh @@ -1,3 +1,3 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index + -u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index -d \ \ No newline at end of file diff --git a/modules/search/examples/create-search-index-payload.sh b/modules/search/examples/create-search-index-payload.sh index 15ee4868f..9e2f5c409 100644 --- a/modules/search/examples/create-search-index-payload.sh +++ b/modules/search/examples/create-search-index-payload.sh @@ -1,5 +1,5 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index + -u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index -d \ '{ "type": "fulltext-index", diff --git a/modules/search/examples/run-search-header.sh b/modules/search/examples/run-search-header.sh index 770a66e39..eda474783 100644 --- a/modules/search/examples/run-search-header.sh +++ b/modules/search/examples/run-search-header.sh @@ -1,2 +1,2 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/$BUCKET-NAME/scope/$SCOPE-NAME/index/$INDEX-NAME/query -d \ \ No newline at end of file + -u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/$BUCKET-NAME/scope/$SCOPE-NAME/index/$INDEX-NAME/query -d \ \ No newline at end of file diff --git a/modules/search/pages/create-search-index-rest-api.adoc b/modules/search/pages/create-search-index-rest-api.adoc index 70c21a9bd..9eba66c16 100644 --- a/modules/search/pages/create-search-index-rest-api.adoc +++ b/modules/search/pages/create-search-index-rest-api.adoc @@ -9,21 +9,23 @@ You must create a Search index before you can xref:simple-search-rest-api.adoc[r == Prerequisites -* You have the Search Service enabled on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have a bucket with scopes and collections in your database. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've installed the Couchbase command-line tool (CLI). - -* You have the hostname or IP address for your database. +* You have the hostname for your database. ++ +For more information about how to find the hostname for your Capella database, see xref:clouds:get-capella-host-name.adoc[]. == Procedure To create a Search index with the REST API: . In your command-line tool, enter a `curl` command with the `XPUT` verb. -. Set your header content to include `Content-Type: application/json`. -. Enter your username, password, and the Search Service endpoint on port `8094` with the name of the index you want to create: +. Set your header content to include `"Content-Type: application/json"`. +. Enter your username, password, and the Search Service endpoint on port `18094` with the name of the index you want to create: + [source,console] ---- @@ -39,12 +41,12 @@ For more information about how to create an index with the UI, see xref:create-s === Example In the following example, the JSON payload creates a simple index named `landmark-content-index` on the `travel-sample` bucket. It creates a type mapping for the `inventory.landmark` collection, with a child field, `content`: -+ + [source,console] ---- include::example$create-search-index-payload.sh[] ---- -+ + For more information about the available JSON properties for a Search index, see xref:search-index-params.adoc[]. == Next Steps diff --git a/modules/search/pages/geo-search-rest-api.adoc b/modules/search/pages/geo-search-rest-api.adoc index 0ccc8670f..c24678381 100644 --- a/modules/search/pages/geo-search-rest-api.adoc +++ b/modules/search/pages/geo-search-rest-api.adoc @@ -39,17 +39,19 @@ include::example$geospatial-search-index.jsonc[] To run a Search query against the Search index: . In your command-line tool, enter a `curl` command with the `XPOST` verb. -. Set your header content to include `Content-Type: application/json`. -. Enter your username, password, and the Search Service endpoint on port `8094` with the name of the index you want to query: +. Set your header content to include `"Content-Type: application/json"`. +. Enter your username, password, and the Search Service endpoint on port `18094` with the name of the index you want to query: + [source,console] ---- include::example$run-search-header.sh[] ---- . Enter the JSON payload for your query. -+ + +=== Example + For example, the following query searches a geospatial field, `geo`, for any locations within a 100 mile radius of the coordinates `-2.235143, 53.482358`: -+ + [source,json] ---- include::example$geospatial-search-query.jsonc[] diff --git a/modules/search/pages/simple-search-rest-api.adoc b/modules/search/pages/simple-search-rest-api.adoc index 3be20867a..ee6b756af 100644 --- a/modules/search/pages/simple-search-rest-api.adoc +++ b/modules/search/pages/simple-search-rest-api.adoc @@ -22,32 +22,32 @@ For more information about how to create a Search index, see xref:create-search- To run a simple search with the REST API: . In your command-line tool, enter a `curl` command with the `XPOST` verb. -. Set your header content to include `Content-Type: application/json`. -. Enter your username, password, and the Search Service endpoint on port `8094` with the name of the index you want to query: +. Set your header content to include `"Content-Type: application/json"`. +. Enter your username, password, and the Search Service endpoint on port `18094` with the name of the index you want to query: + [source,console] ---- include::example$run-search-header.sh[] ---- -+ -To use SSL, use the `https` protocol in the Search Service endpoint URL and port `18094`. + . Enter the JSON payload for your query. + TIP: You can copy the JSON for a Query Request from the Couchbase Server or Couchbase Capella UI to use in your REST API call. For more information about how to perform a search with the UI, see xref:simple-search-ui.adoc[]. -+ +=== Example + In the following example, the JSON payload queries an index named `landmark-content-index` for the strings `view`, `food`, and `beach`: -+ + [source,console] ---- include::example$run-search-payload.sh[] ---- -+ + For more information about the available properties for a Search query JSON payload, see xref:search-request-params.adoc[]. == Next Steps -If you don't get the search results you were expecting, you can change the JSON payload xref:search-index-params.adoc[for your Search index] or xref:search-request-params.adoc[for your Search query]. +If you do not get the search results you were expecting, you can change the JSON payload xref:search-index-params.adoc[for your Search index] or xref:search-request-params.adoc[for your Search query]. You can also xref:customize-index.adoc[]. diff --git a/modules/vector-search/examples/create-vector-search-index-header.sh b/modules/vector-search/examples/create-vector-search-index-header.sh index a3d0546d8..40a8e4df3 100644 --- a/modules/vector-search/examples/create-vector-search-index-header.sh +++ b/modules/vector-search/examples/create-vector-search-index-header.sh @@ -1,3 +1,3 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/$BUCKET_NAME/scope/$SCOPE_NAME/index/$INDEX_NAME + -u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/$BUCKET_NAME/scope/$SCOPE_NAME/index/$INDEX_NAME -d \ \ No newline at end of file diff --git a/modules/vector-search/examples/create-vector-search-index-payload.sh b/modules/vector-search/examples/create-vector-search-index-payload.sh index 05a768a8f..695d1057e 100644 --- a/modules/vector-search/examples/create-vector-search-index-payload.sh +++ b/modules/vector-search/examples/create-vector-search-index-payload.sh @@ -1,5 +1,5 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/vector-sample/scope/color/index/color-index + -u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/vector-sample/scope/color/index/color-index -d \ '{ "type": "fulltext-index", @@ -8,7 +8,7 @@ curl -s -XPUT -H "Content-Type: application/json" \ "sourceName": "vector-sample", "sourceUUID": "789365cccdf940ee2814a5dd2752040a", "planParams": { - "maxPartitionsPerPIndex": 1024, + "maxPartitionsPerPIndex": 512, "indexPartitions": 1 }, "params": { @@ -33,44 +33,95 @@ curl -s -XPUT -H "Content-Type: application/json" \ "store_dynamic": false, "type_field": "_type", "types": { - "color.rgb": { - "dynamic": false, - "enabled": true, - "properties": { - "colorvect_dot": { - "dynamic": false, - "enabled": true, - "fields": [ - { - "dims": 3, - "index": true, - "name": "colorvect_dot", - "similarity": "dot_product", - "type": "vector" - } - ] - }, - "colorvect_l2": { - "dynamic": false, - "enabled": true, - "fields": [ - { - "dims": 3, - "index": true, - "name": "colorvect_l2", - "similarity": "l2_norm", - "type": "vector" - } - ] - } + "color.rgb": { + "dynamic": false, + "enabled": true, + "properties": { + "brightness": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "index": true, + "name": "brightness", + "store": true, + "type": "number" + } + ] + }, + "color": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "analyzer": "en", + "index": true, + "name": "color", + "store": true, + "type": "text" + } + ] + }, + "colorvect_dot": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 3, + "index": true, + "name": "colorvect_dot", + "similarity": "dot_product", + "type": "vector" + } + ] + }, + "colorvect_l2": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 3, + "index": true, + "name": "colorvect_l2", + "similarity": "l2_norm", + "type": "vector" + } + ] + }, + "description": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "analyzer": "en", + "index": true, + "name": "description", + "store": true, + "type": "text" + } + ] + }, + "embedding_vector_dot": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 1536, + "index": true, + "name": "embedding_vector_dot", + "similarity": "dot_product", + "type": "vector" + } + ] } } } - }, - "store": { - "indexType": "scorch", - "segmentVersion": 16 } }, - "sourceParams": {} - } \ No newline at end of file + "store": { + "indexType": "scorch", + "segmentVersion": 16 + } + }, + "sourceParams": {} +}' \ No newline at end of file diff --git a/modules/vector-search/examples/run-vector-search-header.sh b/modules/vector-search/examples/run-vector-search-header.sh index 45d695f17..aeb431f85 100644 --- a/modules/vector-search/examples/run-vector-search-header.sh +++ b/modules/vector-search/examples/run-vector-search-header.sh @@ -1,2 +1,2 @@ curl -s -XPUT -H "Content-Type: application/json" \ --u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/$BUCKET_NAME/scope/$SCOPE_NAME/index/$INDEX_NAME/query -d \ \ No newline at end of file +-u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/$BUCKET_NAME/scope/$SCOPE_NAME/index/$INDEX_NAME/query -d \ \ No newline at end of file diff --git a/modules/vector-search/examples/run-vector-search-payload-ui.jsonc b/modules/vector-search/examples/run-vector-search-payload-ui.jsonc index 1eb6e9f25..0fd107d45 100644 --- a/modules/vector-search/examples/run-vector-search-payload-ui.jsonc +++ b/modules/vector-search/examples/run-vector-search-payload-ui.jsonc @@ -6,8 +6,8 @@ "knn": [ { "k": 2, - "field": "colorvect_dot", - "vector": [ 0.707106781186548, 0, 0.707106781186548 ] + "field": "colorvect_l2", + "vector": [ 0, 0, 128 ] } ] } \ No newline at end of file diff --git a/modules/vector-search/examples/run-vector-search-payload.sh b/modules/vector-search/examples/run-vector-search-payload.sh index 463f263b3..f72ffbd59 100644 --- a/modules/vector-search/examples/run-vector-search-payload.sh +++ b/modules/vector-search/examples/run-vector-search-payload.sh @@ -1,5 +1,5 @@ curl -XPOST -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/vector-sample/scope/color/index/color-index/query \ + -u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/vector-sample/scope/color/index/color-index/query \ -d '{ "fields": ["*"], "query": { @@ -12,8 +12,8 @@ curl -XPOST -H "Content-Type: application/json" \ "knn": [ { "k": 10, - "field": "colorvect_dot", - "vector": [ 0.707106781186548, 0, 0.707106781186548 ] + "field": "colorvect_l2", + "vector": [ 176, 0, 176 ] } ] }' \ No newline at end of file diff --git a/modules/vector-search/examples/vector-search-index-payload.jsonc b/modules/vector-search/examples/vector-search-index-payload.jsonc index 96133d9d5..fe4ccce3a 100644 --- a/modules/vector-search/examples/vector-search-index-payload.jsonc +++ b/modules/vector-search/examples/vector-search-index-payload.jsonc @@ -1,74 +1,123 @@ { - "type": "fulltext-index", - "name": "vector-sample.color.color-test", - "uuid": "5b23225c54c55b64", - "sourceType": "gocbcore", - "sourceName": "vector-sample", - "sourceUUID": "789365cccdf940ee2814a5dd2752040a", - "planParams": { - "maxPartitionsPerPIndex": 1024, - "indexPartitions": 1 + "type": "fulltext-index", + "name": "vector-sample.color.color-index", + "sourceType": "gocbcore", + "sourceName": "vector-sample", + "planParams": { + "maxPartitionsPerPIndex": 512, + "indexPartitions": 1 + }, + "params": { + "doc_config": { + "docid_prefix_delim": "", + "docid_regexp": "", + "mode": "scope.collection.type_field", + "type_field": "type" }, - "params": { - "doc_config": { - "docid_prefix_delim": "", - "docid_regexp": "", - "mode": "scope.collection.type_field", - "type_field": "type" + "mapping": { + "analysis": {}, + "default_analyzer": "standard", + "default_datetime_parser": "dateTimeOptional", + "default_field": "_all", + "default_mapping": { + "dynamic": false, + "enabled": false }, - "mapping": { - "analysis": {}, - "default_analyzer": "standard", - "default_datetime_parser": "dateTimeOptional", - "default_field": "_all", - "default_mapping": { + "default_type": "_default", + "docvalues_dynamic": false, + "index_dynamic": false, + "store_dynamic": false, + "type_field": "_type", + "types": { + "color.rgb": { "dynamic": false, - "enabled": false - }, - "default_type": "_default", - "docvalues_dynamic": false, - "index_dynamic": false, - "store_dynamic": false, - "type_field": "_type", - "types": { - "color.rgb": { - "dynamic": false, - "enabled": true, - "properties": { - "colorvect_dot": { - "dynamic": false, - "enabled": true, - "fields": [ - { - "dims": 3, - "index": true, - "name": "colorvect_dot", - "similarity": "dot_product", - "type": "vector" - } - ] - }, - "colorvect_l2": { - "dynamic": false, - "enabled": true, - "fields": [ - { - "dims": 3, - "index": true, - "name": "colorvect_l2", - "similarity": "l2_norm", - "type": "vector" - } - ] - } + "enabled": true, + "properties": { + "brightness": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "index": true, + "name": "brightness", + "store": true, + "type": "number" + } + ] + }, + "color": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "analyzer": "en", + "index": true, + "name": "color", + "store": true, + "type": "text" + } + ] + }, + "colorvect_dot": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 3, + "index": true, + "name": "colorvect_dot", + "similarity": "dot_product", + "type": "vector" + } + ] + }, + "colorvect_l2": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 3, + "index": true, + "name": "colorvect_l2", + "similarity": "l2_norm", + "type": "vector" + } + ] + }, + "description": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "analyzer": "en", + "index": true, + "name": "description", + "store": true, + "type": "text" + } + ] + }, + "embedding_vector_dot": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 1536, + "index": true, + "name": "embedding_vector_dot", + "similarity": "dot_product", + "type": "vector" + } + ] } } } - }, - "store": { - "indexType": "scorch", - "segmentVersion": 16 } }, - "sourceParams": {} - } \ No newline at end of file + "store": { + "indexType": "scorch", + "segmentVersion": 16 + } + }, + "sourceParams": {} +} \ No newline at end of file diff --git a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc index edcaade69..a5a94b97f 100644 --- a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc +++ b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc @@ -18,9 +18,9 @@ For more information about how to change Services on your database, see xref:clo * You have a bucket with scopes and collections in your database. For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You have documents in a keyspace inside your bucket that contain vector embeddings. - -* You have installed the Couchbase command-line tool (CLI). +* You have documents in a keyspace inside your bucket that contain vector embeddings. ++ +You can https://cbc-remote-execution-examples-prod.s3.amazonaws.com/color_data_2vectors.zip[download a sample dataset] to use with the procedure on this page. * You have the hostname for your database. + @@ -31,8 +31,8 @@ For more information about how to find the hostname for your Capella database, s To create a Search index with the REST API and Couchbase Capella: . In your command-line tool, enter a `curl` command with the `XPUT` verb. -. Set your header content to include `Content-Type: application/json`. -. Enter your username, password, and the Search Service endpoint on port `8094` with the name of the index you want to create: +. Set your header content to include `"Content-Type: application/json"`. +. Enter your username, password, and the Search Service endpoint on port `18094` with the name of the index you want to create: + [source,console] ---- @@ -47,13 +47,16 @@ For more information about how to create a Vector Search index with the UI, see === Example -In the following example, the JSON payload creates an index named `color-index` on the `vector-sample` bucket. It creates two child field mappings, `colorvect_dot` and `colorvect_l2` on two different vector fields in the documents under the `color.rgb` keyspace: -+ +In the following example, the JSON payload creates an index named `color-index` on the `vector-sample.color.rgb` keyspace. +It creates two child field mappings, `colorvect_l2` and `embedding_vector_dot` on two different vector fields in the keyspace's documents. + +It also adds 3 normal Search index fields (`brightness`, `color`, and `description`) to add more usable data to the Vector Search index: + [source,console] ---- include::example$create-vector-search-index-payload.sh[] ---- -+ + For more information about all the available JSON properties for a Search index, see xref:search:search-index-params.adoc[]. == Next Steps diff --git a/modules/vector-search/pages/create-vector-search-index-ui.adoc b/modules/vector-search/pages/create-vector-search-index-ui.adoc index f45c5bcae..0733f5a27 100644 --- a/modules/vector-search/pages/create-vector-search-index-ui.adoc +++ b/modules/vector-search/pages/create-vector-search-index-ui.adoc @@ -8,7 +8,7 @@ Quick Mode works best when you need to create a basic Search index to start testing and prototyping with the Search Service. You must use Advanced Mode to have greater control over how the Search Service returns such results, such as changing your xref:customize-index.adoc#analyzers[analyzer]. -For more information about how to create a Search index in Advanced Mode, see xref:create-search-index-ui.adoc[]. +For more information about how to create a Search index in Advanced Mode, see xref:search:create-search-index-ui.adoc[]. TIP: Vector Search indexes can include all the same features and settings as a Search index. For more information about Search indexes, see the xref:search:search.adoc[Search documentation]. @@ -24,6 +24,8 @@ For more information about how to change Services on your database, see xref:clo For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. * You have documents in a keyspace inside your bucket that contain vector embeddings. ++ +You can https://cbc-remote-execution-examples-prod.s3.amazonaws.com/color_data_2vectors.zip[download a sample dataset] to use with the procedure on this page. * You have logged in to the Couchbase Capella UI. @@ -50,8 +52,8 @@ For Couchbase Server version 7.6 and later, your index name must be unique insid . In your document schema, select the child field that contains your vector embeddings. + Vectors must be represented as an array of floating point numbers. +They appear as `\{field-name} [ number ]` in the Capella Quick Mode editor. . Configure the options for your new child field as follows: -.. In the *Type* list, select *vector*. .. In the *Dimension* field, check that the value matches the total number of elements in your vector field's array. + The Search Service supports arrays up to 2048 elements. @@ -64,13 +66,15 @@ For more information, see xref:search:quick-index-field-options.adoc#quick-field For more information, see xref:search:quick-index-field-options.adoc#quick-field[Quick Field Type Mapping Options]. .. Select *Index*. . Click btn:[Submit]. -. (Optional) Add additional collections or child field type mappings to your index. +. (Optional) Add additional collections or child field type mappings to your index. ++ +For example, you could add the text field that you used to generate your vector embeddings. . Click btn:[Create Index]. == Next Steps This basic Vector Search index includes the vector embeddings from the child field you specified in your type mapping. -If you chose to add additional child fields, the Search Service can also return data from those fields when you run a Vector Search query. +If you chose to add additional child fields and enabled *Include in search results*, the Search Service can also return data from those fields when you run a Vector Search query. You can customize your Vector Search index like any other Search index to add additional data and improve search results. For more information about how to customize an index, see xref:search:customize-index.adoc[]. diff --git a/modules/vector-search/pages/run-vector-search-rest-api.adoc b/modules/vector-search/pages/run-vector-search-rest-api.adoc index fdb33afc7..393d9f6fd 100644 --- a/modules/vector-search/pages/run-vector-search-rest-api.adoc +++ b/modules/vector-search/pages/run-vector-search-rest-api.adoc @@ -10,10 +10,6 @@ * You have the Search Service enabled on a node in your database. For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* Your user account has the *Search Admin* or *Search Reader* role. - -* You have installed the Couchbase command-line tool (CLI). - * You have the hostname for your database. + For more information about how to find the hostname for your Capella database, see xref:clouds:get-capella-host-name.adoc[]. @@ -27,29 +23,33 @@ For more information about how to create a Vector Search index, see xref:create- To run a Vector search with the REST API: . In your command-line tool, enter a `curl` command with the `XPOST` verb. -. Set your header content to include `Content-Type: application/json`. -. Enter your username, password, and the Search Service endpoint on port `8094` with the name of the Vector Search index you want to query: +. Set your header content to include `"Content-Type: application/json"`. +. Enter your username, password, and the Search Service endpoint on port `18094` with the name of the Vector Search index you want to query: + [source,console] ---- include::example$run-vector-search-header.sh[] ---- + -To use SSL, use the `https` protocol in the Search Service endpoint URL and port `18094`. . Enter the JSON payload for your query. + TIP: You can copy the JSON for a Query Request from the Capella UI to use in your REST API call. For more information about how to perform a search with the UI, see xref:search:simple-search-ui.adoc[]. -+ -In the following example, the JSON payload uses a hybrid Vector Search and Search query on an index named `color-index`. -The query searches for a specified normalized color vector, but uses the regular query parameters to limit the brightness of the returned color to the range of `70-80`: -+ +=== Example + +In the following example, the JSON payload uses both a `query` and `knn` object to run both a Vector Search and traditional Search query on an index named `color-index`. + +The query searches for a specified Euclidean distance color vector, but uses the `query` object to search for a color with a `brightness` value in the range of `70-80`: + [source,console] ---- include::example$run-vector-search-payload.sh[] ---- -+ + +The Search Service combines the Vector search results from the `knn` object with the traditional `query` object by using an `OR` function. +If the same documents match the `knn` and `query` objects, the Search Service ranks those documents higher in search results. + For more information about the available properties for a Search query JSON payload, see xref:search:search-request-params.adoc[]. == Next Steps diff --git a/modules/vector-search/pages/run-vector-search-ui.adoc b/modules/vector-search/pages/run-vector-search-ui.adoc index bd8d7aa5b..0e5994144 100644 --- a/modules/vector-search/pages/run-vector-search-ui.adoc +++ b/modules/vector-search/pages/run-vector-search-ui.adoc @@ -22,19 +22,24 @@ To run a Vector Search with the Capella UI: . On the *Databases* page, select the database where you created your Search index. . Go to menu:Data Tools[Search]. . Next to your Vector Search index, click btn:[Search]. -. In the *Search* field, enter a search query. -+ -For example, the following query searches for the top 2 vectors similar to the vector `[ 0.707106781186548, 0, 0.707106781186548 ]` in the `colorvect_dot` field: -+ +. In the *Search* field, enter a search query. +. Press kbd:[Enter] or click btn:[Search]. +. . (Optional) To view a document and its source collection, click a document name in the search results list. + +=== Example + +For example, the following query searches for the top 2 vectors similar to the vector `[ 0, 0, 128 ]` in the `colorvect_l2` field: + [source,json] ---- include::example$run-vector-search-payload-ui.jsonc[] ---- -+ -By using the special `match_none` query in the `query` field, the Search query only returns the `k` number of similar vectors. -. Press kbd:[Enter]. -. (Optional) To view a document and its source collection, click a document name in the search results list. +By using the special `match_none` query in the `query` field, the Search query is only a Vector Search query. +It only returns the `k` number of similar vectors. + +The Search Service combines the Vector search results from a `knn` object with the traditional `query` object by using an `OR` function. +If the same documents match the `knn` and `query` objects, the Search Service ranks those documents higher in search results. == Next Steps diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc index ea6b74474..6510f581f 100644 --- a/modules/vector-search/pages/vector-search.adoc +++ b/modules/vector-search/pages/vector-search.adoc @@ -10,28 +10,33 @@ Vector Search builds on Couchbase Capella's xref:search:search.adoc[Search Service] to provide vector index support. You can use these new Vector Search indexes for Retrieval Augmented Generation (RAG) with an existing Large Language Model (LLM). -Using Vector Search, Capella, an embedding model, and your chosen LLM, you can develop AI applications while giving context and up-to-date information from your own data. +Using Capella's Vector Search, an embedding model, and your chosen LLM, you can develop AI applications while giving context and up-to-date information from your own data. You can develop applications that include: -* *Similarity search:* Search for documents, products, images, and more that are similar to a given query. -Using vector embeddings to represent data, your end users can search based on descriptions, rather than using specific keywords. +* *Similarity search:* Search for documents, products, images, and more that are similar to a given query using vector embeddings. +By using vector embeddings, you can search based on descriptions, rather than using specific keywords and get intuitive and relevant results across data types. -* *Generative AI:* Create new text, images, and more based on a prompt given to the LLM. +* *Semantic search*: Use natural language processing to deliver more accurate results based on an understanding of the intent and context behind a given search query, rather than simple keyword matches. + +* *Generative AI:* Create new original content, such as text and images, based on a prompt or a vector search given to an LLM. +Use generative AI to get tailored and dynamic responses across your applications. Vector Search supports integrations with frameworks like https://python.langchain.com/docs/get_started/introduction[LangChain^] to support AI application development. For more information about all frameworks and integrations supported by Vector Search and Capella, see xref:third-party:integrations.adoc[]. == Using Vector Search Indexes -You can get started using Vector Search in Capella by: +To get started using Vector Search in Capella: -. Using a Capella database to store the data you want to use to provide context to a LLM. -. Generating vector embeddings using your preferred embedding model, then storing the embeddings in an array inside your documents. -. Creating a Vector Search index to use the vector embeddings from your data and find matches from a search query's generated vectors. +. *Store data*: Store the data you want to use for your search or AI project in a Capella database. +. *Generate embeddings*: Generate vector embeddings from your data with your preferred embedding model. +. *Store your embeddings*: Store your vector embeddings in an array inside the documents in your Capella database. +. *Create a Vector Search index*: Create an index to use your embeddings and identify similar documents with vector similarity. -In addition to supporting integrations with frameworks like LangChain, you can also use the API for an existing LLM and one of their embedding models to generate these embeddings for your data. +In addition to supporting integrations with frameworks like LangChain and LlamaIndex, you can also use the API for an existing LLM and one of their embedding models to generate vector embeddings for your data. For example, the OpenAI `embeddings` endpoint can generate embeddings for a text string using a specified embedding model. +You can then store that embedding as a new field in your documents. For more information about how to generate and obtain embeddings for text strings using the OpenAI API, see the https://platform.openai.com/docs/guides/embeddings/what-are-embeddings[Embeddings documentation]. NOTE: When you create a Vector Search index, the xref:search:child-field-options-reference.adoc#dimension[dimension] of your data vector embeddings must match the dimension for any search query vectors. From 17637f4f400f0799dd25b437d45998f405b7517b Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Mon, 11 Mar 2024 16:29:22 -0400 Subject: [PATCH 39/59] [AV-68571] More updates from SME review --- .../pages/create-search-index-rest-api.adoc | 4 +- .../search/pages/simple-search-rest-api.adoc | 4 +- .../run-vector-search-long-payload-ui.jsonc | 1553 +++++++++++++++++ .../create-vector-search-index-rest-api.adoc | 6 +- .../pages/create-vector-search-index-ui.adoc | 2 +- .../pages/run-vector-search-rest-api.adoc | 4 +- .../pages/run-vector-search-ui.adoc | 42 +- .../partials/download-sample-partial.adoc | 15 + 8 files changed, 1619 insertions(+), 11 deletions(-) create mode 100644 modules/vector-search/examples/run-vector-search-long-payload-ui.jsonc create mode 100644 modules/vector-search/partials/download-sample-partial.adoc diff --git a/modules/search/pages/create-search-index-rest-api.adoc b/modules/search/pages/create-search-index-rest-api.adoc index 9eba66c16..8e4a01c43 100644 --- a/modules/search/pages/create-search-index-rest-api.adoc +++ b/modules/search/pages/create-search-index-rest-api.adoc @@ -15,9 +15,9 @@ For more information about how to change Services on your database, see xref:clo * You have a bucket with scopes and collections in your database. For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You have the hostname for your database. +* You have the hostname for the node in your database that's running the Search Service. + -For more information about how to find the hostname for your Capella database, see xref:clouds:get-capella-host-name.adoc[]. +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. == Procedure diff --git a/modules/search/pages/simple-search-rest-api.adoc b/modules/search/pages/simple-search-rest-api.adoc index ee6b756af..691037e23 100644 --- a/modules/search/pages/simple-search-rest-api.adoc +++ b/modules/search/pages/simple-search-rest-api.adoc @@ -11,7 +11,9 @@ * You installed the Couchbase command-line tool (CLI). -* You have the hostname or IP address for your database. +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. * You've created a Search index. + diff --git a/modules/vector-search/examples/run-vector-search-long-payload-ui.jsonc b/modules/vector-search/examples/run-vector-search-long-payload-ui.jsonc new file mode 100644 index 000000000..d7c2d9a93 --- /dev/null +++ b/modules/vector-search/examples/run-vector-search-long-payload-ui.jsonc @@ -0,0 +1,1553 @@ +// tag::partial[] +{ + "fields": ["*"], + "knn": [ + { + "field": "embedding_vector_dot", + "k": 3, + "vector": [ + 0.024032991379499435, + -0.009131478145718575, + 0.013961897231638432, + -0.024734394624829292, + -0.020605377852916718, + 0.006739427801221609, + -0.012539239600300789, + 0.0063192471861839294, + 0.000004374724539957242, + -0.030252983793616295, + -0.010944539681077003, + -0.0012845275923609734, + 0.0059850881807506084, + -0.006388725712895393, + -0.016304319724440575, + 0.03046472743153572, + 0.029988301917910576, + -0.013121536932885647, + 0.01815708354115486, + -0.011096730828285217, + -0.0423753522336483, + -0.0023523480631411076, + -0.00022332418302539736, + -0.0024681459181010723, + -0.02911485731601715, +// end::partial[] + -0.01983780413866043, + 0.025263754650950432, + -0.02444324642419815, + 0.001819678582251072, + -0.01301566418260336, + 0.021015632897615433, + -0.006084343418478966, + -0.027394432574510574, + -0.02503877691924572, + -0.0030520972795784473, + 0.0020049549639225006, + -0.006124045234173536, + 0.01994367688894272, + 0.0008941238629631698, + 0.005230748560279608, + 0.004694770555943251, + 0.0074507566168904305, + 0.007325033191591501, + -0.01629108563065529, + -0.000024400234906352125, + 0.002851932542398572, + 0.007550011854618788, + -0.005564908031374216, + -0.002355656586587429, + -0.0024466405156999826, + 0.025422563776373863, + 0.042851775884628296, + -0.016185212880373, + -0.02908838912844658, + -0.002714629517868161, + 0.006143896374851465, + -0.011930474080145359, + 0.027685582637786865, + -0.008377138525247574, + 0.0012580595212057233, + 0.0012944531626999378, + -0.02789732627570629, + -0.02935306914150715, + -0.000022397038264898583, + -0.015377937816083431, + -0.00907854177057743, + -0.022788992151618004, + 0.027526773512363434, + 0.00034181008231826127, + 0.018964359536767006, + 0.010951156727969646, + 0.01720423437654972, + 0.007000799756497145, + 0.018104147166013718, + 0.02603132836520672, + 0.0017601253930479288, + -0.012863473035395145, + 0.0016286122845485806, + 0.012876707129180431, + -0.000903222244232893, + 0.004208420403301716, + -0.0030984163749963045, + -0.021280312910676003, + 0.010673241689801216, + 0.015801426023244858, + -0.004473100882023573, + -0.015788191929459572, + -0.0008205096237361431, + -0.0007990042795427144, + -0.005336620844900608, + -0.01697925478219986, + 0.013975131325423717, + 0.014120705425739288, + 0.016820447519421577, + -0.013187706470489502, + 0.01565585285425186, + 0.021968482062220573, + 0.020353931933641434, + 0.006928012706339359, + 0.006497906520962715, + 0.02543579787015915, + 0.005733641795814037, + -0.028611963614821434, + -0.005538439843803644, + -0.01806444488465786, + -0.000736969814170152, + -0.002863512374460697, + -0.0022233163472265005, + 0.012605409137904644, + 0.0006480537122115493, + -0.04531330242753029, + 0.02698417939245701, + 0.02319924719631672, + -0.07024621218442917, + 0.011996644549071789, + -0.018024742603302002, + 0.018448231741786003, + -0.020909760147333145, + 0.012962727807462215, + -0.028876643627882004, + -0.006253077182918787, + 0.02842668816447258, + 0.04396343231201172, + -0.011407730169594288, + 0.01753508485853672, + -0.008132308721542358, + -0.003953665029257536, + -0.023013969883322716, + -0.033482085913419724, + -0.0037981653586030006, + 0.017455680295825005, + 0.0035533360205590725, + -0.012499537318944931, + 0.012605409137904644, + 0.007960266433656216, + 0.005336620844900608, + -0.01569555513560772, + 0.020181888714432716, + 0.01462359819561243, + -0.003325049066916108, + 0.022087588906288147, + 0.025713711977005005, + -0.00886679720133543, + 0.005432567559182644, + -0.017415978014469147, + 0.01001815777271986, + 0.006309321615844965, + 0.006451587658375502, + 0.0034673146437853575, + -0.00044209917541593313, + 0.028082601726055145, + -0.008165393956005573, + 0.04393696412444115, + 0.014200109988451004, + -0.010223285295069218, + 0.020380400121212006, + -0.006474747322499752, + 0.012651728466153145, + -0.02260371670126915, + 0.007060352712869644, + -0.008562414906919003, + -0.006196832749992609, + 0.011639325879514217, + 0.0041852607391774654, + 0.015589682385325432, + 0.017455680295825005, + 0.009535115212202072, + 0.00853594671934843, + 0.0044929515570402145, + 0.015457342378795147, + -0.029564812779426575, + 0.004856887273490429, + -0.01381632313132286, + 0.016926318407058716, + 0.01664840430021286, + 0.03329680860042572, + 0.03544072061777115, + 0.0004168718005530536, + -0.001428447663784027, + -0.0028668208979070187, + -0.008350670337677002, + 0.015100023709237576, + 0.02648128569126129, + -0.015430874191224575, + 0.004459866788238287, + 0.011136433109641075, + 0.042957648634910583, + -0.016211681067943573, + 0.0009454057435505092, + -0.0006463994504883885, + 0.002449949039146304, + 0.019229039549827576, + -0.011374644935131073, + -0.025978391990065575, + -0.651749312877655, + -0.010838666930794716, + -0.020115718245506287, + -0.013445770367980003, + 0.022418439388275146, + 0.022775758057832718, + 0.02382124587893486, + 0.010554135777056217, + -0.022749289870262146, + -0.02069801650941372, + -0.0025756722316145897, + 0.010620305314660072, + 0.004258047789335251, + -0.004555813502520323, + -0.025687243789434433, + -0.017164532095193863, + 0.006683182902634144, + -0.004039686173200607, + 0.006755969952791929, + -0.001948710298165679, + 0.014491258189082146, + 0.02924719639122486, + -0.03925212100148201, + -0.013101685792207718, + 0.013856025412678719, + 0.014822108671069145, + -0.0013556604972109199, + -0.002663347637280822, + -0.01099085807800293, + 0.0030239748302847147, + -0.017098361626267433, + 0.02619013749063015, + -0.011288624256849289, + 0.007768373005092144, + 0.052724357694387436, + -0.005472269840538502, + -0.027950262650847435, + 0.02911485731601715, + -0.004092622548341751, + 0.047298409044742584, + 0.007298565004020929, + 0.00609426898881793, + 0.008403606712818146, + -0.02016865462064743, + -0.0014532614732161164, + 0.020486271008849144, + 0.004552504979074001, + -0.028109069913625717, + 0.025343159213662148, + -0.012843621894717216, + 0.026269542053341866, + -0.005389557220041752, + 0.005214205943048, + 0.012625260278582573, + 0.010527667589485645, + -0.009455711580812931, + 0.026110732927918434, + -0.016661638393998146, + -0.017230700701475143, + 0.021769972518086433, + 0.01143419835716486, + 0.01828942447900772, + -0.005220822989940643, + -0.01529853418469429, + -0.01018358301371336, + 0.044333986937999725, + -0.029723621904850006, + -0.009257201105356216, + -0.009667456150054932, + -0.01136802788823843, + -0.016674872487783432, + 0.018315890803933144, + -0.013617812655866146, + -0.006435045041143894, + 0.00004895555684925057, + -0.0009371344349347055, + 0.009925519116222858, + 0.019930442795157433, + -0.00006508452497655526, + 0.01435891818255186, + -0.00855579786002636, + -0.019559890031814575, + -0.008330819196999073, + 0.0074309054762125015, + 0.022524312138557434, + -0.0004739435389637947, + -0.029750090092420578, + -0.03112642839550972, + 0.01635725609958172, + 0.015735257416963577, + 0.03221162036061287, + 0.01529853418469429, + -0.0031728576868772507, + -0.018170317634940147, + -0.018368827179074287, + 0.019599592313170433, + -0.01393542904406786, + -0.013842791318893433, + -0.016463128849864006, + -0.01426627952605486, + -0.006759278476238251, + -0.03874922916293144, + -0.022286098450422287, + 0.009422626346349716, + 0.009462328627705574, + 0.007966883480548859, + 0.0047907172702252865, + 0.02681213617324829, + 0.008403606712818146, + -0.017191000282764435, + 0.015470576472580433, + -0.003626123070716858, + -0.015576448291540146, + 0.010110795497894287, + 0.01095777377486229, + -0.03300565853714943, + 0.015576448291540146, + 0.004181952215731144, + -0.005713790655136108, + -0.031470514833927155, + 0.021822908893227577, + -0.013882492668926716, + -0.0054855034686625, + -0.0008444962440989912, + 0.004516111221164465, + 0.013339897617697716, + -0.008999137207865715, + -0.007232395000755787, + 0.016939552500844002, + -0.0021885770838707685, + 0.02667979523539543, + 0.024866733700037003, + 0.010640156455338001, + 0.00876754242926836, + -0.008648435585200787, + -0.011626091785728931, + -0.021756738424301147, + -0.032158683985471725, + -0.0016699685947969556, + -0.016171978786587715, + -0.003808090928941965, + -0.006808906327933073, + -0.011096730828285217, + 0.0059156096540391445, + -0.011242304928600788, + -0.04578972980380058, + -0.027235625311732292, + -0.001535146962851286, + -0.030120642855763435, + -0.005214205943048, + -0.02289486490190029, + -0.013121536932885647, + -0.024337373673915863, + 0.040019694715738297, + 0.011447432450950146, + 0.0007717091357335448, + -0.02802966721355915, + -0.009779945015907288, + 0.0006215856410562992, + 0.011817985214293003, + 0.006484672427177429, + 0.02089652605354786, + -0.025157883763313293, + 0.011725346557796001, + -0.00974024273455143, + -0.035652466118335724, + -0.011996644549071789, + 0.037425823509693146, + -0.02694447711110115, + -0.027447368949651718, + 0.008747691288590431, + 0.009965221397578716, + 0.007530160713940859, + 0.01865997724235058, + 0.011844453401863575, + 0.015523511916399002, + -0.015589682385325432, + -0.02718268893659115, + -0.007788224145770073, + -0.024403544142842293, + 0.01283038780093193, + 0.013829557225108147, + -0.03345561772584915, + -0.009018988348543644, + 0.060982391238212585, + 0.000804794195573777, + 0.009680690243840218, + 0.023609502241015434, + 0.006474747322499752, + -0.004674919415265322, + 0.006090960465371609, + 0.000341189734172076, + -0.0031877460423856974, + 0.007569862529635429, + -0.005925535224378109, + 0.0037981653586030006, + 0.006110811606049538, + 0.00162695802282542, + 0.011566538363695145, + 0.013009047135710716, + 0.015192661434412003, + -0.02388741634786129, + 0.01214221864938736, + 0.0037518462631851435, + 0.02109503746032715, + -0.0071596079505980015, + 0.01191062293946743, + -0.012367197312414646, + 0.009191030636429787, + 0.036711186170578, + 0.015139725059270859, + -0.01605287380516529, + -0.004539270885288715, + -0.028903111815452576, + 0.023940352723002434, + 0.00964098796248436, + 0.001996683655306697, + 0.027950262650847435, + -0.00865505263209343, + -0.006564076989889145, + 0.030226515606045723, + 0.024138862267136574, + 0.020062783733010292, + 0.006431736517697573, + 0.007556628901511431, + -0.004145558457821608, + 0.021611163392663002, + -0.01704542525112629, + -0.01731010526418686, + -0.011870920658111572, + -0.010712943971157074, + 0.021637631580233574, + -0.012724515981972218, + 0.0007100881775841117, + -0.021412653848528862, + -0.003788239788264036, + 0.021637631580233574, + -0.025343159213662148, + 0.016952786594629288, + -0.005138110369443893, + -0.009462328627705574, + 0.011321709491312504, + -0.00962113682180643, + -0.003376330714672804, + 0.04375169053673744, + 0.0035764954518526793, + 0.016608702018857002, + 0.008674903772771358, + -0.02470792643725872, + -0.01157977245748043, + -0.0007167052244767547, + 0.010090944357216358, + 0.0015897373668849468, + -0.012856855988502502, + 0.015483810566365719, + 0.006226608995348215, + 0.03046472743153572, + 0.0045194197446107864, + 0.00031430809758603573, + 0.01122245378792286, + 0.02826787903904915, + -0.01930844411253929, + -0.013095068745315075, + -0.02158469520509243, + 0.012737750075757504, + -0.00013575215416494757, + -0.0029578048270195723, + -0.0021836142987012863, + 0.0074441395699977875, + -0.017429212108254433, + -0.009700540453195572, + -0.002613720018416643, + 0.005379631649702787, + -0.02964421734213829, + 0.013095068745315075, + 0.003794856835156679, + -0.002913139993324876, + 0.0006728675216436386, + 0.02487996779382229, + 0.0037055271677672863, + -0.008066138252615929, + -0.03287332132458687, + 0.023371288552880287, + 0.03676412254571915, + -0.01757478527724743, + -0.02871783636510372, + -0.03162932023406029, + 0.009270435199141502, + -0.006329172756522894, + 0.035652466118335724, + 0.009965221397578716, + 0.014173641800880432, + -0.007338267285376787, + -0.008165393956005573, + -0.010878369212150574, + -0.01648959517478943, + 0.020539207383990288, + 0.006051258184015751, + 0.010501199401915073, + 0.016185212880373, + 0.020446570590138435, + -0.020737718790769577, + -0.01711159572005272, + 0.011361410841345787, + 0.02362273633480072, + 0.020406868308782578, + 0.00044127204455435276, + 0.006491289474070072, + 0.007397820241749287, + -0.009316754527390003, + -0.0038610270712524652, + -0.0034640063531696796, + -0.004406930413097143, + -0.0034640063531696796, + 0.031735192984342575, + 0.004211728926748037, + -0.010600454173982143, + -0.007463990710675716, + 0.04571032524108887, + -0.014941215515136719, + 0.010924688540399075, + -0.023305119946599007, + -0.02665332704782486, + -0.003354825545102358, + 0.07643973082304001, + -0.006398651283234358, + -0.007695585954934359, + 0.012439983896911144, + -0.009091775864362717, + -0.009085158817470074, + -0.03761110082268715, + -0.00696771452203393, + 0.018183551728725433, + 0.022947801277041435, + 0.019268741831183434, + -0.00004192498090560548, + -0.01859380677342415, + 0.011950325220823288, + 0.006415193900465965, + -0.0043870797380805016, + 0.013425919227302074, + -0.024668224155902863, + -0.02056567557156086, + -0.018540870398283005, + -0.0038279418367892504, + 0.0013523519737645984, + -0.0003715866187121719, + 0.01462359819561243, + -0.0010727832559496164, + 0.019851038232445717, + 0.029961833730340004, + 0.012532622553408146, + 0.015616150572896004, + -0.01605287380516529, + 0.002143912250176072, + 0.01839529536664486, + -0.01921580545604229, + 0.012512771412730217, + -0.007027267478406429, + -0.006315938662737608, + -0.01957312412559986, + 0.015510277822613716, + 0.009746859781444073, + -0.01924227364361286, + 0.028320815414190292, + 0.00909839291125536, + 0.0005922226700931787, + -0.0003314709756523371, + 0.015126491896808147, + -0.009224115870893002, + 0.004671610891819, + 0.0048535787500441074, + -0.0031000706367194653, + -0.00907854177057743, + 0.0035401019267737865, + -0.02415209636092186, + -0.013167855329811573, + 0.013432536274194717, + 0.006841991096735001, + 0.021597929298877716, + 0.0002160868316423148, + -0.0038180164992809296, + 0.0008908153395168483, + -0.018315890803933144, + -0.019652528688311577, + -0.008893265388906002, + -0.0016608702717348933, + -0.018540870398283005, + -0.014438321813941002, + -0.04007263109087944, + 0.005839514080435038, + -0.03292625769972801, + -0.01275098416954279, + 0.013240642845630646, + 0.010243135504424572, + -0.01592053286731243, + -0.034699615091085434, + 0.013617812655866146, + 0.031496983021497726, + 0.014332449994981289, + 0.0037386121693998575, + -0.01435891818255186, + 0.007258863188326359, + -0.010971006937325, + 0.013194323517382145, + -0.014901513233780861, + 0.0035698784049600363, + -0.016436660662293434, + 0.003374676452949643, + -0.01809091307222843, + 0.004377154167741537, + -0.000208849465707317, + -0.019599592313170433, + 0.013908960856497288, + 0.011215836741030216, + -0.020089251920580864, + 0.034302596002817154, + -0.01934814639389515, + -0.031708724796772, + -0.009773327969014645, + -0.009554966352880001, + 0.003394527593627572, + -0.01961282640695572, + -0.005234057083725929, + 0.005826279986649752, + -0.02036716602742672, + -0.012439983896911144, + -0.022722821682691574, + 0.008218330331146717, + -0.010196817107498646, + -0.010699709877371788, + 0.010646773502230644, + -0.02898251637816429, + 0.012029729783535004, + 0.007199310231953859, + -0.039728544652462006, + -0.007199310231953859, + 0.008330819196999073, + 0.0016079341294243932, + -0.004658377263695002, + -0.024694692343473434, + 0.031099960207939148, + -0.005783269181847572, + -0.025475500151515007, + 0.014305981807410717, + -0.03345561772584915, + 0.005670779850333929, + 0.01757478527724743, + -0.030358854681253433, + 0.018607040867209435, + 0.004691462032496929, + -0.005588067229837179, + -0.007550011854618788, + 0.009965221397578716, + -0.011235687881708145, + 0.013220791704952717, + -0.03250276669859886, + -0.01655576564371586, + -0.030808812007308006, + -0.006772512570023537, + -0.012598792091012001, + -0.012975961901247501, + -0.016436660662293434, + -0.009224115870893002, + 0.010679858736693859, + 0.02233903482556343, + -0.009958604350686073, + -0.020009847357869148, + -0.02417856454849243, + -0.03385263681411743, + 0.011990027502179146, + 0.01905699633061886, + -0.01799827441573143, + 0.02368890680372715, + -0.006666640285402536, + -0.015457342378795147, + -0.011440815404057503, + 0.00642511947080493, + 0.004370537120848894, + -0.0013837828300893307, + 0.0035764954518526793, + 0.0028883260674774647, + 0.0052903019823133945, + -0.010732795111835003, + 0.003748537739738822, + 0.030517663806676865, + 0.033905573189258575, + 0.004787408746778965, + -0.004565739072859287, + 0.010110795497894287, + -0.004837036598473787, + -0.0074375225231051445, + -0.0013746843906119466, + 0.018104147166013718, + 0.005558290984481573, + 0.017693892121315002, + 0.0018974284175783396, + 0.022166993468999863, + 0.007841160520911217, + 0.032026343047618866, + 0.0003720001841429621, + -0.006302704568952322, + -0.02720915712416172, + -0.05537116527557373, + -0.006709651090204716, + 0.0031447354704141617, + 0.0028122304938733578, + 0.014530960470438004, + -0.009713774546980858, + -0.0021968481596559286, + 0.01786593534052372, + -0.013399451039731503, + -0.021200908347964287, + 0.0005690631223842502, + 0.021611163392663002, + 0.013035515323281288, + 0.04565738886594772, + -0.03599654883146286, + 0.014583896845579147, + -0.007953649386763573, + -0.018104147166013718, + -0.0044995686039328575, + -0.013492089696228504, + 0.002600486157462001, + -0.00028287729946896434, + 0.022034652531147003, + -0.004033069126307964, + 0.014067769050598145, + 0.014319215901196003, + 0.007583096623420715, + 0.006626938469707966, + 0.00865505263209343, + 0.010024774819612503, + -0.014279513619840145, + -0.022299332544207573, + -0.030279450118541718, + -0.025118181481957436, + -0.01421334408223629, + 0.014094237238168716, + 0.003205942688509822, + -0.024919670075178146, + 0.022921333089470863, + -0.02089652605354786, + 0.0013085142709314823, + 0.02260371670126915, + -0.011334942653775215, + 0.022881630808115005, + -0.0006025617476552725, + 0.0510568730533123, + -0.003523559309542179, + -0.016410192474722862, + 0.008251414634287357, + 0.004138941410928965, + -0.010507816448807716, + -0.0068552251905202866, + 0.022656651213765144, + 0.004645143169909716, + -0.004919749218970537, + -0.006673257332295179, + 0.02566077560186386, + 0.007364735472947359, + -0.02789732627570629, + -0.008800626732409, + 0.009846115484833717, + 0.01329357922077179, + 0.00919764768332243, + -0.02105533517897129, + 0.008085989393293858, + -0.011090113781392574, + 0.02362273633480072, + 0.004691462032496929, + -0.0005132320802658796, + 0.01157977245748043, + -0.018276190385222435, + 0.0013589690206572413, + 0.03192047029733658, + -0.008416840806603432, + 0.01852763630449772, + -0.012876707129180431, + -0.029935365542769432, + -0.015669086948037148, + -0.012711281888186932, + -0.021703802049160004, + -0.004674919415265322, + -0.0062630027532577515, + 0.03642003983259201, + 0.0009627753752283752, + 0.0018461465369910002, + 0.015205895528197289, + -0.0014615326654165983, + -0.0010810544481500983, + -0.013505322858691216, + 0.006703034043312073, + 0.027659114450216293, + -0.007583096623420715, + 0.0034805487375706434, + 0.0031397726852446795, + -0.008039670996367931, + 0.019586358219385147, + -0.021796440705657005, + 0.0024830340407788754, + -0.006514449138194323, + -0.01181136816740036, + -0.013882492668926716, + 0.019229039549827576, + -0.0074507566168904305, + -0.014345684088766575, + 0.017124829813838005, + -0.0029710386879742146, + -0.0005082692950963974, + -0.03914624825119972, + -0.007100054994225502, + 0.009832881391048431, + -0.011685644276440144, + -0.026071030646562576, + -0.024363841861486435, + -0.0008626930648460984, + 0.0023937043733894825, + -0.008674903772771358, + -0.0371876135468483, + -0.01569555513560772, + 0.010243135504424572, + -0.031470514833927155, + 0.02059214375913143, + 0.00004500085560721345, + 0.017958572134375572, + -0.021333249285817146, + 0.00919764768332243, + -0.0006501215393655002, + -0.01562938466668129, + 0.025157883763313293, + 0.010951156727969646, + -0.03491136059165001, + -0.015417640097439289, + 0.005095100030303001, + 0.0034739316906780005, + 0.02319924719631672, + -0.014226577244699001, + -0.022299332544207573, + 0.023609502241015434, + 0.01862027496099472, + 0.0018411838682368398, + -0.008516095578670502, + 0.003245644737035036, + -0.013035515323281288, + 0.0021273696329444647, + 0.010534284636378288, + -0.034540805965662, + 0.004119090735912323, + -0.016463128849864006, + -0.025806350633502007, + -0.009733625687658787, + -0.030358854681253433, + -0.0068684592843055725, + -0.010309305973351002, + 0.03327034041285515, + 0.007258863188326359, + -0.013114919885993004, + 0.009052073583006859, + -0.005879215896129608, + -0.011301858350634575, + 0.019520187750458717, + -0.0032820384949445724, + 0.00023759211762808263, + -0.010679858736693859, + 0.016727808862924576, + 0.03504370152950287, + 0.014279513619840145, + -0.022775758057832718, + -0.024602053686976433, + -0.024324139580130577, + -0.005022312980145216, + -0.020115718245506287, + 0.014583896845579147, + 0.0023242258466780186, + 0.027553241699934006, + 0.0035996551159769297, + -0.04375169053673744, + -0.014994150958955288, + 0.014610364101827145, + -0.022908098995685577, + -0.028585495427250862, + 0.006683182902634144, + 0.014385386370122433, + 0.008562414906919003, + 0.023715374991297722, + 0.0015450725331902504, + 0.01921580545604229, + -0.00023200901341624558, + 0.0017535084625706077, + -0.015245597809553146, + 0.02718268893659115, + -0.03006770648062229, + -0.04261356219649315, + 0.0012952802935615182, + -0.007913947105407715, + -0.02016865462064743, + -0.009607902728021145, + 0.01625138334929943, + -0.003731995355337858, + 0.0004545060801319778, + 0.01957312412559986, + -0.01468976866453886, + 0.0017402743687853217, + -0.0008258859161287546, + 0.02882370725274086, + 0.03583774343132973, + 0.015854362398386, + 0.0024334064219146967, + -0.028479622676968575, + 0.0027940336149185896, + -0.004939599893987179, + 0.008899882435798645, + -0.03366736322641373, + 0.017601253464818, + 0.007861011661589146, + 0.002329188631847501, + -0.015933766961097717, + -0.007093437947332859, + 0.007126522716134787, + -0.00860211718827486, + 0.009124861098825932, + 0.022616950795054436, + 0.010110795497894287, + -0.024800565093755722, + 0.020049549639225006, + 0.012102516368031502, + 0.0019503645598888397, + -0.011407730169594288, + -0.023119842633605003, + 0.010382093489170074, + -0.020539207383990288, + 0.013419302180409431, + -0.026163669303059578, + 0.025885755196213722, + -0.03557306155562401, + 0.0066037788055837154, + -0.01181136816740036, + -0.019109932705760002, + -0.011116581968963146, + -0.011851070448756218, + -0.01447802409529686, + 0.02300073765218258, + 0.01115628331899643, + -0.013346514664590359, + 0.014279513619840145, + -0.03395850956439972, + -0.014279513619840145, + -0.011282007209956646, + 0.004674919415265322, + -0.011070262640714645, + -0.02286839671432972, + 0.0014052881160750985, + -0.009105009958148003, + -0.013002430088818073, + -0.01393542904406786, + 0.0022994119208306074, + -0.020406868308782578, + 0.022709587588906288, + 0.010600454173982143, + 0.21873198449611664, + 0.018911423161625862, + -0.014504492282867432, + 0.042719434946775436, + 0.01435891818255186, + 0.004827111028134823, + 0.029300132766366005, + 0.0004511975566856563, + -0.008747691288590431, + 0.02977655827999115, + 0.011798134073615074, + 0.016264617443084717, + -0.02497260645031929, + 0.012433366850018501, + 0.0011579772690311074, + -0.013657514937222004, + -0.01332666352391243, + -0.009495413862168789, + -0.02922072820365429, + -0.023172779008746147, + 0.007986734621226788, + -0.01625138334929943, + -0.006537608802318573, + -0.022855162620544434, + 0.029591280966997147, + 0.019851038232445717, + 0.012870090082287788, + -0.003679059213027358, + -0.0006195178139023483, + 0.011182751506567001, + -0.024813799187541008, + 0.001568232080899179, + 0.01258555892854929, + 0.005058706272393465, + -0.019652528688311577, + -0.0051314933225512505, + -0.013253876939415932, + -0.013313430361449718, + 0.010481348261237144, + 0.021280312910676003, + 0.0018841944402083755, + 0.019652528688311577, + -0.0052903019823133945, + 0.007788224145770073, + 0.019864272326231003, + 0.01819678582251072, + -0.021703802049160004, + -0.007291948422789574, + 0.011738580651581287, + 0.005330003798007965, + -0.027394432574510574, + 0.010838666930794716, + 0.012168686836957932, + 0.03329680860042572, + -0.002430097898468375, + 0.01207604818046093, + 0.019454017281532288, + -0.0028750922065228224, + -0.006239843089133501, + -0.013697216287255287, + -0.02293456718325615, + 0.026891540735960007, + 0.0013457350432872772, + 0.01846146583557129, + 0.012248090468347073, + 0.030517663806676865, + -0.02718268893659115, + -0.03631416708230972, + -0.0065409173257648945, + 0.007325033191591501, + 0.02674596570432186, + 0.0019652529153972864, + -0.013240642845630646, + -0.0008949509938247502, + 0.003151352284476161, + -0.01535146962851286, + 0.018607040867209435, + 0.01589406467974186, + 0.01523236371576786, + 0.01737627573311329, + -0.016820447519421577, + 0.003904037643224001, + -0.006755969952791929, + 0.023384522646665573, + 0.006097577512264252, + -0.02484026551246643, + -0.013591344468295574, + -0.00997183844447136, + 0.0009396158275194466, + 0.012552473694086075, + 0.013313430361449718, + 0.008846946060657501, + -0.009091775864362717, + 0.007192693185061216, + 0.015669086948037148, + 0.0004962759558111429, + 0.01596023514866829, + 0.007205926813185215, + -0.022855162620544434, + -0.013386216945946217, + -0.015311767347157001, + 0.01110334787517786, + 0.029882431030273438, + 0.04017850384116173, + -0.007172842044383287, + 0.010620305314660072, + -0.011857687495648861, + -0.009581434540450573, + 0.005191046744585037, + -0.023781543597579002, + -0.0014648411888629198, + -0.01421334408223629, + -0.005157961510121822, + -0.019731933251023293, + -0.015338235534727573, + 0.00039578007999807596, + -0.0022530928254127502, + -0.026494519785046577, + 0.009071924723684788, + -0.0018180243205279112, + -0.0018494550604373217, + -0.026415115222334862, + 0.0005885005812160671, + 0.01717776618897915, + 0.013492089696228504, + -0.013710450381040573, + -0.030385322868824005, + 0.016661638393998146, + 0.0007212543860077858, + -0.04346054047346115, + 0.013518556952476501, + -0.009515264071524143, + 0.02457558549940586, + -0.014345684088766575, + -0.012135601602494717, + 0.004807259887456894, + 0.01770712621510029, + -0.013220791704952717, + 0.006557459942996502, + -0.0002204292395617813, + -0.013518556952476501, + -0.010653390549123287, + 0.02280222624540329, + 0.004962759558111429, + 0.009105009958148003, + -0.03329680860042572, + -0.02464175596833229, + -0.012466452084481716, + -0.02306690625846386, + -0.02098916471004486, + -0.0017634339164942503, + -0.026401881128549576, + -0.008939584717154503, + -0.018712911754846573, + 0.0027592943515628576, + 0.0013399451272562146, + -0.010084327310323715, + 0.009151328355073929, + 0.01245321799069643, + 0.03030591830611229, + -0.018607040867209435, + 0.019586358219385147, + 0.05807090550661087, + -0.01572202332317829, + -0.005684013944119215, + -0.0054027908481657505, + -0.16833680868148804, + 0.041501905769109726, + 0.014663300476968288, + -0.022841928526759148, + -0.005048780702054501, + 0.003020666306838393, + 0.006190215703099966, + -0.009799796156585217, + -0.017654189839959145, + 0.013657514937222004, + 0.010746029205620289, + 0.0064714387990534306, + -0.03597008064389229, + -0.008522712625563145, + -0.010732795111835003, + 0.003278729971498251, + -0.007470607757568359, + 0.00561122689396143, + 0.014160407707095146, + 0.001894119894132018, + 0.020671548321843147, + -0.02648128569126129, + 0.021280312910676003, + -0.0006691454327665269, + -0.004516111221164465, + -0.013280345126986504, + -0.015219129621982574, + 0.015616150572896004, + -0.0029164485167711973, + -0.009508647955954075, + -0.013134771026670933, + -0.002851932542398572, + 0.00010080605716211721, + 0.015047087334096432, + -0.005561599507927895, + 0.008972669020295143, + -0.016621936112642288, + -0.018911423161625862, + -0.012526005506515503, + 0.006315938662737608, + 0.014861810952425003, + 0.010646773502230644, + 0.02266988530755043, + 0.004324217792600393, + -0.010342391207814217, + 0.04168717935681343, + 0.04343407228589058, + 0.007113288622349501, + 0.02685183845460415, + -0.02056567557156086, + -0.005144727416336536, + -0.002031422918662429, + 0.015192661434412003, + -0.0022712897043675184, + 0.012069431133568287, + 0.027712050825357437, + 0.01072617806494236, + 0.028320815414190292, + 0.004899898078292608, + -0.014769172295928001, + -0.001813061535358429, + -0.011255539022386074, + 0.001765088178217411, + 0.006392034236341715, + 0.02023482508957386, + 0.006679874379187822, + 0.00832420215010643, + 0.004582281224429607, + 0.011731963604688644, + 0.009025605395436287, + 0.014345684088766575, + -0.022788992151618004, + 0.012856855988502502, + -0.013009047135710716, + 0.01885848678648472, + 0.0074441395699977875, + -0.011235687881708145, + 0.008688137866556644, + -0.014345684088766575, + -0.0009172834106720984, + -0.020512739196419716, + 0.013359748758375645, + -0.019983379170298576, + -0.0054855034686625, + -0.003305197926238179, + 0.02398005500435829, + -0.01535146962851286, + 0.003318432020023465, + -0.005419333465397358, + -0.005538439843803644, + 0.017561553046107292, + -0.035811275243759155, + -0.017389509826898575, + -0.02517111785709858, + -0.006676565855741501, + -0.005942077375948429, + -0.0035268678329885006, + 0.02076418697834015, + 0.0006819659029133618, + -0.014875045046210289, + 0.02030099555850029, + -0.011672411113977432, + 0.0013465621741488576, + -0.0037187612615525723, + 0.04385755956172943, + 0.01970546506345272, + -0.013803089037537575, + 0.017323339357972145, + 0.04213713854551315, + 0.034196723252534866, + -0.02379477769136429, + 0.03422319144010544, + 0.0415283739566803, + 0.001535146962851286, + -0.030835280194878578, + 0.011996644549071789, + 0.009535115212202072, + -0.02253754623234272, + 0.00016025577497202903, + 0.004344068933278322, + 0.03663178160786629, + -0.003272112924605608, + -0.0056310780346393585, + -0.0014276205329224467, + -0.007748521864414215, + -0.021902313455939293, + -0.10444293171167374, + 0.007861011661589146, + -0.008707989007234573, + -0.011182751506567001, + 0.010944539681077003, + 0.02102886699140072, + -0.006203449796885252, + -0.014980916865170002, + 0.0035731869284063578, + 0.026891540735960007, + -0.005561599507927895, + -0.017217466607689857, + -0.0008002449758350849, + -0.019136400893330574, + 0.01914963498711586, + -0.0007324205944314599, + 0.012883324176073074, + -0.011420964263379574, + -0.015444108285009861, + 0.012711281888186932, + -0.002456566086038947, + -0.01629108563065529, + -0.011526836082339287, + -0.017217466607689857, + 0.01611904427409172, + -0.014173641800880432, + -0.026137201115489006, + 0.0017435828922316432, + 0.007113288622349501, + 0.002567400922998786, + 0.012360580265522003, + -0.016304319724440575, + -0.004135632887482643, + -0.01937461458146572, + -0.02678566798567772, + -0.016966020688414574, + -0.016436660662293434, + -0.004092622548341751, + 0.005581450182944536, + -0.018051210790872574, + 0.0062431516125798225, + 0.020486271008849144, + 0.020446570590138435, + -0.03332327678799629, + 0.021637631580233574, + -0.0037783144507557154, + 0.004651760216802359, + 0.003000815398991108, + 0.01914963498711586, + 0.0014648411888629198, + -0.014438321813941002, + -0.01115628331899643, + -0.01678074523806572, + -0.01711159572005272, + 0.015470576472580433, + -0.0034110702108591795, + 0.01569555513560772, + 0.005300227086991072, + -0.02043333649635315, + 0.010673241689801216, + -0.002790725091472268, + 0.017349807545542717, + -0.020049549639225006, + 0.02415209636092186, + -0.0071529909037053585, + -0.008145542815327644, + 0.005862673278898001, + -0.005184429697692394, + 0.02619013749063015, + -0.04547211155295372, + -0.041634246706962585, + -0.0004208006721455604, + -0.018434997648000717, + 0.02490643598139286, + -0.041369564831256866, + 0.006405268330127001, + -0.029670685529708862, + -0.002838698448613286, + 0.016039639711380005, + -0.006815523374825716, + -0.009839498437941074, + -0.010540901683270931, + 0.0033498627599328756, + -0.027817921712994576, + 0.008992520160973072, + 0.04348700866103172, + -0.005750184413045645, + 0.001955327345058322, + 0.0018560721073299646, + -0.021902313455939293, + -0.0027890708297491074, + 0.01872614584863186, + 0.021306781098246574, + -0.01426627952605486, + 0.002851932542398572, + 0.025224052369594574, + 0.013366365805268288, + -0.004913132172077894, + 0.02247137576341629, + 0.013432536274194717, + -0.01462359819561243, + 0.0004383771156426519, + -0.052194997668266296, + 0.032688044011592865, + -0.004585589747875929, + -0.015060321427881718, + 0.01572202332317829, + -0.00045037042582407594, + 0.02174350433051586, + -0.02898251637816429, + -0.01832912489771843, + -0.008138925768435001, + -0.0026666561607271433, + 0.017429212108254433, + -0.009224115870893002, + -0.013068600557744503, + -0.03504370152950287, + 0.016185212880373, + 0.014464790001511574, + -0.011070262640714645, + 0.014464790001511574, + 0.010878369212150574, + -0.020618611946702003, + 0.0003798578982241452, + 0.022219929844141006, + -0.007861011661589146, + -0.028109069913625717, + -0.008039670996367931, + -0.006815523374825716, + -0.010157114826142788, + -0.02326541766524315, + 0.00040053605334833264, + 0.024165330454707146, + -0.021333249285817146, + 0.008066138252615929, + 0.07749845832586288, + -0.005760109517723322, + -0.03409085050225258, + 0.002539278706535697, + 0.03046472743153572, + 0.02333158813416958, + -0.022484609857201576, + -0.012942877598106861, + -0.018964359536767006, + -0.005455727223306894, + -0.035228975117206573, + -0.016873382031917572, + 0.02922072820365429, + -0.02355656586587429, + 0.00033105743932537735, + 0.024813799187541008, + 0.00412901584059, + 0.00717945909127593, + 0.02039363421499729, + 0.010382093489170074, + -0.01400159951299429, + 0.013028898276388645, + -0.023583034053444862, + 0.0017948647728189826, + 0.017482148483395576, + 0.019255507737398148, + -0.007768373005092144, + 0.0015343198319897056, + 0.01700572296977043, + 0.005250599700957537, + -0.021280312910676003, + 0.020711250603199005, + -0.007854394614696503, + -0.00907854177057743, + 0.0051314933225512505, + -0.004327526316046715, + -0.025488734245300293, + -0.01549704372882843, + 0.011546687223017216, + 0.009647605009377003, + 0.004430090077221394, + 0.02592545561492443, + -0.01865997724235058, + 0.028373751789331436, + -0.02289486490190029, + -0.0005934633663855493, + 0.03684352710843086, + -0.02924719639122486, + -0.004909823648631573, + -0.03475255146622658, + 0.020062783733010292, + 0.04428105056285858, + 0.004774174652993679, + -0.018540870398283005, + -0.0002136054536094889, + -0.00023903959663584828, + 0.012089282274246216, + 0.016410192474722862, + -0.0011257192818447948, + -0.014822108671069145, + 0.009502030909061432, + -0.01750861667096615, + -0.014676534570753574, + -0.009601285681128502, + -0.018990827724337578, + -0.02579311653971672, + 0.03925212100148201, + -0.0030388631857931614, + -0.003904037643224001, + 0.009389541111886501, + -0.04668964445590973, + -0.007999968715012074, + -0.0042282710783183575, + -0.010249752551317215, + -0.012215006165206432, + -0.01093792263418436, + 0.010236518457531929, + 0.010540901683270931, + -0.004684844985604286, + -0.0028254645876586437, + 0.0071463738568127155, + 0.003768388880416751, + -0.013339897617697716, + -0.02098916471004486, + -0.02398005500435829, + -0.011877537705004215, + 0.02346392720937729, + 0.013260493986308575, + 0.028903111815452576, + 0.025263754650950432, + 0.016820447519421577, + -0.010124029591679573, + -0.016595467925071716, + 0.01136802788823843, + -0.009343221783638, + 0.018633509054780006, + 0.017415978014469147, + 0.015205895528197289, + -0.018898189067840576, + -0.023993289098143578, + -0.001014884328469634, + -0.022762523964047432, + -0.0063060130923986435, + -0.005253908224403858, + 0.01629108563065529, + -0.010554135777056217, + 0.06299396604299545, + 0.02734149806201458, + -0.019070230424404144, + 0.018302656710147858, + -0.009839498437941074, + 0.01855410449206829, + 0.01700572296977043, + -0.0015103331534191966, + -0.009382924064993858, + -0.01066000759601593, + 0.00022849372180644423, + -0.00116790272295475, + 0.028850175440311432, + -0.006408576853573322, + -0.02789732627570629, + 0.01393542904406786, + -0.026468051597476006, + -0.002397012896835804, + -0.018342358991503716, + -0.013975131325423717, + 0.004026452545076609, + 0.0065409173257648945, + 0.013736918568611145, + 0.00021009016199968755, + -0.006292779464274645, + -0.025846052914857864, + 0.008906499482691288, + 0.0015591336414217949, + -0.008304351009428501, + -0.0012588866520673037, + 0.02869136817753315, + 0.014650066383183002, + -0.03316446766257286, + -0.01691308431327343, + 0.010468114167451859, + -0.02539609558880329, + 0.008780776523053646, + -0.014517726376652718, + 0.003695601597428322, + 0.02244490757584572, + -0.025885755196213722, + -0.0031447354704141617, + -0.004803951364010572, + -0.034699615091085434, + -0.005455727223306894, + 0.00227459822781384, + 0.0015450725331902504, + 0.010461497120559216, + 0.0034673146437853575 + ] + } + ], + "query": { + "match_none": {} + }, + "sort": ["-_score"] + } \ No newline at end of file diff --git a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc index a5a94b97f..60875b3fd 100644 --- a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc +++ b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc @@ -20,11 +20,11 @@ For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[] * You have documents in a keyspace inside your bucket that contain vector embeddings. + -You can https://cbc-remote-execution-examples-prod.s3.amazonaws.com/color_data_2vectors.zip[download a sample dataset] to use with the procedure on this page. +include::partial$download-sample-partial.adoc[] -* You have the hostname for your database. +* You have the hostname for the node in your database that's running the Search Service. + -For more information about how to find the hostname for your Capella database, see xref:clouds:get-capella-host-name.adoc[]. +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. == Procedure diff --git a/modules/vector-search/pages/create-vector-search-index-ui.adoc b/modules/vector-search/pages/create-vector-search-index-ui.adoc index 0733f5a27..af3e955dc 100644 --- a/modules/vector-search/pages/create-vector-search-index-ui.adoc +++ b/modules/vector-search/pages/create-vector-search-index-ui.adoc @@ -25,7 +25,7 @@ For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[] * You have documents in a keyspace inside your bucket that contain vector embeddings. + -You can https://cbc-remote-execution-examples-prod.s3.amazonaws.com/color_data_2vectors.zip[download a sample dataset] to use with the procedure on this page. +include::partial$download-sample-partial.adoc[] * You have logged in to the Couchbase Capella UI. diff --git a/modules/vector-search/pages/run-vector-search-rest-api.adoc b/modules/vector-search/pages/run-vector-search-rest-api.adoc index 393d9f6fd..9756e0581 100644 --- a/modules/vector-search/pages/run-vector-search-rest-api.adoc +++ b/modules/vector-search/pages/run-vector-search-rest-api.adoc @@ -10,9 +10,9 @@ * You have the Search Service enabled on a node in your database. For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have the hostname for your database. +* You have the hostname for the node in your database that's running the Search Service. + -For more information about how to find the hostname for your Capella database, see xref:clouds:get-capella-host-name.adoc[]. +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. * You have created a Vector Search index. + diff --git a/modules/vector-search/pages/run-vector-search-ui.adoc b/modules/vector-search/pages/run-vector-search-ui.adoc index 0e5994144..19f41330f 100644 --- a/modules/vector-search/pages/run-vector-search-ui.adoc +++ b/modules/vector-search/pages/run-vector-search-ui.adoc @@ -26,7 +26,7 @@ To run a Vector Search with the Capella UI: . Press kbd:[Enter] or click btn:[Search]. . . (Optional) To view a document and its source collection, click a document name in the search results list. -=== Example +=== Example: Running a Simple Vector Similarity Query For example, the following query searches for the top 2 vectors similar to the vector `[ 0, 0, 128 ]` in the `colorvect_l2` field: @@ -41,8 +41,46 @@ It only returns the `k` number of similar vectors. The Search Service combines the Vector search results from a `knn` object with the traditional `query` object by using an `OR` function. If the same documents match the `knn` and `query` objects, the Search Service ranks those documents higher in search results. +=== Example: Running a Semantic Search Query with a Large Embedding Vector + +TIP: This example uses the `rgb_questions.json` sample data, which provides ready-made vector embeddings and queries that you can use for searches. +You can find a link to download this sample dataset from xref:create-vector-search-index-ui.adoc[] or xref:create-vector-search-index-rest-api.adoc[]. + +The following query searches for matches to a large embedding vector, generated by the https://platform.openai.com/docs/guides/embeddings[OpenAI embedding model^], `text-embedding-ada-002-v2`. + +The embedding vector for the query was generated from the text in the `question` field in `rgb_questions.json`. +You can also find the full contents of the query in the `couchbase_search_query` field in the same file. +You can find generated embedding vectors for each color's `question` field in `rgb_questions.json`, and generated embedding vectors for each `description` in `rgb.json`. + +This query should return the document for the color `navy`, based on a generated embedding vector for: + +---- +What is a classic, refined hue that exudes elegance and is often linked to power and stability? +---- + +The following shows part of the sample Search query: + +[source,json] +---- +include::example$run-vector-search-long-payload-ui.jsonc[tag=partial] +---- +[NOTE] +==== +Due to the size of the embedding vector, only part of the full query is being displayed in the documentation. + +Click btn:[View on GitHub] to view and copy the entire Vector Search query payload. +Make sure you remove the lines for `// tag::partial[]` and `// end::partial[]`. + +You can also copy the query by going to menu:Data Tools[Documents] and copying the full contents from the `couchbase_search_query` field. + +If you configured your data import to use the `id` field as the document identifier, you can locate the document for `navy` with the ID `#000080`. +==== + == Next Steps If you do not get the search results you were expecting, you can change the xref:search:search-request-params.adoc[JSON payload for your Search query]. -You can also xref:search:search:customize-index.adoc[add additional features to your Search index]. \ No newline at end of file +You can also xref:search:search:customize-index.adoc[add additional features to your Search index]. + +Either of the example queries on this page can also be used with the xref:run-vector-search-rest-api.adoc[REST API] to run a search. +You can find more example embedding vectors and full queries inside the `couchbase_search_query` field for any color in the `rgb_questions.json` sample file. \ No newline at end of file diff --git a/modules/vector-search/partials/download-sample-partial.adoc b/modules/vector-search/partials/download-sample-partial.adoc new file mode 100644 index 000000000..a33206ea2 --- /dev/null +++ b/modules/vector-search/partials/download-sample-partial.adoc @@ -0,0 +1,15 @@ +[TIP] +==== +You can download a sample dataset to use with the procedure on this page: + +https://cbc-remote-execution-examples-prod.s3.amazonaws.com/color_data_2vectors.zip[Download color_data_2vectors.zip] + +To get the best results with using the sample data with the examples in this documentation, xref:clusters:data-service/import-data-documents.adoc[import the sample files] from the dataset into your database with the following settings: + +* Use a bucket called `vector-sample`. +* Use a scope called `color`. +* Use a collection called `rgb` for `rgb.json`. +* Use a collection called `rgb_questions` for `rgb_questions.json`. +* To set your document keys, use the value of the `id` field from each JSON document. + +==== \ No newline at end of file From 228be7e52d6a72a8f6f5891dc470b465217d6cbe Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Tue, 12 Mar 2024 13:10:49 -0400 Subject: [PATCH 40/59] [AV-68571] More SME review updates - adding procedure for running a vector search with Python SDK --- .../run-vector-search-generate-embed.py | 40 +++++++++ .../run-vector-search-simple-color.py | 34 ++++++++ .../pages/run-vector-search-python-sdk.adoc | 81 +++++++++++++++++++ .../pages/run-vector-search-rest-api.adoc | 2 + .../pages/run-vector-search-ui.adoc | 2 + modules/vector-search/partials/nav.adoc | 1 + 6 files changed, 160 insertions(+) create mode 100644 modules/vector-search/examples/run-vector-search-generate-embed.py create mode 100644 modules/vector-search/examples/run-vector-search-simple-color.py create mode 100644 modules/vector-search/pages/run-vector-search-python-sdk.adoc diff --git a/modules/vector-search/examples/run-vector-search-generate-embed.py b/modules/vector-search/examples/run-vector-search-generate-embed.py new file mode 100644 index 000000000..df07d93be --- /dev/null +++ b/modules/vector-search/examples/run-vector-search-generate-embed.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python3 + +import os +import sys +from couchbase.cluster import Cluster +from couchbase.options import ClusterOptions +from couchbase.auth import PasswordAuthenticator +from couchbase.exceptions import CouchbaseException +import couchbase.search as search +from couchbase.options import SearchOptions +from couchbase.vector_search import VectorQuery, VectorSearch +from openai import OpenAI + +# Change the question as desired +question = "What color hides everything like the night?" + +# Make sure to replace OPENAI_API_KEY with your own API Key +openai_api_key = os.getenv("OPENAI_API_KEY") +client = OpenAI() + +# Make sure to change CB_USERNAME, CB_PASSWORD, and CB_HOSTNAME to the username, password, and hostname for your database. +pa = PasswordAuthenticator(os.getenv("CB_USERNAME"), os.getenv("CB_PASSWORD")) +cluster = Cluster("couchbases://" + os.getenv("CB_HOSTNAME") + "/?ssl=no_verify", ClusterOptions(pa)) +# Make sure to change the bucket, scope, and index names to match where you stored the sample data in your database. +bucket = cluster.bucket("vector-sample") +scope = bucket.scope("color") +search_index = "color-index" +try: + vector = client.embeddings.create(input = [question], model="text-embedding-ada-002").data[0].embedding + search_req = search.SearchRequest.create(search.MatchNoneQuery()).with_vector_search( + VectorSearch.from_vector_query(VectorQuery('embedding_vector_dot', vector, num_candidates=2))) + # Change the limit value to return more results. Change the fields array to return different fields from your Search index. + result = scope.search(search_index, search_req, SearchOptions(limit=13,fields=["color", "description"])) + for row in result.rows(): + print("Found row: {}".format(row)) + print("Reported total rows: {}".format( + result.metadata().metrics().total_rows())) +except CouchbaseException as ex: + import traceback + traceback.print_exc() \ No newline at end of file diff --git a/modules/vector-search/examples/run-vector-search-simple-color.py b/modules/vector-search/examples/run-vector-search-simple-color.py new file mode 100644 index 000000000..ee0869c13 --- /dev/null +++ b/modules/vector-search/examples/run-vector-search-simple-color.py @@ -0,0 +1,34 @@ +#!/usr/bin/env python3 + +import os +import sys +from couchbase.cluster import Cluster +from couchbase.options import ClusterOptions +from couchbase.auth import PasswordAuthenticator +from couchbase.exceptions import CouchbaseException +import couchbase.search as search +from couchbase.options import SearchOptions +from couchbase.vector_search import VectorQuery, VectorSearch + +# You can change the RGB values to search for a different color +vector = [0.0,0.0,128.0] + +# Make sure to change CB_USERNAME, CB_PASSWORD, and CB_HOSTNAME to the username, password, and hostname for your database. +pa = PasswordAuthenticator(os.getenv("CB_USERNAME"), os.getenv("CB_PASSWORD")) +cluster = Cluster("couchbases://" + os.getenv("CB_HOSTNAME") + "/?ssl=no_verify", ClusterOptions(pa)) +# Make sure to change the bucket, scope, and index names to match where you stored the sample data in your database. +bucket = cluster.bucket("vector-sample") +scope = bucket.scope("color") +search_index = "color-index" +try: + search_req = search.SearchRequest.create(search.MatchNoneQuery()).with_vector_search( + VectorSearch.from_vector_query(VectorQuery('colorvect_l2', vector, num_candidates=3))) + # Change the limit value to return more results. Change the fields array to return different fields from your Search index. + result = scope.search(search_index, search_req, SearchOptions(limit=13,fields=["color", "id"])) + for row in result.rows(): + print("Found row: {}".format(row)) + print("Reported total rows: {}".format( + result.metadata().metrics().total_rows())) +except CouchbaseException as ex: + import traceback + traceback.print_exc() \ No newline at end of file diff --git a/modules/vector-search/pages/run-vector-search-python-sdk.adoc b/modules/vector-search/pages/run-vector-search-python-sdk.adoc new file mode 100644 index 000000000..f89341b32 --- /dev/null +++ b/modules/vector-search/pages/run-vector-search-python-sdk.adoc @@ -0,0 +1,81 @@ += Run a Vector Search with the Python SDK +:page-topic-type: guide +:description: Using Couchbase's Python SDK, you can run a simple or more complex vector search against a Vector Search index. + +[abstract] +{description} + + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. + +* You have created a Vector Search index. ++ +For more information about how to create a Vector Search index, see xref:create-vector-search-index-ui.adoc[] or xref:create-vector-search-index-rest-api.adoc[]. ++ +include::partial$download-sample-partial.adoc[] + +* Create and activate a virtual environment using `venv` and install packages. ++ +For more information about how to set up your virtual environment, see https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/[the Python Packaging User Guide]. ++ +Make sure to install all dependencies for the Couchbase Python SDK and make a `requirements.txt` file with the following: ++ +---- +couchbase==4.1.12 +openai==1.12.0 +---- + +== Example: Searching for a Similar Color Vector with a Python Program + +The sample dataset inside `rgb.json` has small embedding vectors inside the `colorvect_l2` field. +These embedding vectors describe a color using RGB values. +For example, the color red has an embedding vector of `[255, 0, 0]`. + +The following example code searches for the color navy (`[0.0, 0.0, 128.0]`) in the `rgb.json` dataset: + +[source,python] +---- +include::example$run-vector-search-simple-color.py[] +---- + +== Example: Semantic Search with Color Descriptions + +[NOTE] +==== +The following code sample requires you to have a paid subscription to the OpenAI API to generate an embedding vector from a sample text string. +For more information about pricing for the OpenAI API, see https://openai.com/pricing[OpenAI's Pricing page^] for embedding models. + +The `rgb_questions.json` and sample data contains ready-made embedding vectors for different questions. +For an example of how to use a ready-made vector with Vector Search, see xref:run-vector-search-rest-api.adoc[] or xref:run-vector-search-ui.adoc[]. +==== + +Still using the sample dataset inside `rgb.json`, you can use the OpenAI API to generate an embedding from any text string. + +The following code generates an embedding vector with the question `What color hides everything like the night?`: + +[source,python] +---- +include::example$run-vector-search-generate-embed.py[] +---- + +You can try other questions from `rgb_questions.json`, such as: + +* `"What is the most romantic color?"` + +* `"What color makes people fall in love?"` + +* `"What is the most depressing color?"` + +== Next Steps + +If you did not get the search results you were expecting, you can try to recreate your Vector Search index xref:create-vector-search-index-rest-api.adoc[with the REST API]. + +Vector Search indexes can use the same settings and features as regular Search indexes. +If you want to add additional fields and features to your index, see xref:search:customize-index.adoc[]. \ No newline at end of file diff --git a/modules/vector-search/pages/run-vector-search-rest-api.adoc b/modules/vector-search/pages/run-vector-search-rest-api.adoc index 9756e0581..4a411b5b0 100644 --- a/modules/vector-search/pages/run-vector-search-rest-api.adoc +++ b/modules/vector-search/pages/run-vector-search-rest-api.adoc @@ -17,6 +17,8 @@ For more information about how to find the hostname for a node in your Capella d * You have created a Vector Search index. + For more information about how to create a Vector Search index, see xref:create-vector-search-index-ui.adoc[] or xref:create-vector-search-index-rest-api.adoc[]. ++ +include::partial$download-sample-partial.adoc[] == Procedure diff --git a/modules/vector-search/pages/run-vector-search-ui.adoc b/modules/vector-search/pages/run-vector-search-ui.adoc index 19f41330f..bc43744b6 100644 --- a/modules/vector-search/pages/run-vector-search-ui.adoc +++ b/modules/vector-search/pages/run-vector-search-ui.adoc @@ -12,6 +12,8 @@ * You have created a Vector Search index. + For more information about how to create a Vector Search index, see xref:create-vector-search-index-ui.adoc[]. ++ +include::partial$download-sample-partial.adoc[] * You have logged in to the Couchbase Capella UI. diff --git a/modules/vector-search/partials/nav.adoc b/modules/vector-search/partials/nav.adoc index 6fcd98a68..f720d81fd 100644 --- a/modules/vector-search/partials/nav.adoc +++ b/modules/vector-search/partials/nav.adoc @@ -3,3 +3,4 @@ ** xref:cloud:vector-search:create-vector-search-index-rest-api.adoc[] ** xref:cloud:vector-search:run-vector-search-ui.adoc[] ** xref:cloud:vector-search:run-vector-search-rest-api.adoc[] +** xref:cloud:vector-search:run-vector-search-python-sdk.adoc[] \ No newline at end of file From b05de77bab857a22e572eaa0b9eb96f825243b3e Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Tue, 12 Mar 2024 14:24:56 -0400 Subject: [PATCH 41/59] [AV-68571] Copying some changes over for consistency. --- .../n1ql/pages/n1ql-language-reference/searchfun.adoc | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc index 87c6b4e40..e1070b2f1 100644 --- a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc @@ -51,6 +51,10 @@ The Search Service then creates a temporary index in memory to perform the searc This process may be slower than using a suitable Search index. -- +NOTE: From Couchbase Server 7.6 and later, when you use the SEARCH function, you do not need to use the xref:search:default-analyzers-reference.adoc#keyword[keyword analyzer] in your Search index to run a non-analytic query. +You also do not need to match the analyzer in a query to the analyzer in the Search index for an analytic query. +For more information about how to set the analyzer for a Search index, see xref:search:set-advanced-settings.adoc[] or the xref:search:search-index-params.adoc#mapping[Mapping Object]. + === Arguments identifier:: @@ -141,7 +145,7 @@ For more information, refer to xref:search:customize-index.adoc[]. | `indexUUID` [Optional] | string -| A string, containing the UUID of a Search index in the keyspace. +| A string, containing the xref:search:search-index-params.adoc#uuid[UUID] of a Search index in the keyspace. This provides an index hint to the Query engine. If the Search index cannot be identified, an error occurs. @@ -368,8 +372,8 @@ WHERE t1.type = "hotel" AND SEARCH(t1.description, "amazing"); If the Full Text Search index being queried has its default mapping disabled and has a custom type mapping defined, the query needs to specify the type explicitly. The above query uses the demonstration index xref:server:fts:fts-demonstration-indexes.adoc#travel-sample-index-hotel-description[travel-sample-index-hotel-description], which has the custom type mapping "hotel". -For more information on defining custom type mappings within the Full Text Search index, refer to xref:search:create-type-mapping.adoc[]. -Note that for {sqlpp} queries, only Full Text Search indexes with one type mapping are searchable. +For more information about defining custom type mappings within a Search index, refer to xref:search:create-type-mapping.adoc[Create a Type Mapping]. +Note that for {sqlpp} queries, only Search indexes with one type mapping are searchable. Also the supported type identifiers at the moment are "type_field" and "docid_prefix"; "docid_regexp" isn't supported yet for SEARCH queries via {sqlpp}. ==== From 796cae397b0187ed8d335062edfa211bfb70369c Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Tue, 12 Mar 2024 15:33:23 -0400 Subject: [PATCH 42/59] [AV-68571] Quick consistency edit. --- modules/search/pages/create-child-field.adoc | 3 +++ modules/search/pages/create-child-mapping.adoc | 3 +++ modules/search/pages/create-search-index-alias.adoc | 7 +++++-- modules/search/pages/create-type-mapping.adoc | 3 +++ modules/search/pages/geo-search-rest-api.adoc | 10 +++++++--- .../search/pages/search-query-auto-complete-code.adoc | 7 ++++++- modules/search/pages/simple-search-rest-api.adoc | 7 +++---- modules/search/pages/simple-search-ui.adoc | 3 ++- .../pages/run-vector-search-rest-api.adoc | 4 +++- modules/vector-search/pages/run-vector-search-ui.adoc | 5 ++++- 10 files changed, 39 insertions(+), 13 deletions(-) diff --git a/modules/search/pages/create-child-field.adoc b/modules/search/pages/create-child-field.adoc index a6315b407..4bd09179d 100644 --- a/modules/search/pages/create-child-field.adoc +++ b/modules/search/pages/create-child-field.adoc @@ -7,6 +7,9 @@ == Prerequisites +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + * You have created an index. For more information, see xref:create-search-index-ui.adoc[]. diff --git a/modules/search/pages/create-child-mapping.adoc b/modules/search/pages/create-child-mapping.adoc index dd5f6fc87..b9297c141 100644 --- a/modules/search/pages/create-child-mapping.adoc +++ b/modules/search/pages/create-child-mapping.adoc @@ -7,6 +7,9 @@ == Prerequisites +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + * You have created an index. For more information, see xref:create-search-index-ui.adoc[]. diff --git a/modules/search/pages/create-search-index-alias.adoc b/modules/search/pages/create-search-index-alias.adoc index fc3b045d3..520786b43 100644 --- a/modules/search/pages/create-search-index-alias.adoc +++ b/modules/search/pages/create-search-index-alias.adoc @@ -9,10 +9,13 @@ For more information about Search index aliases, see xref:index-aliases.adoc[]. == Prerequisites -* You've created at least one Search index. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have created at least one Search index. For more information, see xref:create-search-index-ui.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase Capella UI. == Procedure diff --git a/modules/search/pages/create-type-mapping.adoc b/modules/search/pages/create-type-mapping.adoc index adcf1d194..ee8c67569 100644 --- a/modules/search/pages/create-type-mapping.adoc +++ b/modules/search/pages/create-type-mapping.adoc @@ -9,6 +9,9 @@ For more information about type mappings, see xref:customize-index.adoc#type-map == Prerequisites +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + * You have started to create or already created a Search index in xref:create-search-index-ui.adoc[Advanced Mode]. * You have logged in to the Couchbase Capella UI. diff --git a/modules/search/pages/geo-search-rest-api.adoc b/modules/search/pages/geo-search-rest-api.adoc index c24678381..04b2d3adb 100644 --- a/modules/search/pages/geo-search-rest-api.adoc +++ b/modules/search/pages/geo-search-rest-api.adoc @@ -7,13 +7,17 @@ == Prerequisites -* You've deployed the Search Service on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. * You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You installed the Couchbase command-line tool (CLI). +* You have documents in your database that contain geospatial data. -* You have the hostname or IP address for your database. +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. == Procedure diff --git a/modules/search/pages/search-query-auto-complete-code.adoc b/modules/search/pages/search-query-auto-complete-code.adoc index 39572edf7..f065b376b 100644 --- a/modules/search/pages/search-query-auto-complete-code.adoc +++ b/modules/search/pages/search-query-auto-complete-code.adoc @@ -16,7 +16,12 @@ For more information about how to change Services on your database, see xref:clo * You have a bucket with scopes and collections in your database. For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You have created a compatible Search index. For more information, see xref:search-query-auto-complete-ui.adoc[]. +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. + +* You have created a compatible Search index. +For more information, see xref:search-query-auto-complete-ui.adoc[]. == Procedure diff --git a/modules/search/pages/simple-search-rest-api.adoc b/modules/search/pages/simple-search-rest-api.adoc index 691037e23..efcdb007e 100644 --- a/modules/search/pages/simple-search-rest-api.adoc +++ b/modules/search/pages/simple-search-rest-api.adoc @@ -7,15 +7,14 @@ == Prerequisites -* You've deployed the Search Service on a node in your database. - -* You installed the Couchbase command-line tool (CLI). +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. * You have the hostname for the node in your database that's running the Search Service. + For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. -* You've created a Search index. +* You have created a Search index. + For more information about how to create a Search index, see xref:create-search-index-ui.adoc[] or xref:create-search-index-rest-api.adoc[]. diff --git a/modules/search/pages/simple-search-ui.adoc b/modules/search/pages/simple-search-ui.adoc index 540bb31d3..5ae7b69cf 100644 --- a/modules/search/pages/simple-search-ui.adoc +++ b/modules/search/pages/simple-search-ui.adoc @@ -7,7 +7,8 @@ == Prerequisites -* You have the Search Service enabled on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. * You have created a Search index. + diff --git a/modules/vector-search/pages/run-vector-search-rest-api.adoc b/modules/vector-search/pages/run-vector-search-rest-api.adoc index 4a411b5b0..fd7c99b31 100644 --- a/modules/vector-search/pages/run-vector-search-rest-api.adoc +++ b/modules/vector-search/pages/run-vector-search-rest-api.adoc @@ -50,7 +50,9 @@ include::example$run-vector-search-payload.sh[] ---- The Search Service combines the Vector search results from the `knn` object with the traditional `query` object by using an `OR` function. -If the same documents match the `knn` and `query` objects, the Search Service ranks those documents higher in search results. +If the same documents match the `knn` and `query` objects, the Search Service ranks those documents higher in search results. + +TIP: For a more complex query, you can copy the `query` object from the example under xref:run-vector-search-ui.adoc#large[Example: Running a Semantic Search Query with a Large Embedding Vector] to use in your REST API call. For more information about the available properties for a Search query JSON payload, see xref:search:search-request-params.adoc[]. diff --git a/modules/vector-search/pages/run-vector-search-ui.adoc b/modules/vector-search/pages/run-vector-search-ui.adoc index bc43744b6..66acc23d7 100644 --- a/modules/vector-search/pages/run-vector-search-ui.adoc +++ b/modules/vector-search/pages/run-vector-search-ui.adoc @@ -7,7 +7,8 @@ == Prerequisites -* You have the Search Service enabled on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. * You have created a Vector Search index. + @@ -28,6 +29,7 @@ To run a Vector Search with the Capella UI: . Press kbd:[Enter] or click btn:[Search]. . . (Optional) To view a document and its source collection, click a document name in the search results list. +[#similarity] === Example: Running a Simple Vector Similarity Query For example, the following query searches for the top 2 vectors similar to the vector `[ 0, 0, 128 ]` in the `colorvect_l2` field: @@ -43,6 +45,7 @@ It only returns the `k` number of similar vectors. The Search Service combines the Vector search results from a `knn` object with the traditional `query` object by using an `OR` function. If the same documents match the `knn` and `query` objects, the Search Service ranks those documents higher in search results. +[#large] === Example: Running a Semantic Search Query with a Large Embedding Vector TIP: This example uses the `rgb_questions.json` sample data, which provides ready-made vector embeddings and queries that you can use for searches. From 12ee94432ad4d0fc98d610a1147bd0af502c3fce Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Tue, 12 Mar 2024 17:02:40 -0400 Subject: [PATCH 43/59] [AV-68571] Consistency edits and incorporating comments from another SME --- .../pages/create-custom-character-filter.adoc | 2 ++ .../pages/create-custom-date-time-parser.adoc | 2 ++ .../pages/create-custom-token-filter.adoc | 2 ++ .../search/pages/create-custom-tokenizer.adoc | 2 ++ .../search/pages/create-custom-wordlist.adoc | 2 ++ .../search/pages/search-request-params.adoc | 5 +++-- .../search/pages/set-advanced-settings.adoc | 2 ++ modules/search/pages/set-type-identifier.adoc | 2 ++ modules/search/pages/simple-search-ui.adoc | 18 ++++++++++-------- .../vector-search-field-descriptions.adoc | 8 +++++++- .../pages/create-vector-search-index-ui.adoc | 1 + 11 files changed, 35 insertions(+), 11 deletions(-) diff --git a/modules/search/pages/create-custom-character-filter.adoc b/modules/search/pages/create-custom-character-filter.adoc index f992fdccf..2a9d40b35 100644 --- a/modules/search/pages/create-custom-character-filter.adoc +++ b/modules/search/pages/create-custom-character-filter.adoc @@ -42,6 +42,8 @@ To continue customizing your Search index, you can also: * xref:set-advanced-settings.adoc[] * xref:set-type-identifier.adoc[] * xref:create-type-mapping.adoc[] +* xref:create-child-field.adoc[] +* xref:create-child-mapping.adoc[] * xref:create-custom-tokenizer.adoc[] * xref:create-custom-token-filter.adoc[] * xref:create-custom-wordlist.adoc[] diff --git a/modules/search/pages/create-custom-date-time-parser.adoc b/modules/search/pages/create-custom-date-time-parser.adoc index a16ced107..b0f82147a 100644 --- a/modules/search/pages/create-custom-date-time-parser.adoc +++ b/modules/search/pages/create-custom-date-time-parser.adoc @@ -47,6 +47,8 @@ To continue customizing your Search index, you can also: * xref:set-advanced-settings.adoc[] * xref:set-type-identifier.adoc[] * xref:create-type-mapping.adoc[] +* xref:create-child-field.adoc[] +* xref:create-child-mapping.adoc[] * xref:create-custom-analyzer.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-tokenizer.adoc[] diff --git a/modules/search/pages/create-custom-token-filter.adoc b/modules/search/pages/create-custom-token-filter.adoc index 67d12b37f..a9a36c94e 100644 --- a/modules/search/pages/create-custom-token-filter.adoc +++ b/modules/search/pages/create-custom-token-filter.adoc @@ -213,6 +213,8 @@ To continue customizing your Search index, you can also: * xref:set-advanced-settings.adoc[] * xref:set-type-identifier.adoc[] * xref:create-type-mapping.adoc[] +* xref:create-child-field.adoc[] +* xref:create-child-mapping.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-tokenizer.adoc[] * xref:create-custom-wordlist.adoc[] diff --git a/modules/search/pages/create-custom-tokenizer.adoc b/modules/search/pages/create-custom-tokenizer.adoc index 267dda30a..4d17d9c78 100644 --- a/modules/search/pages/create-custom-tokenizer.adoc +++ b/modules/search/pages/create-custom-tokenizer.adoc @@ -77,6 +77,8 @@ To continue customizing your Search index, you can also: * xref:set-advanced-settings.adoc[] * xref:set-type-identifier.adoc[] * xref:create-type-mapping.adoc[] +* xref:create-child-field.adoc[] +* xref:create-child-mapping.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-token-filter.adoc[] * xref:create-custom-wordlist.adoc[] diff --git a/modules/search/pages/create-custom-wordlist.adoc b/modules/search/pages/create-custom-wordlist.adoc index 619c3ed38..9d4a47cbd 100644 --- a/modules/search/pages/create-custom-wordlist.adoc +++ b/modules/search/pages/create-custom-wordlist.adoc @@ -47,6 +47,8 @@ To continue customizing your Search index, you can also: * xref:set-advanced-settings.adoc[] * xref:set-type-identifier.adoc[] * xref:create-type-mapping.adoc[] +* xref:create-child-field.adoc[] +* xref:create-child-mapping.adoc[] * xref:create-custom-analyzer.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-token-filter.adoc[] diff --git a/modules/search/pages/search-request-params.adoc b/modules/search/pages/search-request-params.adoc index 81e02cca4..5d4cdfdeb 100644 --- a/modules/search/pages/search-request-params.adoc +++ b/modules/search/pages/search-request-params.adoc @@ -151,7 +151,7 @@ include::example$run-search-full-request.jsonc[tag=knn] ---- NOTE: To run a Vector Search query, you must still include a `query` object with your Search request. -To return only results from your Vector Search query, you can set the `query` object to a <>. +To return only results from your Vector Search query, you can set the `query` object to a <> query. To run a hybrid query that uses regular Search Service parameters together with Vector Search to return results, see the <> section. An object in the `knn` array can contain the following properties: @@ -166,7 +166,7 @@ Enter the total number of results that you want to return from your Vector Searc The Search Service returns the `k` closest vectors to the vector given in `vector`. -NOTE: The value of `k` overrides any value set in the <> if you use a <> query. +NOTE: The <> overrides any value set in `k`. |field |String |Yes a| @@ -179,6 +179,7 @@ Enter the vector that you want to compare to the vector data in `field`. The Search Service uses the similarity metric defined in the xref:search-index-params.adoc#similarity[Search index definition] to return the `k` closest vectors from the Search index. NOTE: The vector in your Search query must match the dimension of the vectors stored in your Search index. +If the dimensions do not match, your Search query does not return any results. For more information about the dimension value, see the xref:search-index-params.adoc#dims[dims property] or the xref:child-field-options-reference.adoc#dimension[Dimension option] in the UI. |==== diff --git a/modules/search/pages/set-advanced-settings.adoc b/modules/search/pages/set-advanced-settings.adoc index 048dd1bbc..663baa4fc 100644 --- a/modules/search/pages/set-advanced-settings.adoc +++ b/modules/search/pages/set-advanced-settings.adoc @@ -74,6 +74,8 @@ After you change the settings for your Search index, you can continue to customi * xref:set-type-identifier.adoc[] * xref:create-type-mapping.adoc[] +* xref:create-child-field.adoc[] +* xref:create-child-mapping.adoc[] * xref:create-custom-analyzer.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-token-filter.adoc[] diff --git a/modules/search/pages/set-type-identifier.adoc b/modules/search/pages/set-type-identifier.adoc index 9ab1cfd1c..7bb55e064 100644 --- a/modules/search/pages/set-type-identifier.adoc +++ b/modules/search/pages/set-type-identifier.adoc @@ -92,6 +92,8 @@ After you set the type identifier for your Search index, you can continue to cus * xref:set-advanced-settings.adoc[] * xref:create-type-mapping.adoc[] +* xref:create-child-field.adoc[] +* xref:create-child-mapping.adoc[] * xref:create-custom-analyzer.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-token-filter.adoc[] diff --git a/modules/search/pages/simple-search-ui.adoc b/modules/search/pages/simple-search-ui.adoc index 5ae7b69cf..51a1ea5c7 100644 --- a/modules/search/pages/simple-search-ui.adoc +++ b/modules/search/pages/simple-search-ui.adoc @@ -24,22 +24,24 @@ To run a simple search with the Capella UI: . Go to menu:Data Tools[Search]. . Next to your Search index or xref:index-aliases.adoc[index alias], click btn:[Search]. . In the *Search* field, enter a search query. -+ +. Press kbd:[Enter]. +. (Optional) To view a document and its source collection, click a document name in the search results list. + +=== Example + For example, the following query searches for the strings `view`, `food`, and `beach`: -+ + [source,json] ---- include::example$run-search-payload-ui.jsonc[] ---- -+ -The query payload enables scoring explanations and term highlighting. It also returns all available fields in the index, and returns 10 results per page. -+ + +The query payload enables scoring explanations and term highlighting. +It also returns all available fields in the index, and returns 10 results per page. + TIP: Use a xref:index-aliases.adoc[Search index alias] to search multiple Search indexes in a single search query. Use the xref:search-request-params.adoc#collections[`collections` parameter] in your request to specify an array of collections to search from the Search index. -. Press kbd:[Enter]. -. (Optional) To view a document and its source collection, click a document name in the search results list. - == Next Steps If you do not get the search results you were expecting, you can change the xref:search-request-params.adoc[JSON payload for your Search query]. diff --git a/modules/search/partials/vector-search-field-descriptions.adoc b/modules/search/partials/vector-search-field-descriptions.adoc index fd7079a25..6aa526c9e 100644 --- a/modules/search/partials/vector-search-field-descriptions.adoc +++ b/modules/search/partials/vector-search-field-descriptions.adoc @@ -2,10 +2,15 @@ For a `vector` child field, choose whether the Search Service should prioritize recall or latency when returning similar vectors in search results: * *recall*: The Search Service prioritizes returning the most accurate result. -This may increase resource usage for Search queries. +This may increase resource usage for Search queries. ++ +The Search Service uses an `nprobe` value to calculate the number of centroids to search when using recall priority. +This value is calculated by taking the square root of the number of centroids in the index. * *latency*: The Search Service prioritizes returning results with lower latency. This may reduce the accuracy of results. ++ +The Search Service uses half the `nprobe` value calculated for *recall* priority. For more information about Vector Search indexes, see xref:vector-search:vector-search.adoc[] or xref:vector-search:create-vector-search-index-ui.adoc[]. // end::optimized_for[] @@ -32,6 +37,7 @@ For more information about Vector Search indexes, see xref:vector-search:vector- For a `vector` child field, enter the total number of elements in the vector embedding array. Vector Search indexes can support arrays with up to 2048 elements. +Arrays can be an array of arrays. For more information about Vector Search indexes, see xref:vector-search:vector-search.adoc[] or xref:vector-search:create-vector-search-index-ui.adoc[]. // end::dimension[] \ No newline at end of file diff --git a/modules/vector-search/pages/create-vector-search-index-ui.adoc b/modules/vector-search/pages/create-vector-search-index-ui.adoc index af3e955dc..19be49ddf 100644 --- a/modules/vector-search/pages/create-vector-search-index-ui.adoc +++ b/modules/vector-search/pages/create-vector-search-index-ui.adoc @@ -53,6 +53,7 @@ For Couchbase Server version 7.6 and later, your index name must be unique insid + Vectors must be represented as an array of floating point numbers. They appear as `\{field-name} [ number ]` in the Capella Quick Mode editor. +The Search Service also supports arrays of arrays. . Configure the options for your new child field as follows: .. In the *Dimension* field, check that the value matches the total number of elements in your vector field's array. + From d1e683dbe92e521d866efb54b7f9a217991fae52 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Tue, 12 Mar 2024 17:32:11 -0400 Subject: [PATCH 44/59] [AV-68571] Adding clarification based on MB-60141 --- modules/search/pages/search-index-params.adoc | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/modules/search/pages/search-index-params.adoc b/modules/search/pages/search-index-params.adoc index 21fb75899..1d66e97b3 100644 --- a/modules/search/pages/search-index-params.adoc +++ b/modules/search/pages/search-index-params.adoc @@ -94,7 +94,12 @@ The `planParams` object contains the following properties: |Property |Type |Required? |Description |maxPartitionsPerPIndex |n/a |No |This setting is deprecated. Use `indexPartitions`, instead. -|indexPartitions |Number |Yes |The number of partitions to split the Search index into, across the nodes you have available in your database with the Search Service enabled. Use index partitions to increase index and query performance on large datasets. +|indexPartitions |Number |Yes a|The number of partitions to split the Search index into, across the nodes you have available in your database with the Search Service enabled. +Use index partitions to increase index and query performance on large datasets. + +NOTE: The scoring calculation for regular Search queries can be affected by the number of partitions in your Search index, and how the Search Service distributes documents across partitions. +This is a limitation of the https://en.wikipedia.org/wiki/Tf%E2%80%93idf[tf-idf^] weighting scheme. + |numReplicas |Number |Yes a| For high-availability, set the number of replicas the Search Service creates for the Search index. From 36d7aaa51e0c389943860284d66ff8fe67f8262b Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Wed, 13 Mar 2024 14:46:21 -0400 Subject: [PATCH 45/59] [AV-68571] Big consistency pass in tandem with branch on Server to clear up any remaining inconsistencies between files. --- .../n1ql-language-reference/searchfun.adoc | 5 ++- .../search/pages/create-custom-analyzer.adoc | 5 +-- .../pages/create-custom-character-filter.adoc | 3 -- .../pages/create-custom-date-time-parser.adoc | 3 -- .../pages/create-custom-token-filter.adoc | 4 +- .../search/pages/create-custom-tokenizer.adoc | 2 - .../search/pages/create-custom-wordlist.adoc | 2 - .../pages/create-search-index-rest-api.adoc | 10 ++++- .../search/pages/create-search-indexes.adoc | 9 ++--- modules/search/pages/customize-index.adoc | 2 +- .../default-token-filters-reference.adoc | 1 + .../pages/field-data-types-reference.adoc | 10 +++-- modules/search/pages/import-search-index.adoc | 2 +- .../quick-index-supported-languages.adoc | 2 +- modules/search/pages/search-index-params.adoc | 9 +++-- .../pages/search-query-auto-complete-ui.adoc | 3 +- .../search/pages/search-request-params.adoc | 37 ++++++++++++++++++- .../search/pages/set-advanced-settings.adoc | 3 +- modules/search/pages/set-type-identifier.adoc | 6 ++- .../search/pages/simple-search-rest-api.adoc | 2 +- 20 files changed, 80 insertions(+), 40 deletions(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc index e1070b2f1..3a08a7c91 100644 --- a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc @@ -101,7 +101,7 @@ For more information about how to format a full search request object, refer to [NOTE] ==== -When specifying a complete Full Text Search request with the {sqlpp} SEARCH() function, if the value of the `size` parameter is greater than the xref:server:fts:fts-response-object-schema.adoc#request[maximum number of full text search results], the query ignores the `size` parameter and returns all matching results. +When specifying a complete Full Text Search request with the {sqlpp} SEARCH() function, if the value of the `size` parameter is greater than the xref:server:fts:fts-response-object-schema.adoc#request[maximum number of Full Text Search results], the query ignores the `size` parameter and returns all matching results. This is different to the behavior of a complete Full Text Search request in the Search Service, where the query returns an error if the value of the `size` parameter is greater than the maximum number of Full Text Search results. ==== @@ -154,7 +154,8 @@ The `indexUUID` field and the `index` field must both identify the same Search i If they identify different Search indexes, or if either of them does not identify a Search index, an error occurs. You can find the UUID of a Search index by viewing the index definition. -Click a Search index name in the Capella UI to open the editor. Click btn:[Index Definition] to view the index definition. +Click a Search index name in the Capella UI to open the editor. +Click btn:[Index Definition] to view the index definition. | `out` [Optional] diff --git a/modules/search/pages/create-custom-analyzer.adoc b/modules/search/pages/create-custom-analyzer.adoc index af132d1bf..67985a841 100644 --- a/modules/search/pages/create-custom-analyzer.adoc +++ b/modules/search/pages/create-custom-analyzer.adoc @@ -15,9 +15,6 @@ For more information, see xref:customize-index.adoc#analyzers[Search Index Featu * You have the Search Service enabled on a node in your database. For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have a bucket with scopes and collections in your database. -For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. - * You have created an index with xref:create-search-index-ui.adoc[Advanced Mode]. * You have logged in to the Couchbase Capella UI. @@ -56,7 +53,7 @@ For more information, see xref:default-token-filters-reference.adoc[] or xref:cr After you create a custom analyzer, you can xref:set-advanced-settings.adoc#default-analyzer[set it as the default analyzer] for your Search index. -You can also use the custom analyzer when you xref:create-type-mapping.adoc[create a type mapping with Advanced Mode]. +You can also use the custom analyzer when you xref:create-type-mapping.adoc[create a type mapping with Advanced Mode], xref:create-child-mapping.adoc[a child mapping] or xref:create-child-field.adoc[a child field]. To continue customizing your Search index, you can also: diff --git a/modules/search/pages/create-custom-character-filter.adoc b/modules/search/pages/create-custom-character-filter.adoc index 2a9d40b35..4424c2d50 100644 --- a/modules/search/pages/create-custom-character-filter.adoc +++ b/modules/search/pages/create-custom-character-filter.adoc @@ -9,9 +9,6 @@ * You have the Search Service enabled on a node in your database. For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. - -* You have a bucket with scopes and collections in your database. -For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. * You have logged in to the Couchbase Capella UI. diff --git a/modules/search/pages/create-custom-date-time-parser.adoc b/modules/search/pages/create-custom-date-time-parser.adoc index b0f82147a..81583b3bb 100644 --- a/modules/search/pages/create-custom-date-time-parser.adoc +++ b/modules/search/pages/create-custom-date-time-parser.adoc @@ -12,9 +12,6 @@ If you store date data in a format other than RFC-3339 (ISO-8601), then you need * You have the Search Service enabled on a node in your database. For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have a bucket with scopes and collections in your database. -For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. - * You have logged in to the Couchbase Capella UI. * You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. diff --git a/modules/search/pages/create-custom-token-filter.adoc b/modules/search/pages/create-custom-token-filter.adoc index a9a36c94e..30479f282 100644 --- a/modules/search/pages/create-custom-token-filter.adoc +++ b/modules/search/pages/create-custom-token-filter.adoc @@ -12,9 +12,6 @@ xref:customize-index.adoc#token-filters[Token filters] can improve your search r * You have the Search Service enabled on a node in your database. For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. - -* You have a bucket with scopes and collections in your database. -For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. * You have logged in to the Couchbase Capella UI. @@ -215,6 +212,7 @@ To continue customizing your Search index, you can also: * xref:create-type-mapping.adoc[] * xref:create-child-field.adoc[] * xref:create-child-mapping.adoc[] +* xref:create-custom-analyzer.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-tokenizer.adoc[] * xref:create-custom-wordlist.adoc[] diff --git a/modules/search/pages/create-custom-tokenizer.adoc b/modules/search/pages/create-custom-tokenizer.adoc index 4d17d9c78..0d780250a 100644 --- a/modules/search/pages/create-custom-tokenizer.adoc +++ b/modules/search/pages/create-custom-tokenizer.adoc @@ -11,8 +11,6 @@ * You have the Search Service enabled on a node in your database. For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have a bucket with scopes and collections in your database. -For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. * You have logged in to the Couchbase Capella UI. diff --git a/modules/search/pages/create-custom-wordlist.adoc b/modules/search/pages/create-custom-wordlist.adoc index 9d4a47cbd..67c52bcde 100644 --- a/modules/search/pages/create-custom-wordlist.adoc +++ b/modules/search/pages/create-custom-wordlist.adoc @@ -14,8 +14,6 @@ For more information about how to create a custom token filter, see xref:create- * You have the Search Service enabled on a node in your database. For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have a bucket with scopes and collections in your database. -For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. * You have logged in to the Couchbase Capella UI. diff --git a/modules/search/pages/create-search-index-rest-api.adoc b/modules/search/pages/create-search-index-rest-api.adoc index 8e4a01c43..3596b8205 100644 --- a/modules/search/pages/create-search-index-rest-api.adoc +++ b/modules/search/pages/create-search-index-rest-api.adoc @@ -31,9 +31,17 @@ To create a Search index with the REST API: ---- include::example$create-search-index-header.sh[] ---- ++ +[NOTE] +==== +Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). + +As of version 7.6, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +==== + . Enter the JSON payload for the settings you want in your index. + -Don't include the xref:search-index-params.adoc#uuid[uuid] or xref:search-index-params.adoc#sourceuuid[sourceUUID] parameters. +Do not include the xref:search-index-params.adoc#uuid[uuid] or xref:search-index-params.adoc#sourceuuid[sourceUUID] parameters. + TIP: If you remove the xref:search-index-params.adoc#uuid[uuid] and xref:search-index-params.adoc#sourceuuid[sourceUUID] parameters, you can copy the Search index definition JSON payload from the Couchbase Capella UI to use in your REST API call. For more information about how to create an index with the UI, see xref:create-search-index-ui.adoc[]. diff --git a/modules/search/pages/create-search-indexes.adoc b/modules/search/pages/create-search-indexes.adoc index d9fc60ad8..c0831ba11 100644 --- a/modules/search/pages/create-search-indexes.adoc +++ b/modules/search/pages/create-search-indexes.adoc @@ -12,7 +12,7 @@ You can create a Search index with: * The <>. TIP: If you're new to developing with the Search Service, <>. -You can export the Search index definition JSON payload from the UI to create your index xref:create-search-index-rest-api.adoc[with the REST API]. +You can export the Search index definition JSON payload from the UI to create your index xref:create-search-index-rest-api.adoc[with the REST API], or xref:import-search-index.adoc[]. [#ui] == Creating a Search Index with the Capella UI @@ -24,12 +24,12 @@ Couchbase Capella supports two editing modes for the Search Service: [NOTE] ==== -For indexes created with version 7.6, index names must be unique inside a bucket and scope. +For indexes created with Couchbase Server version 7.6 and later, index names must be unique inside a bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope on a Capella database running version 7.6. The Capella UI marks indexes as scoped or not scoped to a specific bucket and scope. -Indexes created with a previous version are not scoped. +Indexes created with a previous version of Couchbase Server are not scoped. ==== Use Quick Mode for a streamlined editing experience. @@ -60,9 +60,6 @@ You can also copy the Search index definition JSON payload from a Search index i [NOTE] ==== -All index names must be unique inside a bucket and scope. -You cannot have 2 indexes with the same name inside the same bucket and scope. - Use the scoped name for an index with the xref:rest-api:rest-fts.adoc[Search Service REST API] for any endpoints that do not include the bucket and scope in their path. For example, you must use `bucket.scope.index_name` as the format for your index name with the `analyzeDoc` endpoint, but not with the new 7.6 `query` endpoint. ==== diff --git a/modules/search/pages/customize-index.adoc b/modules/search/pages/customize-index.adoc index 532274e67..d3e5a41aa 100644 --- a/modules/search/pages/customize-index.adoc +++ b/modules/search/pages/customize-index.adoc @@ -7,7 +7,7 @@ Some features are only available in xref:create-search-index-ui.adoc[Advanced Mode]. -You can use the following components and configure the following options for a Search index: +You can add the following components and configure the following options for a Search index: [cols="2,1,1,4"] |==== diff --git a/modules/search/pages/default-token-filters-reference.adoc b/modules/search/pages/default-token-filters-reference.adoc index 967ac438b..e6451d08d 100644 --- a/modules/search/pages/default-token-filters-reference.adoc +++ b/modules/search/pages/default-token-filters-reference.adoc @@ -17,6 +17,7 @@ The following token filters are available: |Token Filter Type |Description |apostrophe | Removes all characters after an apostrophe (') from tokenizer results. Also removes the apostrophe. +For example, the token `Couchbase's` becomes `Couchbase`. |camelCase a| Splits text in camelCase inside tokenizer results into separate tokens. diff --git a/modules/search/pages/field-data-types-reference.adoc b/modules/search/pages/field-data-types-reference.adoc index 68607e872..af21e16b2 100644 --- a/modules/search/pages/field-data-types-reference.adoc +++ b/modules/search/pages/field-data-types-reference.adoc @@ -12,18 +12,22 @@ For example, if you created a xref:customize-index.adoc#type-mappings[dynamic ty The following field data types are available: +[cols="1,2"] |==== |Field Data Type |Description -|text |The field contains a string. The string can contain numbers and special characters. +|text |The field contains a string. +The string can contain numbers and special characters. -|number |The field contains a number. It does not contain any alphabetic characters. +|number |The field contains a number. +It does not contain any alphabetic characters. |datetime |The field contains a date/time value that matches the format of a xref:customize-index.adoc#date-time[Date/Time Parser] in the index. |boolean |The field contains a true or false value. -|disabled | This field data type is deprecated. It's included for compatibility only. +|disabled | This field data type is deprecated. +It's included for compatibility only. |geopoint a| diff --git a/modules/search/pages/import-search-index.adoc b/modules/search/pages/import-search-index.adoc index 94cfa148f..b49cee636 100644 --- a/modules/search/pages/import-search-index.adoc +++ b/modules/search/pages/import-search-index.adoc @@ -34,6 +34,6 @@ To import a xref:create-search-indexes.adoc[Search index definition] or xref:ind == Next Steps -To change the settings for your imported Search index, see xref:create-quick-index.adoc[] or xref:create-search-index-ui.adoc. +To add additional features to your imported Search index, see xref:customize-index.adoc[]. To run a search with your Search index or index alias, see xref:simple-search-ui.adoc[] or xref:simple-search-rest-api.adoc[]. \ No newline at end of file diff --git a/modules/search/pages/quick-index-supported-languages.adoc b/modules/search/pages/quick-index-supported-languages.adoc index a108e7a34..d407b86f7 100644 --- a/modules/search/pages/quick-index-supported-languages.adoc +++ b/modules/search/pages/quick-index-supported-languages.adoc @@ -12,7 +12,7 @@ The following language options are available for Quick Index text fields: |==== |Language Option |Description -|Unknown/Various |Select this option if the text in the field or fields contains multiple languages, or doesn't match another language option. +|Unknown/Various |Select this option if the text in the field or fields contains multiple languages, or does not match another language option. |English |For text in English. diff --git a/modules/search/pages/search-index-params.adoc b/modules/search/pages/search-index-params.adoc index 1d66e97b3..92a16e2a0 100644 --- a/modules/search/pages/search-index-params.adoc +++ b/modules/search/pages/search-index-params.adoc @@ -46,12 +46,15 @@ The UUID for the Search index. The Search Service automatically generates a UUID for a Search index. -If you use an existing UUID, the Search Service updates the existing Search index. Don't include the `uuid` property when you want to copy an index to a different cluster or create a new index. +If you use an existing UUID, the Search Service updates the existing Search index. +Do not include the `uuid` property when you want to copy an index to a different cluster or create a new index. + +View the UUID for an existing index from the Capella UI by selecting an existing index and clicking btn:[Index Definition]. +The UUID displays in the Index Definition on the Update Index page. |sourceType |String |Yes |The `sourceType` is always `"gocbcore"`. -|sourceName |String |Yes a| -The name of the bucket where you want to create the Search index. +|sourceName |String |Yes |The name of the bucket where you want to create the Search index. |[[sourceuuid]]sourceUUID |String |No a| diff --git a/modules/search/pages/search-query-auto-complete-ui.adoc b/modules/search/pages/search-query-auto-complete-ui.adoc index 6d1563126..d7787b55e 100644 --- a/modules/search/pages/search-query-auto-complete-ui.adoc +++ b/modules/search/pages/search-query-auto-complete-ui.adoc @@ -35,7 +35,8 @@ To create the Search index in the Capella UI with Advanced Mode: ==== Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). -For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. +You cannot have 2 indexes with the same name inside the same bucket and scope. ==== . Under *Advanced Settings*, expand *Analyzers*. . xref:create-custom-analyzer.adoc[] with the following settings: diff --git a/modules/search/pages/search-request-params.adoc b/modules/search/pages/search-request-params.adoc index 5d4cdfdeb..bf6a6b25e 100644 --- a/modules/search/pages/search-request-params.adoc +++ b/modules/search/pages/search-request-params.adoc @@ -500,6 +500,8 @@ Analytic queries use an analyzer to analyze the contents of your Search query, a For more information about analyzers, see xref:customize-index.adoc#analyzers[Customize a Search Index with the Capella UI]. +NOTE: From Couchbase Server 7.6 and later, when you run a Search query with the xref:n1ql:n1ql-language-reference/searchfun.adoc[{sqlpp} Search Function], the analyzer used in your Analytic query does not need to match the analyzer used in the Search index. + [cols="1,1,4,4"] |==== |Property |Type |Description |Examples @@ -553,6 +555,8 @@ Use Non-Analytic queries to run a search query without using an analyzer on the For more information about analyzers, see xref:customize-index.adoc#analyzers[Customize a Search Index with the Capella UI]. +NOTE: From Couchbase Server 7.6 and later, when you run a Search query with the xref:n1ql:n1ql-language-reference/searchfun.adoc[{sqlpp} Search Function], you do not need to use the xref:default-analyzers-reference.adoc#keyword[keyword analyzer] in your Search index to run a non-analytic query. + [cols="1,1,4,4"] |==== |Property |Type |Description |Examples @@ -838,6 +842,35 @@ If you do not set the `inclusive_max` value, by default, `max` is exclusive to t |==== +=== IP Address Range Queries + +Use an IP Address Range query to search for IP address data in your Search index. + +You can use IPv4 or IPv6 CIDR syntax in your query. + +For example, the following query searches for any IP addresses in the range `2.7.13.141/32` and `2001:4860:4860::8888/24`: + +[source,json] +---- +include::example$query-ip-range.jsonc[] +---- + +NOTE: If your IP range includes over 1024 IP addresses, you must update the `bleveMaxResultWindow` setting to accommodate your results. +For more information about how to change this setting, see xref:server:fts:fts-advanced-settings-bleveMaxResultWindow.adoc[]. + +[cols="1,1,1,2"] +|==== +|Property |Type |Required? |Description + +|cidr +|String Containing An IPv4 or IPv6 IP Address +|Yes +a| Enter an IP address range or single IP address, in IPv4 or IPv6 CIDR notation. + +The Search Service returns documents with IP addresses that fall inside the specified range or match the specified IP address. + +|==== + [#geopoint-queries-distance] === Distance/Radius-Based Geopoint Queries @@ -1666,7 +1699,9 @@ include::example$query-special.jsonc[tag=match_all] | [[match_none]]match_none | Object -| Use the `match_none` object as the only property in your `query` object to return no documents from the Search index in search results. +a| Use the `match_none` object as the only property in your `query` object to return no documents from the Search index in search results. + +If you're using the <>, the `match_none` object returns only matches to the Vector Search query inside the `knn` object. a| .Return no documents from the Search index [source,json] diff --git a/modules/search/pages/set-advanced-settings.adoc b/modules/search/pages/set-advanced-settings.adoc index 663baa4fc..7d5605b24 100644 --- a/modules/search/pages/set-advanced-settings.adoc +++ b/modules/search/pages/set-advanced-settings.adoc @@ -30,7 +30,8 @@ To set general settings for a Search index with the Capella UI: |==== |Option |Description -|Default Type |Change the default type assigned to documents in the index. The default value is `_default`. +|Default Type |Change the default type assigned to documents in the index. +The default value is `_default`. |[[default-analyzer]]Default Analyzer a| diff --git a/modules/search/pages/set-type-identifier.adoc b/modules/search/pages/set-type-identifier.adoc index 7bb55e064..bd9787d35 100644 --- a/modules/search/pages/set-type-identifier.adoc +++ b/modules/search/pages/set-type-identifier.adoc @@ -16,10 +16,14 @@ For more information about how to change Services on your database, see xref:clo * You have a bucket with scopes and collections in your database. For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You have logged in to the Couchbase Capella UI. * You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. +* You have created at least one type mapping in your Search index. +For more information, see xref:create-type-mapping.adoc[]. + +* You have logged in to the Couchbase Capella UI. + == Procedure To set a type identifier for a Search index with the Capella UI: diff --git a/modules/search/pages/simple-search-rest-api.adoc b/modules/search/pages/simple-search-rest-api.adoc index efcdb007e..b87022cfd 100644 --- a/modules/search/pages/simple-search-rest-api.adoc +++ b/modules/search/pages/simple-search-rest-api.adoc @@ -51,4 +51,4 @@ For more information about the available properties for a Search query JSON payl If you do not get the search results you were expecting, you can change the JSON payload xref:search-index-params.adoc[for your Search index] or xref:search-request-params.adoc[for your Search query]. -You can also xref:customize-index.adoc[]. +You can also xref:customize-index.adoc[]. \ No newline at end of file From f1f771890198a9001ffaf3be3d4291934028512b Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Wed, 13 Mar 2024 14:50:45 -0400 Subject: [PATCH 46/59] [AV-68965] Updating integrations page for LangChain Per Richard Smedley, copying over missed update to add in Terraform as well. --- modules/third-party/pages/integrations.adoc | 24 +++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/modules/third-party/pages/integrations.adoc b/modules/third-party/pages/integrations.adoc index 7f9b3563b..c6b7f5a40 100644 --- a/modules/third-party/pages/integrations.adoc +++ b/modules/third-party/pages/integrations.adoc @@ -24,6 +24,30 @@ These are examples of integrations that Couchbase or partners have developed to |=== | Integration | Summary | Links | Capella | Self-managed | <<#support-model,Support Model>> +| LangChain +a| Use the LangChain integrations to quickly start developing AI applications with Capella or Couchbase Server: + +* Use the document loader to get documents from your Couchbase database. +* Use the vector stores integration to create a Search index, store vectors, and run vector searches. +a| +* https://python.langchain.com/docs/integrations/document_loaders/couchbase[LangChain - Python Document Loader] +* https://python.langchain.com/docs/integrations/vectorstores/couchbase[LangChain - Python Vector Stores] +* https://js.langchain.com/docs/integrations/document_loaders/web_loaders/couchbase[LangChain - JavaScript Document Loader] +* https://js.langchain.com/docs/integrations/vectorstores/couchbase[LangChain - JavaScript Vector Stores] +| ✔ +| ✔ +| Partner Supported + +| Hashicorp Terraform +| Deploy and manage your clusters with declarative configuration. +a| +* https://github.com/couchbasecloud/terraform-provider-couchbase-capella/[Couchbase Capella Terraform Provider repo] +* xref:cloud:terraform:index.adoc[Couchbase Capella Terraform Provider docs] +* https://registry.terraform.io/providers/couchbasecloud/couchbase-capella/latest[Capella Provider on the Hashicorp Registry site] +| ✔ +| ✖ +| Partner Supported + | Hashicorp Vault | Securely manage your credentials for database access. a| From 1228c0399a65a1989322a96a90d9f786c4e3de76 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Wed, 13 Mar 2024 16:37:56 -0400 Subject: [PATCH 47/59] [AV-68571] Adding caution to all vector search creation pages based on feedback from SRE --- modules/search/pages/create-quick-index.adoc | 5 ++++- modules/search/pages/create-search-index-rest-api.adoc | 4 +++- modules/search/pages/create-search-index-ui.adoc | 2 ++ modules/search/pages/create-search-indexes.adoc | 4 +++- modules/search/pages/field-data-types-reference.adoc | 2 +- modules/search/pages/geo-search-ui.adoc | 2 ++ modules/search/pages/import-search-index.adoc | 2 ++ .../search/pages/search-query-auto-complete-ui.adoc | 2 ++ modules/search/partials/large-index-caution.adoc | 10 ++++++++++ .../pages/create-vector-search-index-rest-api.adoc | 2 ++ .../pages/create-vector-search-index-ui.adoc | 2 ++ 11 files changed, 33 insertions(+), 4 deletions(-) create mode 100644 modules/search/partials/large-index-caution.adoc diff --git a/modules/search/pages/create-quick-index.adoc b/modules/search/pages/create-quick-index.adoc index 8fa0c038d..fdd1c3b75 100644 --- a/modules/search/pages/create-quick-index.adoc +++ b/modules/search/pages/create-quick-index.adoc @@ -37,7 +37,8 @@ By default, *Quick Mode* should be on. ==== Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). -For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. +You cannot have 2 indexes with the same name inside the same bucket and scope. ==== . Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. . In the *Scope* list, select the scope where you want to create the index. @@ -50,6 +51,8 @@ For more information about the available options, see xref:create-type-mapping.a . (Optional) Add additional collections or field type mappings to your index. . Click btn:[Create Index]. +include::partial$large-index-caution.adoc[tag=regular-search] + == Next Steps If you want to add more advanced settings and customization to your Search index, you can xref:create-search-index-ui.adoc[use Advanced Mode]. diff --git a/modules/search/pages/create-search-index-rest-api.adoc b/modules/search/pages/create-search-index-rest-api.adoc index 3596b8205..577628e35 100644 --- a/modules/search/pages/create-search-index-rest-api.adoc +++ b/modules/search/pages/create-search-index-rest-api.adoc @@ -44,7 +44,9 @@ As of version 7.6, your index name must be unique inside your selected bucket an Do not include the xref:search-index-params.adoc#uuid[uuid] or xref:search-index-params.adoc#sourceuuid[sourceUUID] parameters. + TIP: If you remove the xref:search-index-params.adoc#uuid[uuid] and xref:search-index-params.adoc#sourceuuid[sourceUUID] parameters, you can copy the Search index definition JSON payload from the Couchbase Capella UI to use in your REST API call. -For more information about how to create an index with the UI, see xref:create-search-index-ui.adoc[]. +For more information about how to create an index with the UI, see xref:create-search-index-ui.adoc[]. + +include::partial$large-index-caution.adoc[tag=regular-search] === Example diff --git a/modules/search/pages/create-search-index-ui.adoc b/modules/search/pages/create-search-index-ui.adoc index d41cda3b9..350b41409 100644 --- a/modules/search/pages/create-search-index-ui.adoc +++ b/modules/search/pages/create-search-index-ui.adoc @@ -48,6 +48,8 @@ For Couchbase Server version 7.6 and later, your index name must be unique insid For more information, see xref:customize-index.adoc[]. . Click btn:[Create Index]. +include::partial$large-index-caution.adoc[tag=regular-search] + == Next Steps If you did not create a static type mapping, this basic index includes all documents from the collection you selected. diff --git a/modules/search/pages/create-search-indexes.adoc b/modules/search/pages/create-search-indexes.adoc index c0831ba11..c5498761e 100644 --- a/modules/search/pages/create-search-indexes.adoc +++ b/modules/search/pages/create-search-indexes.adoc @@ -11,6 +11,8 @@ You can create a Search index with: // * The <>. * The <>. +include::partial$large-index-caution.adoc[tag=regular-search] + TIP: If you're new to developing with the Search Service, <>. You can export the Search index definition JSON payload from the UI to create your index xref:create-search-index-rest-api.adoc[with the REST API], or xref:import-search-index.adoc[]. @@ -60,7 +62,7 @@ You can also copy the Search index definition JSON payload from a Search index i [NOTE] ==== -Use the scoped name for an index with the xref:rest-api:rest-fts.adoc[Search Service REST API] for any endpoints that do not include the bucket and scope in their path. +Use the scoped name for an index with the xref:server:rest-api:rest-fts.adoc[Search Service REST API] for any endpoints that do not include the bucket and scope in their path. For example, you must use `bucket.scope.index_name` as the format for your index name with the `analyzeDoc` endpoint, but not with the new 7.6 `query` endpoint. ==== diff --git a/modules/search/pages/field-data-types-reference.adoc b/modules/search/pages/field-data-types-reference.adoc index af21e16b2..a6c72afb3 100644 --- a/modules/search/pages/field-data-types-reference.adoc +++ b/modules/search/pages/field-data-types-reference.adoc @@ -76,7 +76,7 @@ For example: } ---- -|vector a| +|[[vector]]vector a| The field contains an array of floating point numbers. diff --git a/modules/search/pages/geo-search-ui.adoc b/modules/search/pages/geo-search-ui.adoc index c944692fc..d146e1656 100644 --- a/modules/search/pages/geo-search-ui.adoc +++ b/modules/search/pages/geo-search-ui.adoc @@ -58,6 +58,8 @@ This field should contain child fields with `lat` and `long` data. .. Click btn:[Add]. . Click btn:[Create Index] +include::partial$large-index-caution.adoc[tag=regular-search] + [#geospatial-query] === Run a Geospatial Search Query diff --git a/modules/search/pages/import-search-index.adoc b/modules/search/pages/import-search-index.adoc index b49cee636..0c1bc9cbf 100644 --- a/modules/search/pages/import-search-index.adoc +++ b/modules/search/pages/import-search-index.adoc @@ -32,6 +32,8 @@ To import a xref:create-search-indexes.adoc[Search index definition] or xref:ind //For more information, see xref:customize-index.adoc[] or xref:create-search-index-alias.adoc[]. //. Click btn:[Create Index] or btn:[Create Index Alias]. +include::partial$large-index-caution.adoc[tag=regular-search] + == Next Steps To add additional features to your imported Search index, see xref:customize-index.adoc[]. diff --git a/modules/search/pages/search-query-auto-complete-ui.adoc b/modules/search/pages/search-query-auto-complete-ui.adoc index d7787b55e..426d8bdeb 100644 --- a/modules/search/pages/search-query-auto-complete-ui.adoc +++ b/modules/search/pages/search-query-auto-complete-ui.adoc @@ -69,6 +69,8 @@ You cannot have 2 indexes with the same name inside the same bucket and scope. . In the *Default Analyzer* list, select your *keyword_to_lower* analyzer. . Click btn:[Create Index]. +include::partial$large-index-caution.adoc[tag=regular-search] + [#api] === Create an Autocomplete Search Index with the REST API diff --git a/modules/search/partials/large-index-caution.adoc b/modules/search/partials/large-index-caution.adoc new file mode 100644 index 000000000..24bc828f9 --- /dev/null +++ b/modules/search/partials/large-index-caution.adoc @@ -0,0 +1,10 @@ +// tag::regular-search[] + +CAUTION: If you create a Search index that includes xref:search:field-data-types-reference.adoc#vector[a field with the vector type] and 5,000,000 documents or more, you must xref:support:manage-support.adoc[contact Couchbase Capella Support]. + +// end::regular-search[] +// tag::vector-search[] + +CAUTION: If you create a Vector Search index that includes 5,000,000 documents or more, you must xref:support:manage-support.adoc[contact Couchbase Capella Support]. + +// end::vector-search[] \ No newline at end of file diff --git a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc index 60875b3fd..083d0e0b5 100644 --- a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc +++ b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc @@ -45,6 +45,8 @@ Do not include the xref:search:search-index-params.adoc#uuid[uuid] or xref:searc TIP: If you remove the xref:search:search-index-params.adoc#uuid[uuid] and xref:search:search-index-params.adoc#sourceuuid[sourceUUID] parameters, you can copy the Search index definition JSON payload from the Capella UI to use in a REST API call. For more information about how to create a Vector Search index with the UI, see xref:create-vector-search-index-ui.adoc[]. +include::search:partial$large-index-caution.adoc[tag=vector-search] + === Example In the following example, the JSON payload creates an index named `color-index` on the `vector-sample.color.rgb` keyspace. diff --git a/modules/vector-search/pages/create-vector-search-index-ui.adoc b/modules/vector-search/pages/create-vector-search-index-ui.adoc index 19be49ddf..e00412629 100644 --- a/modules/vector-search/pages/create-vector-search-index-ui.adoc +++ b/modules/vector-search/pages/create-vector-search-index-ui.adoc @@ -72,6 +72,8 @@ For more information, see xref:search:quick-index-field-options.adoc#quick-field For example, you could add the text field that you used to generate your vector embeddings. . Click btn:[Create Index]. +include::search:partial$large-index-caution.adoc[tag=vector-search] + == Next Steps This basic Vector Search index includes the vector embeddings from the child field you specified in your type mapping. From abdde93870b2a959c65ac862252b680a67c732d6 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 14 Mar 2024 10:45:15 -0400 Subject: [PATCH 48/59] [AV-68571] Adding attributes to handle product and UI names to future-proof updates. Adding proper expansion formatting to variables in REST API paths/curl commands --- antora.yml | 6 +++- .../examples/create-search-index-header.sh | 2 +- .../examples/create-search-index-payload.sh | 2 +- modules/search/examples/run-search-header.sh | 2 +- modules/search/examples/run-search-payload.sh | 2 +- .../pages/child-field-options-reference.adoc | 4 ++- modules/search/pages/create-child-field.adoc | 8 ++++-- .../search/pages/create-child-mapping.adoc | 8 ++++-- .../search/pages/create-custom-analyzer.adoc | 8 ++++-- .../pages/create-custom-character-filter.adoc | 8 ++++-- .../pages/create-custom-date-time-parser.adoc | 8 ++++-- .../pages/create-custom-token-filter.adoc | 28 ++++++++++--------- .../search/pages/create-custom-tokenizer.adoc | 10 ++++--- .../search/pages/create-custom-wordlist.adoc | 8 ++++-- modules/search/pages/create-quick-index.adoc | 4 ++- .../pages/create-search-index-alias.adoc | 8 ++++-- .../pages/create-search-index-rest-api.adoc | 4 ++- .../search/pages/create-search-index-ui.adoc | 4 ++- .../search/pages/create-search-indexes.adoc | 14 ++++++---- modules/search/pages/create-type-mapping.adoc | 8 ++++-- modules/search/pages/customize-index.adoc | 2 ++ .../pages/default-analyzers-reference.adoc | 2 ++ .../default-character-filters-reference.adoc | 2 ++ .../default-token-filters-reference.adoc | 2 ++ .../pages/default-tokenizers-reference.adoc | 2 ++ .../pages/default-wordlists-reference.adoc | 2 ++ .../pages/field-data-types-reference.adoc | 2 ++ modules/search/pages/geo-search-rest-api.adoc | 4 ++- modules/search/pages/geo-search-ui.adoc | 14 ++++++---- modules/search/pages/import-search-index.adoc | 10 ++++--- modules/search/pages/index-aliases.adoc | 2 ++ .../pages/quick-index-field-options.adoc | 4 ++- .../quick-index-supported-languages.adoc | 4 ++- modules/search/pages/run-searches.adoc | 12 ++++---- modules/search/pages/search-index-params.adoc | 4 ++- .../search-query-auto-complete-code.adoc | 4 ++- .../pages/search-query-auto-complete-ui.adoc | 14 ++++++---- .../pages/search-query-auto-complete.adoc | 2 ++ .../search/pages/search-request-params.adoc | 6 ++-- modules/search/pages/search.adoc | 6 ++-- .../search/pages/set-advanced-settings.adoc | 8 ++++-- modules/search/pages/set-type-identifier.adoc | 6 ++-- .../search/pages/simple-search-rest-api.adoc | 4 ++- modules/search/pages/simple-search-ui.adoc | 10 ++++--- .../create-vector-search-index-header.sh | 2 +- .../create-vector-search-index-payload.sh | 2 +- .../examples/run-vector-search-header.sh | 2 +- .../examples/run-vector-search-payload.sh | 2 +- .../create-vector-search-index-rest-api.adoc | 4 ++- .../pages/create-vector-search-index-ui.adoc | 4 ++- .../pages/run-vector-search-python-sdk.adoc | 2 ++ .../pages/run-vector-search-rest-api.adoc | 4 ++- .../pages/run-vector-search-ui.adoc | 10 ++++--- .../vector-search/pages/vector-search.adoc | 16 ++++++----- 54 files changed, 208 insertions(+), 114 deletions(-) diff --git a/antora.yml b/antora.yml index b1093ef32..c55430021 100644 --- a/antora.yml +++ b/antora.yml @@ -1,2 +1,6 @@ name: cloud -version: ~ \ No newline at end of file +version: ~ +asciidoc: + attributes: + ui-name: Capella UI + product-name: Capella \ No newline at end of file diff --git a/modules/search/examples/create-search-index-header.sh b/modules/search/examples/create-search-index-header.sh index 97a651624..2056b8e36 100644 --- a/modules/search/examples/create-search-index-header.sh +++ b/modules/search/examples/create-search-index-header.sh @@ -1,3 +1,3 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index -d \ \ No newline at end of file diff --git a/modules/search/examples/create-search-index-payload.sh b/modules/search/examples/create-search-index-payload.sh index 9e2f5c409..8ee2df3a4 100644 --- a/modules/search/examples/create-search-index-payload.sh +++ b/modules/search/examples/create-search-index-payload.sh @@ -1,5 +1,5 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index -d \ '{ "type": "fulltext-index", diff --git a/modules/search/examples/run-search-header.sh b/modules/search/examples/run-search-header.sh index eda474783..805dfcff3 100644 --- a/modules/search/examples/run-search-header.sh +++ b/modules/search/examples/run-search-header.sh @@ -1,2 +1,2 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/$BUCKET-NAME/scope/$SCOPE-NAME/index/$INDEX-NAME/query -d \ \ No newline at end of file + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/${BUCKET-NAME}/scope/${SCOPE-NAME}/index/${INDEX-NAME}/query -d \ \ No newline at end of file diff --git a/modules/search/examples/run-search-payload.sh b/modules/search/examples/run-search-payload.sh index 0eb9dce95..3ccc0d6b8 100644 --- a/modules/search/examples/run-search-payload.sh +++ b/modules/search/examples/run-search-payload.sh @@ -1,5 +1,5 @@ curl -XPOST -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index/query \ + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index/query \ -d '{ "explain": true, "fields": [ diff --git a/modules/search/pages/child-field-options-reference.adoc b/modules/search/pages/child-field-options-reference.adoc index 0c2d3be94..b48f19392 100644 --- a/modules/search/pages/child-field-options-reference.adoc +++ b/modules/search/pages/child-field-options-reference.adoc @@ -1,6 +1,8 @@ = Child Field Options :page-topic-type: reference -:description: You can configure a child field from the Couchbase Capella UI's Advanced Mode to control how its data is added to your Search index. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: You can configure a child field from the Couchbase {page-ui-name}'s Advanced Mode to control how its data is added to your Search index. [abstract] {description} diff --git a/modules/search/pages/create-child-field.adoc b/modules/search/pages/create-child-field.adoc index 4bd09179d..a3415222b 100644 --- a/modules/search/pages/create-child-field.adoc +++ b/modules/search/pages/create-child-field.adoc @@ -1,6 +1,8 @@ = Create a Child Field in Advanced Mode :page-topic-type: guide -:description: Create a child field with the Couchbase Capella UI's Advanced Mode to add or remove a specific field's content from a Search index. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a child field with the Couchbase {page-ui-name}'s Advanced Mode to add or remove a specific field's content from a Search index. [abstract] {description} @@ -16,13 +18,13 @@ For more information, see xref:create-search-index-ui.adoc[]. * You have created a type mapping. For more information about how to create a type mapping on an index, see xref:create-type-mapping.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. * You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To add or remove a child field from a Search index with the Capella UI: +To add or remove a child field from a Search index with the {page-ui-name}: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/create-child-mapping.adoc b/modules/search/pages/create-child-mapping.adoc index b9297c141..6befabe96 100644 --- a/modules/search/pages/create-child-mapping.adoc +++ b/modules/search/pages/create-child-mapping.adoc @@ -1,6 +1,8 @@ = Create a Child Mapping in Advanced Mode :page-topic-type: guide -:description: Create a child mapping with the Couchbase Capella UI's Advanced Mode to add or remove a field that contains a JSON object from a Search index. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a child mapping with the Couchbase {page-ui-name}'s Advanced Mode to add or remove a field that contains a JSON object from a Search index. [abstract] {description} @@ -16,13 +18,13 @@ For more information, see xref:create-search-index-ui.adoc[]. * You have created a type mapping. For more information about how to create a type mapping on an index, see xref:create-type-mapping.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. * You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a child mapping with the Capella UI: +To create a child mapping with the {page-ui-name}: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/create-custom-analyzer.adoc b/modules/search/pages/create-custom-analyzer.adoc index 67985a841..c5e38fe63 100644 --- a/modules/search/pages/create-custom-analyzer.adoc +++ b/modules/search/pages/create-custom-analyzer.adoc @@ -1,6 +1,8 @@ = Create a Custom Analyzer :page-topic-type: guide -:description: Create a custom analyzer with the Couchbase Capella UI's Advanced Mode to modify the input text from a Search query or Search index and improve search results. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a custom analyzer with the Couchbase {page-ui-name}'s Advanced Mode to modify the input text from a Search query or Search index and improve search results. [abstract] {description} @@ -17,11 +19,11 @@ For more information about how to change Services on your database, see xref:clo * You have created an index with xref:create-search-index-ui.adoc[Advanced Mode]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To create a custom analyzer with the Capella UI in Advanced Mode: +To create a custom analyzer with the {page-ui-name} in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/create-custom-character-filter.adoc b/modules/search/pages/create-custom-character-filter.adoc index 4424c2d50..db2d3e534 100644 --- a/modules/search/pages/create-custom-character-filter.adoc +++ b/modules/search/pages/create-custom-character-filter.adoc @@ -1,6 +1,8 @@ = Create a Custom Character Filter :page-topic-type: guide -:description: Create a custom character filter with the Couchbase Capella UI to remove unwanted characters from a Search query or the contents of a Search index. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a custom character filter with the Couchbase {page-ui-name} to remove unwanted characters from a Search query or the contents of a Search index. [abstract] {description} @@ -10,13 +12,13 @@ * You have the Search Service enabled on a node in your database. For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. * You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a custom character filter with the Capella UI in Advanced Mode: +To create a custom character filter with the {page-ui-name} in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/create-custom-date-time-parser.adoc b/modules/search/pages/create-custom-date-time-parser.adoc index 81583b3bb..755015c4d 100644 --- a/modules/search/pages/create-custom-date-time-parser.adoc +++ b/modules/search/pages/create-custom-date-time-parser.adoc @@ -1,6 +1,8 @@ = Create a Custom Date/Time Parser :page-topic-type: guide -:description: Create a custom date/time parser with the Couchbase Capella UI to tell the Search Service how to process a new date/time format. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a custom date/time parser with the Couchbase {page-ui-name} to tell the Search Service how to process a new date/time format. [abstract] {description} @@ -12,13 +14,13 @@ If you store date data in a format other than RFC-3339 (ISO-8601), then you need * You have the Search Service enabled on a node in your database. For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. * You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a custom date/time parser with the Capella UI in Advanced Mode: +To create a custom date/time parser with the {page-ui-name} in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/create-custom-token-filter.adoc b/modules/search/pages/create-custom-token-filter.adoc index 30479f282..524b2d2a6 100644 --- a/modules/search/pages/create-custom-token-filter.adoc +++ b/modules/search/pages/create-custom-token-filter.adoc @@ -1,6 +1,8 @@ = Create a Custom Token Filter :page-topic-type: guide -:description: Create a custom token filter with the Couchbase Capella UI to change how the Search Service creates tokens from Search index content and Search queries. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a custom token filter with the Couchbase {page-ui-name} to change how the Search Service creates tokens from Search index content and Search queries. :page-toclevels: 3 [abstract] @@ -13,13 +15,13 @@ xref:customize-index.adoc#token-filters[Token filters] can improve your search r * You have the Search Service enabled on a node in your database. For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. * You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a custom token filter with the Capella UI in Advanced Mode: +To create a custom token filter with the {page-ui-name} in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. @@ -72,7 +74,7 @@ You can create any of the following custom token filters: include::partial$custom-token-filters-descriptions.adoc[tags=dict;!dict_example] -To create a new `dict_compound` token filter with the Capella UI in Advanced Mode: +To create a new `dict_compound` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *dict_compound*. . In the *Sub Words* list, select the wordlist to use to find subwords in input tokens. @@ -84,7 +86,7 @@ You can choose your own xref:create-custom-wordlist.adoc[custom wordlist] or a x === Create a Custom `edge_ngram` Token Filter include::partial$custom-token-filters-descriptions.adoc[tags=edge;!edge_example] -To create a new `edge_ngram` token filter with the Capella UI in Advanced Mode: +To create a new `edge_ngram` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *edge_ngram*. . Do one of the following: @@ -99,7 +101,7 @@ To create a new `edge_ngram` token filter with the Capella UI in Advanced Mode: include::partial$custom-token-filters-descriptions.adoc[tags=elision;!elision_example] -To create a new `elision` token filter with the Capella UI in Advanced Mode: +To create a new `elision` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *elision*. . In the *Articles* list, select a wordlist to use to find elisions in input tokens. @@ -112,7 +114,7 @@ You can choose your own xref:create-custom-wordlist.adoc[custom wordlist] or a x include::partial$custom-token-filters-descriptions.adoc[tags=keyword;!keyword_example] -To create a new `keyword_marker` token filter with the Capella UI in Advanced Mode: +To create a new `keyword_marker` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *keyword_marker*. . In the *Articles* list, select a wordlist to use to find keywords to create tokens. @@ -125,7 +127,7 @@ You can choose your own xref:create-custom-wordlist.adoc[custom wordlist] or a x include::partial$custom-token-filters-descriptions.adoc[tags=length;!length_example] -To create a new `length` token filter with the Capella UI in Advanced Mode: +To create a new `length` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *length*. . In the *Min* box, enter the minimum character length for a new token. @@ -137,7 +139,7 @@ To create a new `length` token filter with the Capella UI in Advanced Mode: include::partial$custom-token-filters-descriptions.adoc[tags=ngram;!ngram_example] -To create a new `ngram` token filter with the Capella UI in Advanced Mode: +To create a new `ngram` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *ngram*. . In the *Min* box, enter the minimum character length for a new token. @@ -149,7 +151,7 @@ To create a new `ngram` token filter with the Capella UI in Advanced Mode: include::partial$custom-token-filters-descriptions.adoc[tags=normalize;!normalize_example] -To create a new `normalize_unicode` token filter with the Capella UI in Advanced Mode: +To create a new `normalize_unicode` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *normalize_unicode*. . In the *Form* list, select the type of Unicode normalization to apply: @@ -165,7 +167,7 @@ To create a new `normalize_unicode` token filter with the Capella UI in Advanced include::partial$custom-token-filters-descriptions.adoc[tags=shingle;!shingle_example] -To create a new `shingle` token filter with the Capella UI in Advanced Mode: +To create a new `shingle` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *shingle*. . In the *Min* box, enter the minimum character length for a new token before concatenation. @@ -182,7 +184,7 @@ To create a new `shingle` token filter with the Capella UI in Advanced Mode: include::partial$custom-token-filters-descriptions.adoc[tags=stop;!stop_example] -To create a new `stop_tokens` token filter with the Capella UI in Advanced Mode: +To create a new `stop_tokens` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *stop_tokens*. . In the *Stop Words* list, select a wordlist to use to remove tokens. @@ -195,7 +197,7 @@ You can choose your own xref:create-custom-wordlist.adoc[custom wordlist] or a x include::partial$custom-token-filters-descriptions.adoc[tags=truncate;!truncate_example] -To create a new `truncate_token` token filter with the Capella UI in Advanced Mode: +To create a new `truncate_token` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *truncate_token*. . In the *Length* box, enter the maximum character length for an output token. diff --git a/modules/search/pages/create-custom-tokenizer.adoc b/modules/search/pages/create-custom-tokenizer.adoc index 0d780250a..98d50f617 100644 --- a/modules/search/pages/create-custom-tokenizer.adoc +++ b/modules/search/pages/create-custom-tokenizer.adoc @@ -1,6 +1,8 @@ = Create a Custom Tokenizer :page-topic-type: guide -:description: Create a custom tokenizer with the Couchbase Capella UI to change how the Search Service creates tokens for matching Search index content to a Search query. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a custom tokenizer with the Couchbase {page-ui-name} to change how the Search Service creates tokens for matching Search index content to a Search query. :page-toclevels: 3 [abstract] @@ -12,7 +14,7 @@ For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. * You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. @@ -32,7 +34,7 @@ You can create 2 types of custom tokenizers: [#regexp] === Create a Regular Expression Tokenizer -To create a regular expression tokenizer with the Capella UI: +To create a regular expression tokenizer with the {page-ui-name}: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. @@ -49,7 +51,7 @@ NOTE: Make sure you use *Advanced Mode*. [#excep] === Create an Exception Custom Tokenizer -To create an exception custom tokenizer with the Capella UI in Advanced Mode: +To create an exception custom tokenizer with the {page-ui-name} in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/create-custom-wordlist.adoc b/modules/search/pages/create-custom-wordlist.adoc index 67c52bcde..166e21074 100644 --- a/modules/search/pages/create-custom-wordlist.adoc +++ b/modules/search/pages/create-custom-wordlist.adoc @@ -1,6 +1,8 @@ = Create a Custom Wordlist :page-topic-type: guide -:description: Create a custom wordlist with the Couchbase Capella UI to use with a custom token filter. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a custom wordlist with the Couchbase {page-ui-name} to use with a custom token filter. [abstract] {description} @@ -15,13 +17,13 @@ For more information about how to create a custom token filter, see xref:create- For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. * You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a custom wordlist with the Capella UI in Advanced Mode: +To create a custom wordlist with the {page-ui-name} in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/create-quick-index.adoc b/modules/search/pages/create-quick-index.adoc index fdd1c3b75..9fac8e6e7 100644 --- a/modules/search/pages/create-quick-index.adoc +++ b/modules/search/pages/create-quick-index.adoc @@ -1,5 +1,7 @@ = Create a Search Index in Quick Mode :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Use Quick Mode to create a Search index in Couchbase Capella with streamlined options. [abstract] @@ -20,7 +22,7 @@ For more information about how to change Services on your database, see xref:clo * You have a bucket with scopes and collections in your database. For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure diff --git a/modules/search/pages/create-search-index-alias.adoc b/modules/search/pages/create-search-index-alias.adoc index 520786b43..fb7f48054 100644 --- a/modules/search/pages/create-search-index-alias.adoc +++ b/modules/search/pages/create-search-index-alias.adoc @@ -1,5 +1,7 @@ -= Create a Search Index Alias with the Capella UI += Create a Search Index Alias with the {page-ui-name} :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Use a Search index alias to run a Search query across multiple buckets, scopes, or Search indexes. [abstract] @@ -15,11 +17,11 @@ For more information about how to change Services on your database, see xref:clo * You have created at least one Search index. For more information, see xref:create-search-index-ui.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To create a Search index alias with the Capella UI: +To create a Search index alias with the {page-ui-name}: . On the *Databases* page, select the database where you want to create a Search index alias. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/create-search-index-rest-api.adoc b/modules/search/pages/create-search-index-rest-api.adoc index 577628e35..307648375 100644 --- a/modules/search/pages/create-search-index-rest-api.adoc +++ b/modules/search/pages/create-search-index-rest-api.adoc @@ -1,5 +1,7 @@ = Create a Search Index with the REST API and curl/HTTP :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: You can create a Search index with the Search Service API. [abstract] @@ -43,7 +45,7 @@ As of version 7.6, your index name must be unique inside your selected bucket an + Do not include the xref:search-index-params.adoc#uuid[uuid] or xref:search-index-params.adoc#sourceuuid[sourceUUID] parameters. + -TIP: If you remove the xref:search-index-params.adoc#uuid[uuid] and xref:search-index-params.adoc#sourceuuid[sourceUUID] parameters, you can copy the Search index definition JSON payload from the Couchbase Capella UI to use in your REST API call. +TIP: If you remove the xref:search-index-params.adoc#uuid[uuid] and xref:search-index-params.adoc#sourceuuid[sourceUUID] parameters, you can copy the Search index definition JSON payload from the Couchbase {page-ui-name} to use in your REST API call. For more information about how to create an index with the UI, see xref:create-search-index-ui.adoc[]. include::partial$large-index-caution.adoc[tag=regular-search] diff --git a/modules/search/pages/create-search-index-ui.adoc b/modules/search/pages/create-search-index-ui.adoc index 350b41409..23e6932b8 100644 --- a/modules/search/pages/create-search-index-ui.adoc +++ b/modules/search/pages/create-search-index-ui.adoc @@ -1,5 +1,7 @@ = Create a Basic Search Index in Advanced Mode :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Use Advanced Mode to create a Search index in Couchbase Capella with full customization and advanced features. [abstract] @@ -17,7 +19,7 @@ For more information about how to change Services on your database, see xref:clo * You have a bucket with scopes and collections in your database. For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure diff --git a/modules/search/pages/create-search-indexes.adoc b/modules/search/pages/create-search-indexes.adoc index c5498761e..71e457227 100644 --- a/modules/search/pages/create-search-indexes.adoc +++ b/modules/search/pages/create-search-indexes.adoc @@ -1,5 +1,7 @@ = Create a Search Index :page-topic-type: concept +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Create a Search index to get started with the Search Service in your database. [abstract] @@ -7,7 +9,7 @@ You can create a Search index with: -* The <>. +* The <>. // * The <>. * The <>. @@ -17,7 +19,7 @@ TIP: If you're new to developing with the Search Service, <>. -Then, <>. +Then, <>. [#geospatial-index] === Create a Search Index with a Geospatial Type Mapping diff --git a/modules/search/pages/geo-search-ui.adoc b/modules/search/pages/geo-search-ui.adoc index d146e1656..39cce600a 100644 --- a/modules/search/pages/geo-search-ui.adoc +++ b/modules/search/pages/geo-search-ui.adoc @@ -1,6 +1,8 @@ -= Run a Geospatial Search Query with the Capella UI += Run a Geospatial Search Query with the {page-ui-name} :page-topic-type: guide -:description: Search for geospatial data in your Couchbase Capella database with a compatible Search index and the Capella UI. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Search for geospatial data in your Couchbase Capella database with a compatible Search index and the {page-ui-name}. [abstract] {description} @@ -15,18 +17,18 @@ For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[] * You have documents in your database that contain geospatial data. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure To run a geospatial Search query, <>. -Then, <>. +Then, <>. [#geospatial-index] === Create a Search Index with a Geospatial Type Mapping -To create the Search index in the Capella UI with Advanced Mode: +To create the Search index in the {page-ui-name} with Advanced Mode: . On the *Databases* page, select the database where you want to create a Search index. . Go to menu:Data Tools[Search]. @@ -63,7 +65,7 @@ include::partial$large-index-caution.adoc[tag=regular-search] [#geospatial-query] === Run a Geospatial Search Query -To run a Search query against the Search index from the Capella UI: +To run a Search query against the Search index from the {page-ui-name}: . Next to your <>, click btn:[Search]. . In the *Search* field, enter a search query for geospatial data. diff --git a/modules/search/pages/import-search-index.adoc b/modules/search/pages/import-search-index.adoc index 0c1bc9cbf..161ff4552 100644 --- a/modules/search/pages/import-search-index.adoc +++ b/modules/search/pages/import-search-index.adoc @@ -1,6 +1,8 @@ -= Import a Search Index Definition with the Capella UI += Import a Search Index Definition with the {page-ui-name} :page-topic-type: guide -:description: Use the Couchbase Capella UI to import a JSON Search index definition or Search index alias. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Use the Couchbase {page-ui-name} to import a JSON Search index definition or Search index alias. [abstract] {description} @@ -13,11 +15,11 @@ For more information about how to change Services on your database, see xref:clo * You have a bucket with scopes and collections in your database. For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To import a xref:create-search-indexes.adoc[Search index definition] or xref:index-aliases.adoc[Search alias] with the Capella UI: +To import a xref:create-search-indexes.adoc[Search index definition] or xref:index-aliases.adoc[Search alias] with the {page-ui-name}: . On the *Databases* page, select the database where you want to import a Search index or index alias. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/index-aliases.adoc b/modules/search/pages/index-aliases.adoc index b92005842..7924e3fff 100644 --- a/modules/search/pages/index-aliases.adoc +++ b/modules/search/pages/index-aliases.adoc @@ -1,5 +1,7 @@ = Create Search Index Aliases :page-topic-type: concept +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: A Search index alias lets you run a Search query against a Search index without using the original Search index name. [abstract] diff --git a/modules/search/pages/quick-index-field-options.adoc b/modules/search/pages/quick-index-field-options.adoc index 5ef063fb9..84a2efc6b 100644 --- a/modules/search/pages/quick-index-field-options.adoc +++ b/modules/search/pages/quick-index-field-options.adoc @@ -1,6 +1,8 @@ = Quick Index Mapping and Field Options :page-topic-type: reference -:description: When you create a Search index with Quick Mode in the Capella UI, you must set options for each field or type mapping you add to the index. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: When you create a Search index with Quick Mode in the {page-ui-name}, you must set options for each field or type mapping you add to the index. [abstract] {description} diff --git a/modules/search/pages/quick-index-supported-languages.adoc b/modules/search/pages/quick-index-supported-languages.adoc index d407b86f7..bcf9a111d 100644 --- a/modules/search/pages/quick-index-supported-languages.adoc +++ b/modules/search/pages/quick-index-supported-languages.adoc @@ -1,6 +1,8 @@ = Quick Mode Supported Languages :page-topic-type: reference -:description: When you create a Search index with Quick Mode in Couchbase Capella, you need to specify a language for your text fields. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: When you create a Search index with Quick Mode in Couchbase {page-product-name}, you need to specify a language for your text fields. [abstract] {description} diff --git a/modules/search/pages/run-searches.adoc b/modules/search/pages/run-searches.adoc index 5101c20e8..ce5bf5888 100644 --- a/modules/search/pages/run-searches.adoc +++ b/modules/search/pages/run-searches.adoc @@ -1,5 +1,7 @@ = Run a Search With a Search Index :page-topic-type: concept +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Run a Search query to search and return the contents of a Search index. [abstract] @@ -9,7 +11,7 @@ NOTE: You need to xref:create-search-indexes.adoc[create a Search index] before You can run a search against a Search index with: -* The <>. +* The <>. * The <>. * A <>. * The Couchbase SDKs: @@ -19,13 +21,13 @@ include::partial$sdks-fts-links.adoc[] To run a Search query against multiple Search indexes at once, xref:create-search-index-alias.adoc[]. [#ui] -== Run a Search with the Capella UI +== Run a Search with the {page-ui-name} -You can use the Capella UI to test your Search index before you integrate search into your application. +You can use the {page-ui-name} to test your Search index before you integrate search into your application. -You can enter a basic search query in the Capella UI, or use a xref:search-request-params.adoc#query[query object] and other JSON properties for a more complex search. +You can enter a basic search query in the {page-ui-name}, or use a xref:search-request-params.adoc#query[query object] and other JSON properties for a more complex search. -For more information about how to run a search with the Capella UI, see xref:simple-search-ui.adoc[]. +For more information about how to run a search with the {page-ui-name}, see xref:simple-search-ui.adoc[]. For more information about how to configure a Search index and search for geospatial data, see xref:geo-search-ui.adoc[]. diff --git a/modules/search/pages/search-index-params.adoc b/modules/search/pages/search-index-params.adoc index 92a16e2a0..fe7f8f70d 100644 --- a/modules/search/pages/search-index-params.adoc +++ b/modules/search/pages/search-index-params.adoc @@ -1,5 +1,7 @@ = Search Index JSON Properties :page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Use a JSON payload to control the settings for a Search index. :page-toclevels: 2 @@ -49,7 +51,7 @@ The Search Service automatically generates a UUID for a Search index. If you use an existing UUID, the Search Service updates the existing Search index. Do not include the `uuid` property when you want to copy an index to a different cluster or create a new index. -View the UUID for an existing index from the Capella UI by selecting an existing index and clicking btn:[Index Definition]. +View the UUID for an existing index from the {page-ui-name} by selecting an existing index and clicking btn:[Index Definition]. The UUID displays in the Index Definition on the Update Index page. |sourceType |String |Yes |The `sourceType` is always `"gocbcore"`. diff --git a/modules/search/pages/search-query-auto-complete-code.adoc b/modules/search/pages/search-query-auto-complete-code.adoc index f065b376b..144323985 100644 --- a/modules/search/pages/search-query-auto-complete-code.adoc +++ b/modules/search/pages/search-query-auto-complete-code.adoc @@ -1,5 +1,7 @@ = Add Autocomplete to Your Application :description: Use autocomplete to add suggestions for a user's Search query as they type in your application. +:page-ui-name: {ui-name} +:page-product-name: {product-name} :page-topic-type: guide :tabs: @@ -29,7 +31,7 @@ To add autocomplete with the Search Service to your application: . To test that your Search index was configured correctly, do one of the following: .. xref:simple-search-rest-api.adoc[Run a Search query from the REST API] with 2-8 characters in the `query` property. -.. xref:simple-search-ui.adoc[Run a Search query from the Capella UI] with 2-8 characters in the *Search* field. +.. xref:simple-search-ui.adoc[Run a Search query from the {page-ui-name}] with 2-8 characters in the *Search* field. + For example, with the `travel-sample` bucket, you could enter the strings `Be`, `Bea`, `Beau`, and `Beauf` to find a document with the text `Beaufort Hotel`. diff --git a/modules/search/pages/search-query-auto-complete-ui.adoc b/modules/search/pages/search-query-auto-complete-ui.adoc index 426d8bdeb..37ba5c19a 100644 --- a/modules/search/pages/search-query-auto-complete-ui.adoc +++ b/modules/search/pages/search-query-auto-complete-ui.adoc @@ -1,6 +1,8 @@ = Configure an Autocomplete Search Index -:page-topic-type: guide -:description: Create a Search index with the Capella UI or the REST API to start using autocomplete with the Search Service. +:page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a Search index with the {page-ui-name} or the REST API to start using autocomplete with the Search Service. [abstract] {description} @@ -13,17 +15,17 @@ For more information about how to change Services on your database, see xref:clo * You have a bucket with scopes and collections in your database. For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure -You can create a compatible Search index with the <> or the <>. +You can create a compatible Search index with the <> or the <>. [#ui] -=== Create an Autocomplete Search Index with the Capella UI +=== Create an Autocomplete Search Index with the {page-ui-name} -To create the Search index in the Capella UI with Advanced Mode: +To create the Search index in the {page-ui-name} with Advanced Mode: . On the *Databases* page, select the database where you want to create the Search index. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/search-query-auto-complete.adoc b/modules/search/pages/search-query-auto-complete.adoc index df89d026e..fac969494 100644 --- a/modules/search/pages/search-query-auto-complete.adoc +++ b/modules/search/pages/search-query-auto-complete.adoc @@ -1,5 +1,7 @@ = Use Autocomplete with the Search Service :page-topic-type: concept +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Add autocomplete to your application to provide a search engine-like experience for your database. [abstract] diff --git a/modules/search/pages/search-request-params.adoc b/modules/search/pages/search-request-params.adoc index bf6a6b25e..8bb7a44ea 100644 --- a/modules/search/pages/search-request-params.adoc +++ b/modules/search/pages/search-request-params.adoc @@ -1,5 +1,7 @@ = Search Request JSON Properties :page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: You can add additional properties to a Search request to control how the Search Service returns results. :page-toclevels: 3 @@ -498,7 +500,7 @@ include::example$query-analytic.jsonc[tag=short_match_phrase] Analytic queries use an analyzer to analyze the contents of your Search query, and find a match in the documents inside your Search index. -For more information about analyzers, see xref:customize-index.adoc#analyzers[Customize a Search Index with the Capella UI]. +For more information about analyzers, see xref:customize-index.adoc#analyzers[Customize a Search Index with the {page-ui-name}]. NOTE: From Couchbase Server 7.6 and later, when you run a Search query with the xref:n1ql:n1ql-language-reference/searchfun.adoc[{sqlpp} Search Function], the analyzer used in your Analytic query does not need to match the analyzer used in the Search index. @@ -553,7 +555,7 @@ include::example$query-analytic.jsonc[tag=match_phrase] Use Non-Analytic queries to run a search query without using an analyzer on the contents of your Search query. -For more information about analyzers, see xref:customize-index.adoc#analyzers[Customize a Search Index with the Capella UI]. +For more information about analyzers, see xref:customize-index.adoc#analyzers[Customize a Search Index with the {page-ui-name}]. NOTE: From Couchbase Server 7.6 and later, when you run a Search query with the xref:n1ql:n1ql-language-reference/searchfun.adoc[{sqlpp} Search Function], you do not need to use the xref:default-analyzers-reference.adoc#keyword[keyword analyzer] in your Search index to run a non-analytic query. diff --git a/modules/search/pages/search.adoc b/modules/search/pages/search.adoc index 00f1f5c60..5d57e1978 100644 --- a/modules/search/pages/search.adoc +++ b/modules/search/pages/search.adoc @@ -1,5 +1,7 @@ = Add Search to Your Application :page-topic-type: concept +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Use the Search Service to create a customizable search experience for your database and your end-user applications. [abstract] @@ -33,7 +35,7 @@ For more information about how to create a Search index, see xref:create-search- You can create a Search index: -* xref:create-search-index-ui.adoc[With the Couchbase Capella UI] +* xref:create-search-index-ui.adoc[With the Couchbase {page-ui-name}] //* With the Couchbase SDKs * xref:create-search-index-rest-api.adoc[From the REST API] @@ -48,7 +50,7 @@ For more information about how you can run a search against a Search index, see You can run a Search query: -* xref:simple-search-ui.adoc[With the Couchbase Capella UI] +* xref:simple-search-ui.adoc[With the Couchbase {page-ui-name}] //* With the Couchbase SDKs * xref:simple-search-rest-api.adoc[From the REST API] diff --git a/modules/search/pages/set-advanced-settings.adoc b/modules/search/pages/set-advanced-settings.adoc index 7d5605b24..6b9ba0dd5 100644 --- a/modules/search/pages/set-advanced-settings.adoc +++ b/modules/search/pages/set-advanced-settings.adoc @@ -1,6 +1,8 @@ = Set Search Index General Settings :page-topic-type: guide -:description: Configure general settings with the Couchbase Capella UI for a Search index to improve an index's search results and performance. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Configure general settings with the Couchbase {page-ui-name} for a Search index to improve an index's search results and performance. [abstract] {description} @@ -15,11 +17,11 @@ For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[] * You have created an index with xref:create-search-index-ui.adoc[Advanced Mode]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To set general settings for a Search index with the Capella UI: +To set general settings for a Search index with the {page-ui-name}: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/set-type-identifier.adoc b/modules/search/pages/set-type-identifier.adoc index bd9787d35..12af81c8c 100644 --- a/modules/search/pages/set-type-identifier.adoc +++ b/modules/search/pages/set-type-identifier.adoc @@ -1,5 +1,7 @@ = Set a Search Index Type Identifier :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Use a type identifier with a type mapping to add an extra filter to the documents you want to include in a Search index. :page-toclevels: 3 @@ -22,11 +24,11 @@ For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[] * You have created at least one type mapping in your Search index. For more information, see xref:create-type-mapping.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To set a type identifier for a Search index with the Capella UI: +To set a type identifier for a Search index with the {page-ui-name}: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/simple-search-rest-api.adoc b/modules/search/pages/simple-search-rest-api.adoc index b87022cfd..3f56aad57 100644 --- a/modules/search/pages/simple-search-rest-api.adoc +++ b/modules/search/pages/simple-search-rest-api.adoc @@ -1,5 +1,7 @@ = Run a Simple Search with the REST API and curl/HTTP :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: You can use the REST API and a curl command to run a search against a Search index. [abstract] @@ -33,7 +35,7 @@ include::example$run-search-header.sh[] . Enter the JSON payload for your query. + -TIP: You can copy the JSON for a Query Request from the Couchbase Server or Couchbase Capella UI to use in your REST API call. +TIP: You can copy the JSON for a Query Request from the Couchbase Server or Couchbase {page-ui-name} to use in your REST API call. For more information about how to perform a search with the UI, see xref:simple-search-ui.adoc[]. === Example diff --git a/modules/search/pages/simple-search-ui.adoc b/modules/search/pages/simple-search-ui.adoc index 51a1ea5c7..2f701c061 100644 --- a/modules/search/pages/simple-search-ui.adoc +++ b/modules/search/pages/simple-search-ui.adoc @@ -1,6 +1,8 @@ -= Run A Simple Search with the Capella UI += Run A Simple Search with the {page-ui-name} :page-topic-type: guide -:description: Run a Search query from the Couchbase Capella UI to preview the search results from a Search index. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Run a Search query from the Couchbase {page-ui-name} to preview the search results from a Search index. [abstract] {description} @@ -14,11 +16,11 @@ For more information about how to change Services on your database, see xref:clo + For more information about how to create a Search index, see xref:create-search-indexes.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To run a simple search with the Capella UI: +To run a simple search with the {page-ui-name}: . On the *Databases* page, select the database where you created your Search index. . Go to menu:Data Tools[Search]. diff --git a/modules/vector-search/examples/create-vector-search-index-header.sh b/modules/vector-search/examples/create-vector-search-index-header.sh index 40a8e4df3..ab11fa4dc 100644 --- a/modules/vector-search/examples/create-vector-search-index-header.sh +++ b/modules/vector-search/examples/create-vector-search-index-header.sh @@ -1,3 +1,3 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/$BUCKET_NAME/scope/$SCOPE_NAME/index/$INDEX_NAME + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/${BUCKET_NAME}/scope/${SCOPE_NAME}/index/${INDEX_NAME} -d \ \ No newline at end of file diff --git a/modules/vector-search/examples/create-vector-search-index-payload.sh b/modules/vector-search/examples/create-vector-search-index-payload.sh index 695d1057e..5e939aceb 100644 --- a/modules/vector-search/examples/create-vector-search-index-payload.sh +++ b/modules/vector-search/examples/create-vector-search-index-payload.sh @@ -1,5 +1,5 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/vector-sample/scope/color/index/color-index + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/vector-sample/scope/color/index/color-index -d \ '{ "type": "fulltext-index", diff --git a/modules/vector-search/examples/run-vector-search-header.sh b/modules/vector-search/examples/run-vector-search-header.sh index aeb431f85..19417565c 100644 --- a/modules/vector-search/examples/run-vector-search-header.sh +++ b/modules/vector-search/examples/run-vector-search-header.sh @@ -1,2 +1,2 @@ curl -s -XPUT -H "Content-Type: application/json" \ --u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/$BUCKET_NAME/scope/$SCOPE_NAME/index/$INDEX_NAME/query -d \ \ No newline at end of file +-u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/${BUCKET_NAM}E/scope/${SCOPE_NAME}/index/${INDEX_NAME}/query -d \ \ No newline at end of file diff --git a/modules/vector-search/examples/run-vector-search-payload.sh b/modules/vector-search/examples/run-vector-search-payload.sh index f72ffbd59..40ff53fc9 100644 --- a/modules/vector-search/examples/run-vector-search-payload.sh +++ b/modules/vector-search/examples/run-vector-search-payload.sh @@ -1,5 +1,5 @@ curl -XPOST -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD https://$CB_HOSTNAME:18094/api/bucket/vector-sample/scope/color/index/color-index/query \ + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/vector-sample/scope/color/index/color-index/query \ -d '{ "fields": ["*"], "query": { diff --git a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc index 083d0e0b5..08449c2fe 100644 --- a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc +++ b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc @@ -1,5 +1,7 @@ = Create a Vector Search Index with the REST API and curl/HTTP :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: You can create a Vector Search index with the Search Service API and Couchbase Capella. [abstract] @@ -42,7 +44,7 @@ include::example$create-vector-search-index-header.sh[] + Do not include the xref:search:search-index-params.adoc#uuid[uuid] or xref:search:search-index-params.adoc#sourceuuid[sourceUUID] parameters. + -TIP: If you remove the xref:search:search-index-params.adoc#uuid[uuid] and xref:search:search-index-params.adoc#sourceuuid[sourceUUID] parameters, you can copy the Search index definition JSON payload from the Capella UI to use in a REST API call. +TIP: If you remove the xref:search:search-index-params.adoc#uuid[uuid] and xref:search:search-index-params.adoc#sourceuuid[sourceUUID] parameters, you can copy the Search index definition JSON payload from the {page-ui-name} to use in a REST API call. For more information about how to create a Vector Search index with the UI, see xref:create-vector-search-index-ui.adoc[]. include::search:partial$large-index-caution.adoc[tag=vector-search] diff --git a/modules/vector-search/pages/create-vector-search-index-ui.adoc b/modules/vector-search/pages/create-vector-search-index-ui.adoc index e00412629..380ccfdee 100644 --- a/modules/vector-search/pages/create-vector-search-index-ui.adoc +++ b/modules/vector-search/pages/create-vector-search-index-ui.adoc @@ -1,5 +1,7 @@ = Create a Vector Search Index in Quick Mode :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Use Quick Mode to create a Vector Search index in Couchbase Capella. [abstract] @@ -27,7 +29,7 @@ For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[] + include::partial$download-sample-partial.adoc[] -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure diff --git a/modules/vector-search/pages/run-vector-search-python-sdk.adoc b/modules/vector-search/pages/run-vector-search-python-sdk.adoc index f89341b32..42b3913e7 100644 --- a/modules/vector-search/pages/run-vector-search-python-sdk.adoc +++ b/modules/vector-search/pages/run-vector-search-python-sdk.adoc @@ -1,5 +1,7 @@ = Run a Vector Search with the Python SDK :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Using Couchbase's Python SDK, you can run a simple or more complex vector search against a Vector Search index. [abstract] diff --git a/modules/vector-search/pages/run-vector-search-rest-api.adoc b/modules/vector-search/pages/run-vector-search-rest-api.adoc index fd7c99b31..310430c16 100644 --- a/modules/vector-search/pages/run-vector-search-rest-api.adoc +++ b/modules/vector-search/pages/run-vector-search-rest-api.adoc @@ -1,5 +1,7 @@ = Run a Vector Search with the REST API and curl/HTTP :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: You can use the REST API and a curl command to run a search against a Vector Search index and return similar vectors from a Couchbase Capella database. [abstract] @@ -35,7 +37,7 @@ include::example$run-vector-search-header.sh[] + . Enter the JSON payload for your query. + -TIP: You can copy the JSON for a Query Request from the Capella UI to use in your REST API call. +TIP: You can copy the JSON for a Query Request from the {page-ui-name} to use in your REST API call. For more information about how to perform a search with the UI, see xref:search:simple-search-ui.adoc[]. === Example diff --git a/modules/vector-search/pages/run-vector-search-ui.adoc b/modules/vector-search/pages/run-vector-search-ui.adoc index 66acc23d7..5174954c6 100644 --- a/modules/vector-search/pages/run-vector-search-ui.adoc +++ b/modules/vector-search/pages/run-vector-search-ui.adoc @@ -1,6 +1,8 @@ -= Run a Vector Search with the Capella UI += Run a Vector Search with the {page-ui-name} :page-topic-type: guide -:description: Run a Vector Search query from the Couchbase Capella UI to preview and test the search results from a Vector Search index. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Run a Vector Search query from the Couchbase {page-ui-name} to preview and test the search results from a Vector Search index. [abstract] {description} @@ -16,11 +18,11 @@ For more information about how to create a Vector Search index, see xref:create- + include::partial$download-sample-partial.adoc[] -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To run a Vector Search with the Capella UI: +To run a Vector Search with the {page-ui-name}: . On the *Databases* page, select the database where you created your Search index. . Go to menu:Data Tools[Search]. diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc index 6510f581f..3fa9feb99 100644 --- a/modules/vector-search/pages/vector-search.adoc +++ b/modules/vector-search/pages/vector-search.adoc @@ -1,16 +1,18 @@ = Use Vector Search for AI Applications :page-topic-type: concept -:description: Use Couchbase Capella's Vector Search features to add fast and accurate semantic search to your applications. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Use Couchbase {page-product-name}'s Vector Search features to add fast and accurate semantic search to your applications. [abstract] {description} == About Vector Search -Vector Search builds on Couchbase Capella's xref:search:search.adoc[Search Service] to provide vector index support. +Vector Search builds on Couchbase {page-product-name}'s xref:search:search.adoc[Search Service] to provide vector index support. You can use these new Vector Search indexes for Retrieval Augmented Generation (RAG) with an existing Large Language Model (LLM). -Using Capella's Vector Search, an embedding model, and your chosen LLM, you can develop AI applications while giving context and up-to-date information from your own data. +Using {page-product-name}'s Vector Search, an embedding model, and your chosen LLM, you can develop AI applications while giving context and up-to-date information from your own data. You can develop applications that include: @@ -23,15 +25,15 @@ By using vector embeddings, you can search based on descriptions, rather than us Use generative AI to get tailored and dynamic responses across your applications. Vector Search supports integrations with frameworks like https://python.langchain.com/docs/get_started/introduction[LangChain^] to support AI application development. -For more information about all frameworks and integrations supported by Vector Search and Capella, see xref:third-party:integrations.adoc[]. +For more information about all frameworks and integrations supported by Vector Search and {page-product-name}, see xref:third-party:integrations.adoc[]. == Using Vector Search Indexes -To get started using Vector Search in Capella: +To get started using Vector Search in {page-product-name}: -. *Store data*: Store the data you want to use for your search or AI project in a Capella database. +. *Store data*: Store the data you want to use for your search or AI project in a {page-product-name} database. . *Generate embeddings*: Generate vector embeddings from your data with your preferred embedding model. -. *Store your embeddings*: Store your vector embeddings in an array inside the documents in your Capella database. +. *Store your embeddings*: Store your vector embeddings in an array inside the documents in your {page-product-name} database. . *Create a Vector Search index*: Create an index to use your embeddings and identify similar documents with vector similarity. In addition to supporting integrations with frameworks like LangChain and LlamaIndex, you can also use the API for an existing LLM and one of their embedding models to generate vector embeddings for your data. From 6594a5138905ca026b0806539605ad39f0e21aa2 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 14 Mar 2024 10:58:53 -0400 Subject: [PATCH 49/59] [AV-68571] May -> Can, per peer review --- modules/n1ql/pages/n1ql-language-reference/searchfun.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc index 3a08a7c91..7f49fc91e 100644 --- a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc @@ -122,7 +122,7 @@ The object may contain the following fields: | `index` [Optional] | string, object -| The `index` field may be a string, containing the name of a Search index in the keyspace. +| The `index` field can be a string, containing the name of a Search index in the keyspace. (This might be a Search index alias, but only if the Search index is in the same keyspace.) This provides an index hint to the Query engine. If the Search index does not exist, an error occurs. From c415b1d873716b75cca29d50cd6433d3b2cd7d31 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 14 Mar 2024 11:50:50 -0400 Subject: [PATCH 50/59] [AV-68571] Updating integrations.adoc --- modules/third-party/pages/integrations.adoc | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/modules/third-party/pages/integrations.adoc b/modules/third-party/pages/integrations.adoc index c6b7f5a40..f3fd73d63 100644 --- a/modules/third-party/pages/integrations.adoc +++ b/modules/third-party/pages/integrations.adoc @@ -25,15 +25,18 @@ These are examples of integrations that Couchbase or partners have developed to | Integration | Summary | Links | Capella | Self-managed | <<#support-model,Support Model>> | LangChain -a| Use the LangChain integrations to quickly start developing AI applications with Capella or Couchbase Server: +a| Use the Couchbase LangChain integrations to quickly start developing AI applications with your Couchbase database. -* Use the document loader to get documents from your Couchbase database. -* Use the vector stores integration to create a Search index, store vectors, and run vector searches. +Use the document loader to get documents from a Couchbase database. + +Use the vector stores integration to create a Search index, store vectors, and run vector searches. a| -* https://python.langchain.com/docs/integrations/document_loaders/couchbase[LangChain - Python Document Loader] -* https://python.langchain.com/docs/integrations/vectorstores/couchbase[LangChain - Python Vector Stores] -* https://js.langchain.com/docs/integrations/document_loaders/web_loaders/couchbase[LangChain - JavaScript Document Loader] -* https://js.langchain.com/docs/integrations/vectorstores/couchbase[LangChain - JavaScript Vector Stores] +* https://python.langchain.com/docs/get_started/installation[Install LangChain for Python] +** https://python.langchain.com/docs/integrations/document_loaders/couchbase[LangChain - Python Document Loader] +** https://python.langchain.com/docs/integrations/vectorstores/couchbase[LangChain - Python Vector Stores] +* https://js.langchain.com/docs/get_started/installation[Install LangChain for JavaScript] +** https://js.langchain.com/docs/integrations/document_loaders/web_loaders/couchbase[LangChain - JavaScript Document Loader] +** https://js.langchain.com/docs/integrations/vectorstores/couchbase[LangChain - JavaScript Vector Stores] | ✔ | ✔ | Partner Supported From 972ff55e4192f495d5f8d1bc6feb32a00cd714b9 Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 14 Mar 2024 12:35:05 -0400 Subject: [PATCH 51/59] [AV-68571] Fixing broken links, adding missing file. --- modules/search/examples/query-ip-range.jsonc | 14 ++++++++++++++ modules/search/pages/search-index-params.adoc | 2 +- modules/search/pages/search-request-params.adoc | 4 ++-- .../custom-token-filters-descriptions.adoc | 2 +- .../pages/create-vector-search-index-ui.adoc | 4 ++-- .../vector-search/pages/run-vector-search-ui.adoc | 2 +- 6 files changed, 21 insertions(+), 7 deletions(-) create mode 100644 modules/search/examples/query-ip-range.jsonc diff --git a/modules/search/examples/query-ip-range.jsonc b/modules/search/examples/query-ip-range.jsonc new file mode 100644 index 000000000..f8bf914f0 --- /dev/null +++ b/modules/search/examples/query-ip-range.jsonc @@ -0,0 +1,14 @@ +{ + "query": { + "conjuncts": [ + { + "cidr": "2.7.13.0/24", + "field": "ipv4" + }, + { + "cidr": "2001:db8:1234:5678::/119", + "field": "ipv6" + } + ] + } +} diff --git a/modules/search/pages/search-index-params.adoc b/modules/search/pages/search-index-params.adoc index fe7f8f70d..d28e6bc4a 100644 --- a/modules/search/pages/search-index-params.adoc +++ b/modules/search/pages/search-index-params.adoc @@ -154,7 +154,7 @@ The `doc_config` object is a child object of the <>. It contains the fo |mode |String |Yes a| -Set a xref:customize-search-index.adoc#type-identifiers[type identifier] for the Search index to filter documents from search results: +Set a xref:customize-index.adoc#type-identifiers[type identifier] for the Search index to filter documents from search results: * `type_field`: Use the value from a specific field in the documents. * `docid_prefix_delim`: Use the leading characters in the documents' ID values, up to but not including a specified separator. diff --git a/modules/search/pages/search-request-params.adoc b/modules/search/pages/search-request-params.adoc index 8bb7a44ea..f5110dda6 100644 --- a/modules/search/pages/search-request-params.adoc +++ b/modules/search/pages/search-request-params.adoc @@ -2139,9 +2139,9 @@ Sets what value to use for the sort: * `id`: Uses the document's ID value. * `score`: Uses the document's score. -* `field`: Uses the value of a specific field to sort. See <>. +* `field`: Uses the value of a specific field to sort. See <>. -|[[field]]field |String |No |When `by` is set to `field`, specify the name of the field to use to sort search results. +|[[by-field]]field |String |No |When `by` is set to `field`, specify the name of the field to use to sort search results. |desc |Boolean |Yes a| diff --git a/modules/search/partials/custom-token-filters-descriptions.adoc b/modules/search/partials/custom-token-filters-descriptions.adoc index d7be2cc56..76067d30f 100644 --- a/modules/search/partials/custom-token-filters-descriptions.adoc +++ b/modules/search/partials/custom-token-filters-descriptions.adoc @@ -193,7 +193,7 @@ include::example$complex-search-index-payload.jsonc[tag=shingle] ---- // end::shingle_example[] -For example, if you use a xref:guides:search/default-tokenizers-reference.adoc#whitespace[whitespace tokenizer], a range with a minimum of two and a maximum of three, and a space as a separator, the token `abc def` becomes `abc`, `def`, and `abc def`. +For example, if you use a xref:search:default-tokenizers-reference.adoc#whitespace[whitespace tokenizer], a range with a minimum of two and a maximum of three, and a space as a separator, the token `abc def` becomes `abc`, `def`, and `abc def`. [plantuml,shingle,svg] .... diff --git a/modules/vector-search/pages/create-vector-search-index-ui.adoc b/modules/vector-search/pages/create-vector-search-index-ui.adoc index 380ccfdee..a7262c9b1 100644 --- a/modules/vector-search/pages/create-vector-search-index-ui.adoc +++ b/modules/vector-search/pages/create-vector-search-index-ui.adoc @@ -8,14 +8,14 @@ {description} Quick Mode works best when you need to create a basic Search index to start testing and prototyping with the Search Service. -You must use Advanced Mode to have greater control over how the Search Service returns such results, such as changing your xref:customize-index.adoc#analyzers[analyzer]. +You must use Advanced Mode to have greater control over how the Search Service returns such results, such as changing your xref:search:customize-index.adoc#analyzers[analyzer]. For more information about how to create a Search index in Advanced Mode, see xref:search:create-search-index-ui.adoc[]. TIP: Vector Search indexes can include all the same features and settings as a Search index. For more information about Search indexes, see the xref:search:search.adoc[Search documentation]. -You must create a Vector Search index before you can xref:search:run-vector-search-ui.adoc[run a search] that supports vector comparisons. +You must create a Vector Search index before you can xref:run-vector-search-ui.adoc[run a search] that supports vector comparisons. == Prerequisites diff --git a/modules/vector-search/pages/run-vector-search-ui.adoc b/modules/vector-search/pages/run-vector-search-ui.adoc index 5174954c6..e87fde477 100644 --- a/modules/vector-search/pages/run-vector-search-ui.adoc +++ b/modules/vector-search/pages/run-vector-search-ui.adoc @@ -87,7 +87,7 @@ If you configured your data import to use the `id` field as the document identif If you do not get the search results you were expecting, you can change the xref:search:search-request-params.adoc[JSON payload for your Search query]. -You can also xref:search:search:customize-index.adoc[add additional features to your Search index]. +You can also xref:search:customize-index.adoc[add additional features to your Search index]. Either of the example queries on this page can also be used with the xref:run-vector-search-rest-api.adoc[REST API] to run a search. You can find more example embedding vectors and full queries inside the `couchbase_search_query` field for any color in the `rgb_questions.json` sample file. \ No newline at end of file From 9c960ca4b6bb22e4331781a11009ef6ac31cca6c Mon Sep 17 00:00:00 2001 From: Sarah Welton Date: Thu, 14 Mar 2024 12:43:19 -0400 Subject: [PATCH 52/59] [AV-68571] Changing integrations.adoc into a partial to pull in from Server --- modules/third-party/pages/integrations.adoc | 381 +------------------- 1 file changed, 1 insertion(+), 380 deletions(-) diff --git a/modules/third-party/pages/integrations.adoc b/modules/third-party/pages/integrations.adoc index f3fd73d63..e76d03ef0 100644 --- a/modules/third-party/pages/integrations.adoc +++ b/modules/third-party/pages/integrations.adoc @@ -1,380 +1 @@ -= Integrations, Connectors, and Tools -:page-toclevels: 2 -:page-topic-type: reference -:page-aliases: develop:integrations.adoc -:description: Integrate Couchbase with your development ecosystem and production and deployment tools. - - -[abstract] -{description} - -Third-party and community products and integrations -- -although not by and large supported by Couchbase (see <<#support-model,Support Model section>>) -- -are a vital tool for most larger development projects. - -These are examples of integrations that Couchbase or partners have developed to enhance Couchbase, Capella, and partner services. - - -[#partner-integrations-with-couchbase] -== Partner Integrations - - -.Partner Integrations List -[cols="20,45,50,20,20,28"] -|=== -| Integration | Summary | Links | Capella | Self-managed | <<#support-model,Support Model>> - -| LangChain -a| Use the Couchbase LangChain integrations to quickly start developing AI applications with your Couchbase database. - -Use the document loader to get documents from a Couchbase database. - -Use the vector stores integration to create a Search index, store vectors, and run vector searches. -a| -* https://python.langchain.com/docs/get_started/installation[Install LangChain for Python] -** https://python.langchain.com/docs/integrations/document_loaders/couchbase[LangChain - Python Document Loader] -** https://python.langchain.com/docs/integrations/vectorstores/couchbase[LangChain - Python Vector Stores] -* https://js.langchain.com/docs/get_started/installation[Install LangChain for JavaScript] -** https://js.langchain.com/docs/integrations/document_loaders/web_loaders/couchbase[LangChain - JavaScript Document Loader] -** https://js.langchain.com/docs/integrations/vectorstores/couchbase[LangChain - JavaScript Vector Stores] -| ✔ -| ✔ -| Partner Supported - -| Hashicorp Terraform -| Deploy and manage your clusters with declarative configuration. -a| -* https://github.com/couchbasecloud/terraform-provider-couchbase-capella/[Couchbase Capella Terraform Provider repo] -* xref:cloud:terraform:index.adoc[Couchbase Capella Terraform Provider docs] -* https://registry.terraform.io/providers/couchbasecloud/couchbase-capella/latest[Capella Provider on the Hashicorp Registry site] -| ✔ -| ✖ -| Partner Supported - -| Hashicorp Vault -| Securely manage your credentials for database access. -a| -* https://www.hashicorp.com/partners/tech/couchbase#vault[Hashicorp Vault plug-in for Capella] -* https://github.com/couchbasecloud/vault-plugin-database-couchbasecapella[Plug-in page on GitHub] -* https://developer.hashicorp.com/vault/docs/secrets/databases/couchbase[Hashicorp partner page for self-managed Couchbase] -| ✔ -| ✔ -| Partner Supported - -| Netlify -| Use Couchbase Capella and Netlify Serverless Functions to handle dynamic content without needing infrastructure. -a| -* https://www.netlify.com/integrations/couchbase/[Netlify partner page] -* https://developer.couchbase.com/tutorial-quickstart-netlify[Tutorial] -| ✔ -| ✖ -| Couchbase Officially Supported - - -| Vercel -| Easy build and deploy for your applications. -a| -* https://vercel.com/integrations/couchbase-capella[Vercel partner page & tutorial] -* https://www.couchbase.com/blog/integrating-vercel-with-couchbase-capella[Blog] -* https://vercel.com/templates/next.js/couchbase-capella[Next.js starter template] -| ✔ -| ✖ -| Couchbase Officially Supported - - -| Molo17 GlueSync -| Real-time, cloud-native data replication suite for NoSQL. -a| -* https://www.couchbase.com/partners/molo17/[Intro & Tutorial] -* https://molo17.com/gluesync/[GlueSync homepage] -| ✔ -| ✔ -| Partner Supported - - -| ODBC / JDBC -| Easily access live Couchbase data from BI, Analytics, Reporting, ETL, & Custom Apps. -a| -* xref:server:connectors:odbc-jdbc-drivers.adoc[Couchbase ODBC and JDBC Drivers] -* https://www.cdata.com/drivers/couchbase/docs/[CData Driver] -* https://www.cdata.com/drivers/couchbase/#section-etl-anchor[Popular Data Warehousing Integrations] -* https://www.cdata.com/drivers/couchbase/#section-data-anchor[Data Management Tools] -* https://www.cdata.com/drivers/couchbase/#section-dev-anchor[Developer Tools] -* https://www.cdata.com/drivers/couchbase/#section-bi-anchor[BI & Data Visualization] -| ✔ -| ✔ -| Partner Supported -|=== - - - -== IDE Integrations - - -.IDE Plugins -[cols="20,45,50,20,20,35"] -|=== -| Integration | Summary | Links | Capella | Self-managed | <<#support-model,Support Model>> - -| VS Code extension for Couchbase -| A seamless experience for Couchbase Server users who want to work within the popular Visual Studio Code editor. -a| -* https://marketplace.visualstudio.com/items?itemName=Couchbase.vscode-couchbase[Visual Studio Marketplace] -* https://github.com/couchbaselabs/VSCode-Couchbase/blob/main/GETTING-STARTED.md[Tutorial] -* https://github.com/couchbaselabs/VSCode-Couchbase/blob/main/SNIPPETS.md[Code snippets] -| ✔ -| ✔ -| Couchbase Community Support - - -| VS Code extension for Couchbase Lite -| A seamless experience for Couchbase Mobile users who want to work within the popular Visual Studio Code editor. -a| -* https://marketplace.visualstudio.com/items?itemName=Couchbase.vscode-cblite[Visual Studio Marketplace] -* https://github.com/couchbaselabs/vscode-cblite[GitHub repo and intro] -| ✔ -| ✔ -| Couchbase Community Support - - -| JetBrains -| Provides support for Couchbase in JetBrains IDEs -- to develop Couchbase applications, complete with code completion, syntax highlighting, and debugging support. -a| -* https://plugins.jetbrains.com/plugin/22131-couchbase[JetBrains Marketplace] -// * [Tutorial] -* https://github.com/couchbaselabs/couchbase_jetbrains_plugin[GitHub source repo] -| ✔ -| ✔ -| Couchbase Community Support -|=== - - - -== Data Tools - -Import, export, and query data from the command line. - -.Data Tools -[cols="20,45,50,20,20,35"] -|=== -| Integration | Summary | Links | Capella | Self-managed | <<#support-model,Support Model>> - - -| Couchbase Shell (cbsh) -| Interactive shell, based upon nushell, which you can use to extract, transform, and load different data formats in Couchbase. -a| -* https://couchbase.sh/docs/[cbsh docs] -* https://github.com/couchbaselabs/couchbase-shell[GitHub repo] -| ✔ -| ✔ -| Couchbase Officially Supported (currently in _Developer Preview_) - - -| cbimport -| Utility for importing data into Couchbase databases. -Included in self-managed Couchbase Server, or downloadable as part of the xref:cloud:reference:command-line-tools.adoc#download-and-install-the-couchbase-command-line-tools[Capella Tools package]. -a| xref:server:tools:cbimport.adoc[cbimport reference] -| ✔ -| ✔ -| Couchbase Officially Supported - - -| cbexport -| Utility for exporting data from Couchbase databases. -Included in self-managed Couchbase Server, or downloadable as part of the xref:cloud:reference:command-line-tools.adoc#download-and-install-the-couchbase-command-line-tools[Capella Tools package]. -a| xref:server:tools:cbexport.adoc[cbexport reference] -| ✔ -| ✔ -| Couchbase Officially Supported - - -| cbbackupmgr -| A utility for backing up and restoring a Couchbase database. -a| -* xref:server:current:backup-restore:cbbackupmgr.adoc[cbbackupmgr reference] -* xref:server:backup-restore:cbbackupmgr-tutorial.adoc[Tutorial] -* xref:cloud:clusters:cli-backup-restore.adoc[Capella examples] -| ✔ -| ✔ -| Couchbase Officially Supported -|=== - - -=== Other Shells - -Some languages, such as Ruby, come with REPLs (interactive shells) as standard -- as is normal with many programming languages. -Others have them available as _Notebooks_, such as Python, with https://jupyter.org/[Jupyter Notebook] (which works with _several_ different languages), through which you can interactively develop functionality for a Couchbase app, or blend with analytic tools and charts. - - - -[big-data-connectors] -== Big Data Connectors - - -Couchbase develops and supports connectors for: - -* Elasticsearch and OpenSearch -* The Kafka Connect Framework -* Apache Spark Data Processing Platform -* The Tableau interactive data visualization platform - -Couchbase ODBC and JDBC drivers enable any application based on the ODBC/JDBC standards, for example Microsoft Excel, QlikView, or SAP Lumira, as well as Tableau, to connect to a Couchbase Server or cluster. - - - -.Big Data Connectors -[cols="28,43,28,30,30,28"] -|=== -| Connector | Summary | Docs | Source | Other Integrations | <<#support-model,Support Model>> - - -| Elasticsearch -| Replicate your documents from Couchbase Server or Capella to Elasticsearch or OpenSearch in near real time. -| xref:elasticsearch-connector:getting-started.adoc[Getting Started docs] -| xref:elasticsearch-connector:release-notes.adoc[Release notes & download links] -| xref:elasticsearch-connector:autonomous-operations.adoc[AO Mode] -| Couchbase Officially Supported - - -| Kafka -| A plug-in for the Kafka Connect framework, which provides source and sink components. -| xref:kafka-connector:index.adoc[Kafka connector docs] -| https://github.com/couchbase/kafka-connect-couchbase[GitHub repo] -| https://kafka.apache.org/uses[Use cases] -| Couchbase Officially Supported - - -| Spark -| Connect to the scalable computing engine for Big Data. -| xref:spark-connector:index.adoc[Couchbase Spark docs] -| https://github.com/couchbase/couchbase-spark-connector[Couchbase Spark Connector on GitHub] -| https://docs.databricks.com/_extras/notebooks/source/couchbase.html[Apache Spark Notebook] -| Couchbase Officially Supported - - -| Tableau -| Integrate Couchbase Server tabular views and the Tableau interactive data visualization platform. -| xref:tableau-connector::index.adoc[Tableau connector docs] -| https://www.tableau.com/products/desktop/download[Tableau Desktop Downloads] -| https://www.tableau.com/products/server[Other Tableau Integrations] -| Couchbase Officially Supported -|=== - - -[#sdk-language-ecosystem] -== SDK Language Ecosystem - -Couchbase has SDKs in ten popular languages -- -xref:c-sdk:hello-world:overview.adoc[C], -xref:dotnet-sdk:hello-world:overview.adoc[C# (.NET)], -xref:go-sdk:hello-world:overview.adoc[Go], -xref:java-sdk:hello-world:overview.adoc[Java], -xref:kotlin-sdk:hello-world:overview.adoc[Kotlin], -xref:nodejs-sdk:hello-world:overview.adoc[Node.js], -xref:php-sdk:hello-world:overview.adoc[PHP], -xref:python-sdk:hello-world:overview.adoc[Python], -xref:ruby-sdk:hello-world:overview.adoc[Ruby], and -xref:scala-sdk:hello-world:overview.adoc[Scala]. - -If you develop in one of these languages, you may be interested in (or already using) other libraries in that language's ecosystem, and wondering if it will integrate with Couchbase. - -There are too many third party integrations to evaluate and list (and absence of a mention in these pages is no judgement on importance or quality), but the following are well worth investigating: - -While Couchbase can't evaluate and list every possible third-party integration, we've provided a list of some you might consider in the sections below. - - -.Partner Integrations List -[cols="24,22,45,30,15,15,28"] -|=== -| Integration | SDK Language / Ecosystem | Summary | Links | Capella | Self-managed | <<#support-model,Support Model>> - -| Spring Data Couchbase -| JVM (Java, Kotlin, Scala) -| A familiar and consistent Spring-based programming model for new datastores with store-specific features and capabilities. -Key functional areas of Spring Data Couchbase are a POJO centric model for interacting with Couchbase Buckets and easily writing a Repository style data access layer. -a| -* https://spring.io/projects/spring-data-couchbase[Spring Data Couchbase] -* xref:java-sdk:hello-world:spring-data-sample-application.adoc[Sample application] -* https://blog.couchbase.com/couchbase-spring-boot-spring-data/[Spring Boot example] -| ✔ -| ✔ -| Couchbase Officially Supported - - -| Ottoman -| Node.js -| ODM (Object Document Mapper) for Couchbase. -Adds an abstraction layer over Couchbase Node.js SDK and significantly reduces the level of boilerplate needed during application development. -a| -* https://ottomanjs.com/[Ottoman docs] -* https://ottomanjs.com/docs/advanced/sdk-comparison[Advantages over pure Node.js] -* https://github.com/couchbaselabs/node-ottoman[GitHub repo] -| ✔ -| ✔ -| Couchbase Officially Supported - - -| Laravel -| PHP -| Extends the original Laravel classes to make an Eloquent model and Query builder with support for Couchbase. -a| https://github.com/friendsofcat/laravel-couchbase[Project website] -| ✔ -| ✔ -| Community Supported - - -| Rails Cache Store -| Ruby-on-Rails -| Speeds up your Rails application with caching -- storing content generated during the request-response cycle and reusing it when responding to similar requests. -a| xref:ruby-sdk:project-docs:third-party-integrations.adoc#across-the-ecosystem[Config Code example] -| ✔ -| ✔ -| Community Supported -|=== - - - - -== Support Model - - - -.Support Model -[cols="20,88"] -|=== -| Support Level | Definitions - -| Partner Supported -| This integration is developed and supported directly through the partner. -Couchbase provides support for the Couchbase components and interfaces. - -| Couchbase Community Supported -a| Open source integration that is developed by Couchbase. -Issues can also be raised in the relevant GitHub repositories or via https://www.couchbase.com/forums/[Couchbase forums] or https://discord.com/channels/915294689681362954/915296010694176808[Discord]. -Support is provided best effort by development team - -| Couchbase Officially Supported -a| Open source integration that is developed by Couchbase and officially supported. -A subscription to the Enterprise Edition of Couchbase includes technical support with service level commitments via our 24/7 support organization. -Issues can also be raised in the relevant GitHub repositories or via https://www.couchbase.com/forums/[Couchbase forums] or https://discord.com/channels/915294689681362954/915296010694176808[Discord]. - -| Community Supported -| Open source integration that is developed and supported directly by third party community contributor. -Issues can also be raised in the relevant GitHub repositories. -|=== - -//// - -== Mobile SDK Ecosystem - -Placeholder for Dart/Flutter/etc - won’t be in first page release. -Docs: https://cbl-dart.dev/documents/ -Install - https://cbl-dart.dev/install/ -Repo ; https://github.com/cbl-dart/cbl-dart - -== Useful Tools - -// Tools - -=== Test Data Generation -//// +include::7.6@server:develop:integrations.adoc[] \ No newline at end of file From 59f37fe9d0f5b4826362a8a83665956339244857 Mon Sep 17 00:00:00 2001 From: Jon Strabala <54073900+jon-strabala@users.noreply.github.com> Date: Thu, 14 Mar 2024 10:53:01 -0700 Subject: [PATCH 53/59] need missing command --- modules/vector-search/pages/run-vector-search-python-sdk.adoc | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/modules/vector-search/pages/run-vector-search-python-sdk.adoc b/modules/vector-search/pages/run-vector-search-python-sdk.adoc index 42b3913e7..0f1500185 100644 --- a/modules/vector-search/pages/run-vector-search-python-sdk.adoc +++ b/modules/vector-search/pages/run-vector-search-python-sdk.adoc @@ -33,6 +33,8 @@ Make sure to install all dependencies for the Couchbase Python SDK and make a `r couchbase==4.1.12 openai==1.12.0 ---- ++ +Run `pip install -r requirements.txt` to pull in all needed dependancies == Example: Searching for a Similar Color Vector with a Python Program @@ -80,4 +82,4 @@ You can try other questions from `rgb_questions.json`, such as: If you did not get the search results you were expecting, you can try to recreate your Vector Search index xref:create-vector-search-index-rest-api.adoc[with the REST API]. Vector Search indexes can use the same settings and features as regular Search indexes. -If you want to add additional fields and features to your index, see xref:search:customize-index.adoc[]. \ No newline at end of file +If you want to add additional fields and features to your index, see xref:search:customize-index.adoc[]. From 3944c0cd448c5991fa447c8cca41414a292a1e5c Mon Sep 17 00:00:00 2001 From: Jon Strabala <54073900+jon-strabala@users.noreply.github.com> Date: Thu, 14 Mar 2024 10:56:18 -0700 Subject: [PATCH 54/59] New Go SDK page --- .../pages/run-vector-search-from-go-sdk.adoc | 72 +++++++++++++++++++ 1 file changed, 72 insertions(+) create mode 100644 modules/vector-search/pages/run-vector-search-from-go-sdk.adoc diff --git a/modules/vector-search/pages/run-vector-search-from-go-sdk.adoc b/modules/vector-search/pages/run-vector-search-from-go-sdk.adoc new file mode 100644 index 000000000..835f66ab8 --- /dev/null +++ b/modules/vector-search/pages/run-vector-search-from-go-sdk.adoc @@ -0,0 +1,72 @@ += Run a Vector Search with the Go SDK +:page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Using Couchbase's Go SDK, you can run a simple or more complex vector search against a Vector Search index. + +[abstract] +{description} + + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. + +* Make sure to install the Couchbase Go SDK ++ +For more information about on installing and using the Couchbase Go SDK, see xref:go-sdk:hello-world:start-using-sdk.adoc[Start Using the Go SDK]. + +== Example: Searching for a Similar Color Vector with a Go Program + +The sample dataset inside `rgb.json` has small embedding vectors inside the `colorvect_l2` field. +These embedding vectors describe a color using RGB values. +For example, the color red has an embedding vector of `[255, 0, 0]`. + +The following example code searches for the color navy (`[0.0, 0.0, 128.0]`) in the `rgb.json` dataset: + +[source,go] +---- +include::example$RunVectorSearchSimpleColor.go[] +---- + +== Example: Semantic Search with Color Descriptions + +[NOTE] +==== +The following code sample requires you to have a paid subscription to the OpenAI API to generate an embedding vector from a sample text string. +For more information about pricing for the OpenAI API, see https://openai.com/pricing[OpenAI's Pricing page^] for embedding models. + +The `rgb_questions.json` and sample data contains ready-made embedding vectors for different questions. +For an example of how to use a ready-made vector with Vector Search, see xref:run-vector-search-rest-api.adoc[] or xref:run-vector-search-ui.adoc[]. +==== + +Still using the sample dataset inside `rgb.json`, you can use the OpenAI API to generate an embedding from any text string. + +The following code generates an embedding vector with the question `What color hides everything like the night?`: + +[source,go] +---- +include::example$RunVectorSearchGenerateEmbed.go[] +---- + +You can try other questions from `rgb_questions.json`, such as: + +* `"What is the most romantic color?"` + +* `"What color makes people fall in love?"` + +* `"What is the most depressing color?"` + +== Next Steps + +Edit and update your Search index, `color-index`, and store the `description` field and rerun the second example. This will also return the `description` paragraph with the results. + +If you did not get the search results you were expecting, you can try to recreate your Vector Search index xref:create-vector-search-index-rest-api.adoc[with the REST API]. + +Vector Search indexes can use the same settings and features as regular Search indexes. +If you want to add additional fields and features to your index, see xref:search:customize-index.adoc[]. From 09117cbdebbe04ef2fc6d60f105516b16edce41e Mon Sep 17 00:00:00 2001 From: Jon Strabala <54073900+jon-strabala@users.noreply.github.com> Date: Thu, 14 Mar 2024 10:59:36 -0700 Subject: [PATCH 55/59] Go SDK examples --- .../examples/RunVectorSearchGenerateEmbed.go | 134 ++++++++++++++++++ .../examples/RunVectorSearchSimpleColor.go | 71 ++++++++++ 2 files changed, 205 insertions(+) create mode 100644 modules/vector-search/examples/RunVectorSearchGenerateEmbed.go create mode 100644 modules/vector-search/examples/RunVectorSearchSimpleColor.go diff --git a/modules/vector-search/examples/RunVectorSearchGenerateEmbed.go b/modules/vector-search/examples/RunVectorSearchGenerateEmbed.go new file mode 100644 index 000000000..e2e8ea964 --- /dev/null +++ b/modules/vector-search/examples/RunVectorSearchGenerateEmbed.go @@ -0,0 +1,134 @@ +package main + +import ( + "fmt" + "log" + "time" + "os" + "github.com/couchbase/gocb/v2" + // "github.com/couchbase/gocb/v2/search" + "github.com/couchbase/gocb/v2/vector" + "bytes" + "encoding/json" + "io/ioutil" + "net/http" +) + +type OpenAIResponse struct { + Data []struct { + Embedding []float32 `json:"embedding"` + } `json:"data"` +} + +// generateVector makes a request to OpenAI's API to get an embedding vector for the given input text. +func generateVector(inputText string) ([]float32, error) { + openaiAPIKey := os.Getenv("OPENAI_API_KEY") + if openaiAPIKey == "" { + return nil, fmt.Errorf("OPENAI_API_KEY environment variable is not set") + } + + requestBody, err := json.Marshal(map[string]interface{}{ + "input": inputText, + "model": "text-embedding-ada-002", + }) + if err != nil { + return nil, fmt.Errorf("error marshaling request body: %w", err) + } + + request, err := http.NewRequest("POST", "https://api.openai.com/v1/embeddings", bytes.NewBuffer(requestBody)) + if err != nil { + return nil, fmt.Errorf("error creating request: %w", err) + } + + request.Header.Set("Content-Type", "application/json") + request.Header.Set("Authorization", "Bearer "+openaiAPIKey) + + client := &http.Client{} + response, err := client.Do(request) + if err != nil { + return nil, fmt.Errorf("error making request: %w", err) + } + defer response.Body.Close() + + if response.StatusCode != http.StatusOK { + bodyBytes, _ := ioutil.ReadAll(response.Body) + return nil, fmt.Errorf("API request failed with status %d: %s", response.StatusCode, string(bodyBytes)) + } + + var openAIResponse OpenAIResponse + if err := json.NewDecoder(response.Body).Decode(&openAIResponse); err != nil { + return nil, fmt.Errorf("error decoding response: %w", err) + } + + if len(openAIResponse.Data) == 0 || len(openAIResponse.Data[0].Embedding) == 0 { + return nil, fmt.Errorf("no embedding vector found in response") + } + + return openAIResponse.Data[0].Embedding, nil +} + +func main() { + connstr := "couchbases://" + os.Getenv("CB_HOSTNAME") + username := os.Getenv("CB_USERNAME") + password := os.Getenv("CB_PASSWORD") + bucket_name := "vector-sample" + scope_name := "color" + + cluster, err := gocb.Connect(connstr, gocb.ClusterOptions{ + Authenticator: gocb.PasswordAuthenticator{ + Username: username, + Password: password, + }, + SecurityConfig: gocb.SecurityConfig{ + TLSSkipVerify: true, // Disables TLS certificate verification + }, + }) + if err != nil { + log.Fatal(err) + } + + bucket := cluster.Bucket(bucket_name) + err = bucket.WaitUntilReady(5*time.Second, nil) + if err != nil { + log.Fatal(err) + } + + scope := bucket.Scope(scope_name) + + question := "What color hides everything like the night?" + vect, err := generateVector(question) + if err != nil { + log.Fatalf("Error generating vector: %v", err) + } + + request := gocb.SearchRequest{ + VectorSearch: vector.NewSearch( + []*vector.Query{ + vector.NewQuery("embedding_vector_dot", vect), + }, + nil, + ), + } + + opts := &gocb.SearchOptions{Limit: 2, Fields: []string{"color","description"}} + + matchResult, err := scope.Search("color-index", request, opts) + if err != nil { + log.Fatal(err) + } + + for matchResult.Next() { + row := matchResult.Row() + docID := row.ID + var fields interface{} + err := row.Fields(&fields) + if err != nil { + log.Fatal(err) + } + fmt.Printf("Document ID: %s, Fields: %v\n", docID, fields) + } + + if err = matchResult.Err(); err != nil { + log.Fatal(err) + } +} diff --git a/modules/vector-search/examples/RunVectorSearchSimpleColor.go b/modules/vector-search/examples/RunVectorSearchSimpleColor.go new file mode 100644 index 000000000..63ba3691d --- /dev/null +++ b/modules/vector-search/examples/RunVectorSearchSimpleColor.go @@ -0,0 +1,71 @@ +package main + +import ( + "fmt" + "log" + "time" + "os" + "github.com/couchbase/gocb/v2" + // "github.com/couchbase/gocb/v2/search" + "github.com/couchbase/gocb/v2/vector" +) + +func main() { + connstr := "couchbases://" + os.Getenv("CB_HOSTNAME") + username := os.Getenv("CB_USERNAME") + password := os.Getenv("CB_PASSWORD") + bucket_name := "vector-sample" + scope_name := "color" + + cluster, err := gocb.Connect(connstr, gocb.ClusterOptions{ + Authenticator: gocb.PasswordAuthenticator{ + Username: username, + Password: password, + }, + SecurityConfig: gocb.SecurityConfig{ + TLSSkipVerify: true, // Disables TLS certificate verification + }, + }) + if err != nil { + log.Fatal(err) + } + + bucket := cluster.Bucket(bucket_name) + err = bucket.WaitUntilReady(5*time.Second, nil) + if err != nil { + log.Fatal(err) + } + + scope := bucket.Scope(scope_name) + + request := gocb.SearchRequest{ + VectorSearch: vector.NewSearch( + []*vector.Query{ + vector.NewQuery("colorvect_l2", []float32{0.0, 0.0, 128.0}), + }, + nil, + ), + } + + opts := &gocb.SearchOptions{Limit: 3, Fields: []string{"color"}} + + matchResult, err := scope.Search("color-index", request, opts) + if err != nil { + log.Fatal(err) + } + + for matchResult.Next() { + row := matchResult.Row() + docID := row.ID + var fields interface{} + err := row.Fields(&fields) + if err != nil { + log.Fatal(err) + } + fmt.Printf("Document ID: %s, Fields: %v\n", docID, fields) + } + + if err = matchResult.Err(); err != nil { + log.Fatal(err) + } +} From 04d77bf20d850231f6a71193795ab0098e8b3c73 Mon Sep 17 00:00:00 2001 From: Jon Strabala <54073900+jon-strabala@users.noreply.github.com> Date: Thu, 14 Mar 2024 11:00:37 -0700 Subject: [PATCH 56/59] Java SDK examples --- .../RunVectorSearchGenerateEmbed.java | 114 ++++++++++++++++++ .../examples/RunVectorSearchSimpleColor.java | 63 ++++++++++ 2 files changed, 177 insertions(+) create mode 100644 modules/vector-search/examples/RunVectorSearchGenerateEmbed.java create mode 100644 modules/vector-search/examples/RunVectorSearchSimpleColor.java diff --git a/modules/vector-search/examples/RunVectorSearchGenerateEmbed.java b/modules/vector-search/examples/RunVectorSearchGenerateEmbed.java new file mode 100644 index 000000000..642ed7766 --- /dev/null +++ b/modules/vector-search/examples/RunVectorSearchGenerateEmbed.java @@ -0,0 +1,114 @@ +import com.couchbase.client.core.error.*; +import com.couchbase.client.java.*; +import com.couchbase.client.java.kv.*; +import com.couchbase.client.java.json.*; +import com.couchbase.client.java.search.*; +import com.couchbase.client.java.search.queries.*; +import com.couchbase.client.java.search.result.*; +import com.couchbase.client.java.search.vector.*; + +import java.io.IOException; +import java.net.URI; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.net.http.HttpResponse.BodyHandlers; +import java.util.List; + +import java.time.Duration; +import java.util.Map; + +public class RunVectorSearchGenerateEmbed { + + private static final String OPENAI_API_KEY = System.getenv("OPENAI_API_KEY"); + private static final String OPENAI_URL = "https://api.openai.com/v1/embeddings"; + + public static float[] generateVector(String inputText) { + HttpClient client = HttpClient.newHttpClient(); + + JsonObject jsonBody = JsonObject.create() + .put("input", inputText) + .put("model", "text-embedding-ada-002"); + + HttpRequest request = HttpRequest.newBuilder() + .uri(URI.create(OPENAI_URL)) + .header("Content-Type", "application/json") + .header("Authorization", "Bearer " + OPENAI_API_KEY) + .POST(HttpRequest.BodyPublishers.ofString(jsonBody.toString())) + .build(); + + try { + HttpResponse response = client.send(request, BodyHandlers.ofString()); + + JsonObject jsonResponse = JsonObject.fromJson(response.body()); + List embeddingList = jsonResponse.getArray("data") + .getObject(0) + .getArray("embedding") + .toList(); + + return toFloatArray(embeddingList); + } catch (IOException | InterruptedException e) { + e.printStackTrace(); + return null; // or handle more gracefully + } + } + + private static float[] toFloatArray(List list) { + float[] result = new float[list.size()]; + for (int i = 0; i < list.size(); i++) { + result[i] = ((Number) list.get(i)).floatValue(); + } + return result; + } + + public static void main(String[] args) { + + String endpoint = "couchbases://" + System.getenv("CB_HOSTNAME") + "?tls_verify=none"; + String username = System.getenv("CB_USERNAME"); + String password = System.getenv("CB_PASSWORD"); + + String bucketName = "vector-sample"; + String scopeName = "color"; + String collectionName = "rgb"; + String searchIndexName = "color-index"; + + try { + // Connect to cluster with specified credentials + Cluster cluster = Cluster.connect( + endpoint, + ClusterOptions.clusterOptions(username, password).environment(env -> { + // Use the pre-configured profile below to avoid latency issues with your connection. + env.applyProfile("wan-development"); + }) + ); + + Bucket bucket = cluster.bucket(bucketName); + bucket.waitUntilReady(Duration.ofSeconds(10)); + Scope scope = bucket.scope(scopeName); + Collection collection = scope.collection(collectionName); + + String question = "What color hides everything like the night?"; + float[] vector = generateVector(question); + + SearchRequest request = SearchRequest + .create(VectorSearch.create( + VectorQuery.create("embedding_vector_dot", vector).numCandidates(2))); + SearchResult result = scope.search("color-index", request, + SearchOptions.searchOptions().limit(3).fields("color","description")); + + for (SearchRow row : result.rows()) { + System.out.println("Found row: " + row); + System.out.println(" Fields: " + row.fieldsAs(Map.class)); + } + + } catch (UnambiguousTimeoutException ex) { + boolean authFailure = ex.toString().contains("Authentication Failure"); + if (authFailure) { + System.out.println("Authentication Failure Detected"); + } else { + System.out.println("Error:"); + System.out.println(ex.getMessage()); + } + } + } +} diff --git a/modules/vector-search/examples/RunVectorSearchSimpleColor.java b/modules/vector-search/examples/RunVectorSearchSimpleColor.java new file mode 100644 index 000000000..e71659c3d --- /dev/null +++ b/modules/vector-search/examples/RunVectorSearchSimpleColor.java @@ -0,0 +1,63 @@ +import com.couchbase.client.core.error.*; +import com.couchbase.client.java.*; +import com.couchbase.client.java.kv.*; +import com.couchbase.client.java.json.*; +import com.couchbase.client.java.search.*; +import com.couchbase.client.java.search.queries.*; +import com.couchbase.client.java.search.result.*; +import com.couchbase.client.java.search.vector.*; + +import java.time.Duration; +import java.util.Map; + +public class RunVectorSearchSimpleColor { + + public static void main(String[] args) { + + String endpoint = "couchbases://" + System.getenv("CB_HOSTNAME") + "?tls_verify=none"; + String username = System.getenv("CB_USERNAME"); + String password = System.getenv("CB_PASSWORD"); + + String bucketName = "vector-sample"; + String scopeName = "color"; + String collectionName = "rgb"; + String searchIndexName = "color-index"; + + try { + // Connect to cluster with specified credentials + Cluster cluster = Cluster.connect( + endpoint, + ClusterOptions.clusterOptions(username, password).environment(env -> { + // Use the pre-configured profile below to avoid latency issues with your connection. + env.applyProfile("wan-development"); + }) + ); + + Bucket bucket = cluster.bucket(bucketName); + bucket.waitUntilReady(Duration.ofSeconds(10)); + Scope scope = bucket.scope(scopeName); + Collection collection = scope.collection(collectionName); + + SearchRequest request = SearchRequest + .create(VectorSearch.create( + VectorQuery.create("colorvect_l2", new float[]{ 0.0f, 0.0f, 128.0f } + ).numCandidates(3))); + SearchResult result = scope.search("color-index", request, + SearchOptions.searchOptions().limit(3).fields("color","brightness")); + + for (SearchRow row : result.rows()) { + System.out.println("Found row: " + row); + System.out.println(" Fields: " + row.fieldsAs(Map.class)); + } + + } catch (UnambiguousTimeoutException ex) { + boolean authFailure = ex.toString().contains("Authentication Failure"); + if (authFailure) { + System.out.println("Authentication Failure Detected"); + } else { + System.out.println("Error:"); + System.out.println(ex.getMessage()); + } + } + } +} From 6e1e1276422c5088cc4a8c2ca0c3b03ec417fd4b Mon Sep 17 00:00:00 2001 From: Jon Strabala <54073900+jon-strabala@users.noreply.github.com> Date: Thu, 14 Mar 2024 11:03:15 -0700 Subject: [PATCH 57/59] Create run-vector-search-from-java-sdk.adoc --- .../run-vector-search-from-java-sdk.adoc | 72 +++++++++++++++++++ 1 file changed, 72 insertions(+) create mode 100644 modules/vector-search/pages/run-vector-search-from-java-sdk.adoc diff --git a/modules/vector-search/pages/run-vector-search-from-java-sdk.adoc b/modules/vector-search/pages/run-vector-search-from-java-sdk.adoc new file mode 100644 index 000000000..e547564db --- /dev/null +++ b/modules/vector-search/pages/run-vector-search-from-java-sdk.adoc @@ -0,0 +1,72 @@ += Run a Vector Search with the Java SDK +:page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Using Couchbase's Java SDK, you can run a simple or more complex vector search against a Vector Search index. + +[abstract] +{description} + + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. + +* Make sure to install the Couchbase Java SDK ++ +For more information about on installing and using the Couchbase Java SDK, see xref:java-sdk:hello-world:start-using-sdk.adoc[Start Using the Java SDK]. + +== Example: Searching for a Similar Color Vector with a Java Program + +The sample dataset inside `rgb.json` has small embedding vectors inside the `colorvect_l2` field. +These embedding vectors describe a color using RGB values. +For example, the color red has an embedding vector of `[255, 0, 0]`. + +The following example code searches for the color navy (`[0.0, 0.0, 128.0]`) in the `rgb.json` dataset: + +[source,java] +---- +include::example$RunVectorSearchSimpleColor.java[] +---- + +== Example: Semantic Search with Color Descriptions + +[NOTE] +==== +The following code sample requires you to have a paid subscription to the OpenAI API to generate an embedding vector from a sample text string. +For more information about pricing for the OpenAI API, see https://openai.com/pricing[OpenAI's Pricing page^] for embedding models. + +The `rgb_questions.json` and sample data contains ready-made embedding vectors for different questions. +For an example of how to use a ready-made vector with Vector Search, see xref:run-vector-search-rest-api.adoc[] or xref:run-vector-search-ui.adoc[]. +==== + +Still using the sample dataset inside `rgb.json`, you can use the OpenAI API to generate an embedding from any text string. + +The following code generates an embedding vector with the question `What color hides everything like the night?`: + +[source,java] +---- +include::example$RunVectorSearchGenerateEmbed.java[] +---- + +You can try other questions from `rgb_questions.json`, such as: + +* `"What is the most romantic color?"` + +* `"What color makes people fall in love?"` + +* `"What is the most depressing color?"` + +== Next Steps + +Edit and update your Search index, `color-index`, and store the `description` field and rerun the second example. This will also return the `description` paragraph with the results. + +If you did not get the search results you were expecting, you can try to recreate your Vector Search index xref:create-vector-search-index-rest-api.adoc[with the REST API]. + +Vector Search indexes can use the same settings and features as regular Search indexes. +If you want to add additional fields and features to your index, see xref:search:customize-index.adoc[]. From 14839a28a667a9722b9c9389a0098a45634e7f3b Mon Sep 17 00:00:00 2001 From: Jon Strabala <54073900+jon-strabala@users.noreply.github.com> Date: Thu, 14 Mar 2024 11:03:56 -0700 Subject: [PATCH 58/59] add directions on how to get the description back --- modules/vector-search/pages/run-vector-search-python-sdk.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/modules/vector-search/pages/run-vector-search-python-sdk.adoc b/modules/vector-search/pages/run-vector-search-python-sdk.adoc index 0f1500185..f0d7dd163 100644 --- a/modules/vector-search/pages/run-vector-search-python-sdk.adoc +++ b/modules/vector-search/pages/run-vector-search-python-sdk.adoc @@ -79,6 +79,8 @@ You can try other questions from `rgb_questions.json`, such as: == Next Steps +Edit and update your Search index, `color-index`, and store the `description` field and rerun the second example. This will also return the `description` paragraph with the results. + If you did not get the search results you were expecting, you can try to recreate your Vector Search index xref:create-vector-search-index-rest-api.adoc[with the REST API]. Vector Search indexes can use the same settings and features as regular Search indexes. From df7889fa09c927d129a6aa879edbeadf794af815 Mon Sep 17 00:00:00 2001 From: Jon Strabala <54073900+jon-strabala@users.noreply.github.com> Date: Thu, 14 Mar 2024 11:04:57 -0700 Subject: [PATCH 59/59] Add nave for go and java SDKs --- modules/vector-search/partials/nav.adoc | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/modules/vector-search/partials/nav.adoc b/modules/vector-search/partials/nav.adoc index f720d81fd..c6a5110ab 100644 --- a/modules/vector-search/partials/nav.adoc +++ b/modules/vector-search/partials/nav.adoc @@ -3,4 +3,6 @@ ** xref:cloud:vector-search:create-vector-search-index-rest-api.adoc[] ** xref:cloud:vector-search:run-vector-search-ui.adoc[] ** xref:cloud:vector-search:run-vector-search-rest-api.adoc[] -** xref:cloud:vector-search:run-vector-search-python-sdk.adoc[] \ No newline at end of file +** xref:cloud:vector-search:run-vector-search-python-sdk.adoc[] +** xref:cloud:vector-search:run-vector-search-go-sdk.adoc[] +** xref:cloud:vector-search:run-vector-search-java-sdk.adoc[]