diff --git a/antora.yml b/antora.yml index b1093ef32..c55430021 100644 --- a/antora.yml +++ b/antora.yml @@ -1,2 +1,6 @@ name: cloud -version: ~ \ No newline at end of file +version: ~ +asciidoc: + attributes: + ui-name: Capella UI + product-name: Capella \ No newline at end of file diff --git a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc index 7013fbf4a..7f49fc91e 100644 --- a/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc +++ b/modules/n1ql/pages/n1ql-language-reference/searchfun.adoc @@ -1,44 +1,44 @@ = Search Functions -:description: Search functions enable you to use full text search (FTS) queries directly within a {sqlpp} query. +:description: Search functions enable you to use Full Text Search (FTS) queries directly within a {sqlpp} query. :page-topic-type: reference :example-caption!: :imagesdir: ../../assets/images :underscore: _ -Search functions enable you to use full text search (xref:server:fts:full-text-intro.adoc[FTS]) queries directly within a {sqlpp} query. +Search functions enable you to use xref:search:search.adoc[Full Text Search (FTS)] queries directly within a {sqlpp} query. [float] === Prerequisites -To use any of the search functions, the Search service must be available on the cluster. -It is also recommended, but not required, that you should create suitable full text indexes for the searches that you need to perform. -For more information, refer to xref:server:fts:fts-quickstart-guide.adoc#preparing-for-full-text-searches[Preparing for Full Text Searches]. +To use any of the search functions, the Search Service must be available on the cluster. +It's recommended that you create a suitable Search index for the searches that you want to run. +For more information, refer to xref:search:create-search-indexes.adoc[]. [NOTE] -- -The examples in this page all assume that demonstration full text indexes have been created, as described in xref:server:fts:fts-demonstration-indexes.adoc[Demonstration Indexes]. +The examples in this page all assume that demonstration Search indexes have been created, as described in xref:server:fts:fts-demonstration-indexes.adoc[Demonstration Indexes]. -- [float] === Authorization -You do not need credentials for the FTS service to be able to use the search functions in a query. +You do not need credentials for the Search Service to be able to use the search functions in a query. The role *Data Admin* must be assigned to those who intend to create indexes; and the role *Data Reader* to those who intend to perform searches. -For information on creating users and assigning roles, see xref:server:learn:security/authorization-overview.adoc[Authorization]. +For information on creating users and roles in Capella, see xref:organizations:organization-projects-overview.adoc[]. [float] === When to Use Search Functions -The search functions are useful when you need to combine a full text search with the power of a {sqlpp} query; for example, combining joins and natural-language search in the same query. +The search functions are useful when you need to combine a Full Text Search with the power of a {sqlpp} query; for example, combining joins and natural-language search in the same query. -If you only need to use the capabilities of a full text search without any {sqlpp} features, consider making use of the Search service directly, through the user interface, the REST API, or an SDK. +If you only need to use the capabilities of a Full Text Search without any {sqlpp} features, consider making use of the Search Service directly, through the user interface, the REST API, or an SDK. [[search,SEARCH()]] == SEARCH(`identifier`, `query`[, `options`]) === Description -This function enables you to use a full text search to filter a result set, or as a join predicate. +This function enables you to use a Full Text Search to filter a result set, or as a join predicate. It is only allowed in the xref:n1ql-language-reference/where.adoc[WHERE] clause or the xref:n1ql-language-reference/join.adoc[ON] clause. If a query contains a SEARCH function, the Query engine analyzes the entire query, including the search specification, to select the best index to use with this search, taking any index hints into account. @@ -46,11 +46,15 @@ The Query engine then passes the search specification over to the Search engine [TIP] -- -If no suitable full text index can be selected, or no full text index exists, the Query engine falls back on a Primary index or qualified GSI index to produce document keys, and then fetches the documents. -The Search service then creates a temporary index in memory to perform the search. -This process may be slower than using a suitable full text index. +If no suitable Search index can be selected, or no Search index exists, the Query engine falls back on a Primary index or qualified GSI index to produce document keys, and then fetches the documents. +The Search Service then creates a temporary index in memory to perform the search. +This process may be slower than using a suitable Search index. -- +NOTE: From Couchbase Server 7.6 and later, when you use the SEARCH function, you do not need to use the xref:search:default-analyzers-reference.adoc#keyword[keyword analyzer] in your Search index to run a non-analytic query. +You also do not need to match the analyzer in a query to the analyzer in the Search index for an analytic query. +For more information about how to set the analyzer for a Search index, see xref:search:set-advanced-settings.adoc[] or the xref:search:search-index-params.adoc#mapping[Mapping Object]. + === Arguments identifier:: @@ -75,7 +79,7 @@ Similarly, if the _query_ argument is a query object, the path is ignored. The _identifier_ argument cannot be replaced by a {sqlpp} query parameter. query:: -[Required] The full text search query. +[Required] The Full Text Search query. This may be one of the following: + [cols="1a,4a", options="header"] @@ -85,21 +89,21 @@ This may be one of the following: | string | A query string. -For more details, refer to xref:server:fts:fts-query-types.adoc#query-string-query-syntax[Query String Query]. +For more information about how to format a query string, refer to xref:search:search-request-params.adoc#query-string-query-syntax[Query String Query]. | object -| The query object within a full text search request. -For more details, refer to xref:fts-supported-queries.adoc[Supported Queries]. +| The `query` object within a Full Text Search request. +For more information about how to format the `query` object, see xref:search:search-request-params.adoc#query-object[Query object]. | object -| A complete full text search request, including sort and pagination options, and so on. -For more details, refer to xref:server:fts:fts-sorting.adoc[Sorting Query Results]. +| A complete Full Text Search request, including sort and pagination options, and so on. +For more information about how to format a full search request object, refer to xref:search:search-request-params.adoc[]. [NOTE] ==== -When specifying a complete full text search request with the {sqlpp} SEARCH() function, if the value of the `size` parameter is greater than the xref:server:fts:fts-response-object-schema.adoc#request[maximum number of full text search results], the query ignores the `size` parameter and returns all matching results. +When specifying a complete Full Text Search request with the {sqlpp} SEARCH() function, if the value of the `size` parameter is greater than the xref:server:fts:fts-response-object-schema.adoc#request[maximum number of Full Text Search results], the query ignores the `size` parameter and returns all matching results. -This is different to the behavior of a complete full text search request in the Search service, where the query returns an error if the value of the `size` parameter is greater than the maximum number of full text search results. +This is different to the behavior of a complete Full Text Search request in the Search Service, where the query returns an error if the value of the `size` parameter is greater than the maximum number of Full Text Search results. ==== |=== + @@ -118,10 +122,10 @@ The object may contain the following fields: | `index` [Optional] | string, object -| The `index` field may be a string, containing the name of a full text index in the keyspace. -(This may be a full text index alias, but only if the full text index is in the same keyspace.) +| The `index` field can be a string, containing the name of a Search index in the keyspace. +(This might be a Search index alias, but only if the Search index is in the same keyspace.) This provides an index hint to the Query engine. -If the full text index does not exist, an error occurs. +If the Search index does not exist, an error occurs. [TIP] -- @@ -131,33 +135,34 @@ This takes precedence over a hint provided by the `index` field. ''' -The `index` field may also be an object, containing an example of a full text index mapping. +The `index` field may also be an object, containing an example of a Search index mapping. This is treated as an input to the index mapping. It overrides the default mapping and is used during index selection and filtering. The object must either have a default mapping with no type mapping, or a single type mapping with the default mapping disabled. -For more information, refer to xref:server:fts:fts-creating-indexes.adoc[Creating Indexes]. +For more information, refer to xref:search:customize-index.adoc[]. | `indexUUID` [Optional] | string -| A string, containing the UUID of a full text index in the keyspace. +| A string, containing the xref:search:search-index-params.adoc#uuid[UUID] of a Search index in the keyspace. This provides an index hint to the Query engine. -If the full text index cannot be identified, an error occurs. +If the Search index cannot be identified, an error occurs. -You can use the `indexUUID` field alongside the `index` field to help identify a full text index. -The `indexUUID` field and the `index` field must both identify the same full text index. -If they identify different full text indexes, or if either of them does not identify a full text index, an error occurs. +You can use the `indexUUID` field alongside the `index` field to help identify a Search index. +The `indexUUID` field and the `index` field must both identify the same Search index. +If they identify different Search indexes, or if either of them does not identify a Search index, an error occurs. -You can find the UUID of a full text index by viewing the index definition. -You can do this using the xref:server:fts:fts-creating-indexes.adoc#using-the-index-definition-preview[Index Definition Preview] in the Query Workbench, or the xref:server:rest-api:rest-fts-indexing.adoc[Index Definition] endpoints provided by the Full Text Search REST API. +You can find the UUID of a Search index by viewing the index definition. +Click a Search index name in the Capella UI to open the editor. +Click btn:[Index Definition] to view the index definition. | `out` [Optional] | string -| A name given to this full text search operation in this keyspace. +| A name given to this Full Text Search operation in this keyspace. You can use this name to refer to this operation using the <> and <> functions. -If this field is omitted, the name of this full text search operation defaults to `"out"`. +If this field is omitted, the name of this Full Text Search operation defaults to `"out"`. | (other) [Optional] @@ -177,7 +182,7 @@ This returns `true` if the search query is found within the input path, or `fals === Limitations -The Query service can select a full text index for efficient search in the following cases: +The Query service can select a Search index for efficient search in the following cases: * If the SEARCH() function is used in a WHERE clause or in an ANSI JOIN. The SEARCH() function must be on the leftmost (first) JOIN. @@ -186,7 +191,7 @@ RIGHT OUTER JOINs are rewritten as LEFT OUTER JOINs. * If the SEARCH() function is evaluated on the `true` condition in positive cases: for example, `SEARCH(_field_, _query_, _options_)`, `SEARCH(_field_, _query_, _options_) = true`, `SEARCH(_field_, _query_, _options_) IN [true, true, true]`, or a condition including one of these with `AND` or `OR`. -The Query service cannot select a full text index for efficient search in the following cases: +The Query service cannot select a Search index for efficient search in the following cases: * If a USE KEYS hint is present; or if the SEARCH() function is used on the inner side of a nested-loop JOIN, a lookup JOIN or lookup NEST, an index JOIN or index NEST, an UNNEST clause, a subquery expression, a subquery result, or a correlated query. @@ -285,7 +290,7 @@ WHERE SEARCH(t1, { The results are unordered, so they may be returned in a different order each time. ==== -.Search using a complete full text search request +.Search using a complete Full Text Search request ==== .Query [source,sqlpp] @@ -336,7 +341,7 @@ This query returns 5 results, and the results are ordered, as specified by the s As an alternative, you could limit the number of results and order them using the {sqlpp} xref:n1ql-language-reference/limit.adoc[LIMIT] and xref:n1ql-language-reference/orderby.adoc[ORDER BY] clauses. ==== -.Search against a full text search index that carries a custom type mapping +.Search against a Full Text Search index that carries a custom type mapping ==== .Query [source,sqlpp] @@ -365,14 +370,56 @@ WHERE t1.type = "hotel" AND SEARCH(t1.description, "amazing"); ] ---- -If the full text search index being queried has its default mapping disabled and has a custom type mapping defined, the query needs to specify the type explicitly. +If the Full Text Search index being queried has its default mapping disabled and has a custom type mapping defined, the query needs to specify the type explicitly. The above query uses the demonstration index xref:server:fts:fts-demonstration-indexes.adoc#travel-sample-index-hotel-description[travel-sample-index-hotel-description], which has the custom type mapping "hotel". -For more information on defining custom type mappings within the full text search index, refer to xref:server:fts:fts-creating-indexes.adoc#specifying-type-mappings[Specifying Type Mappings]. -Note that for {sqlpp} queries, only full text search indexes with one type mapping are searchable. +For more information about defining custom type mappings within a Search index, refer to xref:search:create-type-mapping.adoc[Create a Type Mapping]. +Note that for {sqlpp} queries, only Search indexes with one type mapping are searchable. Also the supported type identifiers at the moment are "type_field" and "docid_prefix"; "docid_regexp" isn't supported yet for SEARCH queries via {sqlpp}. ==== +.Search against a Vector Search index for the closest 2 vectors +==== +NOTE: This example does not use the travel sample data or query context mentioned before, as it requires documents that contain vector data. + +.Query +[source,sqlpp] +---- +SELECT t1.color +FROM rgb AS t1 +WHERE SEARCH(t1, + { + "fields": ["*"], + "query": { + "match_none": "" + }, + "knn": [ + { + "k": 2, + "field": "colorvect_dot", + "vector": [ 0.707106781186548, 0, 0.707106781186548 ] + } + ] + } +) +---- + +.Results +---- +[ + { + "color": "magenta / fuchsia" + }, + { + "color": "dark lavender" + } +] +---- +Using a Search index that contains vectors that describe colors, the query returns the color names of the closest `k` vectors to the vector in the `knn` object. + +For more information about Vector Search and the Search Services, see xref:vector-search:vector-search.adoc[]. +==== + [[search_meta,SEARCH_META()]] == SEARCH_META([`identifier`]) @@ -380,12 +427,12 @@ Also the supported type identifiers at the moment are "type_field" and "docid_pr This function is intended to be used in a query which contains a <> function. It returns the metadata given by the Search engine for each document found by the <> function. -If there is no <> function in the query, or if a full text index was not used to evaluate the search, the function returns MISSING. +If there is no <> function in the query, or if a Search index was not used to evaluate the search, the function returns MISSING. === Arguments identifier:: -[Optional] An expression in the form `{startsb}__keyspaceAlias__.{endsb}__outname__`, consisting of the keyspace or keyspace alias in which the full text search operation was performed, followed by the outname of the full text search operation, using dot notation. +[Optional] An expression in the form `{startsb}__keyspaceAlias__.{endsb}__outname__`, consisting of the keyspace or keyspace alias in which the Full Text Search operation was performed, followed by the outname of the Full Text Search operation, using dot notation. [NOTE] -- @@ -430,7 +477,7 @@ LIMIT 3; ---- <1> There is only one <> function in this query, so the SEARCH_META() function does not need to specify the outname. -<2> The full text search specifies that locations should be included in the search result metadata. +<2> The Full Text Search specifies that locations should be included in the search result metadata. .Result [source,json] @@ -505,14 +552,14 @@ This query only uses one data source, so there is no need to specify the keyspac This function is intended to be used in a query which contains a <> function. It returns the score given by the Search engine for each document found by the <> function. -If there is no <> function in the query, or if a full text index was not used to evaluate the search, the function returns MISSING. +If there is no <> function in the query, or if a Search index was not used to evaluate the search, the function returns MISSING. This function is the same as <>. === Arguments identifier:: -[Optional] An expression in the form `{startsb}__keyspaceAlias__.{endsb}__outname__`, consisting of the keyspace or keyspace alias in which the full text search operation was performed, followed by the outname of the full text search operation, using dot notation. +[Optional] An expression in the form `{startsb}__keyspaceAlias__.{endsb}__outname__`, consisting of the keyspace or keyspace alias in which the Full Text Search operation was performed, followed by the outname of the Full Text Search operation, using dot notation. [NOTE] -- @@ -572,4 +619,6 @@ LIMIT 3; == Related Links -xref:server:fts:full-text-intro.adoc[Full Text Search] +* xref:search:search.adoc[] +* xref:vector-search:vector-search.adoc[] + diff --git a/modules/search/examples/create-search-index-header.sh b/modules/search/examples/create-search-index-header.sh index 7b32f5f28..2056b8e36 100644 --- a/modules/search/examples/create-search-index-header.sh +++ b/modules/search/examples/create-search-index-header.sh @@ -1,3 +1,3 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/index/landmark-content-index + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index -d \ \ No newline at end of file diff --git a/modules/search/examples/create-search-index-payload.sh b/modules/search/examples/create-search-index-payload.sh index be736d834..8ee2df3a4 100644 --- a/modules/search/examples/create-search-index-payload.sh +++ b/modules/search/examples/create-search-index-payload.sh @@ -1,5 +1,5 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/index/landmark-content-index + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index -d \ '{ "type": "fulltext-index", diff --git a/modules/search/examples/query-geojson.jsonc b/modules/search/examples/query-geojson.jsonc index 7d79b8d63..fbc32802c 100644 --- a/modules/search/examples/query-geojson.jsonc +++ b/modules/search/examples/query-geojson.jsonc @@ -4,7 +4,7 @@ "field": "geojson", "geometry": { "shape": { - "type": "point", + "type": "Point", "coordinates": [0.47482593026924746, 51.31232878073189] }, "relation": "intersects" @@ -18,7 +18,7 @@ "field": "geojson", "geometry": { "shape": { - "type": "linestring", + "type": "LineString", "coordinates": [ [-2.753735609842721, 53.94860827535115], [-2.599898256093695,53.65007434185782] @@ -138,17 +138,17 @@ "field": "geojson", "geometry": { "shape": { - "type": "geometrycollection", + "type": "GeometryCollection", "geometries": [ { - "type": "linestring", + "type": "LineString", "coordinates": [ [-2.753735609842721, 53.94860827535115], [-2.599898256093695, 53.65007434185782] ] }, { - "type": "multipolygon", + "type": "MultiPolygon", "coordinates": [ [[ [-1.8167959002718135, 53.8626654046235], @@ -183,7 +183,7 @@ "geometry": { "shape": { "coordinates": [-2.2450285424707204, 53.48503270828377], - "type": "circle", + "type": "Circle", "radius": "100mi" }, "relation": "within" diff --git a/modules/search/examples/query-ip-range.jsonc b/modules/search/examples/query-ip-range.jsonc new file mode 100644 index 000000000..f8bf914f0 --- /dev/null +++ b/modules/search/examples/query-ip-range.jsonc @@ -0,0 +1,14 @@ +{ + "query": { + "conjuncts": [ + { + "cidr": "2.7.13.0/24", + "field": "ipv4" + }, + { + "cidr": "2001:db8:1234:5678::/119", + "field": "ipv6" + } + ] + } +} diff --git a/modules/search/examples/run-search-full-request.jsonc b/modules/search/examples/run-search-full-request.jsonc index e689da111..4a2ab1bdd 100644 --- a/modules/search/examples/run-search-full-request.jsonc +++ b/modules/search/examples/run-search-full-request.jsonc @@ -41,7 +41,15 @@ } }, // end::query[] - + // tag::knn[] + "knn": [ + { + "k": 10, + "field": "vector_field", + "vector": [ 0.707106781186548, 0, 0.707106781186548 ] + } + ], + // end::knn[] // tag::ctl[] "ctl": { "timeout": 10000, diff --git a/modules/search/examples/run-search-header.sh b/modules/search/examples/run-search-header.sh index d014a4b57..805dfcff3 100644 --- a/modules/search/examples/run-search-header.sh +++ b/modules/search/examples/run-search-header.sh @@ -1,2 +1,2 @@ curl -s -XPUT -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/index/$INDEX-NAME/query -d \ \ No newline at end of file + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/${BUCKET-NAME}/scope/${SCOPE-NAME}/index/${INDEX-NAME}/query -d \ \ No newline at end of file diff --git a/modules/search/examples/run-search-payload.sh b/modules/search/examples/run-search-payload.sh index 5101e2fa0..3ccc0d6b8 100644 --- a/modules/search/examples/run-search-payload.sh +++ b/modules/search/examples/run-search-payload.sh @@ -1,5 +1,5 @@ curl -XPOST -H "Content-Type: application/json" \ - -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOSTNAME:8094/api/index/landmark-content-index/query \ + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/travel-sample/scope/inventory/index/landmark-content-index/query \ -d '{ "explain": true, "fields": [ diff --git a/modules/search/pages/child-field-options-reference.adoc b/modules/search/pages/child-field-options-reference.adoc new file mode 100644 index 000000000..b48f19392 --- /dev/null +++ b/modules/search/pages/child-field-options-reference.adoc @@ -0,0 +1,102 @@ += Child Field Options +:page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: You can configure a child field from the Couchbase {page-ui-name}'s Advanced Mode to control how its data is added to your Search index. + +[abstract] +{description} + +For more information about how to create a child field using Advanced Mode, see xref:create-child-field.adoc[]. + +For more information about creating an index and child fields in Quick Mode, see xref:create-quick-index.adoc[]. + +You can configure the following options for a child field in Advanced Mode, based on its type: + + +|==== +|Option |Description + +|Dimension (Vector Fields Only) a| + +include::partial$vector-search-field-descriptions.adoc[tag=dimension] + +|Similarity Metric (Vector Fields Only) a| + +include::partial$vector-search-field-descriptions.adoc[tag=similarity_metric] + +|Optimized For (Vector Fields Only) a| + +include::partial$vector-search-field-descriptions.adoc[tag=optimized_for] + +|Searchable As a| + +Set a different name that you can use to search the field's contents in a query. + +The default value is the value set in the *Field* field when you xref:create-child-field.adoc#field[create a child field]. + +|Analyzer a| + +Choose an analyzer for the content in the child field. + +You can choose a default analyzer or a custom analyzer. + +For more information about the available default analyzers, see xref:default-analyzers-reference.adoc[]. + +For more information about how to create a custom analyzer, see xref:create-custom-analyzer.adoc[]. + +|Index a| + +To include the child field in the index, select *Index*. + +To remove the child field from the index, clear *Index*. + +If you remove the child field from the index, it cannot appear in search results. + +|[[store]]Store a| + +To store the values from the field in the index and return them in search results, select *Store*. +This increases your index's size and indexing time. + +To remove the field's values from the index, clear *Store*. + +|Include in _all field a| + +The `_all` field is a composite field that has the content from multiple fields in an index. +It allows searches to query the content of a field without specifying the field's name. + +To include this field in the `_all` field, select *Include in _all field*. + +To exclude this field from the `_all` field, clear *Include in _all field*. + +TIP: To change the name of the `_all` field, see xref:set-advanced-settings.adoc#all-field[Default Field]. + +|[[term-vectors]]Include Term Vectors a| + +Term vectors store the location of terms in a field for an index. +You can use term vectors to xref:search-request-params.adoc#highlight[highlight matching search terms in search results], and xref:search-request-params.adoc#match_phrase[perform phrase searches]. +Term vectors increase your index's size and indexing time. + +To enable term vectors for this field, select *Include Term Vectors*. + +To turn off term vectors, clear *Include Term Vectors*. + +NOTE: To enable term vectors, you must also enable <>. + +|Doc Values a| + +Use doc values for a field for Search xref:search-request-params.adoc#facets[Facets] and sorting your search results. + +To store doc values and enable faceting and sorting for this field, select *Doc Values*. + +To exclude doc values from the index for this field, clear *Doc Values*. +|==== + +== See Also + +* xref:create-child-field.adoc[] +* xref:create-search-index-ui.adoc[] +* xref:vector-search:create-vector-search-index-ui.adoc[] +* xref:set-type-identifier.adoc[] +* xref:create-type-mapping.adoc[] +* xref:set-advanced-settings.adoc[] \ No newline at end of file diff --git a/modules/search/pages/create-child-field.adoc b/modules/search/pages/create-child-field.adoc index 0dac7c824..a3415222b 100644 --- a/modules/search/pages/create-child-field.adoc +++ b/modules/search/pages/create-child-field.adoc @@ -1,23 +1,30 @@ -= Create a Child Field += Create a Child Field in Advanced Mode :page-topic-type: guide -:description: Create a child field with the Couchbase Capella UI to add or remove a specific field's content from a Search index. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a child field with the Couchbase {page-ui-name}'s Advanced Mode to add or remove a specific field's content from a Search index. [abstract] {description} == Prerequisites -* You've created an index. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have created an index. For more information, see xref:create-search-index-ui.adoc[]. -* You've created a type mapping. +* You have created a type mapping. For more information about how to create a type mapping on an index, see xref:create-type-mapping.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To add or remove a child field from a Search index with the Capella UI: +To add or remove a child field from a Search index with the {page-ui-name}: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. @@ -30,72 +37,9 @@ NOTE: The field must contain either a single value or an array. If the field con + For more information about the available data types, see xref:field-data-types-reference.adoc[]. -. Configure optional settings for the child field: +. Configure optional settings for the child field. + -|==== -|Option |Description - -|Searchable As a| - -Set a different name that you can use to search the field's contents in a query. - -The default value is the value set in *Field*. - -|Analyzer a| - -Choose an analyzer for the content in the child field. - -You can choose a default analyzer or a custom analyzer. - -For more information about the available default analyzers, see xref:default-analyzers-reference.adoc[]. - -For more information about how to create a custom analyzer, see xref:create-custom-analyzer.adoc[]. - -|Index a| - -To include the child field in the index, select *Index*. - -To remove the child field from the index, clear *Index*. - -|[[store]]Store a| - -To store the values from the field in the index and return them in search results, select *Store*. -This increases your index's size and indexing time. - -To remove the field's values from the index, clear *Store*. - -|Include in _all field a| - -The `_all` field is a composite field that has the content from multiple fields in an index. -It allows searches to query the content of a field without specifying the field's name. - -To include this field in the `_all` field, select *Include in _all field*. - -To exclude this field from the `_all` field, clear *Include in _all field*. - -TIP: To change the name of the `_all` field, see xref:set-advanced-settings.adoc#all-field[Default Field]. - -|[[term-vectors]]Include Term Vectors a| - -Term vectors store the location of terms in a field for an index. -You can use term vectors to highlighting matching search terms in search results, and perform phrase searches. -Term vectors increase your index's size and indexing time. - -To enable term vectors for this field, select *Include Term Vectors*. - -To turn off term vectors, clear *Include Term Vectors*. - -NOTE: To enable term vectors, you must also enable <>. - -|Doc Values a| - -Doc values are the value for each instance of the field in an index. -Use doc values for Search xref:search-request-params.adoc#facets[Facets] and sorting search results. - -To store doc values, select *Doc Values*. - -To exclude doc values from the index, clear *Doc Values*. -|==== +For more information about the available settings for child fields, see xref:child-field-options-reference.adoc[]. . Click btn:[Submit]. . Click btn:[Update Index]. diff --git a/modules/search/pages/create-child-mapping.adoc b/modules/search/pages/create-child-mapping.adoc index faa9e2f66..6befabe96 100644 --- a/modules/search/pages/create-child-mapping.adoc +++ b/modules/search/pages/create-child-mapping.adoc @@ -1,23 +1,30 @@ -= Create a Child Mapping += Create a Child Mapping in Advanced Mode :page-topic-type: guide -:description: Create a child mapping with the Couchbase Capella UI to add or remove a field that contains a JSON object from a Search index. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a child mapping with the Couchbase {page-ui-name}'s Advanced Mode to add or remove a field that contains a JSON object from a Search index. [abstract] {description} == Prerequisites -* You've created an index. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have created an index. For more information, see xref:create-search-index-ui.adoc[]. -* You've created a type mapping. +* You have created a type mapping. For more information about how to create a type mapping on an index, see xref:create-type-mapping.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a child mapping with the Capella UI: +To create a child mapping with the {page-ui-name}: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/create-custom-analyzer.adoc b/modules/search/pages/create-custom-analyzer.adoc index 7b3fc0af1..c5e38fe63 100644 --- a/modules/search/pages/create-custom-analyzer.adoc +++ b/modules/search/pages/create-custom-analyzer.adoc @@ -1,27 +1,36 @@ = Create a Custom Analyzer :page-topic-type: guide -:description: Create a custom analyzer with the Couchbase Capella UI to modify the input text from a Search query or Search index and improve search results. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a custom analyzer with the Couchbase {page-ui-name}'s Advanced Mode to modify the input text from a Search query or Search index and improve search results. [abstract] {description} -For more information, see xref:customize-index.adoc#analyzers[]. +NOTE: You must use Advanced Mode to add a custom analyzer to your Search index. +For more information, see xref:create-search-index-ui.adoc[]. + +For more information, see xref:customize-index.adoc#analyzers[Search Index Features]. == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have created an index with xref:create-search-index-ui.adoc[Advanced Mode]. + +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To create a custom analyzer with the Capella UI: +To create a custom analyzer with the {page-ui-name} in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index where you want to create a custom analyzer. -. Expand *Analyzers*. +. Under *Advanced Settings*, expand *Analyzers*. ++ +NOTE: Make sure you use *Advanced Mode*. . Click btn:[Add Analyzer]. . In the *Name* field, enter a name for the new custom analyzer. . To remove specific characters from search input before tokenizing, in the *Character Filters* list, do one of the following: @@ -46,12 +55,13 @@ For more information, see xref:default-token-filters-reference.adoc[] or xref:cr After you create a custom analyzer, you can xref:set-advanced-settings.adoc#default-analyzer[set it as the default analyzer] for your Search index. -You can also use the custom analyzer when you xref:create-type-mapping.adoc[], xref:create-child-mapping.adoc[], and xref:create-child-field.adoc[]. +You can also use the custom analyzer when you xref:create-type-mapping.adoc[create a type mapping with Advanced Mode], xref:create-child-mapping.adoc[a child mapping] or xref:create-child-field.adoc[a child field]. To continue customizing your Search index, you can also: * xref:set-advanced-settings.adoc[] * xref:set-type-identifier.adoc[] +* xref:create-type-mapping.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-tokenizer.adoc[] * xref:create-custom-token-filter.adoc[] diff --git a/modules/search/pages/create-custom-character-filter.adoc b/modules/search/pages/create-custom-character-filter.adoc index 3277676ff..db2d3e534 100644 --- a/modules/search/pages/create-custom-character-filter.adoc +++ b/modules/search/pages/create-custom-character-filter.adoc @@ -1,25 +1,31 @@ = Create a Custom Character Filter :page-topic-type: guide -:description: Create a custom character filter with the Couchbase Capella UI to remove unwanted characters from a Search query or the contents of a Search index. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a custom character filter with the Couchbase {page-ui-name} to remove unwanted characters from a Search query or the contents of a Search index. [abstract] {description} == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have logged in to the Couchbase {page-ui-name}. -* You've logged in to the Couchbase Capella UI. +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a custom character filter with the Capella UI: +To create a custom character filter with the {page-ui-name} in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index where you want to create a custom character filter. -. Expand *Custom Filters*. +. Under *Advanced Settings*, expand *Custom Filters*. ++ +NOTE: Make sure you use *Advanced Mode*. . Click btn:[Add Character Filter]. . In the *Name* field, enter a name for the character filter. . In the *Regular Expression* field, enter the regular expression for the character filter. diff --git a/modules/search/pages/create-custom-date-time-parser.adoc b/modules/search/pages/create-custom-date-time-parser.adoc index 7317adf67..755015c4d 100644 --- a/modules/search/pages/create-custom-date-time-parser.adoc +++ b/modules/search/pages/create-custom-date-time-parser.adoc @@ -1,6 +1,8 @@ = Create a Custom Date/Time Parser :page-topic-type: guide -:description: Create a custom date/time parser with the Couchbase Capella UI to tell the Search Service how to process a new date/time format. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a custom date/time parser with the Couchbase {page-ui-name} to tell the Search Service how to process a new date/time format. [abstract] {description} @@ -9,19 +11,23 @@ If you store date data in a format other than RFC-3339 (ISO-8601), then you need == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a custom date/time parser with the Capella UI: +To create a custom date/time parser with the {page-ui-name} in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index where you want to create a custom date/time parser. -. Expand *Date/Time Parsers*. +. Under *Advanced Settings*, expand *Date Time Parsers*. ++ +NOTE: Make sure you use *Advanced Mode*. . Click btn:[Add Date/Time Parser] . In the *Name* field, enter a name for the date/time parser. . In the *New Word* field, enter a date/time layout with Go syntax. diff --git a/modules/search/pages/create-custom-token-filter.adoc b/modules/search/pages/create-custom-token-filter.adoc index 2857c889f..524b2d2a6 100644 --- a/modules/search/pages/create-custom-token-filter.adoc +++ b/modules/search/pages/create-custom-token-filter.adoc @@ -1,6 +1,8 @@ = Create a Custom Token Filter :page-topic-type: guide -:description: Create a custom token filter with the Couchbase Capella UI to change how the Search Service creates tokens from Search index content and Search queries. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a custom token filter with the Couchbase {page-ui-name} to change how the Search Service creates tokens from Search index content and Search queries. :page-toclevels: 3 [abstract] @@ -10,19 +12,23 @@ xref:customize-index.adoc#token-filters[Token filters] can improve your search r == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have logged in to the Couchbase {page-ui-name}. -* You've logged in to the Couchbase Capella UI. +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a custom token filter with the Capella UI: +To create a custom token filter with the {page-ui-name} in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index where you want to create a custom token filter. -. Expand *Custom Filters*. +. Under *Advanced Settings*, expand *Custom Filters*. ++ +NOTE: Make sure you use *Advanced Mode*. . Click btn:[Add Token Filter]. . In the *Name* field, enter a name for the token filter. @@ -68,7 +74,7 @@ You can create any of the following custom token filters: include::partial$custom-token-filters-descriptions.adoc[tags=dict;!dict_example] -To create a new `dict_compound` token filter with the Capella UI: +To create a new `dict_compound` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *dict_compound*. . In the *Sub Words* list, select the wordlist to use to find subwords in input tokens. @@ -80,7 +86,7 @@ You can choose your own xref:create-custom-wordlist.adoc[custom wordlist] or a x === Create a Custom `edge_ngram` Token Filter include::partial$custom-token-filters-descriptions.adoc[tags=edge;!edge_example] -To create a new `edge_ngram` token filter with the Capella UI: +To create a new `edge_ngram` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *edge_ngram*. . Do one of the following: @@ -95,7 +101,7 @@ To create a new `edge_ngram` token filter with the Capella UI: include::partial$custom-token-filters-descriptions.adoc[tags=elision;!elision_example] -To create a new `elision` token filter with the Capella UI: +To create a new `elision` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *elision*. . In the *Articles* list, select a wordlist to use to find elisions in input tokens. @@ -108,7 +114,7 @@ You can choose your own xref:create-custom-wordlist.adoc[custom wordlist] or a x include::partial$custom-token-filters-descriptions.adoc[tags=keyword;!keyword_example] -To create a new `keyword_marker` token filter with the Capella UI: +To create a new `keyword_marker` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *keyword_marker*. . In the *Articles* list, select a wordlist to use to find keywords to create tokens. @@ -121,7 +127,7 @@ You can choose your own xref:create-custom-wordlist.adoc[custom wordlist] or a x include::partial$custom-token-filters-descriptions.adoc[tags=length;!length_example] -To create a new `length` token filter with the Capella UI: +To create a new `length` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *length*. . In the *Min* box, enter the minimum character length for a new token. @@ -133,7 +139,7 @@ To create a new `length` token filter with the Capella UI: include::partial$custom-token-filters-descriptions.adoc[tags=ngram;!ngram_example] -To create a new `ngram` token filter with the Capella UI: +To create a new `ngram` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *ngram*. . In the *Min* box, enter the minimum character length for a new token. @@ -145,7 +151,7 @@ To create a new `ngram` token filter with the Capella UI: include::partial$custom-token-filters-descriptions.adoc[tags=normalize;!normalize_example] -To create a new `normalize_unicode` token filter with the Capella UI: +To create a new `normalize_unicode` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *normalize_unicode*. . In the *Form* list, select the type of Unicode normalization to apply: @@ -161,7 +167,7 @@ To create a new `normalize_unicode` token filter with the Capella UI: include::partial$custom-token-filters-descriptions.adoc[tags=shingle;!shingle_example] -To create a new `shingle` token filter with the Capella UI: +To create a new `shingle` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *shingle*. . In the *Min* box, enter the minimum character length for a new token before concatenation. @@ -178,7 +184,7 @@ To create a new `shingle` token filter with the Capella UI: include::partial$custom-token-filters-descriptions.adoc[tags=stop;!stop_example] -To create a new `stop_tokens` token filter with the Capella UI: +To create a new `stop_tokens` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *stop_tokens*. . In the *Stop Words* list, select a wordlist to use to remove tokens. @@ -191,7 +197,7 @@ You can choose your own xref:create-custom-wordlist.adoc[custom wordlist] or a x include::partial$custom-token-filters-descriptions.adoc[tags=truncate;!truncate_example] -To create a new `truncate_token` token filter with the Capella UI: +To create a new `truncate_token` token filter with the {page-ui-name} in Advanced Mode: . In the *Type* list, select *truncate_token*. . In the *Length* box, enter the maximum character length for an output token. @@ -208,6 +214,7 @@ To continue customizing your Search index, you can also: * xref:create-type-mapping.adoc[] * xref:create-child-field.adoc[] * xref:create-child-mapping.adoc[] +* xref:create-custom-analyzer.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-tokenizer.adoc[] * xref:create-custom-wordlist.adoc[] diff --git a/modules/search/pages/create-custom-tokenizer.adoc b/modules/search/pages/create-custom-tokenizer.adoc index 0a02fe663..98d50f617 100644 --- a/modules/search/pages/create-custom-tokenizer.adoc +++ b/modules/search/pages/create-custom-tokenizer.adoc @@ -1,6 +1,8 @@ = Create a Custom Tokenizer :page-topic-type: guide -:description: Create a custom tokenizer with the Couchbase Capella UI to change how the Search Service creates tokens for matching Search index content to a Search query. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a custom tokenizer with the Couchbase {page-ui-name} to change how the Search Service creates tokens for matching Search index content to a Search query. :page-toclevels: 3 [abstract] @@ -8,10 +10,13 @@ == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You've logged in to the Couchbase Capella UI. + +* You have logged in to the Couchbase {page-ui-name}. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure @@ -29,12 +34,14 @@ You can create 2 types of custom tokenizers: [#regexp] === Create a Regular Expression Tokenizer -To create a regular expression tokenizer with the Capella UI: +To create a regular expression tokenizer with the {page-ui-name}: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index where you want to create a custom tokenizer. -. Expand *Custom Filters*. +. Under *Advanced Settings*, expand *Custom Filters*. ++ +NOTE: Make sure you use *Advanced Mode*. . Click btn:[Add Tokenizer]. . In the *Name* field, enter a name for the custom tokenizer. . In the *Type* list, select *regexp*. @@ -44,7 +51,7 @@ To create a regular expression tokenizer with the Capella UI: [#excep] === Create an Exception Custom Tokenizer -To create an exception custom tokenizer with the Capella UI: +To create an exception custom tokenizer with the {page-ui-name} in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/create-custom-wordlist.adoc b/modules/search/pages/create-custom-wordlist.adoc index 9310468e6..166e21074 100644 --- a/modules/search/pages/create-custom-wordlist.adoc +++ b/modules/search/pages/create-custom-wordlist.adoc @@ -1,6 +1,8 @@ = Create a Custom Wordlist :page-topic-type: guide -:description: Create a custom wordlist with the Couchbase Capella UI to use with a custom token filter. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a custom wordlist with the Couchbase {page-ui-name} to use with a custom token filter. [abstract] {description} @@ -11,19 +13,24 @@ For more information about how to create a custom token filter, see xref:create- == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You've logged in to the Couchbase Capella UI. + +* You have logged in to the Couchbase {page-ui-name}. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. == Procedure -To create a custom wordlist with the Capella UI: +To create a custom wordlist with the {page-ui-name} in Advanced Mode: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index where you want to create a custom wordlist. -. Expand *Custom Filters*. +. Under *Advanced Settings*, expand *Custom Filters*. ++ +NOTE: Make sure you use *Advanced Mode*. . Click btn:[Add Wordlist]. . In the *Name* field, enter a name for the wordlist. . In the *New Word* field, enter a word you want to add to the wordlist. diff --git a/modules/search/pages/create-quick-index.adoc b/modules/search/pages/create-quick-index.adoc new file mode 100644 index 000000000..9fac8e6e7 --- /dev/null +++ b/modules/search/pages/create-quick-index.adoc @@ -0,0 +1,64 @@ += Create a Search Index in Quick Mode +:page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Use Quick Mode to create a Search index in Couchbase Capella with streamlined options. + +[abstract] +{description} + +Quick Mode works best when you need to create a basic Search index to start testing and prototyping with the Search Service. +You must use Advanced Mode to have greater control over how the Search Service returns such results, such as changing your xref:customize-index.adoc#analyzers[analyzer]. + +For more information about how to create a Search index in Advanced Mode, see xref:create-search-index-ui.adoc[]. + +You must create a Search index before you can xref:simple-search-ui.adoc[run a search] with the Search Service. + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. + +* You have logged in to the Couchbase {page-ui-name}. + +== Procedure + +To use Quick Mode in Couchbase Capella to create a Search index: + +. On the *Databases* page, select the database where you want to create a Search index. +. Go to menu:Data Tools[Search]. +. Click btn:[Create Search Index]. ++ +By default, *Quick Mode* should be on. +. In the *Index Name* field, enter a name for the index. ++ +[NOTE] +==== +Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). + +For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. +You cannot have 2 indexes with the same name inside the same bucket and scope. +==== +. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. +. In the *Scope* list, select the scope where you want to create the index. +. Choose a type mapping to add to your index: +.. To add an entire collection as a type mapping, select a collection name. +.. To only add a single field as a type mapping, expand a collection, then select a field from your document schema. +. Configure the options for your new type mapping. +For more information about the available options, see xref:create-type-mapping.adoc[]. +. Click btn:[Add]. +. (Optional) Add additional collections or field type mappings to your index. +. Click btn:[Create Index]. + +include::partial$large-index-caution.adoc[tag=regular-search] + +== Next Steps + +If you want to add more advanced settings and customization to your Search index, you can xref:create-search-index-ui.adoc[use Advanced Mode]. + +CAUTION: If you edit your Search index in Advanced Mode, you cannot return to Quick Mode and keep any Advanced Mode settings. + +To run a search and test the contents of your Search index, see xref:simple-search-ui.adoc[] or xref:simple-search-rest-api.adoc[]. \ No newline at end of file diff --git a/modules/search/pages/create-search-index-alias.adoc b/modules/search/pages/create-search-index-alias.adoc index fc3b045d3..fb7f48054 100644 --- a/modules/search/pages/create-search-index-alias.adoc +++ b/modules/search/pages/create-search-index-alias.adoc @@ -1,5 +1,7 @@ -= Create a Search Index Alias with the Capella UI += Create a Search Index Alias with the {page-ui-name} :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Use a Search index alias to run a Search query across multiple buckets, scopes, or Search indexes. [abstract] @@ -9,14 +11,17 @@ For more information about Search index aliases, see xref:index-aliases.adoc[]. == Prerequisites -* You've created at least one Search index. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have created at least one Search index. For more information, see xref:create-search-index-ui.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To create a Search index alias with the Capella UI: +To create a Search index alias with the {page-ui-name}: . On the *Databases* page, select the database where you want to create a Search index alias. . Go to menu:Data Tools[Search]. diff --git a/modules/search/pages/create-search-index-rest-api.adoc b/modules/search/pages/create-search-index-rest-api.adoc index 5e85c1094..307648375 100644 --- a/modules/search/pages/create-search-index-rest-api.adoc +++ b/modules/search/pages/create-search-index-rest-api.adoc @@ -1,5 +1,7 @@ = Create a Search Index with the REST API and curl/HTTP :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: You can create a Search index with the Search Service API. [abstract] @@ -9,40 +11,54 @@ You must create a Search index before you can xref:simple-search-rest-api.adoc[r == Prerequisites -* You have the Search Service enabled on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have a bucket with scopes and collections in your database. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've installed the Couchbase command-line tool (CLI). - -* You have the hostname or IP address for your database. +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. == Procedure To create a Search index with the REST API: . In your command-line tool, enter a `curl` command with the `XPUT` verb. -. Set your header content to include `Content-Type: application/json`. -. Enter your username, password, and the Search Service endpoint on port `8094` with the name of the index you want to create: +. Set your header content to include `"Content-Type: application/json"`. +. Enter your username, password, and the Search Service endpoint on port `18094` with the name of the index you want to create: + [source,console] ---- include::example$create-search-index-header.sh[] ---- -. Enter the JSON payload for the settings you want in your index. + -Don't include the xref:search-index-params.adoc#uuid[uuid] or xref:search-index-params.adoc#sourceuuid[sourceUUID] parameters. +[NOTE] +==== +Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). + +As of version 7.6, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +==== + +. Enter the JSON payload for the settings you want in your index. + -TIP: If you remove the xref:search-index-params.adoc#uuid[uuid] and xref:search-index-params.adoc#sourceuuid[sourceUUID] parameters, you can copy the Search index definition JSON payload from the Couchbase Capella UI to use in your REST API call. -For more information about how to create an index with the UI, see xref:create-search-index-ui.adoc[]. +Do not include the xref:search-index-params.adoc#uuid[uuid] or xref:search-index-params.adoc#sourceuuid[sourceUUID] parameters. + +TIP: If you remove the xref:search-index-params.adoc#uuid[uuid] and xref:search-index-params.adoc#sourceuuid[sourceUUID] parameters, you can copy the Search index definition JSON payload from the Couchbase {page-ui-name} to use in your REST API call. +For more information about how to create an index with the UI, see xref:create-search-index-ui.adoc[]. + +include::partial$large-index-caution.adoc[tag=regular-search] + +=== Example + In the following example, the JSON payload creates a simple index named `landmark-content-index` on the `travel-sample` bucket. It creates a type mapping for the `inventory.landmark` collection, with a child field, `content`: -+ + [source,console] ---- include::example$create-search-index-payload.sh[] ---- -+ + For more information about the available JSON properties for a Search index, see xref:search-index-params.adoc[]. == Next Steps diff --git a/modules/search/pages/create-search-index-ui.adoc b/modules/search/pages/create-search-index-ui.adoc index 198c4eaf7..23e6932b8 100644 --- a/modules/search/pages/create-search-index-ui.adoc +++ b/modules/search/pages/create-search-index-ui.adoc @@ -1,45 +1,62 @@ -= Create a Basic Search Index with the Capella UI += Create a Basic Search Index in Advanced Mode :page-topic-type: guide -:description: You can create a Search index with the Couchbase Capella UI. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Use Advanced Mode to create a Search index in Couchbase Capella with full customization and advanced features. [abstract] {description} +Use xref:create-quick-index.adoc[Quick Mode] if you do not need full customization or advanced settings to quickly create a Search index and xref:customize-index.adoc#type-mappings[type mappings]. + You must create a Search index before you can xref:simple-search-ui.adoc[run a search] with the Search Service. == Prerequisites -* You've deployed the Search Service on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have a bucket with scopes and collections in your database. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To create a Search index with the Capella UI: +To use Advanced Mode in Couchbase Capella to create a Search index: . On the *Databases* page, select the database where you want to create a Search index. . Go to menu:Data Tools[Search]. . Click btn:[Create Search Index]. +. Click btn:[Advanced Mode]. . In the *Index Name* field, enter a name for the Search index. + -NOTE: Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). +[NOTE] +==== +Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). + +For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +==== -. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create a type mapping. -. In the *Scope* list, select the scope where you want to create a type mapping. +. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. +. In the *Scope* list, select the scope where you want to create your Search index. . Click btn:[Add Mapping]. -. In the *Collection* list, select the collection for the type mapping. +. In the *Collection* list, select a collection for the type mapping. +. (Optional) To switch from a xref:customize-index.adoc#type-mappings[dynamic to a static type mapping], click *Index only specified fields*. . Click btn:[Submit]. +. (Optional) Do any of the following: +.. If you created a xref:customize-index.adoc#type-mappings[static type mapping], then you should xref:create-child-field.adoc[add a child field] or xref:create-child-mapping.adoc[add a child mapping] to your type mapping to specify the documents you want to include in your index. +.. Add additional features to your Search index to improve performance and search results. +For more information, see xref:customize-index.adoc[]. . Click btn:[Create Index]. -TIP: To view and copy the JSON definition for your Search index to use xref:create-search-index-rest-api.adoc[with the REST API], click btn:[Index Definition]. +include::partial$large-index-caution.adoc[tag=regular-search] == Next Steps -This basic index includes all documents from the collection you selected in the Data Insights area. +If you did not create a static type mapping, this basic index includes all documents from the collection you selected. You can run a search against this index, but it's recommended that you customize your index to improve performance and reduce the index size. -For more information about how to customize an index, see xref:customize-index.adoc[]. +For more information about the different features you can add to your Search index to improve performance and search results, see xref:customize-index.adoc[]. For more information about how to run a search, see xref:simple-search-ui.adoc[] or xref:simple-search-rest-api.adoc[]. diff --git a/modules/search/pages/create-search-indexes.adoc b/modules/search/pages/create-search-indexes.adoc index 3cf984b69..71e457227 100644 --- a/modules/search/pages/create-search-indexes.adoc +++ b/modules/search/pages/create-search-indexes.adoc @@ -1,5 +1,7 @@ = Create a Search Index :page-topic-type: concept +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Create a Search index to get started with the Search Service in your database. [abstract] @@ -7,27 +9,41 @@ You can create a Search index with: -* The <>. +* The <>. // * The <>. * The <>. -TIP: If you're new to developing with the Search Service, xref:create-search-index-ui.adoc[create a Search index with the UI]. -You can copy the Search index definition JSON payload from the UI to create your index xref:create-search-index-rest-api.adoc[with the REST API]. +include::partial$large-index-caution.adoc[tag=regular-search] + +TIP: If you're new to developing with the Search Service, <>. +You can export the Search index definition JSON payload from the UI to create your index xref:create-search-index-rest-api.adoc[with the REST API], or xref:import-search-index.adoc[]. [#ui] -== Creating a Search Index with the Capella UI +== Creating a Search Index with the {page-ui-name} + +Couchbase Capella supports two editing modes for the Search Service: + +* xref:create-quick-index.adoc[Quick Mode] +* xref:create-search-index-ui.adoc[Advanced Mode] -To xref:create-search-index-ui.adoc[create a basic Search index], provide the following information: +[NOTE] +==== +For indexes created with Couchbase Server version 7.6 and later, index names must be unique inside a bucket and scope. +You cannot have 2 indexes with the same name inside the same bucket and scope on a Capella database running version 7.6. -* The name of the index. -* The bucket, scope and collection or collections where you want to create the index. -* A xref:customize-index.adoc#type-mappings[type mapping] for the documents that you want to add to the index. +The {page-ui-name} marks indexes as scoped or not scoped to a specific bucket and scope. + +Indexes created with a previous version of Couchbase Server are not scoped. +==== + +Use Quick Mode for a streamlined editing experience. +Use Advanced Mode to fully customize your Search index with advanced settings and features. After you create a Search index, the Search Service streams data from your chosen collection or collections, and any document mutations, into the index builder. Before your index finishes building, you can run a search and return partial results. You can also customize a Search index to improve search results and performance. -For more information about how you can customize a Search index with the Capella UI, see xref:customize-index.adoc[]. +For more information about how you can customize a Search index with the {page-ui-name}, see xref:customize-index.adoc[]. //[#sdks] //== Creating a Search Index with Couchbase SDKs @@ -43,8 +59,14 @@ For more information about how you can customize a Search index with the Capella You can create a Search index with the REST API through a JSON payload. -Most properties in the JSON payload correspond to settings in the Capella UI. -You can also copy the Search index definition JSON payload from a Search index in the Capella UI to use in a REST API call. +Most properties in the JSON payload correspond to settings in the {page-ui-name}. +You can also copy the Search index definition JSON payload from a Search index in the {page-ui-name} to use in a REST API call. + +[NOTE] +==== +Use the scoped name for an index with the xref:server:rest-api:rest-fts.adoc[Search Service REST API] for any endpoints that do not include the bucket and scope in their path. +For example, you must use `bucket.scope.index_name` as the format for your index name with the `analyzeDoc` endpoint, but not with the new 7.6 `query` endpoint. +==== For more information about how to use the REST API to create a Search index, see xref:create-search-index-rest-api.adoc[]. diff --git a/modules/search/pages/create-type-mapping.adoc b/modules/search/pages/create-type-mapping.adoc index 4c3db04a5..c8acb66d0 100644 --- a/modules/search/pages/create-type-mapping.adoc +++ b/modules/search/pages/create-type-mapping.adoc @@ -1,63 +1,62 @@ -= Create a Type Mapping -:page-topic-type: guide -:description: Create a type mapping with the Couchbase Capella UI to control what documents are included or excluded from a Search index. += Create a Type Mapping in Advanced Mode +:page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Add a type mapping to your Search index using the Couchbase {page-ui-name}'s Advanced Mode to control what documents are added to your index. [abstract] {description} -For more information, see xref:customize-index.adoc#type-mappings[Customize a Search Index with the Capella UI]. +For more information about type mappings, see xref:customize-index.adoc#type-mappings[Search Index Features]. -== Prerequisites +== Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. - -* You've logged in to the Couchbase Capella UI. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -== Procedure +* You have started to create or already created a Search index in xref:create-search-index-ui.adoc[Advanced Mode]. -To create a type mapping with the Capella UI: +* You have logged in to the Couchbase {page-ui-name}. -. On the *Databases* page, select the database that has the Search index you want to edit. +== Procedure + +To create a type mapping with the {page-ui-name}: + +. On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. -. Click the index where you want to create a type mapping. -. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create a type mapping. -. In the *Scope* list, select the scope where you want to create a type mapping. +. Click the index where you want to create a type mapping. +. Under *Type Mappings*, in the Bucket list, select the bucket where you want to create a type mapping. +. In the *Scope* list, select the scope where you want to create a type mapping. . Click btn:[Add Mapping]. -. In the *Collection* list, select the collection for the type mapping. +. In the *Collection* list, select the collection to use for the type mapping. . (Optional) Do any of the following: -.. To use a specific analyzer for documents in the type mapping, in the *Analyzer* list, select an analyzer. +.. To use a specific analyzer for documents added from this type mapping, in the *Analyzer* list, select an analyzer. ++ +By default, the mapping inherits the default analyzer you set for the Search index. For more information about analyzers, see xref:customize-index.adoc#analyzers[Search Index Features]. +.. To switch from a xref:customize-index.adoc#type-mappings[dynamic type mapping to a static type mapping], select *Only index specified fields*. +.. To only include documents of a specific type from a collection, in the *Name* field, add the document type to the end of the collection. + -By default, the mapping inherits the default analyzer you set for the Search index. -For more information about analyzers, see xref:customize-index.adoc#analyzers[Customize a Search Index with the Capella UI]. -.. To switch from a xref:customize-index.adoc#type-mappings[dynamic type mapping to a static type mapping], select *Only index specified fields*. +For example, `collection.document_type`. +For more information, see xref:set-type-identifier.adoc[]. . Click btn:[Submit]. -[TIP] -==== -You can also create type mappings with the Data Insights area: - -. Expand the scope where you want to create a type mapping. -. Next to the collection where you want to create the type mapping, go to menu:More Options (⋮)[Map Collection to Index]. -. (Optional) To edit the analyzer or switch to a static type mapping, next to the type mapping, go to menu:More Options (⋮)[Edit]. -==== - -== Next Steps +== Next Steps After you create a static type mapping, you can xref:create-child-field.adoc[] to add or remove specific document fields from your Search index. To add an extra filter to the documents selected by your type mapping, you can also xref:set-type-identifier.adoc[]. -If you have a document field that contains a JSON object, xref:create-child-mapping.adoc[] for that field. +If you have a document field that contains a JSON object, xref:create-child-mapping.adoc[] for that field. -To continue customizing your Search index, you can: +To continue customizing your Search index, you can: * xref:set-advanced-settings.adoc[] * xref:set-type-identifier.adoc[] * xref:create-custom-analyzer.adoc[] * xref:create-custom-character-filter.adoc[] * xref:create-custom-token-filter.adoc[] -* xref:create-custom-tokenizer.adoc[] +* xref:create-custom-tokenizer.adoc[] * xref:create-custom-wordlist.adoc[] -To run a search and test the contents of your Search index, see xref:simple-search-ui.adoc[] or xref:simple-search-rest-api.adoc[]. \ No newline at end of file + +To run a search and test the contents of your Search index, see xref:simple-search-ui.adoc[] or xref:simple-search-rest-api.adoc[]. diff --git a/modules/search/pages/customize-index.adoc b/modules/search/pages/customize-index.adoc index 5f923df81..0a7587486 100644 --- a/modules/search/pages/customize-index.adoc +++ b/modules/search/pages/customize-index.adoc @@ -1,51 +1,71 @@ -= Customize a Search Index with the Capella UI += Search Index Features :page-topic-type: concept -:description: Configure additional options for a Search index to improve performance and fine tune your search results. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Search indexes in Couchbase Capella have multiple features that you can configure to improve performance and fine tune your search results. [abstract] {description} +Some features are only available in xref:create-search-index-ui.adoc[Advanced Mode]. + You can add the following components and configure the following options for a Search index: -[cols="1,2"] +[cols="2,1,1,4"] |==== -|Option |Description +|Option |Quick Mode |Advanced Mode |Description -|[[type-mappings]]Mappings a| +|[[type-mappings]]Type Mappings and Mappings +|✓ +|✓ +a|Use a type mapping to include or exclude specific documents in a collection from an index. -Use a type mapping to include or exclude specific documents in a scope or collection from an index, based on their type. +Type mappings can also set a field's data type and other settings. -You can create two types of type mappings: +You can create two types of type mappings with the Search Service: -* *Dynamic type mappings*: Add all available fields from a matching document type to an index. -* *Static type mappings*: Add only specific fields from a matching document type to an index. +* *Dynamic type mappings*: When you do not know the structure of your data fields ahead of time, use a dynamic type mapping to add all available fields from a matching document type to an index. +For example, you could create a dynamic type mapping to include all documents from the `hotel` collection in your Search index, or include all fields under a JSON object from your document schema. ++ +Configure this type of mapping by selecting a collection in xref:create-quick-index.adoc[Quick Mode] or by clearing *Only index specified fields* when you xref:create-type-mapping.adoc[]. -By default, all indexes have a dynamic type mapping that includes all documents from the *_default* scope and *_default* collection in a bucket. +* *Static type mappings*: When your data fields are stable and unlikely to change, use a static type mapping to add and define only specific fields from a matching document type to an index. +For example, you could create a static type mapping to only include the contents of the `city` field from the `hotel` collection in your Search index, as a text field with an `en` analyzer. ++ +Configure this type of mapping by selecting a field in your document schema in xref:create-quick-index.adoc[Quick Mode]. ++ +In Advanced Mode, xref:create-type-mapping.adoc[create a type mapping] and xref:create-child-field.adoc[add a child field mapping] to create a static type mapping. -Add xref:create-child-field.adoc[child fields] to a type mapping to create a static type mapping. -Child fields set the specific fields from a document that you want to include or exclude from an index. +Type mappings start at the collection level. +Create additional mappings for child fields or JSON objects under a collection's type mapping to restrict the documents added to your index. +This can improve Search index performance over indexing entire collections. -For more information about how to add a type mapping to an index, see xref:create-type-mapping.adoc[]. +For more information about how to configure settings for mappings and type mappings in Quick Mode, see xref:quick-index-field-options.adoc[]. -|General Settings a| +For more information about how to configure a type mapping in Advanced Mode, see xref:create-type-mapping.adoc[]. -Set general settings to change your index's default analyzer, replication, and more. +|Advanced Settings +| +|✓ +a|Set advanced settings with xref:create-search-index-ui.adoc[Advanced Mode] to change your index's default analyzer, replication, and more. For more information about how to change general settings, see xref:set-advanced-settings.adoc[]. -|[[type-identifiers]]Type Identifier a| - -Set a type identifier to add a filter to the documents added to your Search index: +|[[type-identifiers]]Document Filter +| +|✓ +a|In xref:create-search-index-ui.adoc[Advanced Mode], configure a document filter to add or remove documents in your Search index that meet certain conditions: * JSON Type Field: Selects only documents that contain a specific field with a specified string value. * Doc ID up to Separator: Selects only documents with an ID or key up to a specific substring. * Doc ID with Regex: Selects only documents with an ID or key that matches a regular expression. -For more information about how to configure a type identifier, see xref:set-type-identifier.adoc[]. - -|[[analyzers]]Analyzers a| +For more information about how to configure a document filter, see xref:set-type-identifier.adoc[]. -Use analyzers to improve and customize the search results in your index. +|[[analyzers]]Analyzers +| +|✓ +a|In xref:create-search-index-ui.adoc[Advanced Mode], use analyzers to improve and customize the search results in your index. Analyzers transform input text into tokens, which give you greater control over your index's text matching. @@ -54,18 +74,19 @@ For more information about how to create a custom analyzer, see xref:create-cust Analyzers have different components that control how text is transformed for search. When you create a custom analyzer, you can choose these components. +For more information, see <>. -Both custom and default analyzers can contain custom filters. - -|[[custom-filters-table]]Custom Filters a| - -Use custom filters to add more customization to a custom analyzer. +|[[custom-filters-table]]Custom Filters +| +|✓ +a|In xref:create-search-index-ui.adoc[Advanced Mode], use custom filters to add more customization to a custom analyzer. For more information about these filters, see the <> section. -|[[date-time]]Date/Time Parsers a| - -If the documents in your index contain date and time data in a format other than RFC-3339 (ISO-8601), then you need to create a date/time parser. +|[[date-time]]Date/Time Parsers +| +|✓ +a|In xref:create-search-index-ui.adoc[Advanced Mode], if the documents in your index contain date and time data in a format other than RFC-3339 (ISO-8601), then you need to create a date/time parser. A custom date/time parser tells the Search index how to interpret date data from your documents. @@ -76,9 +97,9 @@ For more information about how to add a custom date/time parser, see xref:create [#custom-filters] == Custom Filters -Custom filters are components of a Search index <>. +Custom filters are components of a Search index <>. -Create and add these components to a custom analyzer to improve search results and performance for an index. +Create and add these components to a custom analyzer to improve search results and performance for an index in xref:create-search-index-ui.adoc[Advanced Mode]. You can create the following custom filters: @@ -141,6 +162,15 @@ For more information about how to create your own wordlist, see xref:create-cust == See Also -* xref:create-search-indexes.adoc[] -* xref:index-aliases.adoc[] -* xref:run-searches.adoc[] \ No newline at end of file +* xref:set-type-identifier.adoc[] +* xref:create-type-mapping.adoc[] +* xref:create-child-field.adoc[] +* xref:create-child-mapping.adoc[] +* xref:create-custom-analyzer.adoc[] +* xref:create-custom-character-filter.adoc[] +* xref:create-custom-tokenizer.adoc[] +* xref:create-custom-token-filter.adoc[] +* xref:create-custom-wordlist.adoc[] +* xref:set-advanced-settings.adoc[] +* xref:run-searches.adoc[] +* xref:index-aliases.adoc[] \ No newline at end of file diff --git a/modules/search/pages/default-analyzers-reference.adoc b/modules/search/pages/default-analyzers-reference.adoc index f42c335ba..0d4ba6816 100644 --- a/modules/search/pages/default-analyzers-reference.adoc +++ b/modules/search/pages/default-analyzers-reference.adoc @@ -1,5 +1,7 @@ = Default Analyzers :page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Use an analyzer to filter and modify search strings to improve matches for search results. @@ -52,7 +54,7 @@ The following default analyzer options are available: |it |The `it` analyzer uses character filters, tokenizers, and token filters designed for Italian language searches. -|keyword a| +|[[keyword]]keyword a| The `keyword` analyzer turns input into a single token. It forces exact matches and preserves whitespace characters like spaces. diff --git a/modules/search/pages/default-character-filters-reference.adoc b/modules/search/pages/default-character-filters-reference.adoc index d5492a60e..b11e22a6d 100644 --- a/modules/search/pages/default-character-filters-reference.adoc +++ b/modules/search/pages/default-character-filters-reference.adoc @@ -1,5 +1,7 @@ = Default Character Filters :page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Character filters remove unwanted characters from your search input. [abstract] diff --git a/modules/search/pages/default-token-filters-reference.adoc b/modules/search/pages/default-token-filters-reference.adoc index 967ac438b..914bcefab 100644 --- a/modules/search/pages/default-token-filters-reference.adoc +++ b/modules/search/pages/default-token-filters-reference.adoc @@ -1,5 +1,7 @@ = Default Token Filters :page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Use a token filter to filter a tokenizer's results and get better search result matches. [abstract] @@ -17,6 +19,7 @@ The following token filters are available: |Token Filter Type |Description |apostrophe | Removes all characters after an apostrophe (') from tokenizer results. Also removes the apostrophe. +For example, the token `Couchbase's` becomes `Couchbase`. |camelCase a| Splits text in camelCase inside tokenizer results into separate tokens. diff --git a/modules/search/pages/default-tokenizers-reference.adoc b/modules/search/pages/default-tokenizers-reference.adoc index 147f1ada5..16a2657ad 100644 --- a/modules/search/pages/default-tokenizers-reference.adoc +++ b/modules/search/pages/default-tokenizers-reference.adoc @@ -1,5 +1,7 @@ = Default Tokenizers :page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Tokenizers control how the Search Service splits input strings into individual tokens. [abstract] diff --git a/modules/search/pages/default-wordlists-reference.adoc b/modules/search/pages/default-wordlists-reference.adoc index 5031e8765..0a7f38149 100644 --- a/modules/search/pages/default-wordlists-reference.adoc +++ b/modules/search/pages/default-wordlists-reference.adoc @@ -1,5 +1,7 @@ = Default Wordlists :page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: A wordlist can contain articles, conjunctions, prepositions, or other terms that limit matches in a search. [abstract] diff --git a/modules/search/pages/field-data-types-reference.adoc b/modules/search/pages/field-data-types-reference.adoc index bb72c1e38..33b300415 100644 --- a/modules/search/pages/field-data-types-reference.adoc +++ b/modules/search/pages/field-data-types-reference.adoc @@ -1,28 +1,35 @@ = Field Data Types :page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: You can assign a data type to a field to tell the Search Service how to analyze its data. [abstract] {description} -When you xref:create-child-field.adoc[create a child field] on a xref:customize-index.adoc#type-mappings[type mapping], you need to set a field's data type. +When you create a mapping for a field in xref:create-quick-index.adoc[Quick Mode] or xref:create-child-field.adoc[Advanced Mode], you need to set the field's data type. -When you create a Search index and don't set a data type for a field, the Search Service automatically assigns a field data type. +If you create a Search index and do not set a data type for a field, the Search Service automatically assigns a field data type. +For example, if you created a xref:customize-index.adoc#type-mappings[dynamic type mapping], the Search Service automatically assigns data types to all fields in the type mapping. The following field data types are available: +[cols="1,2"] |==== |Field Data Type |Description -|text |The field contains a string. The string can contain numbers and special characters. +|text |The field contains a string. +The string can contain numbers and special characters. -|number |The field contains a number. It doesn't contain any alphabetic characters. +|number |The field contains a number. +It does not contain any alphabetic characters. |datetime |The field contains a date/time value that matches the format of a xref:customize-index.adoc#date-time[Date/Time Parser] in the index. |boolean |The field contains a true or false value. -|disabled | This field data type is deprecated. It's included for compatibility only. +|disabled | This field data type is deprecated. +It's included for compatibility only. |geopoint a| @@ -33,5 +40,49 @@ The field contains geopoint (latitude and longitude) data, represented as either * An array, as two floating point integers. * A JSON object, with the properties `lon`/`lng` and `lat`. -|geoshape |The field contains a GeoJSON object. For more information about GeoJSON objects, see xref:server:fts:fts-supported-queries-geojson-spatial.adoc[]. +|geoshape a| + +The field contains a GeoJSON object. + +A GeoJSON object describes a shape made of floating point coordinates with the following JSON properties: + +* A `type` string, for the type of GeoJSON object. +For example, a `point` or `MultiLineString`. + +* A `coordinates` array of floating point numbers, for each latitude and longitude coordinate point in the GeoJSON shape. + +For example: +---- +{ + "type": "LineString", + "coordinates": [ + + [-2.753735609842721, 53.94860827535115], + [-2.599898256093695,53.65007434185782] + ] +} +---- +This JSON object describes a `LineString` GeoJSON object with 2 latitude and longitude coordinates. + +For more information about GeoJSON queries, see xref:search-request-params.adoc#query-object[the Query object]. + +|ip a| + +The field contains an IP address, formatted in IPv4 or IPv6 CIDR syntax. + +For example: +---- +{ + "ipv4": "4.7.44.162", + "ipv6": "2001:4800:0000:0000:0000:0000:0000:0000" +} +---- + +|[[vector]]vector a| + +The field contains an array of floating point numbers. + +Use the `vector` type to perform vector similarity searches with Vector Search. + +For more information about Vector Search, see xref:vector-search:vector-search.adoc[]. |==== \ No newline at end of file diff --git a/modules/search/pages/geo-search-rest-api.adoc b/modules/search/pages/geo-search-rest-api.adoc index 0ccc8670f..06cf897d4 100644 --- a/modules/search/pages/geo-search-rest-api.adoc +++ b/modules/search/pages/geo-search-rest-api.adoc @@ -1,5 +1,7 @@ = Run a Geospatial Search Query with the REST API and curl/HTTP :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Search for geospatial data in your Couchbase Capella database with a compatible Search index, the REST API and curl/HTTP. [abstract] @@ -7,19 +9,23 @@ == Prerequisites -* You've deployed the Search Service on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. * You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You installed the Couchbase command-line tool (CLI). +* You have documents in your database that contain geospatial data. -* You have the hostname or IP address for your database. +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. == Procedure To run a geospatial Search query, <>. -Then, <>. +Then, <>. [#geospatial-index] === Create a Search Index with a Geospatial Type Mapping @@ -39,17 +45,19 @@ include::example$geospatial-search-index.jsonc[] To run a Search query against the Search index: . In your command-line tool, enter a `curl` command with the `XPOST` verb. -. Set your header content to include `Content-Type: application/json`. -. Enter your username, password, and the Search Service endpoint on port `8094` with the name of the index you want to query: +. Set your header content to include `"Content-Type: application/json"`. +. Enter your username, password, and the Search Service endpoint on port `18094` with the name of the index you want to query: + [source,console] ---- include::example$run-search-header.sh[] ---- . Enter the JSON payload for your query. -+ + +=== Example + For example, the following query searches a geospatial field, `geo`, for any locations within a 100 mile radius of the coordinates `-2.235143, 53.482358`: -+ + [source,json] ---- include::example$geospatial-search-query.jsonc[] diff --git a/modules/search/pages/geo-search-ui.adoc b/modules/search/pages/geo-search-ui.adoc index 05163c2df..39cce600a 100644 --- a/modules/search/pages/geo-search-ui.adoc +++ b/modules/search/pages/geo-search-ui.adoc @@ -1,48 +1,71 @@ -= Run a Geospatial Search Query with the Capella UI += Run a Geospatial Search Query with the {page-ui-name} :page-topic-type: guide -:description: Search for geospatial data in your Couchbase Capella database with a compatible Search index and the Capella UI. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Search for geospatial data in your Couchbase Capella database with a compatible Search index and the {page-ui-name}. [abstract] {description} == Prerequisites -* You've deployed the Search Service on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. * You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have documents in your database that contain geospatial data. + +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To run a geospatial Search query, <>. +To run a geospatial Search query, <>. -Then, <>. +Then, <>. [#geospatial-index] === Create a Search Index with a Geospatial Type Mapping -To create the Search index from the Capella UI: +To create the Search index in the {page-ui-name} with Advanced Mode: -. On the *Databases* page, select the database where you want to create the Search index. +. On the *Databases* page, select the database where you want to create a Search index. . Go to menu:Data Tools[Search]. . Click btn:[Create Search Index]. +. Click the btn:[Advanced Mode] toggle. . In the *Index Name* field, enter a name for the Search index. + -NOTE: Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). -. xref:create-type-mapping.adoc[] on the scope and collection in your database that you want to search. -. xref:create-child-field.adoc[] on the new type mapping with the following settings: -.. In the *Field* field, enter the name of the field in your documents that contains the geospatial data you want to search. -.. In the *Type* list, select *geopoint*. -.. Select *Index*. +[NOTE] +==== +Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). + +For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +==== + +. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. +. In the *Scope* list, select the scope where you want to create the index. +. In the *Filter by collection or field* field, enter the name of the collection you want to search. +.. When the collection appears, click the collection name. +.. Click btn:[Add]. +. In the *Filter by collection or field* field, enter the name of the field that contains the geospatial data you want to search. ++ +This field should contain child fields with `lat` and `long` data. ++ +.. When the field appears, click the field name. +.. Click the btn:[Child] toggle. +.. In the *Type* list, select *Geopoint*. +.. Select *Index this field*. .. Select *Include in _all field*. -. Click btn:[Submit]. -. Click btn:[Create Index]. +.. Click btn:[Add]. +. Click btn:[Create Index] + +include::partial$large-index-caution.adoc[tag=regular-search] [#geospatial-query] === Run a Geospatial Search Query -To run a Search query against the Search index from the Capella UI: +To run a Search query against the Search index from the {page-ui-name}: . Next to your <>, click btn:[Search]. . In the *Search* field, enter a search query for geospatial data. @@ -56,6 +79,6 @@ include::example$geospatial-search-query.jsonc[] == Next Steps -You can xref:customize-index.adoc[customize your Search index] to improve search results and performance. +For more information about the different features you can add to your Search index to improve performance and search results, see xref:customize-index.adoc[]. If you want to add autocomplete to your database's search, see xref:search-query-auto-complete.adoc[]. diff --git a/modules/search/pages/import-search-index.adoc b/modules/search/pages/import-search-index.adoc new file mode 100644 index 000000000..161ff4552 --- /dev/null +++ b/modules/search/pages/import-search-index.adoc @@ -0,0 +1,43 @@ += Import a Search Index Definition with the {page-ui-name} +:page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Use the Couchbase {page-ui-name} to import a JSON Search index definition or Search index alias. + +[abstract] +{description} + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. + +* You have logged in to the Couchbase {page-ui-name}. + +== Procedure + +To import a xref:create-search-indexes.adoc[Search index definition] or xref:index-aliases.adoc[Search alias] with the {page-ui-name}: + +. On the *Databases* page, select the database where you want to import a Search index or index alias. +. Go to menu:Data Tools[Search]. +. Do one of the following: +.. To import a Search index definition, click btn:[Import Search Index]. +.. To import a Search index alias definition, click btn:[Import Search Alias]. +. In the *Index Name* or *Alias Name* field, enter a name for your new Search index or alias. +. Upload a JSON file smaller than 40 MB that contains your Search index or alias definition. +. Click btn:[Import Search Index] or btn:[Import Search Alias]. +//. (Optional) Make any changes to your Search index or index alias settings. +//+ +//For more information, see xref:customize-index.adoc[] or xref:create-search-index-alias.adoc[]. +//. Click btn:[Create Index] or btn:[Create Index Alias]. + +include::partial$large-index-caution.adoc[tag=regular-search] + +== Next Steps + +To add additional features to your imported Search index, see xref:customize-index.adoc[]. + +To run a search with your Search index or index alias, see xref:simple-search-ui.adoc[] or xref:simple-search-rest-api.adoc[]. \ No newline at end of file diff --git a/modules/search/pages/index-aliases.adoc b/modules/search/pages/index-aliases.adoc index e0b204a4f..7924e3fff 100644 --- a/modules/search/pages/index-aliases.adoc +++ b/modules/search/pages/index-aliases.adoc @@ -1,5 +1,7 @@ = Create Search Index Aliases :page-topic-type: concept +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: A Search index alias lets you run a Search query against a Search index without using the original Search index name. [abstract] @@ -22,4 +24,5 @@ For more information about how to create a Search index alias, see xref:create-s == See Also * xref:create-search-index-alias.adoc[] +* xref:import-search-index.adoc[] diff --git a/modules/search/pages/quick-index-field-options.adoc b/modules/search/pages/quick-index-field-options.adoc new file mode 100644 index 000000000..84a2efc6b --- /dev/null +++ b/modules/search/pages/quick-index-field-options.adoc @@ -0,0 +1,131 @@ += Quick Index Mapping and Field Options +:page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: When you create a Search index with Quick Mode in the {page-ui-name}, you must set options for each field or type mapping you add to the index. + +[abstract] +{description} + +You can create the following types of type mappings in Quick Mode: + +* <> +* <> + +TIP: Indexing an entire collection in Quick Mode creates a dynamic type mapping for that collection. +Indexing a field creates a static type mapping for the parent collection. +For more information about static and dynamic type mappings, see xref:customize-index.adoc#type-mappings[Search Index Features]. + +For more information about how to create a Search index with Quick Mode, see xref:create-quick-index.adoc[]. + +[#quick-collection] +== Quick Collection Type Mapping Options + +Configure an entire collection in Quick Mode to add or remove all documents in that collection from your Search index. + +You can configure the following settings in Quick Mode for a type mapping that uses an entire collection: + +|==== +|Option |Description + +| Index everything from collection: "`$COLLECTION_NAME`" a| + +You must select *Index everything from collection* to add a collection type mapping in Quick Mode. + +| Index text fields as identifiers a| + +To index any *Text* type fields with the xref:default-analyzers-reference.adoc#keyword[keyword analyzer], select *Index text fields as identifiers*. + +To use other analyzers or settings on text fields, clear *Index text fields as identifiers*. + +For more information about analyzers, see xref:customize-index.adoc#analyzers[Search Index Features]. + +|Analyzer/Language a| + +Select the language for the content inside any *Text* fields. + +The Search Service automatically applies an xref:customize-index.adoc#analyzers[analyzer] to the field's contents based on the selected language. + +For more information about the available language options, see xref:quick-index-supported-languages.adoc[]. + +|==== + +[#quick-field] +== Quick Field Type Mapping Options + +Configure a single field in Quick Mode to add or remove it from your Search index. + +You can configure the following settings in Quick Mode for a type mapping that uses a single field: + +|==== +|Option |Description + +|Index this field as an identifier (Text Fields Only) a| + +To index this field with the xref:default-analyzers-reference.adoc#keyword[keyword analyzer], select *Index this field as an identifier*. + +To use other analyzers or settings on this text field, clear *Index this field as an identifier*. + +For more information about analyzers, see xref:customize-index.adoc#analyzers.adoc[Search Index Features]. + +|Language (Text Fields Only) a| + +Select the language for the content inside a text field. + +The Search Service automatically applies an xref:customize-index.adoc#analyzers[analyzer] to the field's contents based on the selected language. + +For more information about the available language options, see xref:quick-index-supported-languages.adoc[]. + +|Dimension (Vector Fields Only) a| + +include::partial$vector-search-field-descriptions.adoc[tag=dimension] + +|Similarity Metric (Vector Fields Only) a| + +include::partial$vector-search-field-descriptions.adoc[tag=similarity_metric] + +|Optimized For (Vector Fields Only) a| + +include::partial$vector-search-field-descriptions.adoc[tag=optimized_for] + +|Include in search results a| + +To include content from the field in search results, select *Include in search results*. + +To exclude the field's content from search results, clear *Include in search results*. + +|Support highlighting a| + +The Search Service can highlight matching search terms in search results from an index. + +To enable highlighting in search results, select *Support highlighting*. + +To turn off highlighting in search results, clear *Support highlighting*. + +NOTE: To enable *Support highlighting*, you must also enable *Include in search results*. + +|Support phrase matching a| + +To support searches for whole phrases, select *Support phrase matching*. + +To turn off phrase matching, clear *Support phrase matching*. + +|Support field agnostic search a| + +To search the field's contents without specifying the field name in a search query, select *Support field agnostic search*. + +To turn off field agnostic search, clear *Support field agnostic search*. + +|Support sorting and faceting a| + +To sort search results and use xref:search-request-params.adoc#facets[facets] with the field's contents, select *Support sorting and faceting*. + +To turn off sorting and facets, clear *Support sorting and faceting*. + +|Searchable As a| + +Set a different name that you can use to search the field's contents in a query. + +The default value is the field's name. + +|==== \ No newline at end of file diff --git a/modules/search/pages/quick-index-supported-languages.adoc b/modules/search/pages/quick-index-supported-languages.adoc new file mode 100644 index 000000000..bcf9a111d --- /dev/null +++ b/modules/search/pages/quick-index-supported-languages.adoc @@ -0,0 +1,64 @@ += Quick Mode Supported Languages +:page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: When you create a Search index with Quick Mode in Couchbase {page-product-name}, you need to specify a language for your text fields. + +[abstract] +{description} + +For more information about how to create a Search index with Quick Mode, see xref:create-quick-index.adoc[]. + +The following language options are available for Quick Index text fields: + +|==== +|Language Option |Description + +|Unknown/Various |Select this option if the text in the field or fields contains multiple languages, or does not match another language option. + +|English |For text in English. + +|Arabic |For text in Arabic. + +|Chinese, Japanese, and Korean |For text in Chinese, Japanese, or Korean. + +|Croatian |For text in Croatian. + +|Danish |For text in Danish. + +|Dutch |For text in Dutch. + +|Finnish |For text in Finnish. + +|French |For text in French. + +|German |For text in German. + +|Hebrew |For text in Hebrew. + +|Hindi |For text in Hindi. + +|Hungarian |For text in Hungarian. + +|Italian |For text in Italian. + +|Norwegian |For text in Norwegian. + +|Persian |For text in Persian. + +|Portuguese |For text in Portuguese. + +|Romanian |For text in Romanian. + +|Russian |For text in Russian. + +|Sorani Kurdish |For text in Sorani. + +|Spanish |For text in Spanish. + +|Swedish |For text in Swedish. + +|Turkish |For text in Turkish. + +|Web |Select this option if the text in the field contains content like URLs, email addresses, Twitter usernames, or hashtags. +|==== \ No newline at end of file diff --git a/modules/search/pages/run-searches.adoc b/modules/search/pages/run-searches.adoc index 5101c20e8..ce5bf5888 100644 --- a/modules/search/pages/run-searches.adoc +++ b/modules/search/pages/run-searches.adoc @@ -1,5 +1,7 @@ = Run a Search With a Search Index :page-topic-type: concept +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Run a Search query to search and return the contents of a Search index. [abstract] @@ -9,7 +11,7 @@ NOTE: You need to xref:create-search-indexes.adoc[create a Search index] before You can run a search against a Search index with: -* The <>. +* The <>. * The <>. * A <>. * The Couchbase SDKs: @@ -19,13 +21,13 @@ include::partial$sdks-fts-links.adoc[] To run a Search query against multiple Search indexes at once, xref:create-search-index-alias.adoc[]. [#ui] -== Run a Search with the Capella UI +== Run a Search with the {page-ui-name} -You can use the Capella UI to test your Search index before you integrate search into your application. +You can use the {page-ui-name} to test your Search index before you integrate search into your application. -You can enter a basic search query in the Capella UI, or use a xref:search-request-params.adoc#query[query object] and other JSON properties for a more complex search. +You can enter a basic search query in the {page-ui-name}, or use a xref:search-request-params.adoc#query[query object] and other JSON properties for a more complex search. -For more information about how to run a search with the Capella UI, see xref:simple-search-ui.adoc[]. +For more information about how to run a search with the {page-ui-name}, see xref:simple-search-ui.adoc[]. For more information about how to configure a Search index and search for geospatial data, see xref:geo-search-ui.adoc[]. diff --git a/modules/search/pages/search-index-params.adoc b/modules/search/pages/search-index-params.adoc index 90245f4ec..d28e6bc4a 100644 --- a/modules/search/pages/search-index-params.adoc +++ b/modules/search/pages/search-index-params.adoc @@ -1,6 +1,9 @@ = Search Index JSON Properties :page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Use a JSON payload to control the settings for a Search index. +:page-toclevels: 2 [abstract] {description} @@ -45,12 +48,15 @@ The UUID for the Search index. The Search Service automatically generates a UUID for a Search index. -If you use an existing UUID, the Search Service updates the existing Search index. Don't include the `uuid` property when you want to copy an index to a different cluster or create a new index. +If you use an existing UUID, the Search Service updates the existing Search index. +Do not include the `uuid` property when you want to copy an index to a different cluster or create a new index. + +View the UUID for an existing index from the {page-ui-name} by selecting an existing index and clicking btn:[Index Definition]. +The UUID displays in the Index Definition on the Update Index page. |sourceType |String |Yes |The `sourceType` is always `"gocbcore"`. -|sourceName |String |Yes a| -The name of the bucket where you want to create the Search index. +|sourceName |String |Yes |The name of the bucket where you want to create the Search index. |[[sourceuuid]]sourceUUID |String |No a| @@ -58,13 +64,13 @@ The UUID of the bucket where you want to create the Search index. The Search Service automatically finds the UUID for the bucket. -Don't include the `sourceUUID` property when you want to copy an index to a different cluster, or create a new index. +Do not include the `sourceUUID` property when you want to copy an index to a different cluster, or create a new index. |sourceParams |Object |No a| This object contains advanced settings for index behavior. -Don't add content into this object unless instructed by Couchbase Support. +Do not add content into this object unless instructed by Couchbase Support. |planParams |Object |Yes |An object that sets the Search index's partitions and replications. For more information, see <>. @@ -93,7 +99,12 @@ The `planParams` object contains the following properties: |Property |Type |Required? |Description |maxPartitionsPerPIndex |n/a |No |This setting is deprecated. Use `indexPartitions`, instead. -|indexPartitions |Number |Yes |The number of partitions to split the Search index into, across the nodes you have available in your database with the Search Service enabled. Use index partitions to increase index and query performance on large datasets. +|indexPartitions |Number |Yes a|The number of partitions to split the Search index into, across the nodes you have available in your database with the Search Service enabled. +Use index partitions to increase index and query performance on large datasets. + +NOTE: The scoring calculation for regular Search queries can be affected by the number of partitions in your Search index, and how the Search Service distributes documents across partitions. +This is a limitation of the https://en.wikipedia.org/wiki/Tf%E2%80%93idf[tf-idf^] weighting scheme. + |numReplicas |Number |Yes a| For high-availability, set the number of replicas the Search Service creates for the Search index. @@ -143,7 +154,7 @@ The `doc_config` object is a child object of the <>. It contains the fo |mode |String |Yes a| -Set a xref:customize-search-index.adoc#type-identifiers[type identifier] for the Search index to filter documents from search results: +Set a xref:customize-index.adoc#type-identifiers[type identifier] for the Search index to filter documents from search results: * `type_field`: Use the value from a specific field in the documents. * `docid_prefix_delim`: Use the leading characters in the documents' ID values, up to but not including a specified separator. @@ -229,7 +240,7 @@ For more information about the properties inside the `default_mapping` object, s |default_type |String |No |This setting is included for compatibility with earlier indexes only. |docvalues_dynamic |Boolean |Yes a| -To include the values for an indexed field in the Search index, set `docvalues_dynamic` to `true`. +To include the value for each instance of an indexed field in the Search index to support xref:search-request-params.adoc#facet-name[facets] and sorting search results, set `docvalues_dynamic` to `true`. To exclude the values for an indexed field in the index, set `docvalues_dynamic` to `false`. @@ -833,7 +844,7 @@ To only index the fields you specify in the type mapping, set `dynamic` to `fals |enabled |Boolean |Yes a| To enable the Search Service's default type mapping, set `enabled` to `true`. -The default type mapping includes all documents in the bucket in the Search index, even if they don't match another configured type mapping. +The default type mapping includes all documents in the bucket in the Search index, even if they do not match another configured type mapping. This can increase index size and indexing time. To disable the default type mapping, set `enabled` to `false`. @@ -871,9 +882,9 @@ TIP: To add a type identifier as an additional filter to your type mapping, add |==== [#scope-collection] -=== \{Scope\}.\{collection\} Object +=== \{Scope\}.\{collection\} Object and JSON Object Field Objects -The `\{scope\}.\{collection\}` object defines a custom type mapping for a Search index: +The `\{scope\}.\{collection\}` object defines a custom type mapping for a Search index, on a specific scope and collection in the database: [source,json] ---- @@ -881,7 +892,12 @@ include::example$simple-search-index-payload.jsonc[tag=scope_collection] ---- A `\{scope\}.\{collection\}` object is a child object of the <>. -It contains the following properties: + +A JSON object field object is a child object of the `\{scope\}.\{collection\}` object. +It defines a mapping for a field that contains a JSON object in your document schema. +It can contain additional mappings as <> under its <> object. + +Both objects can contain the following properties: [cols="1,1,1,2"] |==== @@ -905,7 +921,7 @@ The `properties` object is only enabled if `dynamic` is set to `false`. Specifies properties for the fields to index in the type mapping. Contains any number of `\{field_name\}` objects. -For more information, see <> +For more information, see <>. |==== @@ -965,6 +981,16 @@ It contains the following properties: |==== |Property |Type |Required? |Description +|analyzer |String |Text Only a| + +If the child field's `type` is `text`, set the analyzer to use for the child field. + +NOTE: If you want to use the default analyzer for the content of this child field, you don't need to include an `analyzer` property. + +|[[dims]]dims |Number |Vector Only a| + +include::partial$vector-search-field-descriptions.adoc[tag=dimension] + |docvalues |Boolean |Yes a| To include the value for each instance of the field in the Search index to support xref:search-request-params.adoc#facet-name[facets] and sorting search results, set `docvalues` to `true`. @@ -997,6 +1023,10 @@ To exclude the child field from the index, set `index` to `false`. |name |String |Yes |The child field's name. +|[[similarity]]similarity |String |Vector Only a| + +include::partial$vector-search-field-descriptions.adoc[tag=similarity_metric] + |store |Boolean |Yes a| To include the content of the child field in the Search index and allow its content to be viewed in search results, set `store` to `true`. @@ -1014,13 +1044,13 @@ The child field's type. Can be one of: * `geopoint` * `geoshape` * `disabled` +* `ip` +* `vector` For more information about the available field data types, see xref:field-data-types-reference.adoc[]. -|analyzer |String |No a| - -If the child field's `type` is `text`, set the analyzer to use for the child field. +|vector_index_optimized_for |String |Vector Only a| -NOTE: If you want to use the default analyzer for the content of this child field, you don't need to include an `analyzer` property. +include::partial$vector-search-field-descriptions.adoc[tag=optimized_for] |==== diff --git a/modules/search/pages/search-query-auto-complete-code.adoc b/modules/search/pages/search-query-auto-complete-code.adoc index bd9f89761..144323985 100644 --- a/modules/search/pages/search-query-auto-complete-code.adoc +++ b/modules/search/pages/search-query-auto-complete-code.adoc @@ -1,5 +1,7 @@ = Add Autocomplete to Your Application :description: Use autocomplete to add suggestions for a user's Search query as they type in your application. +:page-ui-name: {ui-name} +:page-product-name: {product-name} :page-topic-type: guide :tabs: @@ -10,11 +12,18 @@ After you xref:search-query-auto-complete-ui.adoc[create and configure a Search == Prerequisites -* You've deployed the Search Service on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. * You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've created a compatible Search index. For more information, see xref:search-query-auto-complete-ui.adoc[]. +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. + +* You have created a compatible Search index. +For more information, see xref:search-query-auto-complete-ui.adoc[]. == Procedure @@ -22,7 +31,7 @@ To add autocomplete with the Search Service to your application: . To test that your Search index was configured correctly, do one of the following: .. xref:simple-search-rest-api.adoc[Run a Search query from the REST API] with 2-8 characters in the `query` property. -.. xref:simple-search-ui.adoc[Run a Search query from the Capella UI] with 2-8 characters in the *Search* field. +.. xref:simple-search-ui.adoc[Run a Search query from the {page-ui-name}] with 2-8 characters in the *Search* field. + For example, with the `travel-sample` bucket, you could enter the strings `Be`, `Bea`, `Beau`, and `Beauf` to find a document with the text `Beaufort Hotel`. @@ -72,5 +81,5 @@ include::example$autocomplete-sample.py[] After you add autocomplete to your application, to improve your search results, you can: -* xref:customize-index.adoc[Customize your Search index with the Capella UI]. +* xref:customize-index.adoc[Add additional features to your Search index from the UI]. * Change the xref:search-index-params.adoc[JSON payload] for your Search index. diff --git a/modules/search/pages/search-query-auto-complete-ui.adoc b/modules/search/pages/search-query-auto-complete-ui.adoc index 567177c2f..37ba5c19a 100644 --- a/modules/search/pages/search-query-auto-complete-ui.adoc +++ b/modules/search/pages/search-query-auto-complete-ui.adoc @@ -1,40 +1,51 @@ = Configure an Autocomplete Search Index -:page-topic-type: guide -:description: Create a Search index with the Capella UI or the REST API to start using autocomplete with the Search Service. +:page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Create a Search index with the {page-ui-name} or the REST API to start using autocomplete with the Search Service. [abstract] {description} == Prerequisites -* You've deployed the Search Service on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. * You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure -You can create a compatible Search index with the <> or the <>. +You can create a compatible Search index with the <> or the <>. [#ui] -=== Create an Autocomplete Search Index with the Capella UI +=== Create an Autocomplete Search Index with the {page-ui-name} -To create the Search index with the Capella UI: +To create the Search index in the {page-ui-name} with Advanced Mode: . On the *Databases* page, select the database where you want to create the Search index. . Go to menu:Data Tools[Search]. . Click btn:[Create Search Index]. +. Click the btn:[Advanced Mode] toggle. . In the *Index Name* field, enter a name for the Search index. + -NOTE: Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). -. Expand *Analyzers*. +[NOTE] +==== +Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). + +For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. +You cannot have 2 indexes with the same name inside the same bucket and scope. +==== +. Under *Advanced Settings*, expand *Analyzers*. . xref:create-custom-analyzer.adoc[] with the following settings: .. In the *Name* field, enter `keyword_to_lower`. .. In the *Tokenizer* list, select *single*. .. In the *Token Filters* list, select and add the *to_lower* token filter. -. Expand *Custom Filters*. +. Under *Advanced Settings*, expand *Custom Filters*. . xref:create-custom-token-filter.adoc#edge-ngram[Create a custom token filter] with the following settings: .. In the *Name* field, enter `edge_ngram_2_8`. .. In the *Type* list, select *edge_ngram*. @@ -44,21 +55,23 @@ NOTE: Your index name must start with an alphabetic character (a-z or A-Z). It c .. In the *Name* field, enter `edge_ngram`. .. In the *Tokenizer* list, select *unicode*. .. In the *Token Filters* list, click both the *to_lower* and your custom *edge_ngram_2_8* token filter. -. xref:create-type-mapping.adoc[] with the following settings: -.. Clear *index all contained fields*. -. (Optional) If the field you want to search with autocomplete is nested inside a JSON object in your documents, xref:create-child-mapping.adoc[] for the JSON object with the following settings: -.. Clear *index all contained fields*. -. Under the type mapping or child mapping, xref:create-child-field.adoc[] with the following settings: -.. In the *Field* field, enter the name of a text field in a document from your selected scope and collection. -.. In the *Analyzer* list, select your *edge_ngram* analyzer. -.. Select *Index*. -.. Select *Store*. -.. Select *Include in _all field*. +. Under *Type Mappings*, in the *Bucket* list, select the bucket where you want to create your Search index. +. In the *Scope* list, select the scope where you want to create the index. +. In the *Filter by collection or field* field, enter the name of the collection you want to search from your Search index. +.. When the collection appears, click the collection name. +.. Clear *Index everything from collection: "$COLLECTION_NAME"*. +.. Click btn:[Add]. +. In the *Filter by collection or field* field, enter the name of a field in your collection's document schema that contains the data you want to search. +. Configure the field: +.. In the *Analyzer/Language* list, select your *edge_ngram* analyzer. +.. Select *Index this field*. +.. Select *Store field values*. +.. Select *Include in _all field*. . Expand *General Settings*. . In the *Default Analyzer* list, select your *keyword_to_lower* analyzer. -. Click btn:[Create Index] +. Click btn:[Create Index]. -TIP: You can use the generated index definition JSON payload from the UI to create a Search index with the REST API. To view the generated JSON, click btn:[Index Definition]. +include::partial$large-index-caution.adoc[tag=regular-search] [#api] === Create an Autocomplete Search Index with the REST API diff --git a/modules/search/pages/search-query-auto-complete.adoc b/modules/search/pages/search-query-auto-complete.adoc index df89d026e..fac969494 100644 --- a/modules/search/pages/search-query-auto-complete.adoc +++ b/modules/search/pages/search-query-auto-complete.adoc @@ -1,5 +1,7 @@ = Use Autocomplete with the Search Service :page-topic-type: concept +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Add autocomplete to your application to provide a search engine-like experience for your database. [abstract] diff --git a/modules/search/pages/search-request-params.adoc b/modules/search/pages/search-request-params.adoc index 4cc2f2138..f5110dda6 100644 --- a/modules/search/pages/search-request-params.adoc +++ b/modules/search/pages/search-request-params.adoc @@ -1,5 +1,7 @@ = Search Request JSON Properties :page-topic-type: reference +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: You can add additional properties to a Search request to control how the Search Service returns results. :page-toclevels: 3 @@ -25,13 +27,19 @@ An object that contains the properties for one of the supported query types. For more information about how to configure the `query` object, see <>. +|[[knn]]knn |Array of Objects |Vector Only a| + +An array that contains objects that describe a xref:vector-search:vector-search.adoc[Vector Search] query. + +For more information about how to configure the objects inside a `knn` array, see <>. + |ctl |Object |No a| An object that contains properties for query consistency. For more information about how to configure Search query consistency inside the `ctl` object, see <>. -|size/limit |Integer |No a| +|[[size-limit]]size/limit |Integer |No a| Set the total number of results to return for a single page of search results. @@ -92,7 +100,7 @@ For more information about the properties for a `sort` JSON object, see < To return the position of each occurrence of a search term inside a document, set `includeLocations` to `true`. NOTE: You must have *Include Term Vectors* enabled or the `include_term_vectors` property set to `true` on a field to use `includeLocations`. -For more information about how to enable term vectors, see xref:create-child-field.adoc#term-vectors[Create a Child Field] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. +For more information about how to enable term vectors, see xref:child-field-options-reference.adoc#term-vectors[Child Field Options] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. To turn off term locations, set `includeLocations` to `false`. @@ -132,6 +140,52 @@ For example, if you had a set of 10 documents to sort based on `_id` values of 1 |==== +[#knn-object] +== Knn Objects + +Each object inside the `knn` array in a Search query describes a xref:vector-search:vector-search.adoc[Vector Search] query. + +Add the `knn` array with at least one object to run a Vector Search query: + +[source,json] +---- +include::example$run-search-full-request.jsonc[tag=knn] +---- + +NOTE: To run a Vector Search query, you must still include a `query` object with your Search request. +To return only results from your Vector Search query, you can set the `query` object to a <> query. +To run a hybrid query that uses regular Search Service parameters together with Vector Search to return results, see the <> section. + +An object in the `knn` array can contain the following properties: + +[cols="1,1,1,4"] +|==== +|Property |Type |Required? |Description + +|k |Integer |Yes a| + +Enter the total number of results that you want to return from your Vector Search query. + +The Search Service returns the `k` closest vectors to the vector given in `vector`. + +NOTE: The <> overrides any value set in `k`. + +|field |String |Yes a| + +The name of the field that contains the vector data you want to search. + +|vector |Array of Floats |Yes a| + +Enter the vector that you want to compare to the vector data in `field`. + +The Search Service uses the similarity metric defined in the xref:search-index-params.adoc#similarity[Search index definition] to return the `k` closest vectors from the Search index. + +NOTE: The vector in your Search query must match the dimension of the vectors stored in your Search index. +If the dimensions do not match, your Search query does not return any results. +For more information about the dimension value, see the xref:search-index-params.adoc#dims[dims property] or the xref:child-field-options-reference.adoc#dimension[Dimension option] in the UI. + +|==== + [#query-object] == Query Object @@ -430,7 +484,7 @@ To use a match phrase query: * You must specify a field to search with the <> or the <> syntax in your search query. * You must have *Include Term Vectors* enabled or the `include_term_vectors` property set to `true` on the field you want to search. -For more information about how to enable term vectors, see xref:create-child-field.adoc#term-vectors[Create a Child Field] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. +For more information about how to enable term vectors, see xref:child-field-options-reference.adoc#term-vectors[Child Field Options] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. The Search Service searches for exact matches to the phrase you specify. @@ -446,7 +500,9 @@ include::example$query-analytic.jsonc[tag=short_match_phrase] Analytic queries use an analyzer to analyze the contents of your Search query, and find a match in the documents inside your Search index. -For more information about analyzers, see xref:customize-index.adoc#analyzers[Customize a Search Index with the Capella UI]. +For more information about analyzers, see xref:customize-index.adoc#analyzers[Customize a Search Index with the {page-ui-name}]. + +NOTE: From Couchbase Server 7.6 and later, when you run a Search query with the xref:n1ql:n1ql-language-reference/searchfun.adoc[{sqlpp} Search Function], the analyzer used in your Analytic query does not need to match the analyzer used in the Search index. [cols="1,1,4,4"] |==== @@ -471,7 +527,7 @@ a| include::example$query-analytic.jsonc[tag=match] ---- -| match_phrase +| [[match_phrase]]match_phrase | String a| Use the `match_phrase` property to run a match phrase query. The Search Service searches for exact matches to the phrase you specify. @@ -483,7 +539,7 @@ To use a match phrase query: * You must specify a field to search with the <> or the <> syntax in your search query. * You must have *Include Term Vectors* enabled or the `include_term_vectors` property set to `true` on the field you want to search. -For more information about how to enable term vectors, see xref:create-child-field.adoc#term-vectors[Create a Child Field] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. +For more information about how to enable term vectors, see xref:child-field-options-reference.adoc#term-vectors[Child Field Options] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. For more information about the additional properties you can specify with a match phrase query, see <>. a| @@ -499,7 +555,9 @@ include::example$query-analytic.jsonc[tag=match_phrase] Use Non-Analytic queries to run a search query without using an analyzer on the contents of your Search query. -For more information about analyzers, see xref:customize-index.adoc#analyzers[Customize a Search Index with the Capella UI]. +For more information about analyzers, see xref:customize-index.adoc#analyzers[Customize a Search Index with the {page-ui-name}]. + +NOTE: From Couchbase Server 7.6 and later, when you run a Search query with the xref:n1ql:n1ql-language-reference/searchfun.adoc[{sqlpp} Search Function], you do not need to use the xref:default-analyzers-reference.adoc#keyword[keyword analyzer] in your Search index to run a non-analytic query. [cols="1,1,4,4"] |==== @@ -589,7 +647,7 @@ To use a phrase query: * You must set the <> in your search query. * You must have *Include Term Vectors* enabled or the `include_term_vectors` property set to `true` on the field you want to search. -For more information about how to enable term vectors, see xref:create-child-field.adoc#term-vectors[Create a Child Field] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. +For more information about how to enable term vectors, see xref:child-field-options-reference.adoc#term-vectors[Child Field Options] or xref:search-index-params.adoc#term-vectors[Search Index JSON Properties]. a| .Search for the phrase "nice view" in the reviews.content field [source,json] @@ -786,6 +844,35 @@ If you do not set the `inclusive_max` value, by default, `max` is exclusive to t |==== +=== IP Address Range Queries + +Use an IP Address Range query to search for IP address data in your Search index. + +You can use IPv4 or IPv6 CIDR syntax in your query. + +For example, the following query searches for any IP addresses in the range `2.7.13.141/32` and `2001:4860:4860::8888/24`: + +[source,json] +---- +include::example$query-ip-range.jsonc[] +---- + +NOTE: If your IP range includes over 1024 IP addresses, you must update the `bleveMaxResultWindow` setting to accommodate your results. +For more information about how to change this setting, see xref:server:fts:fts-advanced-settings-bleveMaxResultWindow.adoc[]. + +[cols="1,1,1,2"] +|==== +|Property |Type |Required? |Description + +|cidr +|String Containing An IPv4 or IPv6 IP Address +|Yes +a| Enter an IP address range or single IP address, in IPv4 or IPv6 CIDR notation. + +The Search Service returns documents with IP addresses that fall inside the specified range or match the specified IP address. + +|==== + [#geopoint-queries-distance] === Distance/Radius-Based Geopoint Queries @@ -955,7 +1042,7 @@ Defines the specific GeoJSON shape to use in the query. |[[point-type]]type |String |Yes -a| Set `type` to `point`. +a| Set `type` to `Point`. |[[point-coordinates]]coordinates |Array @@ -1023,7 +1110,7 @@ Defines the specific GeoJSON shape to use in the query. |[[linestring-type]]type |String |Yes -a| Set `type` to `linestring`. +a| Set `type` to `LineString`. |[[linestring-coordinates]]coordinates |Array of Arrays @@ -1091,7 +1178,7 @@ Defines the specific GeoJSON shape to use in the query. |[[polygon-type]]type |String |Yes -a| Set `type` to `polygon`. +a| Set `type` to `Polygon`. |[[polygon-coordinates]]coordinates |Array of Arrays @@ -1162,7 +1249,7 @@ Defines the specific GeoJSON shape to use in the query. |[[multipoint-type]]type |String |Yes -a| Set `type` to `multipoint`. +a| Set `type` to `MultiPoint`. |[[multipoint-coordinates]]coordinates |Array of Arrays @@ -1232,7 +1319,7 @@ Defines the specific GeoJSON shape to use in the query. |[[multilinestring-type]]type |String |Yes -a| Set `type` to `multilinestring`. +a| Set `type` to `MultiLineString`. |[[multilinestring-coordinates]]coordinates |Array of Arrays @@ -1313,7 +1400,7 @@ Defines the specific GeoJSON shape to use in the query. |[[multipolygon-type]]type |String |Yes -a| Set `type` to `multipolygon`. +a| Set `type` to `MultiPolygon`. |[[multipolygon-coordinates]]coordinates |Array of Arrays @@ -1384,7 +1471,7 @@ a| Contains the <> and <> -* <> +* <> * <> -* <> -* <> -* <> +* <> +* <> +* <> |[[geometrycollection-coordinates]]coordinates |Array or Array of Arrays @@ -1472,7 +1559,7 @@ Defines the specific GeoJSON shape to use in the query. |[[circle-type]]type |String |Yes -a| Set `type` to `circle`. +a| Set `type` to `Circle`. |[[circle-coordinates]]coordinates |Array @@ -1559,7 +1646,7 @@ Defines the specific GeoJSON shape to use in the query. |[[envelope-type]]type |String |Yes -a| Set `type` to `envelope`. +a| Set `type` to `Envelope`. |[[envelope-coordinates]]coordinates |Array of 2 Arrays @@ -1612,9 +1699,11 @@ a| include::example$query-special.jsonc[tag=match_all] ---- -| match_none +| [[match_none]]match_none | Object -| Use the `match_none` object as the only property in your `query` object to return no documents from the Search index in search results. +a| Use the `match_none` object as the only property in your `query` object to return no documents from the Search index in search results. + +If you're using the <>, the `match_none` object returns only matches to the Vector Search query inside the `knn` object. a| .Return no documents from the Search index [source,json] @@ -2050,9 +2139,9 @@ Sets what value to use for the sort: * `id`: Uses the document's ID value. * `score`: Uses the document's score. -* `field`: Uses the value of a specific field to sort. See <>. +* `field`: Uses the value of a specific field to sort. See <>. -|[[field]]field |String |No |When `by` is set to `field`, specify the name of the field to use to sort search results. +|[[by-field]]field |String |No |When `by` is set to `field`, specify the name of the field to use to sort search results. |desc |Boolean |Yes a| diff --git a/modules/search/pages/search.adoc b/modules/search/pages/search.adoc index 036fb0dc5..5d57e1978 100644 --- a/modules/search/pages/search.adoc +++ b/modules/search/pages/search.adoc @@ -1,5 +1,7 @@ = Add Search to Your Application :page-topic-type: concept +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Use the Search Service to create a customizable search experience for your database and your end-user applications. [abstract] @@ -33,7 +35,7 @@ For more information about how to create a Search index, see xref:create-search- You can create a Search index: -* xref:create-search-index-ui.adoc[With the Couchbase Capella UI] +* xref:create-search-index-ui.adoc[With the Couchbase {page-ui-name}] //* With the Couchbase SDKs * xref:create-search-index-rest-api.adoc[From the REST API] @@ -48,13 +50,23 @@ For more information about how you can run a search against a Search index, see You can run a Search query: -* xref:simple-search-ui.adoc[With the Couchbase Capella UI] +* xref:simple-search-ui.adoc[With the Couchbase {page-ui-name}] //* With the Couchbase SDKs * xref:simple-search-rest-api.adoc[From the REST API] +== Vector Search for AI Applications + +Vector Search builds on Capella's Search Service to provide vector index support for Retrieval Augmented Generation (RAG) with an existing Large Language Model (LLM). + +Vector Search adds a new index type to the Search Service to support AI application development, known as a Vector Search index. +Using Vector Search and Couchbase Capella, you can develop applications with an existing LLM while giving context and up-to-date information from your own data. + +For more information about Vector Search, see xref:vector-search:vector-search.adoc[]. + == See Also * xref:create-search-indexes.adoc[] * xref:customize-index.adoc[] * xref:index-aliases.adoc[] * xref:run-searches.adoc[] +* xref:vector-search:vector-search.adoc[] diff --git a/modules/search/pages/set-advanced-settings.adoc b/modules/search/pages/set-advanced-settings.adoc index 4c971da0c..6b9ba0dd5 100644 --- a/modules/search/pages/set-advanced-settings.adoc +++ b/modules/search/pages/set-advanced-settings.adoc @@ -1,37 +1,39 @@ = Set Search Index General Settings :page-topic-type: guide -:description: Configure general settings with the Couchbase Capella UI for a Search index to improve an index's search results and performance. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Configure general settings with the Couchbase {page-ui-name} for a Search index to improve an index's search results and performance. [abstract] {description} == Prerequisites -* You've deployed the Search Service on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You have a bucket with scopes and collections in your database. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. - -* You've logged in to the Couchbase Capella UI. +* You have created an index with xref:create-search-index-ui.adoc[Advanced Mode]. + +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To set general settings for a Search index with the Capella UI: +To set general settings for a Search index with the {page-ui-name}: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. . Click the index you want to edit. -. Expand *General Settings*. +. Under *Advanced Settings*, expand *General Settings*. . Configure any of the following general settings for your index: + |==== |Option |Description -|Type Identifier |Change how the Search Service sets a document's type. For more information, see xref:set-type-identifier.adoc[]. - -|Default Type |Change the default type assigned to documents in the index. The default value is `_default`. +|Default Type |Change the default type assigned to documents in the index. +The default value is `_default`. |[[default-analyzer]]Default Analyzer a| @@ -45,7 +47,7 @@ For more information about how to create your own custom analyzer, see xref:crea |[[all-field]]Default Field a| -When you xref:create-child-field.adoc[create a child field] in a type mapping, you can choose to include that field in an `_all` field. +When you xref:create-child-field.adoc[create a mapping for a child field], you can choose to include that field in an `_all` field. You can add fields to the `_all` field to search their contents without specifying their field name in your search query. @@ -55,7 +57,7 @@ Enter a value in the *Default Field* field to change the name of this default fi |Index Dynamic Fields |Select *Index Dynamic Fields* to include fields from a xref:customize-index.adoc#type-mappings[dynamic type mapping] in the index. -|DocValues for Dynamic Fields |Select *DocValues for Dynamic Fields* to include the values of each field from a xref:customize-index.adoc#type-mappings[dynamic type mapping] in the index. +|DocValues for Dynamic Fields |Select *DocValues for Dynamic Fields* to include the values of each field from a xref:customize-index.adoc#type-mappings[dynamic type mapping] in the index for xref:search-request-params.adoc#facets[Facets] and sorting search results. |Number of Replicas a| diff --git a/modules/search/pages/set-type-identifier.adoc b/modules/search/pages/set-type-identifier.adoc index aff500fb4..12af81c8c 100644 --- a/modules/search/pages/set-type-identifier.adoc +++ b/modules/search/pages/set-type-identifier.adoc @@ -1,32 +1,40 @@ -= Set a Search Index Type Identifier += Set a Search Index Type Identifier :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: Use a type identifier with a type mapping to add an extra filter to the documents you want to include in a Search index. :page-toclevels: 3 [abstract] {description} -For more information about type identifiers and type mappings, see xref:customize-index.adoc#type-identifiers[Customize a Search Index with the Capella UI]. +For more information about type identifiers and type mappings, see xref:customize-index.adoc#type-identifiers[Search Index Features]. == Prerequisites -* You've created an index. -For more information, see xref:create-search-index-ui.adoc[]. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You've created at least one type mapping in your Search index. -For more information, see xref:create-type-mapping.adoc[]. +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. -* You've logged in to the Couchbase Capella UI. + +* You have started to create or already created an index in xref:create-search-index-ui.adoc[Advanced Mode]. + +* You have created at least one type mapping in your Search index. +For more information, see xref:create-type-mapping.adoc[]. + +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To set a type identifier for a Search index with the Capella UI: +To set a type identifier for a Search index with the {page-ui-name}: . On the *Databases* page, select the database that has the Search index you want to edit. . Go to menu:Data Tools[Search]. -. Click the index where you want to change the type identifier. -. Expand *General Settings*. -. Do one of the following: +. Click the index where you want to create a type identifier. +. Expand *General Settings*. +. Do one of the following: .. <> .. <> .. <> @@ -36,17 +44,17 @@ To set a type identifier for a Search index with the Capella UI: To only add documents to your Search index that contain a specific field with a specified string value: -. Click the *JSON type field* tab. +. Click the *JSON Type Field* tab. . In the *JSON Type Field* field, enter the name of the field in your documents that you want to use to filter documents in your Search index. + For example, if your documents include a `type` field, you could enter `type` in the *JSON Type Field* field. + -NOTE: You can't use a field as a type identifier if the field name contains a period (.). -. Under *Type Mappings*, next to the type mapping where you want to add the type identifier, go to menu:More Options (⋮)[Edit]. -. In the *Name* field, add a period (.) to the end of the current type mapping name. -. After the period, add the exact string from the document field that you want to use as a filter. +NOTE: You cannot use a field as a type identifier if the field name contains a period (.). +. Under *Type Mappings*, next to the type mapping where you want to add the type identifier, go to menu:More Options (⋮)[Edit]. +. In the *Name* field, add a period (.) to the end of the current type mapping name. +. After the period, add the exact string from the document field that you want to use as a filter. + -For example, if you wanted your Search index to only return documents that had a `type` value of `hotel`, you could enter `scope.collection.hotel` in the type mapping *Name* field. +For example, if you wanted your Search index to only return documents that had a `type` value of `hotel`, you could enter `hotel`. . Click btn:[Submit]. . Click btn:[Update Index]. @@ -56,35 +64,34 @@ For example, if you wanted your Search index to only return documents that had a To only add documents to your Search index that have IDs that match a specified prefix: . Click the *Doc ID up to Separator* tab. -. In the *Doc ID up to Separator* field, enter the ID prefix, up to the separator character, that you want to use to filter documents in your Search index. +. In the *Doc ID up to Separator* field, enter the separator character from the ID prefix in your document ID values. + -For example, if you know all of your document ID values are prefixed by a string and an underscore, enter `_`. -. Under *Type Mappings*, next to the type mapping where you want to add the type identifier, go to menu:More Options (⋮)[Edit]. -. In the *Name* field, add a period (.) to the end of the current type mapping name. +For example, if you know all of your document ID values are prefixed by a string and an underscore (_), enter `_`. +. Under *Type Mappings*, next to the type mapping where you want to add the type identifier, go to menu:More Options (⋮)[Edit]. +. In the *Name* field, add a period (.) to the end of the current type mapping name. . After the period, add the exact prefix from the document's ID value that you want to use as a filter. + -For example, if you wanted your Search index to only return documents that have an prefix of `landmark_`, you could enter `scope.collection.landmark` in the type mapping *Name* field. +For example, if you wanted your Search index to only return documents that have an prefix of `landmark_`, you could enter `landmark`. . Click btn:[Submit]. . Click btn:[Update Index]. [#doc-id-regex] -=== Create a Doc ID with Regex Type Identifier +=== Create a Doc ID with Regex Type Identifier To only add documents to your Search index that have IDs that match a specified https://github.com/google/re2/wiki/Syntax[RE2] regular expression: . Click the *Doc ID with Regex* tab. . In the *Doc ID with Regex* field, enter the regular expression that you want to use to filter documents in your Search index. + -For example, if you wanted only documents with ID values that contained `_40`, you could enter `_[3-5]0` as your regular expression. -. Under *Type Mappings*, next to the type mapping where you want to add the type identifier, go to menu:More Options (⋮)[Edit]. -. In the *Name* field, add a period (.) to the end of the current type mapping name. -. After the period, add a match for the regular expression from the document's ID value that you want to use as a filter. +For example, if you wanted documents with ID values that contained `_40`, you could enter `_[3-5]0` as your regular expression. +. Under *Type Mappings*, next to the type mapping where you want to add the type identifier, go to menu:More Options (⋮)[Edit]. +. In the *Name* field, add a period (.) to the end of the current type mapping name. +. After the period, add a match for the regular expression from the document's ID value that you want to use as a filter. + -For example, if you wanted your Search index to only return documents with ID values that contained `_40`, you could enter `scope.collection._40` in the type mapping *Name* field. +For example, if you wanted your Search index to only return documents with ID values that contained `_40`, you could enter `_40`. . Click btn:[Submit]. . Click btn:[Update Index]. - == Next Steps After you set the type identifier for your Search index, you can continue to customize your Search index: diff --git a/modules/search/pages/simple-search-rest-api.adoc b/modules/search/pages/simple-search-rest-api.adoc index 3be20867a..3f56aad57 100644 --- a/modules/search/pages/simple-search-rest-api.adoc +++ b/modules/search/pages/simple-search-rest-api.adoc @@ -1,5 +1,7 @@ = Run a Simple Search with the REST API and curl/HTTP :page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} :description: You can use the REST API and a curl command to run a search against a Search index. [abstract] @@ -7,13 +9,14 @@ == Prerequisites -* You've deployed the Search Service on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. -* You installed the Couchbase command-line tool (CLI). - -* You have the hostname or IP address for your database. +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. -* You've created a Search index. +* You have created a Search index. + For more information about how to create a Search index, see xref:create-search-index-ui.adoc[] or xref:create-search-index-rest-api.adoc[]. @@ -22,32 +25,32 @@ For more information about how to create a Search index, see xref:create-search- To run a simple search with the REST API: . In your command-line tool, enter a `curl` command with the `XPOST` verb. -. Set your header content to include `Content-Type: application/json`. -. Enter your username, password, and the Search Service endpoint on port `8094` with the name of the index you want to query: +. Set your header content to include `"Content-Type: application/json"`. +. Enter your username, password, and the Search Service endpoint on port `18094` with the name of the index you want to query: + [source,console] ---- include::example$run-search-header.sh[] ---- -+ -To use SSL, use the `https` protocol in the Search Service endpoint URL and port `18094`. + . Enter the JSON payload for your query. + -TIP: You can copy the JSON for a Query Request from the Couchbase Server or Couchbase Capella UI to use in your REST API call. +TIP: You can copy the JSON for a Query Request from the Couchbase Server or Couchbase {page-ui-name} to use in your REST API call. For more information about how to perform a search with the UI, see xref:simple-search-ui.adoc[]. -+ +=== Example + In the following example, the JSON payload queries an index named `landmark-content-index` for the strings `view`, `food`, and `beach`: -+ + [source,console] ---- include::example$run-search-payload.sh[] ---- -+ + For more information about the available properties for a Search query JSON payload, see xref:search-request-params.adoc[]. == Next Steps -If you don't get the search results you were expecting, you can change the JSON payload xref:search-index-params.adoc[for your Search index] or xref:search-request-params.adoc[for your Search query]. +If you do not get the search results you were expecting, you can change the JSON payload xref:search-index-params.adoc[for your Search index] or xref:search-request-params.adoc[for your Search query]. -You can also xref:customize-index.adoc[]. +You can also xref:customize-index.adoc[]. \ No newline at end of file diff --git a/modules/search/pages/simple-search-ui.adoc b/modules/search/pages/simple-search-ui.adoc index 540bb31d3..2f701c061 100644 --- a/modules/search/pages/simple-search-ui.adoc +++ b/modules/search/pages/simple-search-ui.adoc @@ -1,44 +1,49 @@ -= Run A Simple Search with the Capella UI += Run A Simple Search with the {page-ui-name} :page-topic-type: guide -:description: Run a Search query from the Couchbase Capella UI to preview the search results from a Search index. +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Run a Search query from the Couchbase {page-ui-name} to preview the search results from a Search index. [abstract] {description} == Prerequisites -* You have the Search Service enabled on a node in your database. +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. * You have created a Search index. + For more information about how to create a Search index, see xref:create-search-indexes.adoc[]. -* You have logged in to the Couchbase Capella UI. +* You have logged in to the Couchbase {page-ui-name}. == Procedure -To run a simple search with the Capella UI: +To run a simple search with the {page-ui-name}: . On the *Databases* page, select the database where you created your Search index. . Go to menu:Data Tools[Search]. . Next to your Search index or xref:index-aliases.adoc[index alias], click btn:[Search]. . In the *Search* field, enter a search query. -+ +. Press kbd:[Enter]. +. (Optional) To view a document and its source collection, click a document name in the search results list. + +=== Example + For example, the following query searches for the strings `view`, `food`, and `beach`: -+ + [source,json] ---- include::example$run-search-payload-ui.jsonc[] ---- -+ -The query payload enables scoring explanations and term highlighting. It also returns all available fields in the index, and returns 10 results per page. -+ + +The query payload enables scoring explanations and term highlighting. +It also returns all available fields in the index, and returns 10 results per page. + TIP: Use a xref:index-aliases.adoc[Search index alias] to search multiple Search indexes in a single search query. Use the xref:search-request-params.adoc#collections[`collections` parameter] in your request to specify an array of collections to search from the Search index. -. Press kbd:[Enter]. -. (Optional) To view a document and its source collection, click a document name in the search results list. - == Next Steps If you do not get the search results you were expecting, you can change the xref:search-request-params.adoc[JSON payload for your Search query]. diff --git a/modules/search/partials/custom-token-filters-descriptions.adoc b/modules/search/partials/custom-token-filters-descriptions.adoc index d7be2cc56..76067d30f 100644 --- a/modules/search/partials/custom-token-filters-descriptions.adoc +++ b/modules/search/partials/custom-token-filters-descriptions.adoc @@ -193,7 +193,7 @@ include::example$complex-search-index-payload.jsonc[tag=shingle] ---- // end::shingle_example[] -For example, if you use a xref:guides:search/default-tokenizers-reference.adoc#whitespace[whitespace tokenizer], a range with a minimum of two and a maximum of three, and a space as a separator, the token `abc def` becomes `abc`, `def`, and `abc def`. +For example, if you use a xref:search:default-tokenizers-reference.adoc#whitespace[whitespace tokenizer], a range with a minimum of two and a maximum of three, and a space as a separator, the token `abc def` becomes `abc`, `def`, and `abc def`. [plantuml,shingle,svg] .... diff --git a/modules/search/partials/large-index-caution.adoc b/modules/search/partials/large-index-caution.adoc new file mode 100644 index 000000000..24bc828f9 --- /dev/null +++ b/modules/search/partials/large-index-caution.adoc @@ -0,0 +1,10 @@ +// tag::regular-search[] + +CAUTION: If you create a Search index that includes xref:search:field-data-types-reference.adoc#vector[a field with the vector type] and 5,000,000 documents or more, you must xref:support:manage-support.adoc[contact Couchbase Capella Support]. + +// end::regular-search[] +// tag::vector-search[] + +CAUTION: If you create a Vector Search index that includes 5,000,000 documents or more, you must xref:support:manage-support.adoc[contact Couchbase Capella Support]. + +// end::vector-search[] \ No newline at end of file diff --git a/modules/search/partials/nav.adoc b/modules/search/partials/nav.adoc index ed0d8e9c1..d957e6967 100644 --- a/modules/search/partials/nav.adoc +++ b/modules/search/partials/nav.adoc @@ -1,8 +1,31 @@ * xref:cloud:search:search.adoc[] ** xref:cloud:search:create-search-indexes.adoc[] - *** xref:cloud:search:create-search-index-ui.adoc[] - *** xref:cloud:search:create-search-index-rest-api.adoc[] - **** xref:cloud:search:search-index-params.adoc[] + *** xref:cloud:search:customize-index.adoc[] + *** xref:cloud:search:field-data-types-reference.adoc[] + *** xref:cloud:search:create-quick-index.adoc[] + **** xref:cloud:search:quick-index-field-options.adoc[] + **** xref:cloud:search:quick-index-supported-languages.adoc[] + *** xref:cloud:search:create-search-index-ui.adoc[] + **** xref:cloud:search:create-type-mapping.adoc[] + **** xref:cloud:search:create-child-field.adoc[] + ***** xref:cloud:search:child-field-options-reference.adoc[] + **** xref:cloud:search:create-child-mapping.adoc[] + **** xref:cloud:search:set-type-identifier.adoc[] + **** xref:cloud:search:set-advanced-settings.adoc[] + **** xref:cloud:search:create-custom-analyzer.adoc[] + **** xref:cloud:search:create-custom-character-filter.adoc[] + **** xref:cloud:search:create-custom-tokenizer.adoc[] + **** xref:cloud:search:create-custom-token-filter.adoc[] + **** xref:cloud:search:create-custom-wordlist.adoc[] + **** xref:cloud:search:create-custom-date-time-parser.adoc[] + **** xref:cloud:search:default-analyzers-reference.adoc[] + **** xref:cloud:search:default-character-filters-reference.adoc[] + **** xref:cloud:search:default-token-filters-reference.adoc[] + **** xref:cloud:search:default-tokenizers-reference.adoc[] + **** xref:cloud:search:default-wordlists-reference.adoc[] + *** xref:cloud:search:import-search-index.adoc[] + *** xref:cloud:search:create-search-index-rest-api.adoc[] + **** xref:cloud:search:search-index-params.adoc[] ** xref:cloud:search:run-searches.adoc[] *** xref:cloud:search:simple-search-ui.adoc[] *** xref:cloud:search:geo-search-ui.adoc[] @@ -14,21 +37,3 @@ *** xref:cloud:search:search-query-auto-complete-code.adoc[] ** xref:cloud:search:index-aliases.adoc[] *** xref:cloud:search:create-search-index-alias.adoc[] - ** xref:cloud:search:customize-index.adoc[] - *** xref:cloud:search:set-advanced-settings.adoc[] - *** xref:cloud:search:set-type-identifier.adoc[] - *** xref:cloud:search:create-type-mapping.adoc[] - *** xref:cloud:search:create-child-field.adoc[] - *** xref:cloud:search:create-child-mapping.adoc[] - *** xref:cloud:search:create-custom-analyzer.adoc[] - *** xref:cloud:search:create-custom-character-filter.adoc[] - *** xref:cloud:search:create-custom-tokenizer.adoc[] - *** xref:cloud:search:create-custom-token-filter.adoc[] - *** xref:cloud:search:create-custom-wordlist.adoc[] - *** xref:cloud:search:create-custom-date-time-parser.adoc[] - *** xref:cloud:search:default-analyzers-reference.adoc[] - *** xref:cloud:search:default-character-filters-reference.adoc[] - *** xref:cloud:search:default-token-filters-reference.adoc[] - *** xref:cloud:search:default-tokenizers-reference.adoc[] - *** xref:cloud:search:default-wordlists-reference.adoc[] - *** xref:cloud:search:field-data-types-reference.adoc[] \ No newline at end of file diff --git a/modules/search/partials/vector-search-field-descriptions.adoc b/modules/search/partials/vector-search-field-descriptions.adoc new file mode 100644 index 000000000..6aa526c9e --- /dev/null +++ b/modules/search/partials/vector-search-field-descriptions.adoc @@ -0,0 +1,43 @@ +// tag::optimized_for[] +For a `vector` child field, choose whether the Search Service should prioritize recall or latency when returning similar vectors in search results: + +* *recall*: The Search Service prioritizes returning the most accurate result. +This may increase resource usage for Search queries. ++ +The Search Service uses an `nprobe` value to calculate the number of centroids to search when using recall priority. +This value is calculated by taking the square root of the number of centroids in the index. + +* *latency*: The Search Service prioritizes returning results with lower latency. +This may reduce the accuracy of results. ++ +The Search Service uses half the `nprobe` value calculated for *recall* priority. + +For more information about Vector Search indexes, see xref:vector-search:vector-search.adoc[] or xref:vector-search:create-vector-search-index-ui.adoc[]. +// end::optimized_for[] +// tag::similarity_metric[] +For a `vector` child field, choose the method to calculate the similarity between the vector embedding in a Vector Search index and the vector embedding in a Vector Search query. + +NOTE: It's recommended to choose the same similarity metric for your Search index as the one used in your embedding model. + +* *dot_product*: Calculated by adding the result of multiplying a vector's components, or the product of the magnitudes of the vectors and the cosine of the angle between them. +The dot product of 2 vectors is affected by the length and direction of each of the vectors, rather than just taking a straight-line distance. ++ +Dot product similarity is commonly used by Large Language Models (LLMs). +Use *dot_product* to get the best results with an embedding model that uses dot product similarity. + +* *l2_norm*: Also known as Euclidean distance. +Uses the straight-line distance between 2 vectors to calculate similarity. +Smaller euclidean distances mean that the values of each coordinate in the vectors are closer together. ++ +It's best to use *l2_norm* similarity when your embeddings contain information about the count or measure of specific things, and your embedding model uses the same similarity metric. + +For more information about Vector Search indexes, see xref:vector-search:vector-search.adoc[] or xref:vector-search:create-vector-search-index-ui.adoc[]. +// end::similarity_metric[] +// tag::dimension[] +For a `vector` child field, enter the total number of elements in the vector embedding array. + +Vector Search indexes can support arrays with up to 2048 elements. +Arrays can be an array of arrays. + +For more information about Vector Search indexes, see xref:vector-search:vector-search.adoc[] or xref:vector-search:create-vector-search-index-ui.adoc[]. +// end::dimension[] \ No newline at end of file diff --git a/modules/third-party/pages/integrations.adoc b/modules/third-party/pages/integrations.adoc index 7f9b3563b..e76d03ef0 100644 --- a/modules/third-party/pages/integrations.adoc +++ b/modules/third-party/pages/integrations.adoc @@ -1,353 +1 @@ -= Integrations, Connectors, and Tools -:page-toclevels: 2 -:page-topic-type: reference -:page-aliases: develop:integrations.adoc -:description: Integrate Couchbase with your development ecosystem and production and deployment tools. - - -[abstract] -{description} - -Third-party and community products and integrations -- -although not by and large supported by Couchbase (see <<#support-model,Support Model section>>) -- -are a vital tool for most larger development projects. - -These are examples of integrations that Couchbase or partners have developed to enhance Couchbase, Capella, and partner services. - - -[#partner-integrations-with-couchbase] -== Partner Integrations - - -.Partner Integrations List -[cols="20,45,50,20,20,28"] -|=== -| Integration | Summary | Links | Capella | Self-managed | <<#support-model,Support Model>> - -| Hashicorp Vault -| Securely manage your credentials for database access. -a| -* https://www.hashicorp.com/partners/tech/couchbase#vault[Hashicorp Vault plug-in for Capella] -* https://github.com/couchbasecloud/vault-plugin-database-couchbasecapella[Plug-in page on GitHub] -* https://developer.hashicorp.com/vault/docs/secrets/databases/couchbase[Hashicorp partner page for self-managed Couchbase] -| ✔ -| ✔ -| Partner Supported - -| Netlify -| Use Couchbase Capella and Netlify Serverless Functions to handle dynamic content without needing infrastructure. -a| -* https://www.netlify.com/integrations/couchbase/[Netlify partner page] -* https://developer.couchbase.com/tutorial-quickstart-netlify[Tutorial] -| ✔ -| ✖ -| Couchbase Officially Supported - - -| Vercel -| Easy build and deploy for your applications. -a| -* https://vercel.com/integrations/couchbase-capella[Vercel partner page & tutorial] -* https://www.couchbase.com/blog/integrating-vercel-with-couchbase-capella[Blog] -* https://vercel.com/templates/next.js/couchbase-capella[Next.js starter template] -| ✔ -| ✖ -| Couchbase Officially Supported - - -| Molo17 GlueSync -| Real-time, cloud-native data replication suite for NoSQL. -a| -* https://www.couchbase.com/partners/molo17/[Intro & Tutorial] -* https://molo17.com/gluesync/[GlueSync homepage] -| ✔ -| ✔ -| Partner Supported - - -| ODBC / JDBC -| Easily access live Couchbase data from BI, Analytics, Reporting, ETL, & Custom Apps. -a| -* xref:server:connectors:odbc-jdbc-drivers.adoc[Couchbase ODBC and JDBC Drivers] -* https://www.cdata.com/drivers/couchbase/docs/[CData Driver] -* https://www.cdata.com/drivers/couchbase/#section-etl-anchor[Popular Data Warehousing Integrations] -* https://www.cdata.com/drivers/couchbase/#section-data-anchor[Data Management Tools] -* https://www.cdata.com/drivers/couchbase/#section-dev-anchor[Developer Tools] -* https://www.cdata.com/drivers/couchbase/#section-bi-anchor[BI & Data Visualization] -| ✔ -| ✔ -| Partner Supported -|=== - - - -== IDE Integrations - - -.IDE Plugins -[cols="20,45,50,20,20,35"] -|=== -| Integration | Summary | Links | Capella | Self-managed | <<#support-model,Support Model>> - -| VS Code extension for Couchbase -| A seamless experience for Couchbase Server users who want to work within the popular Visual Studio Code editor. -a| -* https://marketplace.visualstudio.com/items?itemName=Couchbase.vscode-couchbase[Visual Studio Marketplace] -* https://github.com/couchbaselabs/VSCode-Couchbase/blob/main/GETTING-STARTED.md[Tutorial] -* https://github.com/couchbaselabs/VSCode-Couchbase/blob/main/SNIPPETS.md[Code snippets] -| ✔ -| ✔ -| Couchbase Community Support - - -| VS Code extension for Couchbase Lite -| A seamless experience for Couchbase Mobile users who want to work within the popular Visual Studio Code editor. -a| -* https://marketplace.visualstudio.com/items?itemName=Couchbase.vscode-cblite[Visual Studio Marketplace] -* https://github.com/couchbaselabs/vscode-cblite[GitHub repo and intro] -| ✔ -| ✔ -| Couchbase Community Support - - -| JetBrains -| Provides support for Couchbase in JetBrains IDEs -- to develop Couchbase applications, complete with code completion, syntax highlighting, and debugging support. -a| -* https://plugins.jetbrains.com/plugin/22131-couchbase[JetBrains Marketplace] -// * [Tutorial] -* https://github.com/couchbaselabs/couchbase_jetbrains_plugin[GitHub source repo] -| ✔ -| ✔ -| Couchbase Community Support -|=== - - - -== Data Tools - -Import, export, and query data from the command line. - -.Data Tools -[cols="20,45,50,20,20,35"] -|=== -| Integration | Summary | Links | Capella | Self-managed | <<#support-model,Support Model>> - - -| Couchbase Shell (cbsh) -| Interactive shell, based upon nushell, which you can use to extract, transform, and load different data formats in Couchbase. -a| -* https://couchbase.sh/docs/[cbsh docs] -* https://github.com/couchbaselabs/couchbase-shell[GitHub repo] -| ✔ -| ✔ -| Couchbase Officially Supported (currently in _Developer Preview_) - - -| cbimport -| Utility for importing data into Couchbase databases. -Included in self-managed Couchbase Server, or downloadable as part of the xref:cloud:reference:command-line-tools.adoc#download-and-install-the-couchbase-command-line-tools[Capella Tools package]. -a| xref:server:tools:cbimport.adoc[cbimport reference] -| ✔ -| ✔ -| Couchbase Officially Supported - - -| cbexport -| Utility for exporting data from Couchbase databases. -Included in self-managed Couchbase Server, or downloadable as part of the xref:cloud:reference:command-line-tools.adoc#download-and-install-the-couchbase-command-line-tools[Capella Tools package]. -a| xref:server:tools:cbexport.adoc[cbexport reference] -| ✔ -| ✔ -| Couchbase Officially Supported - - -| cbbackupmgr -| A utility for backing up and restoring a Couchbase database. -a| -* xref:server:current:backup-restore:cbbackupmgr.adoc[cbbackupmgr reference] -* xref:server:backup-restore:cbbackupmgr-tutorial.adoc[Tutorial] -* xref:cloud:clusters:cli-backup-restore.adoc[Capella examples] -| ✔ -| ✔ -| Couchbase Officially Supported -|=== - - -=== Other Shells - -Some languages, such as Ruby, come with REPLs (interactive shells) as standard -- as is normal with many programming languages. -Others have them available as _Notebooks_, such as Python, with https://jupyter.org/[Jupyter Notebook] (which works with _several_ different languages), through which you can interactively develop functionality for a Couchbase app, or blend with analytic tools and charts. - - - -[big-data-connectors] -== Big Data Connectors - - -Couchbase develops and supports connectors for: - -* Elasticsearch and OpenSearch -* The Kafka Connect Framework -* Apache Spark Data Processing Platform -* The Tableau interactive data visualization platform - -Couchbase ODBC and JDBC drivers enable any application based on the ODBC/JDBC standards, for example Microsoft Excel, QlikView, or SAP Lumira, as well as Tableau, to connect to a Couchbase Server or cluster. - - - -.Big Data Connectors -[cols="28,43,28,30,30,28"] -|=== -| Connector | Summary | Docs | Source | Other Integrations | <<#support-model,Support Model>> - - -| Elasticsearch -| Replicate your documents from Couchbase Server or Capella to Elasticsearch or OpenSearch in near real time. -| xref:elasticsearch-connector:getting-started.adoc[Getting Started docs] -| xref:elasticsearch-connector:release-notes.adoc[Release notes & download links] -| xref:elasticsearch-connector:autonomous-operations.adoc[AO Mode] -| Couchbase Officially Supported - - -| Kafka -| A plug-in for the Kafka Connect framework, which provides source and sink components. -| xref:kafka-connector:index.adoc[Kafka connector docs] -| https://github.com/couchbase/kafka-connect-couchbase[GitHub repo] -| https://kafka.apache.org/uses[Use cases] -| Couchbase Officially Supported - - -| Spark -| Connect to the scalable computing engine for Big Data. -| xref:spark-connector:index.adoc[Couchbase Spark docs] -| https://github.com/couchbase/couchbase-spark-connector[Couchbase Spark Connector on GitHub] -| https://docs.databricks.com/_extras/notebooks/source/couchbase.html[Apache Spark Notebook] -| Couchbase Officially Supported - - -| Tableau -| Integrate Couchbase Server tabular views and the Tableau interactive data visualization platform. -| xref:tableau-connector::index.adoc[Tableau connector docs] -| https://www.tableau.com/products/desktop/download[Tableau Desktop Downloads] -| https://www.tableau.com/products/server[Other Tableau Integrations] -| Couchbase Officially Supported -|=== - - -[#sdk-language-ecosystem] -== SDK Language Ecosystem - -Couchbase has SDKs in ten popular languages -- -xref:c-sdk:hello-world:overview.adoc[C], -xref:dotnet-sdk:hello-world:overview.adoc[C# (.NET)], -xref:go-sdk:hello-world:overview.adoc[Go], -xref:java-sdk:hello-world:overview.adoc[Java], -xref:kotlin-sdk:hello-world:overview.adoc[Kotlin], -xref:nodejs-sdk:hello-world:overview.adoc[Node.js], -xref:php-sdk:hello-world:overview.adoc[PHP], -xref:python-sdk:hello-world:overview.adoc[Python], -xref:ruby-sdk:hello-world:overview.adoc[Ruby], and -xref:scala-sdk:hello-world:overview.adoc[Scala]. - -If you develop in one of these languages, you may be interested in (or already using) other libraries in that language's ecosystem, and wondering if it will integrate with Couchbase. - -There are too many third party integrations to evaluate and list (and absence of a mention in these pages is no judgement on importance or quality), but the following are well worth investigating: - -While Couchbase can't evaluate and list every possible third-party integration, we've provided a list of some you might consider in the sections below. - - -.Partner Integrations List -[cols="24,22,45,30,15,15,28"] -|=== -| Integration | SDK Language / Ecosystem | Summary | Links | Capella | Self-managed | <<#support-model,Support Model>> - -| Spring Data Couchbase -| JVM (Java, Kotlin, Scala) -| A familiar and consistent Spring-based programming model for new datastores with store-specific features and capabilities. -Key functional areas of Spring Data Couchbase are a POJO centric model for interacting with Couchbase Buckets and easily writing a Repository style data access layer. -a| -* https://spring.io/projects/spring-data-couchbase[Spring Data Couchbase] -* xref:java-sdk:hello-world:spring-data-sample-application.adoc[Sample application] -* https://blog.couchbase.com/couchbase-spring-boot-spring-data/[Spring Boot example] -| ✔ -| ✔ -| Couchbase Officially Supported - - -| Ottoman -| Node.js -| ODM (Object Document Mapper) for Couchbase. -Adds an abstraction layer over Couchbase Node.js SDK and significantly reduces the level of boilerplate needed during application development. -a| -* https://ottomanjs.com/[Ottoman docs] -* https://ottomanjs.com/docs/advanced/sdk-comparison[Advantages over pure Node.js] -* https://github.com/couchbaselabs/node-ottoman[GitHub repo] -| ✔ -| ✔ -| Couchbase Officially Supported - - -| Laravel -| PHP -| Extends the original Laravel classes to make an Eloquent model and Query builder with support for Couchbase. -a| https://github.com/friendsofcat/laravel-couchbase[Project website] -| ✔ -| ✔ -| Community Supported - - -| Rails Cache Store -| Ruby-on-Rails -| Speeds up your Rails application with caching -- storing content generated during the request-response cycle and reusing it when responding to similar requests. -a| xref:ruby-sdk:project-docs:third-party-integrations.adoc#across-the-ecosystem[Config Code example] -| ✔ -| ✔ -| Community Supported -|=== - - - - -== Support Model - - - -.Support Model -[cols="20,88"] -|=== -| Support Level | Definitions - -| Partner Supported -| This integration is developed and supported directly through the partner. -Couchbase provides support for the Couchbase components and interfaces. - -| Couchbase Community Supported -a| Open source integration that is developed by Couchbase. -Issues can also be raised in the relevant GitHub repositories or via https://www.couchbase.com/forums/[Couchbase forums] or https://discord.com/channels/915294689681362954/915296010694176808[Discord]. -Support is provided best effort by development team - -| Couchbase Officially Supported -a| Open source integration that is developed by Couchbase and officially supported. -A subscription to the Enterprise Edition of Couchbase includes technical support with service level commitments via our 24/7 support organization. -Issues can also be raised in the relevant GitHub repositories or via https://www.couchbase.com/forums/[Couchbase forums] or https://discord.com/channels/915294689681362954/915296010694176808[Discord]. - -| Community Supported -| Open source integration that is developed and supported directly by third party community contributor. -Issues can also be raised in the relevant GitHub repositories. -|=== - -//// - -== Mobile SDK Ecosystem - -Placeholder for Dart/Flutter/etc - won’t be in first page release. -Docs: https://cbl-dart.dev/documents/ -Install - https://cbl-dart.dev/install/ -Repo ; https://github.com/cbl-dart/cbl-dart - -== Useful Tools - -// Tools - -=== Test Data Generation -//// +include::7.6@server:develop:integrations.adoc[] \ No newline at end of file diff --git a/modules/vector-search/examples/RunVectorSearchGenerateEmbed.go b/modules/vector-search/examples/RunVectorSearchGenerateEmbed.go new file mode 100644 index 000000000..e2e8ea964 --- /dev/null +++ b/modules/vector-search/examples/RunVectorSearchGenerateEmbed.go @@ -0,0 +1,134 @@ +package main + +import ( + "fmt" + "log" + "time" + "os" + "github.com/couchbase/gocb/v2" + // "github.com/couchbase/gocb/v2/search" + "github.com/couchbase/gocb/v2/vector" + "bytes" + "encoding/json" + "io/ioutil" + "net/http" +) + +type OpenAIResponse struct { + Data []struct { + Embedding []float32 `json:"embedding"` + } `json:"data"` +} + +// generateVector makes a request to OpenAI's API to get an embedding vector for the given input text. +func generateVector(inputText string) ([]float32, error) { + openaiAPIKey := os.Getenv("OPENAI_API_KEY") + if openaiAPIKey == "" { + return nil, fmt.Errorf("OPENAI_API_KEY environment variable is not set") + } + + requestBody, err := json.Marshal(map[string]interface{}{ + "input": inputText, + "model": "text-embedding-ada-002", + }) + if err != nil { + return nil, fmt.Errorf("error marshaling request body: %w", err) + } + + request, err := http.NewRequest("POST", "https://api.openai.com/v1/embeddings", bytes.NewBuffer(requestBody)) + if err != nil { + return nil, fmt.Errorf("error creating request: %w", err) + } + + request.Header.Set("Content-Type", "application/json") + request.Header.Set("Authorization", "Bearer "+openaiAPIKey) + + client := &http.Client{} + response, err := client.Do(request) + if err != nil { + return nil, fmt.Errorf("error making request: %w", err) + } + defer response.Body.Close() + + if response.StatusCode != http.StatusOK { + bodyBytes, _ := ioutil.ReadAll(response.Body) + return nil, fmt.Errorf("API request failed with status %d: %s", response.StatusCode, string(bodyBytes)) + } + + var openAIResponse OpenAIResponse + if err := json.NewDecoder(response.Body).Decode(&openAIResponse); err != nil { + return nil, fmt.Errorf("error decoding response: %w", err) + } + + if len(openAIResponse.Data) == 0 || len(openAIResponse.Data[0].Embedding) == 0 { + return nil, fmt.Errorf("no embedding vector found in response") + } + + return openAIResponse.Data[0].Embedding, nil +} + +func main() { + connstr := "couchbases://" + os.Getenv("CB_HOSTNAME") + username := os.Getenv("CB_USERNAME") + password := os.Getenv("CB_PASSWORD") + bucket_name := "vector-sample" + scope_name := "color" + + cluster, err := gocb.Connect(connstr, gocb.ClusterOptions{ + Authenticator: gocb.PasswordAuthenticator{ + Username: username, + Password: password, + }, + SecurityConfig: gocb.SecurityConfig{ + TLSSkipVerify: true, // Disables TLS certificate verification + }, + }) + if err != nil { + log.Fatal(err) + } + + bucket := cluster.Bucket(bucket_name) + err = bucket.WaitUntilReady(5*time.Second, nil) + if err != nil { + log.Fatal(err) + } + + scope := bucket.Scope(scope_name) + + question := "What color hides everything like the night?" + vect, err := generateVector(question) + if err != nil { + log.Fatalf("Error generating vector: %v", err) + } + + request := gocb.SearchRequest{ + VectorSearch: vector.NewSearch( + []*vector.Query{ + vector.NewQuery("embedding_vector_dot", vect), + }, + nil, + ), + } + + opts := &gocb.SearchOptions{Limit: 2, Fields: []string{"color","description"}} + + matchResult, err := scope.Search("color-index", request, opts) + if err != nil { + log.Fatal(err) + } + + for matchResult.Next() { + row := matchResult.Row() + docID := row.ID + var fields interface{} + err := row.Fields(&fields) + if err != nil { + log.Fatal(err) + } + fmt.Printf("Document ID: %s, Fields: %v\n", docID, fields) + } + + if err = matchResult.Err(); err != nil { + log.Fatal(err) + } +} diff --git a/modules/vector-search/examples/RunVectorSearchGenerateEmbed.java b/modules/vector-search/examples/RunVectorSearchGenerateEmbed.java new file mode 100644 index 000000000..642ed7766 --- /dev/null +++ b/modules/vector-search/examples/RunVectorSearchGenerateEmbed.java @@ -0,0 +1,114 @@ +import com.couchbase.client.core.error.*; +import com.couchbase.client.java.*; +import com.couchbase.client.java.kv.*; +import com.couchbase.client.java.json.*; +import com.couchbase.client.java.search.*; +import com.couchbase.client.java.search.queries.*; +import com.couchbase.client.java.search.result.*; +import com.couchbase.client.java.search.vector.*; + +import java.io.IOException; +import java.net.URI; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.net.http.HttpResponse.BodyHandlers; +import java.util.List; + +import java.time.Duration; +import java.util.Map; + +public class RunVectorSearchGenerateEmbed { + + private static final String OPENAI_API_KEY = System.getenv("OPENAI_API_KEY"); + private static final String OPENAI_URL = "https://api.openai.com/v1/embeddings"; + + public static float[] generateVector(String inputText) { + HttpClient client = HttpClient.newHttpClient(); + + JsonObject jsonBody = JsonObject.create() + .put("input", inputText) + .put("model", "text-embedding-ada-002"); + + HttpRequest request = HttpRequest.newBuilder() + .uri(URI.create(OPENAI_URL)) + .header("Content-Type", "application/json") + .header("Authorization", "Bearer " + OPENAI_API_KEY) + .POST(HttpRequest.BodyPublishers.ofString(jsonBody.toString())) + .build(); + + try { + HttpResponse response = client.send(request, BodyHandlers.ofString()); + + JsonObject jsonResponse = JsonObject.fromJson(response.body()); + List embeddingList = jsonResponse.getArray("data") + .getObject(0) + .getArray("embedding") + .toList(); + + return toFloatArray(embeddingList); + } catch (IOException | InterruptedException e) { + e.printStackTrace(); + return null; // or handle more gracefully + } + } + + private static float[] toFloatArray(List list) { + float[] result = new float[list.size()]; + for (int i = 0; i < list.size(); i++) { + result[i] = ((Number) list.get(i)).floatValue(); + } + return result; + } + + public static void main(String[] args) { + + String endpoint = "couchbases://" + System.getenv("CB_HOSTNAME") + "?tls_verify=none"; + String username = System.getenv("CB_USERNAME"); + String password = System.getenv("CB_PASSWORD"); + + String bucketName = "vector-sample"; + String scopeName = "color"; + String collectionName = "rgb"; + String searchIndexName = "color-index"; + + try { + // Connect to cluster with specified credentials + Cluster cluster = Cluster.connect( + endpoint, + ClusterOptions.clusterOptions(username, password).environment(env -> { + // Use the pre-configured profile below to avoid latency issues with your connection. + env.applyProfile("wan-development"); + }) + ); + + Bucket bucket = cluster.bucket(bucketName); + bucket.waitUntilReady(Duration.ofSeconds(10)); + Scope scope = bucket.scope(scopeName); + Collection collection = scope.collection(collectionName); + + String question = "What color hides everything like the night?"; + float[] vector = generateVector(question); + + SearchRequest request = SearchRequest + .create(VectorSearch.create( + VectorQuery.create("embedding_vector_dot", vector).numCandidates(2))); + SearchResult result = scope.search("color-index", request, + SearchOptions.searchOptions().limit(3).fields("color","description")); + + for (SearchRow row : result.rows()) { + System.out.println("Found row: " + row); + System.out.println(" Fields: " + row.fieldsAs(Map.class)); + } + + } catch (UnambiguousTimeoutException ex) { + boolean authFailure = ex.toString().contains("Authentication Failure"); + if (authFailure) { + System.out.println("Authentication Failure Detected"); + } else { + System.out.println("Error:"); + System.out.println(ex.getMessage()); + } + } + } +} diff --git a/modules/vector-search/examples/RunVectorSearchSimpleColor.go b/modules/vector-search/examples/RunVectorSearchSimpleColor.go new file mode 100644 index 000000000..63ba3691d --- /dev/null +++ b/modules/vector-search/examples/RunVectorSearchSimpleColor.go @@ -0,0 +1,71 @@ +package main + +import ( + "fmt" + "log" + "time" + "os" + "github.com/couchbase/gocb/v2" + // "github.com/couchbase/gocb/v2/search" + "github.com/couchbase/gocb/v2/vector" +) + +func main() { + connstr := "couchbases://" + os.Getenv("CB_HOSTNAME") + username := os.Getenv("CB_USERNAME") + password := os.Getenv("CB_PASSWORD") + bucket_name := "vector-sample" + scope_name := "color" + + cluster, err := gocb.Connect(connstr, gocb.ClusterOptions{ + Authenticator: gocb.PasswordAuthenticator{ + Username: username, + Password: password, + }, + SecurityConfig: gocb.SecurityConfig{ + TLSSkipVerify: true, // Disables TLS certificate verification + }, + }) + if err != nil { + log.Fatal(err) + } + + bucket := cluster.Bucket(bucket_name) + err = bucket.WaitUntilReady(5*time.Second, nil) + if err != nil { + log.Fatal(err) + } + + scope := bucket.Scope(scope_name) + + request := gocb.SearchRequest{ + VectorSearch: vector.NewSearch( + []*vector.Query{ + vector.NewQuery("colorvect_l2", []float32{0.0, 0.0, 128.0}), + }, + nil, + ), + } + + opts := &gocb.SearchOptions{Limit: 3, Fields: []string{"color"}} + + matchResult, err := scope.Search("color-index", request, opts) + if err != nil { + log.Fatal(err) + } + + for matchResult.Next() { + row := matchResult.Row() + docID := row.ID + var fields interface{} + err := row.Fields(&fields) + if err != nil { + log.Fatal(err) + } + fmt.Printf("Document ID: %s, Fields: %v\n", docID, fields) + } + + if err = matchResult.Err(); err != nil { + log.Fatal(err) + } +} diff --git a/modules/vector-search/examples/RunVectorSearchSimpleColor.java b/modules/vector-search/examples/RunVectorSearchSimpleColor.java new file mode 100644 index 000000000..e71659c3d --- /dev/null +++ b/modules/vector-search/examples/RunVectorSearchSimpleColor.java @@ -0,0 +1,63 @@ +import com.couchbase.client.core.error.*; +import com.couchbase.client.java.*; +import com.couchbase.client.java.kv.*; +import com.couchbase.client.java.json.*; +import com.couchbase.client.java.search.*; +import com.couchbase.client.java.search.queries.*; +import com.couchbase.client.java.search.result.*; +import com.couchbase.client.java.search.vector.*; + +import java.time.Duration; +import java.util.Map; + +public class RunVectorSearchSimpleColor { + + public static void main(String[] args) { + + String endpoint = "couchbases://" + System.getenv("CB_HOSTNAME") + "?tls_verify=none"; + String username = System.getenv("CB_USERNAME"); + String password = System.getenv("CB_PASSWORD"); + + String bucketName = "vector-sample"; + String scopeName = "color"; + String collectionName = "rgb"; + String searchIndexName = "color-index"; + + try { + // Connect to cluster with specified credentials + Cluster cluster = Cluster.connect( + endpoint, + ClusterOptions.clusterOptions(username, password).environment(env -> { + // Use the pre-configured profile below to avoid latency issues with your connection. + env.applyProfile("wan-development"); + }) + ); + + Bucket bucket = cluster.bucket(bucketName); + bucket.waitUntilReady(Duration.ofSeconds(10)); + Scope scope = bucket.scope(scopeName); + Collection collection = scope.collection(collectionName); + + SearchRequest request = SearchRequest + .create(VectorSearch.create( + VectorQuery.create("colorvect_l2", new float[]{ 0.0f, 0.0f, 128.0f } + ).numCandidates(3))); + SearchResult result = scope.search("color-index", request, + SearchOptions.searchOptions().limit(3).fields("color","brightness")); + + for (SearchRow row : result.rows()) { + System.out.println("Found row: " + row); + System.out.println(" Fields: " + row.fieldsAs(Map.class)); + } + + } catch (UnambiguousTimeoutException ex) { + boolean authFailure = ex.toString().contains("Authentication Failure"); + if (authFailure) { + System.out.println("Authentication Failure Detected"); + } else { + System.out.println("Error:"); + System.out.println(ex.getMessage()); + } + } + } +} diff --git a/modules/vector-search/examples/create-vector-search-index-header.sh b/modules/vector-search/examples/create-vector-search-index-header.sh new file mode 100644 index 000000000..ab11fa4dc --- /dev/null +++ b/modules/vector-search/examples/create-vector-search-index-header.sh @@ -0,0 +1,3 @@ +curl -s -XPUT -H "Content-Type: application/json" \ + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/${BUCKET_NAME}/scope/${SCOPE_NAME}/index/${INDEX_NAME} + -d \ \ No newline at end of file diff --git a/modules/vector-search/examples/create-vector-search-index-payload.sh b/modules/vector-search/examples/create-vector-search-index-payload.sh new file mode 100644 index 000000000..5e939aceb --- /dev/null +++ b/modules/vector-search/examples/create-vector-search-index-payload.sh @@ -0,0 +1,127 @@ +curl -s -XPUT -H "Content-Type: application/json" \ + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/vector-sample/scope/color/index/color-index + -d \ + '{ + "type": "fulltext-index", + "name": "vector-sample.color.color-index", + "sourceType": "gocbcore", + "sourceName": "vector-sample", + "sourceUUID": "789365cccdf940ee2814a5dd2752040a", + "planParams": { + "maxPartitionsPerPIndex": 512, + "indexPartitions": 1 + }, + "params": { + "doc_config": { + "docid_prefix_delim": "", + "docid_regexp": "", + "mode": "scope.collection.type_field", + "type_field": "type" + }, + "mapping": { + "analysis": {}, + "default_analyzer": "standard", + "default_datetime_parser": "dateTimeOptional", + "default_field": "_all", + "default_mapping": { + "dynamic": false, + "enabled": false + }, + "default_type": "_default", + "docvalues_dynamic": false, + "index_dynamic": false, + "store_dynamic": false, + "type_field": "_type", + "types": { + "color.rgb": { + "dynamic": false, + "enabled": true, + "properties": { + "brightness": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "index": true, + "name": "brightness", + "store": true, + "type": "number" + } + ] + }, + "color": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "analyzer": "en", + "index": true, + "name": "color", + "store": true, + "type": "text" + } + ] + }, + "colorvect_dot": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 3, + "index": true, + "name": "colorvect_dot", + "similarity": "dot_product", + "type": "vector" + } + ] + }, + "colorvect_l2": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 3, + "index": true, + "name": "colorvect_l2", + "similarity": "l2_norm", + "type": "vector" + } + ] + }, + "description": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "analyzer": "en", + "index": true, + "name": "description", + "store": true, + "type": "text" + } + ] + }, + "embedding_vector_dot": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 1536, + "index": true, + "name": "embedding_vector_dot", + "similarity": "dot_product", + "type": "vector" + } + ] + } + } + } + } + }, + "store": { + "indexType": "scorch", + "segmentVersion": 16 + } + }, + "sourceParams": {} +}' \ No newline at end of file diff --git a/modules/vector-search/examples/run-vector-search-generate-embed.py b/modules/vector-search/examples/run-vector-search-generate-embed.py new file mode 100644 index 000000000..df07d93be --- /dev/null +++ b/modules/vector-search/examples/run-vector-search-generate-embed.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python3 + +import os +import sys +from couchbase.cluster import Cluster +from couchbase.options import ClusterOptions +from couchbase.auth import PasswordAuthenticator +from couchbase.exceptions import CouchbaseException +import couchbase.search as search +from couchbase.options import SearchOptions +from couchbase.vector_search import VectorQuery, VectorSearch +from openai import OpenAI + +# Change the question as desired +question = "What color hides everything like the night?" + +# Make sure to replace OPENAI_API_KEY with your own API Key +openai_api_key = os.getenv("OPENAI_API_KEY") +client = OpenAI() + +# Make sure to change CB_USERNAME, CB_PASSWORD, and CB_HOSTNAME to the username, password, and hostname for your database. +pa = PasswordAuthenticator(os.getenv("CB_USERNAME"), os.getenv("CB_PASSWORD")) +cluster = Cluster("couchbases://" + os.getenv("CB_HOSTNAME") + "/?ssl=no_verify", ClusterOptions(pa)) +# Make sure to change the bucket, scope, and index names to match where you stored the sample data in your database. +bucket = cluster.bucket("vector-sample") +scope = bucket.scope("color") +search_index = "color-index" +try: + vector = client.embeddings.create(input = [question], model="text-embedding-ada-002").data[0].embedding + search_req = search.SearchRequest.create(search.MatchNoneQuery()).with_vector_search( + VectorSearch.from_vector_query(VectorQuery('embedding_vector_dot', vector, num_candidates=2))) + # Change the limit value to return more results. Change the fields array to return different fields from your Search index. + result = scope.search(search_index, search_req, SearchOptions(limit=13,fields=["color", "description"])) + for row in result.rows(): + print("Found row: {}".format(row)) + print("Reported total rows: {}".format( + result.metadata().metrics().total_rows())) +except CouchbaseException as ex: + import traceback + traceback.print_exc() \ No newline at end of file diff --git a/modules/vector-search/examples/run-vector-search-header.sh b/modules/vector-search/examples/run-vector-search-header.sh new file mode 100644 index 000000000..19417565c --- /dev/null +++ b/modules/vector-search/examples/run-vector-search-header.sh @@ -0,0 +1,2 @@ +curl -s -XPUT -H "Content-Type: application/json" \ +-u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/${BUCKET_NAM}E/scope/${SCOPE_NAME}/index/${INDEX_NAME}/query -d \ \ No newline at end of file diff --git a/modules/vector-search/examples/run-vector-search-long-payload-ui.jsonc b/modules/vector-search/examples/run-vector-search-long-payload-ui.jsonc new file mode 100644 index 000000000..d7c2d9a93 --- /dev/null +++ b/modules/vector-search/examples/run-vector-search-long-payload-ui.jsonc @@ -0,0 +1,1553 @@ +// tag::partial[] +{ + "fields": ["*"], + "knn": [ + { + "field": "embedding_vector_dot", + "k": 3, + "vector": [ + 0.024032991379499435, + -0.009131478145718575, + 0.013961897231638432, + -0.024734394624829292, + -0.020605377852916718, + 0.006739427801221609, + -0.012539239600300789, + 0.0063192471861839294, + 0.000004374724539957242, + -0.030252983793616295, + -0.010944539681077003, + -0.0012845275923609734, + 0.0059850881807506084, + -0.006388725712895393, + -0.016304319724440575, + 0.03046472743153572, + 0.029988301917910576, + -0.013121536932885647, + 0.01815708354115486, + -0.011096730828285217, + -0.0423753522336483, + -0.0023523480631411076, + -0.00022332418302539736, + -0.0024681459181010723, + -0.02911485731601715, +// end::partial[] + -0.01983780413866043, + 0.025263754650950432, + -0.02444324642419815, + 0.001819678582251072, + -0.01301566418260336, + 0.021015632897615433, + -0.006084343418478966, + -0.027394432574510574, + -0.02503877691924572, + -0.0030520972795784473, + 0.0020049549639225006, + -0.006124045234173536, + 0.01994367688894272, + 0.0008941238629631698, + 0.005230748560279608, + 0.004694770555943251, + 0.0074507566168904305, + 0.007325033191591501, + -0.01629108563065529, + -0.000024400234906352125, + 0.002851932542398572, + 0.007550011854618788, + -0.005564908031374216, + -0.002355656586587429, + -0.0024466405156999826, + 0.025422563776373863, + 0.042851775884628296, + -0.016185212880373, + -0.02908838912844658, + -0.002714629517868161, + 0.006143896374851465, + -0.011930474080145359, + 0.027685582637786865, + -0.008377138525247574, + 0.0012580595212057233, + 0.0012944531626999378, + -0.02789732627570629, + -0.02935306914150715, + -0.000022397038264898583, + -0.015377937816083431, + -0.00907854177057743, + -0.022788992151618004, + 0.027526773512363434, + 0.00034181008231826127, + 0.018964359536767006, + 0.010951156727969646, + 0.01720423437654972, + 0.007000799756497145, + 0.018104147166013718, + 0.02603132836520672, + 0.0017601253930479288, + -0.012863473035395145, + 0.0016286122845485806, + 0.012876707129180431, + -0.000903222244232893, + 0.004208420403301716, + -0.0030984163749963045, + -0.021280312910676003, + 0.010673241689801216, + 0.015801426023244858, + -0.004473100882023573, + -0.015788191929459572, + -0.0008205096237361431, + -0.0007990042795427144, + -0.005336620844900608, + -0.01697925478219986, + 0.013975131325423717, + 0.014120705425739288, + 0.016820447519421577, + -0.013187706470489502, + 0.01565585285425186, + 0.021968482062220573, + 0.020353931933641434, + 0.006928012706339359, + 0.006497906520962715, + 0.02543579787015915, + 0.005733641795814037, + -0.028611963614821434, + -0.005538439843803644, + -0.01806444488465786, + -0.000736969814170152, + -0.002863512374460697, + -0.0022233163472265005, + 0.012605409137904644, + 0.0006480537122115493, + -0.04531330242753029, + 0.02698417939245701, + 0.02319924719631672, + -0.07024621218442917, + 0.011996644549071789, + -0.018024742603302002, + 0.018448231741786003, + -0.020909760147333145, + 0.012962727807462215, + -0.028876643627882004, + -0.006253077182918787, + 0.02842668816447258, + 0.04396343231201172, + -0.011407730169594288, + 0.01753508485853672, + -0.008132308721542358, + -0.003953665029257536, + -0.023013969883322716, + -0.033482085913419724, + -0.0037981653586030006, + 0.017455680295825005, + 0.0035533360205590725, + -0.012499537318944931, + 0.012605409137904644, + 0.007960266433656216, + 0.005336620844900608, + -0.01569555513560772, + 0.020181888714432716, + 0.01462359819561243, + -0.003325049066916108, + 0.022087588906288147, + 0.025713711977005005, + -0.00886679720133543, + 0.005432567559182644, + -0.017415978014469147, + 0.01001815777271986, + 0.006309321615844965, + 0.006451587658375502, + 0.0034673146437853575, + -0.00044209917541593313, + 0.028082601726055145, + -0.008165393956005573, + 0.04393696412444115, + 0.014200109988451004, + -0.010223285295069218, + 0.020380400121212006, + -0.006474747322499752, + 0.012651728466153145, + -0.02260371670126915, + 0.007060352712869644, + -0.008562414906919003, + -0.006196832749992609, + 0.011639325879514217, + 0.0041852607391774654, + 0.015589682385325432, + 0.017455680295825005, + 0.009535115212202072, + 0.00853594671934843, + 0.0044929515570402145, + 0.015457342378795147, + -0.029564812779426575, + 0.004856887273490429, + -0.01381632313132286, + 0.016926318407058716, + 0.01664840430021286, + 0.03329680860042572, + 0.03544072061777115, + 0.0004168718005530536, + -0.001428447663784027, + -0.0028668208979070187, + -0.008350670337677002, + 0.015100023709237576, + 0.02648128569126129, + -0.015430874191224575, + 0.004459866788238287, + 0.011136433109641075, + 0.042957648634910583, + -0.016211681067943573, + 0.0009454057435505092, + -0.0006463994504883885, + 0.002449949039146304, + 0.019229039549827576, + -0.011374644935131073, + -0.025978391990065575, + -0.651749312877655, + -0.010838666930794716, + -0.020115718245506287, + -0.013445770367980003, + 0.022418439388275146, + 0.022775758057832718, + 0.02382124587893486, + 0.010554135777056217, + -0.022749289870262146, + -0.02069801650941372, + -0.0025756722316145897, + 0.010620305314660072, + 0.004258047789335251, + -0.004555813502520323, + -0.025687243789434433, + -0.017164532095193863, + 0.006683182902634144, + -0.004039686173200607, + 0.006755969952791929, + -0.001948710298165679, + 0.014491258189082146, + 0.02924719639122486, + -0.03925212100148201, + -0.013101685792207718, + 0.013856025412678719, + 0.014822108671069145, + -0.0013556604972109199, + -0.002663347637280822, + -0.01099085807800293, + 0.0030239748302847147, + -0.017098361626267433, + 0.02619013749063015, + -0.011288624256849289, + 0.007768373005092144, + 0.052724357694387436, + -0.005472269840538502, + -0.027950262650847435, + 0.02911485731601715, + -0.004092622548341751, + 0.047298409044742584, + 0.007298565004020929, + 0.00609426898881793, + 0.008403606712818146, + -0.02016865462064743, + -0.0014532614732161164, + 0.020486271008849144, + 0.004552504979074001, + -0.028109069913625717, + 0.025343159213662148, + -0.012843621894717216, + 0.026269542053341866, + -0.005389557220041752, + 0.005214205943048, + 0.012625260278582573, + 0.010527667589485645, + -0.009455711580812931, + 0.026110732927918434, + -0.016661638393998146, + -0.017230700701475143, + 0.021769972518086433, + 0.01143419835716486, + 0.01828942447900772, + -0.005220822989940643, + -0.01529853418469429, + -0.01018358301371336, + 0.044333986937999725, + -0.029723621904850006, + -0.009257201105356216, + -0.009667456150054932, + -0.01136802788823843, + -0.016674872487783432, + 0.018315890803933144, + -0.013617812655866146, + -0.006435045041143894, + 0.00004895555684925057, + -0.0009371344349347055, + 0.009925519116222858, + 0.019930442795157433, + -0.00006508452497655526, + 0.01435891818255186, + -0.00855579786002636, + -0.019559890031814575, + -0.008330819196999073, + 0.0074309054762125015, + 0.022524312138557434, + -0.0004739435389637947, + -0.029750090092420578, + -0.03112642839550972, + 0.01635725609958172, + 0.015735257416963577, + 0.03221162036061287, + 0.01529853418469429, + -0.0031728576868772507, + -0.018170317634940147, + -0.018368827179074287, + 0.019599592313170433, + -0.01393542904406786, + -0.013842791318893433, + -0.016463128849864006, + -0.01426627952605486, + -0.006759278476238251, + -0.03874922916293144, + -0.022286098450422287, + 0.009422626346349716, + 0.009462328627705574, + 0.007966883480548859, + 0.0047907172702252865, + 0.02681213617324829, + 0.008403606712818146, + -0.017191000282764435, + 0.015470576472580433, + -0.003626123070716858, + -0.015576448291540146, + 0.010110795497894287, + 0.01095777377486229, + -0.03300565853714943, + 0.015576448291540146, + 0.004181952215731144, + -0.005713790655136108, + -0.031470514833927155, + 0.021822908893227577, + -0.013882492668926716, + -0.0054855034686625, + -0.0008444962440989912, + 0.004516111221164465, + 0.013339897617697716, + -0.008999137207865715, + -0.007232395000755787, + 0.016939552500844002, + -0.0021885770838707685, + 0.02667979523539543, + 0.024866733700037003, + 0.010640156455338001, + 0.00876754242926836, + -0.008648435585200787, + -0.011626091785728931, + -0.021756738424301147, + -0.032158683985471725, + -0.0016699685947969556, + -0.016171978786587715, + -0.003808090928941965, + -0.006808906327933073, + -0.011096730828285217, + 0.0059156096540391445, + -0.011242304928600788, + -0.04578972980380058, + -0.027235625311732292, + -0.001535146962851286, + -0.030120642855763435, + -0.005214205943048, + -0.02289486490190029, + -0.013121536932885647, + -0.024337373673915863, + 0.040019694715738297, + 0.011447432450950146, + 0.0007717091357335448, + -0.02802966721355915, + -0.009779945015907288, + 0.0006215856410562992, + 0.011817985214293003, + 0.006484672427177429, + 0.02089652605354786, + -0.025157883763313293, + 0.011725346557796001, + -0.00974024273455143, + -0.035652466118335724, + -0.011996644549071789, + 0.037425823509693146, + -0.02694447711110115, + -0.027447368949651718, + 0.008747691288590431, + 0.009965221397578716, + 0.007530160713940859, + 0.01865997724235058, + 0.011844453401863575, + 0.015523511916399002, + -0.015589682385325432, + -0.02718268893659115, + -0.007788224145770073, + -0.024403544142842293, + 0.01283038780093193, + 0.013829557225108147, + -0.03345561772584915, + -0.009018988348543644, + 0.060982391238212585, + 0.000804794195573777, + 0.009680690243840218, + 0.023609502241015434, + 0.006474747322499752, + -0.004674919415265322, + 0.006090960465371609, + 0.000341189734172076, + -0.0031877460423856974, + 0.007569862529635429, + -0.005925535224378109, + 0.0037981653586030006, + 0.006110811606049538, + 0.00162695802282542, + 0.011566538363695145, + 0.013009047135710716, + 0.015192661434412003, + -0.02388741634786129, + 0.01214221864938736, + 0.0037518462631851435, + 0.02109503746032715, + -0.0071596079505980015, + 0.01191062293946743, + -0.012367197312414646, + 0.009191030636429787, + 0.036711186170578, + 0.015139725059270859, + -0.01605287380516529, + -0.004539270885288715, + -0.028903111815452576, + 0.023940352723002434, + 0.00964098796248436, + 0.001996683655306697, + 0.027950262650847435, + -0.00865505263209343, + -0.006564076989889145, + 0.030226515606045723, + 0.024138862267136574, + 0.020062783733010292, + 0.006431736517697573, + 0.007556628901511431, + -0.004145558457821608, + 0.021611163392663002, + -0.01704542525112629, + -0.01731010526418686, + -0.011870920658111572, + -0.010712943971157074, + 0.021637631580233574, + -0.012724515981972218, + 0.0007100881775841117, + -0.021412653848528862, + -0.003788239788264036, + 0.021637631580233574, + -0.025343159213662148, + 0.016952786594629288, + -0.005138110369443893, + -0.009462328627705574, + 0.011321709491312504, + -0.00962113682180643, + -0.003376330714672804, + 0.04375169053673744, + 0.0035764954518526793, + 0.016608702018857002, + 0.008674903772771358, + -0.02470792643725872, + -0.01157977245748043, + -0.0007167052244767547, + 0.010090944357216358, + 0.0015897373668849468, + -0.012856855988502502, + 0.015483810566365719, + 0.006226608995348215, + 0.03046472743153572, + 0.0045194197446107864, + 0.00031430809758603573, + 0.01122245378792286, + 0.02826787903904915, + -0.01930844411253929, + -0.013095068745315075, + -0.02158469520509243, + 0.012737750075757504, + -0.00013575215416494757, + -0.0029578048270195723, + -0.0021836142987012863, + 0.0074441395699977875, + -0.017429212108254433, + -0.009700540453195572, + -0.002613720018416643, + 0.005379631649702787, + -0.02964421734213829, + 0.013095068745315075, + 0.003794856835156679, + -0.002913139993324876, + 0.0006728675216436386, + 0.02487996779382229, + 0.0037055271677672863, + -0.008066138252615929, + -0.03287332132458687, + 0.023371288552880287, + 0.03676412254571915, + -0.01757478527724743, + -0.02871783636510372, + -0.03162932023406029, + 0.009270435199141502, + -0.006329172756522894, + 0.035652466118335724, + 0.009965221397578716, + 0.014173641800880432, + -0.007338267285376787, + -0.008165393956005573, + -0.010878369212150574, + -0.01648959517478943, + 0.020539207383990288, + 0.006051258184015751, + 0.010501199401915073, + 0.016185212880373, + 0.020446570590138435, + -0.020737718790769577, + -0.01711159572005272, + 0.011361410841345787, + 0.02362273633480072, + 0.020406868308782578, + 0.00044127204455435276, + 0.006491289474070072, + 0.007397820241749287, + -0.009316754527390003, + -0.0038610270712524652, + -0.0034640063531696796, + -0.004406930413097143, + -0.0034640063531696796, + 0.031735192984342575, + 0.004211728926748037, + -0.010600454173982143, + -0.007463990710675716, + 0.04571032524108887, + -0.014941215515136719, + 0.010924688540399075, + -0.023305119946599007, + -0.02665332704782486, + -0.003354825545102358, + 0.07643973082304001, + -0.006398651283234358, + -0.007695585954934359, + 0.012439983896911144, + -0.009091775864362717, + -0.009085158817470074, + -0.03761110082268715, + -0.00696771452203393, + 0.018183551728725433, + 0.022947801277041435, + 0.019268741831183434, + -0.00004192498090560548, + -0.01859380677342415, + 0.011950325220823288, + 0.006415193900465965, + -0.0043870797380805016, + 0.013425919227302074, + -0.024668224155902863, + -0.02056567557156086, + -0.018540870398283005, + -0.0038279418367892504, + 0.0013523519737645984, + -0.0003715866187121719, + 0.01462359819561243, + -0.0010727832559496164, + 0.019851038232445717, + 0.029961833730340004, + 0.012532622553408146, + 0.015616150572896004, + -0.01605287380516529, + 0.002143912250176072, + 0.01839529536664486, + -0.01921580545604229, + 0.012512771412730217, + -0.007027267478406429, + -0.006315938662737608, + -0.01957312412559986, + 0.015510277822613716, + 0.009746859781444073, + -0.01924227364361286, + 0.028320815414190292, + 0.00909839291125536, + 0.0005922226700931787, + -0.0003314709756523371, + 0.015126491896808147, + -0.009224115870893002, + 0.004671610891819, + 0.0048535787500441074, + -0.0031000706367194653, + -0.00907854177057743, + 0.0035401019267737865, + -0.02415209636092186, + -0.013167855329811573, + 0.013432536274194717, + 0.006841991096735001, + 0.021597929298877716, + 0.0002160868316423148, + -0.0038180164992809296, + 0.0008908153395168483, + -0.018315890803933144, + -0.019652528688311577, + -0.008893265388906002, + -0.0016608702717348933, + -0.018540870398283005, + -0.014438321813941002, + -0.04007263109087944, + 0.005839514080435038, + -0.03292625769972801, + -0.01275098416954279, + 0.013240642845630646, + 0.010243135504424572, + -0.01592053286731243, + -0.034699615091085434, + 0.013617812655866146, + 0.031496983021497726, + 0.014332449994981289, + 0.0037386121693998575, + -0.01435891818255186, + 0.007258863188326359, + -0.010971006937325, + 0.013194323517382145, + -0.014901513233780861, + 0.0035698784049600363, + -0.016436660662293434, + 0.003374676452949643, + -0.01809091307222843, + 0.004377154167741537, + -0.000208849465707317, + -0.019599592313170433, + 0.013908960856497288, + 0.011215836741030216, + -0.020089251920580864, + 0.034302596002817154, + -0.01934814639389515, + -0.031708724796772, + -0.009773327969014645, + -0.009554966352880001, + 0.003394527593627572, + -0.01961282640695572, + -0.005234057083725929, + 0.005826279986649752, + -0.02036716602742672, + -0.012439983896911144, + -0.022722821682691574, + 0.008218330331146717, + -0.010196817107498646, + -0.010699709877371788, + 0.010646773502230644, + -0.02898251637816429, + 0.012029729783535004, + 0.007199310231953859, + -0.039728544652462006, + -0.007199310231953859, + 0.008330819196999073, + 0.0016079341294243932, + -0.004658377263695002, + -0.024694692343473434, + 0.031099960207939148, + -0.005783269181847572, + -0.025475500151515007, + 0.014305981807410717, + -0.03345561772584915, + 0.005670779850333929, + 0.01757478527724743, + -0.030358854681253433, + 0.018607040867209435, + 0.004691462032496929, + -0.005588067229837179, + -0.007550011854618788, + 0.009965221397578716, + -0.011235687881708145, + 0.013220791704952717, + -0.03250276669859886, + -0.01655576564371586, + -0.030808812007308006, + -0.006772512570023537, + -0.012598792091012001, + -0.012975961901247501, + -0.016436660662293434, + -0.009224115870893002, + 0.010679858736693859, + 0.02233903482556343, + -0.009958604350686073, + -0.020009847357869148, + -0.02417856454849243, + -0.03385263681411743, + 0.011990027502179146, + 0.01905699633061886, + -0.01799827441573143, + 0.02368890680372715, + -0.006666640285402536, + -0.015457342378795147, + -0.011440815404057503, + 0.00642511947080493, + 0.004370537120848894, + -0.0013837828300893307, + 0.0035764954518526793, + 0.0028883260674774647, + 0.0052903019823133945, + -0.010732795111835003, + 0.003748537739738822, + 0.030517663806676865, + 0.033905573189258575, + 0.004787408746778965, + -0.004565739072859287, + 0.010110795497894287, + -0.004837036598473787, + -0.0074375225231051445, + -0.0013746843906119466, + 0.018104147166013718, + 0.005558290984481573, + 0.017693892121315002, + 0.0018974284175783396, + 0.022166993468999863, + 0.007841160520911217, + 0.032026343047618866, + 0.0003720001841429621, + -0.006302704568952322, + -0.02720915712416172, + -0.05537116527557373, + -0.006709651090204716, + 0.0031447354704141617, + 0.0028122304938733578, + 0.014530960470438004, + -0.009713774546980858, + -0.0021968481596559286, + 0.01786593534052372, + -0.013399451039731503, + -0.021200908347964287, + 0.0005690631223842502, + 0.021611163392663002, + 0.013035515323281288, + 0.04565738886594772, + -0.03599654883146286, + 0.014583896845579147, + -0.007953649386763573, + -0.018104147166013718, + -0.0044995686039328575, + -0.013492089696228504, + 0.002600486157462001, + -0.00028287729946896434, + 0.022034652531147003, + -0.004033069126307964, + 0.014067769050598145, + 0.014319215901196003, + 0.007583096623420715, + 0.006626938469707966, + 0.00865505263209343, + 0.010024774819612503, + -0.014279513619840145, + -0.022299332544207573, + -0.030279450118541718, + -0.025118181481957436, + -0.01421334408223629, + 0.014094237238168716, + 0.003205942688509822, + -0.024919670075178146, + 0.022921333089470863, + -0.02089652605354786, + 0.0013085142709314823, + 0.02260371670126915, + -0.011334942653775215, + 0.022881630808115005, + -0.0006025617476552725, + 0.0510568730533123, + -0.003523559309542179, + -0.016410192474722862, + 0.008251414634287357, + 0.004138941410928965, + -0.010507816448807716, + -0.0068552251905202866, + 0.022656651213765144, + 0.004645143169909716, + -0.004919749218970537, + -0.006673257332295179, + 0.02566077560186386, + 0.007364735472947359, + -0.02789732627570629, + -0.008800626732409, + 0.009846115484833717, + 0.01329357922077179, + 0.00919764768332243, + -0.02105533517897129, + 0.008085989393293858, + -0.011090113781392574, + 0.02362273633480072, + 0.004691462032496929, + -0.0005132320802658796, + 0.01157977245748043, + -0.018276190385222435, + 0.0013589690206572413, + 0.03192047029733658, + -0.008416840806603432, + 0.01852763630449772, + -0.012876707129180431, + -0.029935365542769432, + -0.015669086948037148, + -0.012711281888186932, + -0.021703802049160004, + -0.004674919415265322, + -0.0062630027532577515, + 0.03642003983259201, + 0.0009627753752283752, + 0.0018461465369910002, + 0.015205895528197289, + -0.0014615326654165983, + -0.0010810544481500983, + -0.013505322858691216, + 0.006703034043312073, + 0.027659114450216293, + -0.007583096623420715, + 0.0034805487375706434, + 0.0031397726852446795, + -0.008039670996367931, + 0.019586358219385147, + -0.021796440705657005, + 0.0024830340407788754, + -0.006514449138194323, + -0.01181136816740036, + -0.013882492668926716, + 0.019229039549827576, + -0.0074507566168904305, + -0.014345684088766575, + 0.017124829813838005, + -0.0029710386879742146, + -0.0005082692950963974, + -0.03914624825119972, + -0.007100054994225502, + 0.009832881391048431, + -0.011685644276440144, + -0.026071030646562576, + -0.024363841861486435, + -0.0008626930648460984, + 0.0023937043733894825, + -0.008674903772771358, + -0.0371876135468483, + -0.01569555513560772, + 0.010243135504424572, + -0.031470514833927155, + 0.02059214375913143, + 0.00004500085560721345, + 0.017958572134375572, + -0.021333249285817146, + 0.00919764768332243, + -0.0006501215393655002, + -0.01562938466668129, + 0.025157883763313293, + 0.010951156727969646, + -0.03491136059165001, + -0.015417640097439289, + 0.005095100030303001, + 0.0034739316906780005, + 0.02319924719631672, + -0.014226577244699001, + -0.022299332544207573, + 0.023609502241015434, + 0.01862027496099472, + 0.0018411838682368398, + -0.008516095578670502, + 0.003245644737035036, + -0.013035515323281288, + 0.0021273696329444647, + 0.010534284636378288, + -0.034540805965662, + 0.004119090735912323, + -0.016463128849864006, + -0.025806350633502007, + -0.009733625687658787, + -0.030358854681253433, + -0.0068684592843055725, + -0.010309305973351002, + 0.03327034041285515, + 0.007258863188326359, + -0.013114919885993004, + 0.009052073583006859, + -0.005879215896129608, + -0.011301858350634575, + 0.019520187750458717, + -0.0032820384949445724, + 0.00023759211762808263, + -0.010679858736693859, + 0.016727808862924576, + 0.03504370152950287, + 0.014279513619840145, + -0.022775758057832718, + -0.024602053686976433, + -0.024324139580130577, + -0.005022312980145216, + -0.020115718245506287, + 0.014583896845579147, + 0.0023242258466780186, + 0.027553241699934006, + 0.0035996551159769297, + -0.04375169053673744, + -0.014994150958955288, + 0.014610364101827145, + -0.022908098995685577, + -0.028585495427250862, + 0.006683182902634144, + 0.014385386370122433, + 0.008562414906919003, + 0.023715374991297722, + 0.0015450725331902504, + 0.01921580545604229, + -0.00023200901341624558, + 0.0017535084625706077, + -0.015245597809553146, + 0.02718268893659115, + -0.03006770648062229, + -0.04261356219649315, + 0.0012952802935615182, + -0.007913947105407715, + -0.02016865462064743, + -0.009607902728021145, + 0.01625138334929943, + -0.003731995355337858, + 0.0004545060801319778, + 0.01957312412559986, + -0.01468976866453886, + 0.0017402743687853217, + -0.0008258859161287546, + 0.02882370725274086, + 0.03583774343132973, + 0.015854362398386, + 0.0024334064219146967, + -0.028479622676968575, + 0.0027940336149185896, + -0.004939599893987179, + 0.008899882435798645, + -0.03366736322641373, + 0.017601253464818, + 0.007861011661589146, + 0.002329188631847501, + -0.015933766961097717, + -0.007093437947332859, + 0.007126522716134787, + -0.00860211718827486, + 0.009124861098825932, + 0.022616950795054436, + 0.010110795497894287, + -0.024800565093755722, + 0.020049549639225006, + 0.012102516368031502, + 0.0019503645598888397, + -0.011407730169594288, + -0.023119842633605003, + 0.010382093489170074, + -0.020539207383990288, + 0.013419302180409431, + -0.026163669303059578, + 0.025885755196213722, + -0.03557306155562401, + 0.0066037788055837154, + -0.01181136816740036, + -0.019109932705760002, + -0.011116581968963146, + -0.011851070448756218, + -0.01447802409529686, + 0.02300073765218258, + 0.01115628331899643, + -0.013346514664590359, + 0.014279513619840145, + -0.03395850956439972, + -0.014279513619840145, + -0.011282007209956646, + 0.004674919415265322, + -0.011070262640714645, + -0.02286839671432972, + 0.0014052881160750985, + -0.009105009958148003, + -0.013002430088818073, + -0.01393542904406786, + 0.0022994119208306074, + -0.020406868308782578, + 0.022709587588906288, + 0.010600454173982143, + 0.21873198449611664, + 0.018911423161625862, + -0.014504492282867432, + 0.042719434946775436, + 0.01435891818255186, + 0.004827111028134823, + 0.029300132766366005, + 0.0004511975566856563, + -0.008747691288590431, + 0.02977655827999115, + 0.011798134073615074, + 0.016264617443084717, + -0.02497260645031929, + 0.012433366850018501, + 0.0011579772690311074, + -0.013657514937222004, + -0.01332666352391243, + -0.009495413862168789, + -0.02922072820365429, + -0.023172779008746147, + 0.007986734621226788, + -0.01625138334929943, + -0.006537608802318573, + -0.022855162620544434, + 0.029591280966997147, + 0.019851038232445717, + 0.012870090082287788, + -0.003679059213027358, + -0.0006195178139023483, + 0.011182751506567001, + -0.024813799187541008, + 0.001568232080899179, + 0.01258555892854929, + 0.005058706272393465, + -0.019652528688311577, + -0.0051314933225512505, + -0.013253876939415932, + -0.013313430361449718, + 0.010481348261237144, + 0.021280312910676003, + 0.0018841944402083755, + 0.019652528688311577, + -0.0052903019823133945, + 0.007788224145770073, + 0.019864272326231003, + 0.01819678582251072, + -0.021703802049160004, + -0.007291948422789574, + 0.011738580651581287, + 0.005330003798007965, + -0.027394432574510574, + 0.010838666930794716, + 0.012168686836957932, + 0.03329680860042572, + -0.002430097898468375, + 0.01207604818046093, + 0.019454017281532288, + -0.0028750922065228224, + -0.006239843089133501, + -0.013697216287255287, + -0.02293456718325615, + 0.026891540735960007, + 0.0013457350432872772, + 0.01846146583557129, + 0.012248090468347073, + 0.030517663806676865, + -0.02718268893659115, + -0.03631416708230972, + -0.0065409173257648945, + 0.007325033191591501, + 0.02674596570432186, + 0.0019652529153972864, + -0.013240642845630646, + -0.0008949509938247502, + 0.003151352284476161, + -0.01535146962851286, + 0.018607040867209435, + 0.01589406467974186, + 0.01523236371576786, + 0.01737627573311329, + -0.016820447519421577, + 0.003904037643224001, + -0.006755969952791929, + 0.023384522646665573, + 0.006097577512264252, + -0.02484026551246643, + -0.013591344468295574, + -0.00997183844447136, + 0.0009396158275194466, + 0.012552473694086075, + 0.013313430361449718, + 0.008846946060657501, + -0.009091775864362717, + 0.007192693185061216, + 0.015669086948037148, + 0.0004962759558111429, + 0.01596023514866829, + 0.007205926813185215, + -0.022855162620544434, + -0.013386216945946217, + -0.015311767347157001, + 0.01110334787517786, + 0.029882431030273438, + 0.04017850384116173, + -0.007172842044383287, + 0.010620305314660072, + -0.011857687495648861, + -0.009581434540450573, + 0.005191046744585037, + -0.023781543597579002, + -0.0014648411888629198, + -0.01421334408223629, + -0.005157961510121822, + -0.019731933251023293, + -0.015338235534727573, + 0.00039578007999807596, + -0.0022530928254127502, + -0.026494519785046577, + 0.009071924723684788, + -0.0018180243205279112, + -0.0018494550604373217, + -0.026415115222334862, + 0.0005885005812160671, + 0.01717776618897915, + 0.013492089696228504, + -0.013710450381040573, + -0.030385322868824005, + 0.016661638393998146, + 0.0007212543860077858, + -0.04346054047346115, + 0.013518556952476501, + -0.009515264071524143, + 0.02457558549940586, + -0.014345684088766575, + -0.012135601602494717, + 0.004807259887456894, + 0.01770712621510029, + -0.013220791704952717, + 0.006557459942996502, + -0.0002204292395617813, + -0.013518556952476501, + -0.010653390549123287, + 0.02280222624540329, + 0.004962759558111429, + 0.009105009958148003, + -0.03329680860042572, + -0.02464175596833229, + -0.012466452084481716, + -0.02306690625846386, + -0.02098916471004486, + -0.0017634339164942503, + -0.026401881128549576, + -0.008939584717154503, + -0.018712911754846573, + 0.0027592943515628576, + 0.0013399451272562146, + -0.010084327310323715, + 0.009151328355073929, + 0.01245321799069643, + 0.03030591830611229, + -0.018607040867209435, + 0.019586358219385147, + 0.05807090550661087, + -0.01572202332317829, + -0.005684013944119215, + -0.0054027908481657505, + -0.16833680868148804, + 0.041501905769109726, + 0.014663300476968288, + -0.022841928526759148, + -0.005048780702054501, + 0.003020666306838393, + 0.006190215703099966, + -0.009799796156585217, + -0.017654189839959145, + 0.013657514937222004, + 0.010746029205620289, + 0.0064714387990534306, + -0.03597008064389229, + -0.008522712625563145, + -0.010732795111835003, + 0.003278729971498251, + -0.007470607757568359, + 0.00561122689396143, + 0.014160407707095146, + 0.001894119894132018, + 0.020671548321843147, + -0.02648128569126129, + 0.021280312910676003, + -0.0006691454327665269, + -0.004516111221164465, + -0.013280345126986504, + -0.015219129621982574, + 0.015616150572896004, + -0.0029164485167711973, + -0.009508647955954075, + -0.013134771026670933, + -0.002851932542398572, + 0.00010080605716211721, + 0.015047087334096432, + -0.005561599507927895, + 0.008972669020295143, + -0.016621936112642288, + -0.018911423161625862, + -0.012526005506515503, + 0.006315938662737608, + 0.014861810952425003, + 0.010646773502230644, + 0.02266988530755043, + 0.004324217792600393, + -0.010342391207814217, + 0.04168717935681343, + 0.04343407228589058, + 0.007113288622349501, + 0.02685183845460415, + -0.02056567557156086, + -0.005144727416336536, + -0.002031422918662429, + 0.015192661434412003, + -0.0022712897043675184, + 0.012069431133568287, + 0.027712050825357437, + 0.01072617806494236, + 0.028320815414190292, + 0.004899898078292608, + -0.014769172295928001, + -0.001813061535358429, + -0.011255539022386074, + 0.001765088178217411, + 0.006392034236341715, + 0.02023482508957386, + 0.006679874379187822, + 0.00832420215010643, + 0.004582281224429607, + 0.011731963604688644, + 0.009025605395436287, + 0.014345684088766575, + -0.022788992151618004, + 0.012856855988502502, + -0.013009047135710716, + 0.01885848678648472, + 0.0074441395699977875, + -0.011235687881708145, + 0.008688137866556644, + -0.014345684088766575, + -0.0009172834106720984, + -0.020512739196419716, + 0.013359748758375645, + -0.019983379170298576, + -0.0054855034686625, + -0.003305197926238179, + 0.02398005500435829, + -0.01535146962851286, + 0.003318432020023465, + -0.005419333465397358, + -0.005538439843803644, + 0.017561553046107292, + -0.035811275243759155, + -0.017389509826898575, + -0.02517111785709858, + -0.006676565855741501, + -0.005942077375948429, + -0.0035268678329885006, + 0.02076418697834015, + 0.0006819659029133618, + -0.014875045046210289, + 0.02030099555850029, + -0.011672411113977432, + 0.0013465621741488576, + -0.0037187612615525723, + 0.04385755956172943, + 0.01970546506345272, + -0.013803089037537575, + 0.017323339357972145, + 0.04213713854551315, + 0.034196723252534866, + -0.02379477769136429, + 0.03422319144010544, + 0.0415283739566803, + 0.001535146962851286, + -0.030835280194878578, + 0.011996644549071789, + 0.009535115212202072, + -0.02253754623234272, + 0.00016025577497202903, + 0.004344068933278322, + 0.03663178160786629, + -0.003272112924605608, + -0.0056310780346393585, + -0.0014276205329224467, + -0.007748521864414215, + -0.021902313455939293, + -0.10444293171167374, + 0.007861011661589146, + -0.008707989007234573, + -0.011182751506567001, + 0.010944539681077003, + 0.02102886699140072, + -0.006203449796885252, + -0.014980916865170002, + 0.0035731869284063578, + 0.026891540735960007, + -0.005561599507927895, + -0.017217466607689857, + -0.0008002449758350849, + -0.019136400893330574, + 0.01914963498711586, + -0.0007324205944314599, + 0.012883324176073074, + -0.011420964263379574, + -0.015444108285009861, + 0.012711281888186932, + -0.002456566086038947, + -0.01629108563065529, + -0.011526836082339287, + -0.017217466607689857, + 0.01611904427409172, + -0.014173641800880432, + -0.026137201115489006, + 0.0017435828922316432, + 0.007113288622349501, + 0.002567400922998786, + 0.012360580265522003, + -0.016304319724440575, + -0.004135632887482643, + -0.01937461458146572, + -0.02678566798567772, + -0.016966020688414574, + -0.016436660662293434, + -0.004092622548341751, + 0.005581450182944536, + -0.018051210790872574, + 0.0062431516125798225, + 0.020486271008849144, + 0.020446570590138435, + -0.03332327678799629, + 0.021637631580233574, + -0.0037783144507557154, + 0.004651760216802359, + 0.003000815398991108, + 0.01914963498711586, + 0.0014648411888629198, + -0.014438321813941002, + -0.01115628331899643, + -0.01678074523806572, + -0.01711159572005272, + 0.015470576472580433, + -0.0034110702108591795, + 0.01569555513560772, + 0.005300227086991072, + -0.02043333649635315, + 0.010673241689801216, + -0.002790725091472268, + 0.017349807545542717, + -0.020049549639225006, + 0.02415209636092186, + -0.0071529909037053585, + -0.008145542815327644, + 0.005862673278898001, + -0.005184429697692394, + 0.02619013749063015, + -0.04547211155295372, + -0.041634246706962585, + -0.0004208006721455604, + -0.018434997648000717, + 0.02490643598139286, + -0.041369564831256866, + 0.006405268330127001, + -0.029670685529708862, + -0.002838698448613286, + 0.016039639711380005, + -0.006815523374825716, + -0.009839498437941074, + -0.010540901683270931, + 0.0033498627599328756, + -0.027817921712994576, + 0.008992520160973072, + 0.04348700866103172, + -0.005750184413045645, + 0.001955327345058322, + 0.0018560721073299646, + -0.021902313455939293, + -0.0027890708297491074, + 0.01872614584863186, + 0.021306781098246574, + -0.01426627952605486, + 0.002851932542398572, + 0.025224052369594574, + 0.013366365805268288, + -0.004913132172077894, + 0.02247137576341629, + 0.013432536274194717, + -0.01462359819561243, + 0.0004383771156426519, + -0.052194997668266296, + 0.032688044011592865, + -0.004585589747875929, + -0.015060321427881718, + 0.01572202332317829, + -0.00045037042582407594, + 0.02174350433051586, + -0.02898251637816429, + -0.01832912489771843, + -0.008138925768435001, + -0.0026666561607271433, + 0.017429212108254433, + -0.009224115870893002, + -0.013068600557744503, + -0.03504370152950287, + 0.016185212880373, + 0.014464790001511574, + -0.011070262640714645, + 0.014464790001511574, + 0.010878369212150574, + -0.020618611946702003, + 0.0003798578982241452, + 0.022219929844141006, + -0.007861011661589146, + -0.028109069913625717, + -0.008039670996367931, + -0.006815523374825716, + -0.010157114826142788, + -0.02326541766524315, + 0.00040053605334833264, + 0.024165330454707146, + -0.021333249285817146, + 0.008066138252615929, + 0.07749845832586288, + -0.005760109517723322, + -0.03409085050225258, + 0.002539278706535697, + 0.03046472743153572, + 0.02333158813416958, + -0.022484609857201576, + -0.012942877598106861, + -0.018964359536767006, + -0.005455727223306894, + -0.035228975117206573, + -0.016873382031917572, + 0.02922072820365429, + -0.02355656586587429, + 0.00033105743932537735, + 0.024813799187541008, + 0.00412901584059, + 0.00717945909127593, + 0.02039363421499729, + 0.010382093489170074, + -0.01400159951299429, + 0.013028898276388645, + -0.023583034053444862, + 0.0017948647728189826, + 0.017482148483395576, + 0.019255507737398148, + -0.007768373005092144, + 0.0015343198319897056, + 0.01700572296977043, + 0.005250599700957537, + -0.021280312910676003, + 0.020711250603199005, + -0.007854394614696503, + -0.00907854177057743, + 0.0051314933225512505, + -0.004327526316046715, + -0.025488734245300293, + -0.01549704372882843, + 0.011546687223017216, + 0.009647605009377003, + 0.004430090077221394, + 0.02592545561492443, + -0.01865997724235058, + 0.028373751789331436, + -0.02289486490190029, + -0.0005934633663855493, + 0.03684352710843086, + -0.02924719639122486, + -0.004909823648631573, + -0.03475255146622658, + 0.020062783733010292, + 0.04428105056285858, + 0.004774174652993679, + -0.018540870398283005, + -0.0002136054536094889, + -0.00023903959663584828, + 0.012089282274246216, + 0.016410192474722862, + -0.0011257192818447948, + -0.014822108671069145, + 0.009502030909061432, + -0.01750861667096615, + -0.014676534570753574, + -0.009601285681128502, + -0.018990827724337578, + -0.02579311653971672, + 0.03925212100148201, + -0.0030388631857931614, + -0.003904037643224001, + 0.009389541111886501, + -0.04668964445590973, + -0.007999968715012074, + -0.0042282710783183575, + -0.010249752551317215, + -0.012215006165206432, + -0.01093792263418436, + 0.010236518457531929, + 0.010540901683270931, + -0.004684844985604286, + -0.0028254645876586437, + 0.0071463738568127155, + 0.003768388880416751, + -0.013339897617697716, + -0.02098916471004486, + -0.02398005500435829, + -0.011877537705004215, + 0.02346392720937729, + 0.013260493986308575, + 0.028903111815452576, + 0.025263754650950432, + 0.016820447519421577, + -0.010124029591679573, + -0.016595467925071716, + 0.01136802788823843, + -0.009343221783638, + 0.018633509054780006, + 0.017415978014469147, + 0.015205895528197289, + -0.018898189067840576, + -0.023993289098143578, + -0.001014884328469634, + -0.022762523964047432, + -0.0063060130923986435, + -0.005253908224403858, + 0.01629108563065529, + -0.010554135777056217, + 0.06299396604299545, + 0.02734149806201458, + -0.019070230424404144, + 0.018302656710147858, + -0.009839498437941074, + 0.01855410449206829, + 0.01700572296977043, + -0.0015103331534191966, + -0.009382924064993858, + -0.01066000759601593, + 0.00022849372180644423, + -0.00116790272295475, + 0.028850175440311432, + -0.006408576853573322, + -0.02789732627570629, + 0.01393542904406786, + -0.026468051597476006, + -0.002397012896835804, + -0.018342358991503716, + -0.013975131325423717, + 0.004026452545076609, + 0.0065409173257648945, + 0.013736918568611145, + 0.00021009016199968755, + -0.006292779464274645, + -0.025846052914857864, + 0.008906499482691288, + 0.0015591336414217949, + -0.008304351009428501, + -0.0012588866520673037, + 0.02869136817753315, + 0.014650066383183002, + -0.03316446766257286, + -0.01691308431327343, + 0.010468114167451859, + -0.02539609558880329, + 0.008780776523053646, + -0.014517726376652718, + 0.003695601597428322, + 0.02244490757584572, + -0.025885755196213722, + -0.0031447354704141617, + -0.004803951364010572, + -0.034699615091085434, + -0.005455727223306894, + 0.00227459822781384, + 0.0015450725331902504, + 0.010461497120559216, + 0.0034673146437853575 + ] + } + ], + "query": { + "match_none": {} + }, + "sort": ["-_score"] + } \ No newline at end of file diff --git a/modules/vector-search/examples/run-vector-search-payload-ui.jsonc b/modules/vector-search/examples/run-vector-search-payload-ui.jsonc new file mode 100644 index 000000000..0fd107d45 --- /dev/null +++ b/modules/vector-search/examples/run-vector-search-payload-ui.jsonc @@ -0,0 +1,13 @@ +{ + "fields": ["*"], + "query": { + "match_none": "" + }, + "knn": [ + { + "k": 2, + "field": "colorvect_l2", + "vector": [ 0, 0, 128 ] + } + ] +} \ No newline at end of file diff --git a/modules/vector-search/examples/run-vector-search-payload.sh b/modules/vector-search/examples/run-vector-search-payload.sh new file mode 100644 index 000000000..40ff53fc9 --- /dev/null +++ b/modules/vector-search/examples/run-vector-search-payload.sh @@ -0,0 +1,19 @@ +curl -XPOST -H "Content-Type: application/json" \ + -u ${CB_USERNAME}:${CB_PASSWORD} https://${CB_HOSTNAME}:18094/api/bucket/vector-sample/scope/color/index/color-index/query \ +-d '{ + "fields": ["*"], + "query": { + "min": 70, + "max": 80, + "inclusive_min": false, + "inclusive_max": true, + "field": "brightness" + }, + "knn": [ + { + "k": 10, + "field": "colorvect_l2", + "vector": [ 176, 0, 176 ] + } + ] + }' \ No newline at end of file diff --git a/modules/vector-search/examples/run-vector-search-simple-color.py b/modules/vector-search/examples/run-vector-search-simple-color.py new file mode 100644 index 000000000..ee0869c13 --- /dev/null +++ b/modules/vector-search/examples/run-vector-search-simple-color.py @@ -0,0 +1,34 @@ +#!/usr/bin/env python3 + +import os +import sys +from couchbase.cluster import Cluster +from couchbase.options import ClusterOptions +from couchbase.auth import PasswordAuthenticator +from couchbase.exceptions import CouchbaseException +import couchbase.search as search +from couchbase.options import SearchOptions +from couchbase.vector_search import VectorQuery, VectorSearch + +# You can change the RGB values to search for a different color +vector = [0.0,0.0,128.0] + +# Make sure to change CB_USERNAME, CB_PASSWORD, and CB_HOSTNAME to the username, password, and hostname for your database. +pa = PasswordAuthenticator(os.getenv("CB_USERNAME"), os.getenv("CB_PASSWORD")) +cluster = Cluster("couchbases://" + os.getenv("CB_HOSTNAME") + "/?ssl=no_verify", ClusterOptions(pa)) +# Make sure to change the bucket, scope, and index names to match where you stored the sample data in your database. +bucket = cluster.bucket("vector-sample") +scope = bucket.scope("color") +search_index = "color-index" +try: + search_req = search.SearchRequest.create(search.MatchNoneQuery()).with_vector_search( + VectorSearch.from_vector_query(VectorQuery('colorvect_l2', vector, num_candidates=3))) + # Change the limit value to return more results. Change the fields array to return different fields from your Search index. + result = scope.search(search_index, search_req, SearchOptions(limit=13,fields=["color", "id"])) + for row in result.rows(): + print("Found row: {}".format(row)) + print("Reported total rows: {}".format( + result.metadata().metrics().total_rows())) +except CouchbaseException as ex: + import traceback + traceback.print_exc() \ No newline at end of file diff --git a/modules/vector-search/examples/vector-search-index-payload.jsonc b/modules/vector-search/examples/vector-search-index-payload.jsonc new file mode 100644 index 000000000..fe4ccce3a --- /dev/null +++ b/modules/vector-search/examples/vector-search-index-payload.jsonc @@ -0,0 +1,123 @@ +{ + "type": "fulltext-index", + "name": "vector-sample.color.color-index", + "sourceType": "gocbcore", + "sourceName": "vector-sample", + "planParams": { + "maxPartitionsPerPIndex": 512, + "indexPartitions": 1 + }, + "params": { + "doc_config": { + "docid_prefix_delim": "", + "docid_regexp": "", + "mode": "scope.collection.type_field", + "type_field": "type" + }, + "mapping": { + "analysis": {}, + "default_analyzer": "standard", + "default_datetime_parser": "dateTimeOptional", + "default_field": "_all", + "default_mapping": { + "dynamic": false, + "enabled": false + }, + "default_type": "_default", + "docvalues_dynamic": false, + "index_dynamic": false, + "store_dynamic": false, + "type_field": "_type", + "types": { + "color.rgb": { + "dynamic": false, + "enabled": true, + "properties": { + "brightness": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "index": true, + "name": "brightness", + "store": true, + "type": "number" + } + ] + }, + "color": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "analyzer": "en", + "index": true, + "name": "color", + "store": true, + "type": "text" + } + ] + }, + "colorvect_dot": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 3, + "index": true, + "name": "colorvect_dot", + "similarity": "dot_product", + "type": "vector" + } + ] + }, + "colorvect_l2": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 3, + "index": true, + "name": "colorvect_l2", + "similarity": "l2_norm", + "type": "vector" + } + ] + }, + "description": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "analyzer": "en", + "index": true, + "name": "description", + "store": true, + "type": "text" + } + ] + }, + "embedding_vector_dot": { + "dynamic": false, + "enabled": true, + "fields": [ + { + "dims": 1536, + "index": true, + "name": "embedding_vector_dot", + "similarity": "dot_product", + "type": "vector" + } + ] + } + } + } + } + }, + "store": { + "indexType": "scorch", + "segmentVersion": 16 + } + }, + "sourceParams": {} +} \ No newline at end of file diff --git a/modules/vector-search/pages/create-vector-search-index-rest-api.adoc b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc new file mode 100644 index 000000000..08449c2fe --- /dev/null +++ b/modules/vector-search/pages/create-vector-search-index-rest-api.adoc @@ -0,0 +1,72 @@ += Create a Vector Search Index with the REST API and curl/HTTP +:page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: You can create a Vector Search index with the Search Service API and Couchbase Capella. + +[abstract] +{description} + +You must create a Search index before you can xref:run-vector-search-rest-api.adoc[run a search] with the Search Service. + +TIP: Vector Search indexes can include all the same features and settings as a Search index. +For more information about Search indexes, see the xref:search:search.adoc[Search documentation]. + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. + +* You have documents in a keyspace inside your bucket that contain vector embeddings. ++ +include::partial$download-sample-partial.adoc[] + +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. + +== Procedure + +To create a Search index with the REST API and Couchbase Capella: + +. In your command-line tool, enter a `curl` command with the `XPUT` verb. +. Set your header content to include `"Content-Type: application/json"`. +. Enter your username, password, and the Search Service endpoint on port `18094` with the name of the index you want to create: ++ +[source,console] +---- +include::example$create-vector-search-index-header.sh[] +---- +. Enter the JSON payload for the settings you want in your index. ++ +Do not include the xref:search:search-index-params.adoc#uuid[uuid] or xref:search:search-index-params.adoc#sourceuuid[sourceUUID] parameters. ++ +TIP: If you remove the xref:search:search-index-params.adoc#uuid[uuid] and xref:search:search-index-params.adoc#sourceuuid[sourceUUID] parameters, you can copy the Search index definition JSON payload from the {page-ui-name} to use in a REST API call. +For more information about how to create a Vector Search index with the UI, see xref:create-vector-search-index-ui.adoc[]. + +include::search:partial$large-index-caution.adoc[tag=vector-search] + +=== Example + +In the following example, the JSON payload creates an index named `color-index` on the `vector-sample.color.rgb` keyspace. +It creates two child field mappings, `colorvect_l2` and `embedding_vector_dot` on two different vector fields in the keyspace's documents. + +It also adds 3 normal Search index fields (`brightness`, `color`, and `description`) to add more usable data to the Vector Search index: + +[source,console] +---- +include::example$create-vector-search-index-payload.sh[] +---- + +For more information about all the available JSON properties for a Search index, see xref:search:search-index-params.adoc[]. + +== Next Steps + +After you create a Search index, you can xref:run-vector-search-rest-api.adoc[] to test your Search index. + +If you want to edit your index with another REST API call, include the xref:search:search-index-params.adoc#uuid[uuid] parameter with the UUID the Search Service assigned to your Vector Search index. + +You can also xref:create-vector-search-index-ui.adoc[]. \ No newline at end of file diff --git a/modules/vector-search/pages/create-vector-search-index-ui.adoc b/modules/vector-search/pages/create-vector-search-index-ui.adoc new file mode 100644 index 000000000..a7262c9b1 --- /dev/null +++ b/modules/vector-search/pages/create-vector-search-index-ui.adoc @@ -0,0 +1,90 @@ += Create a Vector Search Index in Quick Mode +:page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Use Quick Mode to create a Vector Search index in Couchbase Capella. + +[abstract] +{description} + +Quick Mode works best when you need to create a basic Search index to start testing and prototyping with the Search Service. +You must use Advanced Mode to have greater control over how the Search Service returns such results, such as changing your xref:search:customize-index.adoc#analyzers[analyzer]. + +For more information about how to create a Search index in Advanced Mode, see xref:search:create-search-index-ui.adoc[]. + +TIP: Vector Search indexes can include all the same features and settings as a Search index. +For more information about Search indexes, see the xref:search:search.adoc[Search documentation]. + +You must create a Vector Search index before you can xref:run-vector-search-ui.adoc[run a search] that supports vector comparisons. + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have a bucket with scopes and collections in your database. +For more information, see xref:cloud:clusters:data-service/manage-buckets.adoc[]. + +* You have documents in a keyspace inside your bucket that contain vector embeddings. ++ +include::partial$download-sample-partial.adoc[] + +* You have logged in to the Couchbase {page-ui-name}. + +== Procedure + +To create a Vector Search index with Quick Mode in Capella: + +. On the *Databases* page, select the database where you want to create a Search index. +. Go to menu:Data Tools[Search]. +. Click btn:[Create Search Index]. ++ +By default, *Quick Mode* should be on. +. In the *Index Name* field, enter a name for the Vector Search index. ++ +[NOTE] +==== +Your index name must start with an alphabetic character (a-z or A-Z). It can only contain alphanumeric characters (a-z, A-Z, or 0-9), hyphens (-), or underscores (_). + +For Couchbase Server version 7.6 and later, your index name must be unique inside your selected bucket and scope. You cannot have 2 indexes with the same name inside the same bucket and scope. +==== +. Under *Type Mappings*, in the *Bucket* list, select the bucket that contains the documents you want to include in your index. +. In the *Scope* list, select the scope that contains these documents. +. Expand the collection that contains these documents. +. In your document schema, select the child field that contains your vector embeddings. ++ +Vectors must be represented as an array of floating point numbers. +They appear as `\{field-name} [ number ]` in the Capella Quick Mode editor. +The Search Service also supports arrays of arrays. +. Configure the options for your new child field as follows: +.. In the *Dimension* field, check that the value matches the total number of elements in your vector field's array. ++ +The Search Service supports arrays up to 2048 elements. +Capella automatically fills in the dimension value for your selected array field. +.. In the *Similarity metric* list, select the method to use to calculate the similarity between search term and Search index vectors. ++ +For more information, see xref:search:quick-index-field-options.adoc#quick-field[Quick Field Type Mapping Options]. +.. In the *Optimized for* list, select whether the Search Service should optimize Search queries for accuracy (*recall*) or speed (*latency*). ++ +For more information, see xref:search:quick-index-field-options.adoc#quick-field[Quick Field Type Mapping Options]. +.. Select *Index*. +. Click btn:[Submit]. +. (Optional) Add additional collections or child field type mappings to your index. ++ +For example, you could add the text field that you used to generate your vector embeddings. +. Click btn:[Create Index]. + +include::search:partial$large-index-caution.adoc[tag=vector-search] + +== Next Steps + +This basic Vector Search index includes the vector embeddings from the child field you specified in your type mapping. +If you chose to add additional child fields and enabled *Include in search results*, the Search Service can also return data from those fields when you run a Vector Search query. + +You can customize your Vector Search index like any other Search index to add additional data and improve search results. +For more information about how to customize an index, see xref:search:customize-index.adoc[]. + +CAUTION: Some Search index features are only available in Advanced Mode. +If you edit your Search index in Advanced Mode, you cannot make any additional edits in Quick Mode without losing all Advanced Mode settings. + +For more information about how to run a search against a Vector Search index, see xref:run-vector-search-ui.adoc[] or xref:run-vector-search-rest-api.adoc[]. \ No newline at end of file diff --git a/modules/vector-search/pages/run-vector-search-from-go-sdk.adoc b/modules/vector-search/pages/run-vector-search-from-go-sdk.adoc new file mode 100644 index 000000000..835f66ab8 --- /dev/null +++ b/modules/vector-search/pages/run-vector-search-from-go-sdk.adoc @@ -0,0 +1,72 @@ += Run a Vector Search with the Go SDK +:page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Using Couchbase's Go SDK, you can run a simple or more complex vector search against a Vector Search index. + +[abstract] +{description} + + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. + +* Make sure to install the Couchbase Go SDK ++ +For more information about on installing and using the Couchbase Go SDK, see xref:go-sdk:hello-world:start-using-sdk.adoc[Start Using the Go SDK]. + +== Example: Searching for a Similar Color Vector with a Go Program + +The sample dataset inside `rgb.json` has small embedding vectors inside the `colorvect_l2` field. +These embedding vectors describe a color using RGB values. +For example, the color red has an embedding vector of `[255, 0, 0]`. + +The following example code searches for the color navy (`[0.0, 0.0, 128.0]`) in the `rgb.json` dataset: + +[source,go] +---- +include::example$RunVectorSearchSimpleColor.go[] +---- + +== Example: Semantic Search with Color Descriptions + +[NOTE] +==== +The following code sample requires you to have a paid subscription to the OpenAI API to generate an embedding vector from a sample text string. +For more information about pricing for the OpenAI API, see https://openai.com/pricing[OpenAI's Pricing page^] for embedding models. + +The `rgb_questions.json` and sample data contains ready-made embedding vectors for different questions. +For an example of how to use a ready-made vector with Vector Search, see xref:run-vector-search-rest-api.adoc[] or xref:run-vector-search-ui.adoc[]. +==== + +Still using the sample dataset inside `rgb.json`, you can use the OpenAI API to generate an embedding from any text string. + +The following code generates an embedding vector with the question `What color hides everything like the night?`: + +[source,go] +---- +include::example$RunVectorSearchGenerateEmbed.go[] +---- + +You can try other questions from `rgb_questions.json`, such as: + +* `"What is the most romantic color?"` + +* `"What color makes people fall in love?"` + +* `"What is the most depressing color?"` + +== Next Steps + +Edit and update your Search index, `color-index`, and store the `description` field and rerun the second example. This will also return the `description` paragraph with the results. + +If you did not get the search results you were expecting, you can try to recreate your Vector Search index xref:create-vector-search-index-rest-api.adoc[with the REST API]. + +Vector Search indexes can use the same settings and features as regular Search indexes. +If you want to add additional fields and features to your index, see xref:search:customize-index.adoc[]. diff --git a/modules/vector-search/pages/run-vector-search-from-java-sdk.adoc b/modules/vector-search/pages/run-vector-search-from-java-sdk.adoc new file mode 100644 index 000000000..e547564db --- /dev/null +++ b/modules/vector-search/pages/run-vector-search-from-java-sdk.adoc @@ -0,0 +1,72 @@ += Run a Vector Search with the Java SDK +:page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Using Couchbase's Java SDK, you can run a simple or more complex vector search against a Vector Search index. + +[abstract] +{description} + + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. + +* Make sure to install the Couchbase Java SDK ++ +For more information about on installing and using the Couchbase Java SDK, see xref:java-sdk:hello-world:start-using-sdk.adoc[Start Using the Java SDK]. + +== Example: Searching for a Similar Color Vector with a Java Program + +The sample dataset inside `rgb.json` has small embedding vectors inside the `colorvect_l2` field. +These embedding vectors describe a color using RGB values. +For example, the color red has an embedding vector of `[255, 0, 0]`. + +The following example code searches for the color navy (`[0.0, 0.0, 128.0]`) in the `rgb.json` dataset: + +[source,java] +---- +include::example$RunVectorSearchSimpleColor.java[] +---- + +== Example: Semantic Search with Color Descriptions + +[NOTE] +==== +The following code sample requires you to have a paid subscription to the OpenAI API to generate an embedding vector from a sample text string. +For more information about pricing for the OpenAI API, see https://openai.com/pricing[OpenAI's Pricing page^] for embedding models. + +The `rgb_questions.json` and sample data contains ready-made embedding vectors for different questions. +For an example of how to use a ready-made vector with Vector Search, see xref:run-vector-search-rest-api.adoc[] or xref:run-vector-search-ui.adoc[]. +==== + +Still using the sample dataset inside `rgb.json`, you can use the OpenAI API to generate an embedding from any text string. + +The following code generates an embedding vector with the question `What color hides everything like the night?`: + +[source,java] +---- +include::example$RunVectorSearchGenerateEmbed.java[] +---- + +You can try other questions from `rgb_questions.json`, such as: + +* `"What is the most romantic color?"` + +* `"What color makes people fall in love?"` + +* `"What is the most depressing color?"` + +== Next Steps + +Edit and update your Search index, `color-index`, and store the `description` field and rerun the second example. This will also return the `description` paragraph with the results. + +If you did not get the search results you were expecting, you can try to recreate your Vector Search index xref:create-vector-search-index-rest-api.adoc[with the REST API]. + +Vector Search indexes can use the same settings and features as regular Search indexes. +If you want to add additional fields and features to your index, see xref:search:customize-index.adoc[]. diff --git a/modules/vector-search/pages/run-vector-search-python-sdk.adoc b/modules/vector-search/pages/run-vector-search-python-sdk.adoc new file mode 100644 index 000000000..f0d7dd163 --- /dev/null +++ b/modules/vector-search/pages/run-vector-search-python-sdk.adoc @@ -0,0 +1,87 @@ += Run a Vector Search with the Python SDK +:page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Using Couchbase's Python SDK, you can run a simple or more complex vector search against a Vector Search index. + +[abstract] +{description} + + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. + +* You have created a Vector Search index. ++ +For more information about how to create a Vector Search index, see xref:create-vector-search-index-ui.adoc[] or xref:create-vector-search-index-rest-api.adoc[]. ++ +include::partial$download-sample-partial.adoc[] + +* Create and activate a virtual environment using `venv` and install packages. ++ +For more information about how to set up your virtual environment, see https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/[the Python Packaging User Guide]. ++ +Make sure to install all dependencies for the Couchbase Python SDK and make a `requirements.txt` file with the following: ++ +---- +couchbase==4.1.12 +openai==1.12.0 +---- ++ +Run `pip install -r requirements.txt` to pull in all needed dependancies + +== Example: Searching for a Similar Color Vector with a Python Program + +The sample dataset inside `rgb.json` has small embedding vectors inside the `colorvect_l2` field. +These embedding vectors describe a color using RGB values. +For example, the color red has an embedding vector of `[255, 0, 0]`. + +The following example code searches for the color navy (`[0.0, 0.0, 128.0]`) in the `rgb.json` dataset: + +[source,python] +---- +include::example$run-vector-search-simple-color.py[] +---- + +== Example: Semantic Search with Color Descriptions + +[NOTE] +==== +The following code sample requires you to have a paid subscription to the OpenAI API to generate an embedding vector from a sample text string. +For more information about pricing for the OpenAI API, see https://openai.com/pricing[OpenAI's Pricing page^] for embedding models. + +The `rgb_questions.json` and sample data contains ready-made embedding vectors for different questions. +For an example of how to use a ready-made vector with Vector Search, see xref:run-vector-search-rest-api.adoc[] or xref:run-vector-search-ui.adoc[]. +==== + +Still using the sample dataset inside `rgb.json`, you can use the OpenAI API to generate an embedding from any text string. + +The following code generates an embedding vector with the question `What color hides everything like the night?`: + +[source,python] +---- +include::example$run-vector-search-generate-embed.py[] +---- + +You can try other questions from `rgb_questions.json`, such as: + +* `"What is the most romantic color?"` + +* `"What color makes people fall in love?"` + +* `"What is the most depressing color?"` + +== Next Steps + +Edit and update your Search index, `color-index`, and store the `description` field and rerun the second example. This will also return the `description` paragraph with the results. + +If you did not get the search results you were expecting, you can try to recreate your Vector Search index xref:create-vector-search-index-rest-api.adoc[with the REST API]. + +Vector Search indexes can use the same settings and features as regular Search indexes. +If you want to add additional fields and features to your index, see xref:search:customize-index.adoc[]. diff --git a/modules/vector-search/pages/run-vector-search-rest-api.adoc b/modules/vector-search/pages/run-vector-search-rest-api.adoc new file mode 100644 index 000000000..310430c16 --- /dev/null +++ b/modules/vector-search/pages/run-vector-search-rest-api.adoc @@ -0,0 +1,65 @@ += Run a Vector Search with the REST API and curl/HTTP +:page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: You can use the REST API and a curl command to run a search against a Vector Search index and return similar vectors from a Couchbase Capella database. + +[abstract] +{description} + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have the hostname for the node in your database that's running the Search Service. ++ +For more information about how to find the hostname for a node in your Capella database, see xref:clouds:get-capella-host-name.adoc[]. + +* You have created a Vector Search index. ++ +For more information about how to create a Vector Search index, see xref:create-vector-search-index-ui.adoc[] or xref:create-vector-search-index-rest-api.adoc[]. ++ +include::partial$download-sample-partial.adoc[] + +== Procedure + +To run a Vector search with the REST API: + +. In your command-line tool, enter a `curl` command with the `XPOST` verb. +. Set your header content to include `"Content-Type: application/json"`. +. Enter your username, password, and the Search Service endpoint on port `18094` with the name of the Vector Search index you want to query: ++ +[source,console] +---- +include::example$run-vector-search-header.sh[] +---- ++ +. Enter the JSON payload for your query. ++ +TIP: You can copy the JSON for a Query Request from the {page-ui-name} to use in your REST API call. +For more information about how to perform a search with the UI, see xref:search:simple-search-ui.adoc[]. + +=== Example + +In the following example, the JSON payload uses both a `query` and `knn` object to run both a Vector Search and traditional Search query on an index named `color-index`. + +The query searches for a specified Euclidean distance color vector, but uses the `query` object to search for a color with a `brightness` value in the range of `70-80`: + +[source,console] +---- +include::example$run-vector-search-payload.sh[] +---- + +The Search Service combines the Vector search results from the `knn` object with the traditional `query` object by using an `OR` function. +If the same documents match the `knn` and `query` objects, the Search Service ranks those documents higher in search results. + +TIP: For a more complex query, you can copy the `query` object from the example under xref:run-vector-search-ui.adoc#large[Example: Running a Semantic Search Query with a Large Embedding Vector] to use in your REST API call. + +For more information about the available properties for a Search query JSON payload, see xref:search:search-request-params.adoc[]. + +== Next Steps + +If you do not get the search results you were expecting, you can change the JSON definition xref:search:search-index-params.adoc[for your Search index] or change the parameters xref:search:search-request-params.adoc[for your Search query]. + +You can also xref:search:customize-index.adoc[]. \ No newline at end of file diff --git a/modules/vector-search/pages/run-vector-search-ui.adoc b/modules/vector-search/pages/run-vector-search-ui.adoc new file mode 100644 index 000000000..e87fde477 --- /dev/null +++ b/modules/vector-search/pages/run-vector-search-ui.adoc @@ -0,0 +1,93 @@ += Run a Vector Search with the {page-ui-name} +:page-topic-type: guide +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Run a Vector Search query from the Couchbase {page-ui-name} to preview and test the search results from a Vector Search index. + +[abstract] +{description} + +== Prerequisites + +* You have the Search Service enabled on a node in your database. +For more information about how to change Services on your database, see xref:cloud:clusters:modify-database.adoc[]. + +* You have created a Vector Search index. ++ +For more information about how to create a Vector Search index, see xref:create-vector-search-index-ui.adoc[]. ++ +include::partial$download-sample-partial.adoc[] + +* You have logged in to the Couchbase {page-ui-name}. + +== Procedure + +To run a Vector Search with the {page-ui-name}: + +. On the *Databases* page, select the database where you created your Search index. +. Go to menu:Data Tools[Search]. +. Next to your Vector Search index, click btn:[Search]. +. In the *Search* field, enter a search query. +. Press kbd:[Enter] or click btn:[Search]. +. . (Optional) To view a document and its source collection, click a document name in the search results list. + +[#similarity] +=== Example: Running a Simple Vector Similarity Query + +For example, the following query searches for the top 2 vectors similar to the vector `[ 0, 0, 128 ]` in the `colorvect_l2` field: + +[source,json] +---- +include::example$run-vector-search-payload-ui.jsonc[] +---- + +By using the special `match_none` query in the `query` field, the Search query is only a Vector Search query. +It only returns the `k` number of similar vectors. + +The Search Service combines the Vector search results from a `knn` object with the traditional `query` object by using an `OR` function. +If the same documents match the `knn` and `query` objects, the Search Service ranks those documents higher in search results. + +[#large] +=== Example: Running a Semantic Search Query with a Large Embedding Vector + +TIP: This example uses the `rgb_questions.json` sample data, which provides ready-made vector embeddings and queries that you can use for searches. +You can find a link to download this sample dataset from xref:create-vector-search-index-ui.adoc[] or xref:create-vector-search-index-rest-api.adoc[]. + +The following query searches for matches to a large embedding vector, generated by the https://platform.openai.com/docs/guides/embeddings[OpenAI embedding model^], `text-embedding-ada-002-v2`. + +The embedding vector for the query was generated from the text in the `question` field in `rgb_questions.json`. +You can also find the full contents of the query in the `couchbase_search_query` field in the same file. +You can find generated embedding vectors for each color's `question` field in `rgb_questions.json`, and generated embedding vectors for each `description` in `rgb.json`. + +This query should return the document for the color `navy`, based on a generated embedding vector for: + +---- +What is a classic, refined hue that exudes elegance and is often linked to power and stability? +---- + +The following shows part of the sample Search query: + +[source,json] +---- +include::example$run-vector-search-long-payload-ui.jsonc[tag=partial] +---- +[NOTE] +==== +Due to the size of the embedding vector, only part of the full query is being displayed in the documentation. + +Click btn:[View on GitHub] to view and copy the entire Vector Search query payload. +Make sure you remove the lines for `// tag::partial[]` and `// end::partial[]`. + +You can also copy the query by going to menu:Data Tools[Documents] and copying the full contents from the `couchbase_search_query` field. + +If you configured your data import to use the `id` field as the document identifier, you can locate the document for `navy` with the ID `#000080`. +==== + +== Next Steps + +If you do not get the search results you were expecting, you can change the xref:search:search-request-params.adoc[JSON payload for your Search query]. + +You can also xref:search:customize-index.adoc[add additional features to your Search index]. + +Either of the example queries on this page can also be used with the xref:run-vector-search-rest-api.adoc[REST API] to run a search. +You can find more example embedding vectors and full queries inside the `couchbase_search_query` field for any color in the `rgb_questions.json` sample file. \ No newline at end of file diff --git a/modules/vector-search/pages/vector-search.adoc b/modules/vector-search/pages/vector-search.adoc new file mode 100644 index 000000000..3fa9feb99 --- /dev/null +++ b/modules/vector-search/pages/vector-search.adoc @@ -0,0 +1,57 @@ += Use Vector Search for AI Applications +:page-topic-type: concept +:page-ui-name: {ui-name} +:page-product-name: {product-name} +:description: Use Couchbase {page-product-name}'s Vector Search features to add fast and accurate semantic search to your applications. + +[abstract] +{description} + +== About Vector Search + +Vector Search builds on Couchbase {page-product-name}'s xref:search:search.adoc[Search Service] to provide vector index support. +You can use these new Vector Search indexes for Retrieval Augmented Generation (RAG) with an existing Large Language Model (LLM). + +Using {page-product-name}'s Vector Search, an embedding model, and your chosen LLM, you can develop AI applications while giving context and up-to-date information from your own data. + +You can develop applications that include: + +* *Similarity search:* Search for documents, products, images, and more that are similar to a given query using vector embeddings. +By using vector embeddings, you can search based on descriptions, rather than using specific keywords and get intuitive and relevant results across data types. + +* *Semantic search*: Use natural language processing to deliver more accurate results based on an understanding of the intent and context behind a given search query, rather than simple keyword matches. + +* *Generative AI:* Create new original content, such as text and images, based on a prompt or a vector search given to an LLM. +Use generative AI to get tailored and dynamic responses across your applications. + +Vector Search supports integrations with frameworks like https://python.langchain.com/docs/get_started/introduction[LangChain^] to support AI application development. +For more information about all frameworks and integrations supported by Vector Search and {page-product-name}, see xref:third-party:integrations.adoc[]. + +== Using Vector Search Indexes + +To get started using Vector Search in {page-product-name}: + +. *Store data*: Store the data you want to use for your search or AI project in a {page-product-name} database. +. *Generate embeddings*: Generate vector embeddings from your data with your preferred embedding model. +. *Store your embeddings*: Store your vector embeddings in an array inside the documents in your {page-product-name} database. +. *Create a Vector Search index*: Create an index to use your embeddings and identify similar documents with vector similarity. + +In addition to supporting integrations with frameworks like LangChain and LlamaIndex, you can also use the API for an existing LLM and one of their embedding models to generate vector embeddings for your data. +For example, the OpenAI `embeddings` endpoint can generate embeddings for a text string using a specified embedding model. +You can then store that embedding as a new field in your documents. +For more information about how to generate and obtain embeddings for text strings using the OpenAI API, see the https://platform.openai.com/docs/guides/embeddings/what-are-embeddings[Embeddings documentation]. + +NOTE: When you create a Vector Search index, the xref:search:child-field-options-reference.adoc#dimension[dimension] of your data vector embeddings must match the dimension for any search query vectors. +Otherwise, a Vector Search query fails to return any results. + +For more information about how to create a Vector Search index, see xref:create-vector-search-index-ui.adoc[] or xref:create-vector-search-index-rest-api.adoc[]. + +For information about how to run a Vector Search query, see xref:run-vector-search-ui.adoc[] or xref:run-vector-search-rest-api.adoc[]. + +== See Also + +* xref:search:search.adoc[] +* xref:create-vector-search-index-ui.adoc[] +* xref:create-vector-search-index-rest-api.adoc[] +* xref:run-vector-search-ui.adoc[] +* xref:run-vector-search-rest-api.adoc[] \ No newline at end of file diff --git a/modules/vector-search/partials/download-sample-partial.adoc b/modules/vector-search/partials/download-sample-partial.adoc new file mode 100644 index 000000000..a33206ea2 --- /dev/null +++ b/modules/vector-search/partials/download-sample-partial.adoc @@ -0,0 +1,15 @@ +[TIP] +==== +You can download a sample dataset to use with the procedure on this page: + +https://cbc-remote-execution-examples-prod.s3.amazonaws.com/color_data_2vectors.zip[Download color_data_2vectors.zip] + +To get the best results with using the sample data with the examples in this documentation, xref:clusters:data-service/import-data-documents.adoc[import the sample files] from the dataset into your database with the following settings: + +* Use a bucket called `vector-sample`. +* Use a scope called `color`. +* Use a collection called `rgb` for `rgb.json`. +* Use a collection called `rgb_questions` for `rgb_questions.json`. +* To set your document keys, use the value of the `id` field from each JSON document. + +==== \ No newline at end of file diff --git a/modules/vector-search/partials/nav.adoc b/modules/vector-search/partials/nav.adoc new file mode 100644 index 000000000..c6a5110ab --- /dev/null +++ b/modules/vector-search/partials/nav.adoc @@ -0,0 +1,8 @@ +* xref:cloud:vector-search:vector-search.adoc[] +** xref:cloud:vector-search:create-vector-search-index-ui.adoc[] +** xref:cloud:vector-search:create-vector-search-index-rest-api.adoc[] +** xref:cloud:vector-search:run-vector-search-ui.adoc[] +** xref:cloud:vector-search:run-vector-search-rest-api.adoc[] +** xref:cloud:vector-search:run-vector-search-python-sdk.adoc[] +** xref:cloud:vector-search:run-vector-search-go-sdk.adoc[] +** xref:cloud:vector-search:run-vector-search-java-sdk.adoc[]