Add hybrid search mode and unify vector, hybrid, and full-text search…

… under the same API (#587)

biome.json

@@ -7,7 +7,6 @@
 	"files": {
 		"ignore": [
 			".vscode/",
-			"**./tap",
 			"**/.docusaurus/",
 			"**/coverage",
 			"**/dist/",

packages/docs/.vitepress/config.mts

@@ -29,6 +29,7 @@ export default defineConfig({
   },
 
   vite: {
+    // @ts-ignore
     plugins: [OramaPlugin()]
   },
 
@@ -74,6 +75,7 @@ export default defineConfig({
               items: [
                 { text: 'Searching with Orama', link: '/open-source/usage/search/introduction.html' },
                 { text: 'Vector Search', link: '/open-source/usage/search/vector-search.html' },
+                { text: 'Hybrid Search', link: '/open-source/usage/search/hybrid-search.html' },
                 { text: 'Fields Boosting', link: '/open-source/usage/search/fields-boosting.html' },
                 { text: 'Facets', link: '/open-source/usage/search/facets.html' },
                 { text: 'Filters', link: '/open-source/usage/search/filters.html' },
@@ -104,6 +106,7 @@ export default defineConfig({
             { text: 'Writing your own plugins', link: '/open-source/plugins/writing-your-own-plugins.html' },
             { text: 'Plugin Vitepress', link: '/open-source/plugins/plugin-vitepress.html' },
             { text: 'Plugin Docusaurus', link: '/open-source/plugins/plugin-docusaurus.html' },
+            { text: 'Plugin Secure Proxy', link: '/open-source/plugins/plugin-secure-proxy.html' },
             { text: 'Plugin Telemetry', link: '/open-source/plugins/plugin-telemetry.html' },
             { text: 'Plugin Astro', link: '/open-source/plugins/plugin-astro.html' },
             { text: 'Plugin Data Persistence', link: '/open-source/plugins/plugin-data-persistence.html' },
@@ -168,15 +171,17 @@ export default defineConfig({
         {
           text: 'Orama AI',
           items: [
-            { text: 'Automatic embeddings generation', link: '/cloud/orama-ai/automatic-embeddings-generation.html' }
+            { text: 'Automatic embeddings generation', link: '/cloud/orama-ai/automatic-embeddings-generation.html' },
+            { text: 'Orama Secure Proxy', link: '/cloud/orama-ai/orama-secure-proxy.html' }
           ]
         },
         {
           text: 'Performing search',
           collapsed: false,
           items: [
             { text: 'Full-text search', link: '/cloud/performing-search/full-text-search.html' },
-            { text: 'Vector search', link: '/cloud/performing-search/vector-search.html' }
+            { text: 'Vector search', link: '/cloud/performing-search/vector-search.html' },
+            { text: 'Hybrid search', link: '/cloud/performing-search/hybrid-search.html' },
           ]
         },
         {

packages/docs/cloud/orama-ai/orama-secure-proxy.md

@@ -0,0 +1,88 @@
+---
+outline: deep
+---
+
+# Plugin Secure Proxy
+
+When running a [vector](/open-source/usage/search/vector-search.html) or [hybrid](/open-source/usage/search/hybrid-search.html) search with Orama (or any other vector database), there is a process called **inference**.
+
+This term refers to the process of using a trained machine learning model to make predictions on new, unseen data. In the context of a vector or hybrid search, this could involve using the model to transform raw data into a vector representation that can then be used in the search process.
+
+A very popular model is OpenAI's [`text-embedding-ada-002`](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings). This model takes a string of maximum `8191` tokens, and returns a vector of `1536` elements, to be used to perform vector search.
+
+When running vector or hybrid search on the front-end, you'll encounter a problem: the need to call the OpenAI API, which exposes your API Key to the end user. If a malicious user gets a hold of this information, they could generate their own embeddings, or even use other OpenAI API offerings - all at your expense. It's important to remember that these APIs are not inexpensive!
+
+Because Orama is a vector database that operates client-side, we offer a secure proxy. This allows you to safely and easily call OpenAI, and soon other services, directly from a browser or any other unsecured source.
+
+## Enabling the secure proxy
+
+The **Orama Secure Proxy** is available with the free plan of Orama Cloud. Before you start, make sure to [create a free account](https://cloud.oramasearch.com) (no credit card required).
+
+Navigating to [https://cloud.oramasearch.com/secure-proxy](https://cloud.oramasearch.com/secure-proxy), you will notice that you'll need to import an OpenAI API Key before you start. This is required for other services too (such as the [automatic embeddings generation](/cloud/orama-ai/automatic-embeddings-generation.html)). Orama will encrypt this information in a secure vault and will never be able to retrieve it again in plaintext.
+
+<ZoomImg
+  src='/plugins/secure-proxy/initial-screen.png'
+  alt='Initial screen of Orama Secure Proxy'
+/>
+
+By going to [https://cloud.oramasearch.com/developer-tools](https://cloud.oramasearch.com/developer-tools), you'll be able to insert a new OpenAI API key. Remember, you will never be able to retrieve it again in plaintext.
+
+<ZoomImg
+  src='/plugins/secure-proxy/openai-key-popup.png'
+  alt='OpenAI API Key popup'
+/>
+
+If you go back to [https://cloud.oramasearch.com/secure-proxy](https://cloud.oramasearch.com/secure-proxy), you will see that you can now enable the Orama Secure Proxy.
+
+<ZoomImg
+  src='/plugins/secure-proxy/activate.png'
+  alt='Enable Orama Secure Proxy'
+/>
+
+After activating the Proxy, you will be presented with the following screen. The **Secure Proxy Public Key** is a **public** API key that you can include in any of your public-facing API requests.
+
+For security reasons, you can regenerate it at any moment. If needed, you can always shut down the secure proxy by just switching the "disable secure proxy" switch.
+
+<ZoomImg
+  src='/plugins/secure-proxy/proxy-key.png'
+  alt='Orama Secure Proxy Key'
+/>
+
+## Configuring the secure proxy
+
+Once the Orama Secure Proxy is enabled, you can finally configure it. Configurations are real-time, and any change is propagated immediately.
+
+<ZoomImg
+  src='/plugins/secure-proxy/configuration.png'
+  alt='Orama Secure Proxy Configuration'
+/>
+
+### Authorized domains
+
+Configuring the secure proxy is pretty straightforward. First of all, you have to explicit all the domains that are authorized to perform an API call to the proxy.
+
+For example, if you only want to authorize API calls coming from `https://oramasearch.com`, write `oramasearch.com` and press enter.
+
+Regular expressions are supported, so that you can test the proxy on dynamic environments such as Vercel or Netlify, where URLs are automatically generated based on branch name. Use them wisely!
+
+### User-level rate limiter
+
+After each connection to the secure proxy, Orama will identify the user and will perform rate limiting based on the data you're expressing in the "User rate limit" field.
+
+With the default configuration, users will be able to perform at most 10 requests every 60 seconds. After performing 10 requests, the system will block any further request.
+
+### System-level rate limiter
+
+The Orama Secure Proxy will perform a second type of rate limiting operation, based on the number of total requests from any number of users in a given time window.
+
+With this configuration, you can tell the secure proxy to stop proxying to OpenAI after a given number of requests has occurred within a specific timespan.
+
+With the default configuration, the users can perform at most 100 requests every 60 seconds.
+
+Make sure to configure these limits accordingly to your traffic patterns!
+
+## Performing hybrid and vector search
+
+Once the Orama Secure Proxy is configured, you can finally perform vector and hybrid search from the frontend securely.
+
+If you're using Orama Open Source, install the `@orama/plugin-secure-proxy` and follow the instructions provided in the [official documentation page](/open-source/plugins/plugin-secure-proxy.html).

packages/docs/cloud/performing-search/hybrid-search.md

@@ -0,0 +1,50 @@
+---
+outline: deep
+---
+
+# Performing hybrid search on Orama Cloud
+
+After deploying your index, Orama will distribute it to over 300 global points of presence across more than 100 countries worldwide. This will guarantee the lowest possible latency for any search query, at any scale.
+
+At the time of this writing, you can execute search queries using our official JavaScript SDK.
+
+This SDK manages connection, cache, telemetry, and type-safety for all your search operations. It is the official method for communicating with Orama Cloud.
+
+::: tip Installing the Orama SDK
+You can find the guide on installing the SDK [here](/cloud/integrating-orama-cloud/javascript-sdk).
+:::
+
+Make sure you have the Orama SDK installed to start performing vector search at the edge!
+
+## What is hybrid search?
+
+Hybrid search is a technique that merges full-text and vector search results into a unified list. It capitalizes on the strengths of both methods by balancing keyword-specific searches with the broader context of the overall query.
+
+## Performing hybrid search
+
+::: info
+The following guide assumes that you have an **Open AI API key** set on Orama Cloud, as it is needed to perform transform text into embeddings at search time.
+:::
+
+To perform a vector search with Orama Cloud, you need to populate an Orama index with vectors. Currently, this can be achieved through [automatic embeddings generation](/cloud/orama-ai/automatic-embeddings-generation) during the deployment process.
+
+As an alternative, you can provide your own vectors while inserting new documents into an Orama index.
+
+Once you have at least one index containing vectors, you can perform hybrid search by using the `search` function:
+
+```ts
+import { OramaClient } from '@oramacloud/client'
+
+const client = new OramaClient({
+  endpoint: '',
+  api_key: ''
+})
+
+const vectorSearchResults = await client.search({
+  term: 'Super Mario videogame',
+  mode: 'hybrid',
+  limit: 5        // How many results to return. Default is 10.
+})
+```
+
+Orama will automatically convert your search term, for instance, `"Super Mario videogame"`, into an embedding using your OpenAI API Key. It will then search through your vectors and ultimately return the full documents in their original format.

packages/docs/cloud/performing-search/vector-search.md

@@ -24,7 +24,9 @@ The following guide assumes that you have an **Open AI API key** set on Orama Cl
 
 To perform a vector search with Orama Cloud, you need to populate an Orama index with vectors. Currently, this can be achieved through [automatic embeddings generation](/cloud/orama-ai/automatic-embeddings-generation) during the deployment process.
 
-Once you have at least one index containing vectors, you can perform vector search by using the `searchVector` function:
+As an alternative, you can provide your own vectors while inserting new documents into an Orama index.
+
+Once you have at least one index containing vectors, you can perform vector search by using the `search` function:
 
 ```ts
 import { OramaClient } from '@oramacloud/client'
@@ -34,10 +36,11 @@ const client = new OramaClient({
   api_key: ''
 })
 
-const vectorSearchResults = await client.searchVector({
+const vectorSearchResults = await client.search({
   term: 'Super Mario videogame',
-  threshold: 0.8, // Minimum similarity, between 0 and 1. Default is 0.8 (80% similar).
-  limit: 5        // How many results to return. Default is 10.
+  mode: 'vector',
+  similarity: 0.8, // Minimum similarity, between 0 and 1. Default is 0.8 (80% similar).
+  limit: 5         // How many results to return. Default is 10.
 })
 ```
 

packages/docs/open-source/plugins/plugin-secure-proxy.md

@@ -0,0 +1,84 @@
+---
+outline: deep
+---
+
+# Plugin Secure Proxy
+
+The **Orama Secure Proxy** plugin is an official Orama plugin that allows you to perform vector and hybrid search securely on your browser by masking OpenAI (and other services soon) API keys when generating embeddings.
+
+::: info You will need a free Orama Cloud account for this
+To use this plugin, you will need a free [Orama Cloud](https://cloud.oramasearch.com) account. If you already have one, follow the [guide](/cloud/orama-ai/orama-secure-proxy.html) to enable the Orama Secure Proxy before continuing.
+:::
+
+## Installation
+
+First of all, install it via npm (or any other package manager of your choice):
+
+```sh
+npm i @orama/plugin-secure-proxy
+```
+
+## Usage
+
+Now, when creating a new Orama Instance, make sure to install the plugin:
+
+```js
+import { create } from '@orama/orama'
+import { pluginSecureProxy } from '@orama/plugin-secure-proxy'
+
+const secureProxy = secureProxyPlugin({
+  apiKey: 'YOUR API KEY',
+  defaultProperty: 'embeddings'
+})
+
+const db = await create({
+  schema: {
+    title: 'string',
+    description: 'string',
+    embeddings: 'vector[1536]'
+  },
+  plugins: [secureProxy]
+})
+```
+
+## Running queries
+
+By telling on which property to perform search by default (in the example above, `'embeddings'`), the plugin will automatically translate your search term into a vector by calling the OpenAI API for you and setting the result into the `vector.value` property.
+
+This will finally allow you to perform hybrid and vector search with the exact same APIs used for full-text search. 
+
+```js
+import { search } from '@orama/orama'
+
+const resultsHybrid = await search(db, {
+  mode: 'hybrid',
+  term: 'Videogame for little kids with a passion about ice cream' // [!code ++]
+  vector: { // [!code --]
+    value: [0.9272643, 0.182738, 0.192031, 0.998761], // [!code --]
+    property: 'embeddings' // [!code --]
+  }
+})
+
+const resultsVector = await search(db, {
+  mode: 'vector',
+  term: 'Videogame for little kids with a passion about ice cream' // [!code ++]
+  vector: { // [!code --]
+    value: [0.9272643, 0.182738, 0.192031, 0.998761], // [!code --]
+    property: 'embeddings' // [!code --]
+  }
+})
+```
+
+## Specifying the vector property
+
+If you have a more complex schema with multiple vector properties, you can always override the vector property to perform search on by using the default `vector` property:
+
+```js
+const resultsVector = await search(db, {
+  mode: 'vector',
+  term: 'Videogame for little kids with a passion about ice cream'
+  vector: {
+    property: 'myAlternativeProperty'
+  }
+})
+```