Add l1norm and l2norm distances for vectors (#44116)

* Add l1norm and l2norm distances for vectors

Add L1norm - Manhattan distance
Add L2norm - Euclidean distance
relates to #37947

* Address Christoph's feedback

- organize vector functions as a separate doc
- increase precision in tests calculations
- add a separate test when sparse doc dims
are bigger and less than query vector dims

* Made examples more realistic

Author: mayya-sharipova

docs/reference/query-dsl/script-score-query.asciidoc

@@ -11,7 +11,6 @@ a function to be used to compute a new score for each document returned
 by the query. For more information on scripting see
 <<modules-scripting, scripting documentation>>.
 
-
 Here is an example of using `script_score` to assign each matched document
 a score equal to the number of likes divided by 10:
 
@@ -32,7 +31,6 @@ GET /_search
 }
 --------------------------------------------------
 // CONSOLE
-// TEST[setup:twitter]
 
 NOTE: The values returned from `script_score` cannot be negative. In general,
 Lucene requires the scores produced by queries to be non-negative in order to
@@ -76,140 +74,6 @@ to be the most efficient by using the internal mechanisms.
 --------------------------------------------------
 // NOTCONSOLE
 
-[role="xpack"]
-[testenv="basic"]
-[[vector-functions]]
-===== Functions for vector fields
-
-experimental[]
-
-These functions are used for
-for <<dense-vector,`dense_vector`>>  and
-<<sparse-vector,`sparse_vector`>> fields.
-
-NOTE: During vector functions' calculation, all matched documents are
-linearly scanned. Thus, expect the query time grow linearly 
-with the number of matched documents. For this reason, we recommend
-to limit the number of matched documents with a `query` parameter.
-
-For dense_vector fields, `cosineSimilarity` calculates the measure of
-cosine similarity between a given query vector and document vectors.
-
-[source,js]
---------------------------------------------------
-{
-  "query": {
-    "script_score": {
-      "query": {
-        "match_all": {}
-      },
-      "script": {
-        "source": "cosineSimilarity(params.query_vector, doc['my_dense_vector']) + 1.0", <1>
-        "params": {
-          "query_vector": [4, 3.4, -0.2]  <2>
-        }
-      }
-    }
-  }
-}
---------------------------------------------------
-// NOTCONSOLE
-<1> The script adds 1.0 to the cosine similarity to prevent the score from being negative.
-<2> To take advantage of the script optimizations, provide a query vector as a script parameter.
-
-Similarly, for sparse_vector fields, `cosineSimilaritySparse` calculates cosine similarity
-between a given query vector and document vectors.
-
-[source,js]
---------------------------------------------------
-{
-  "query": {
-    "script_score": {
-      "query": {
-        "match_all": {}
-      },
-      "script": {
-        "source": "cosineSimilaritySparse(params.query_vector, doc['my_sparse_vector']) + 1.0",
-        "params": {
-          "query_vector": {"2": 0.5, "10" : 111.3, "50": -1.3, "113": 14.8, "4545": 156.0}
-        }
-      }
-    }
-  }
-}
---------------------------------------------------
-// NOTCONSOLE
-
-For dense_vector fields, `dotProduct` calculates the measure of
-dot product between a given query vector and document vectors.
-
-[source,js]
---------------------------------------------------
-{
-  "query": {
-    "script_score": {
-      "query": {
-        "match_all": {}
-      },
-      "script": {
-        "source": """
-          double value = dotProduct(params.query_vector, doc['my_vector']);
-          return sigmoid(1, Math.E, -value); <1>
-        """,
-        "params": {
-          "query_vector": [4, 3.4, -0.2]
-        }
-      }
-    }
-  }
-}
---------------------------------------------------
-// NOTCONSOLE
-
-<1> Using the standard sigmoid function prevents scores from being negative.
-
-Similarly, for sparse_vector fields, `dotProductSparse` calculates dot product
-between a given query vector and document vectors.
-
-[source,js]
---------------------------------------------------
-{
-  "query": {
-    "script_score": {
-      "query": {
-        "match_all": {}
-      },
-      "script": {
-        "source": """
-          double value = dotProductSparse(params.query_vector, doc['my_sparse_vector']);
-          return sigmoid(1, Math.E, -value);
-        """,
-         "params": {
-          "query_vector": {"2": 0.5, "10" : 111.3, "50": -1.3, "113": 14.8, "4545": 156.0}
-        }
-      }
-    }
-  }
-}
---------------------------------------------------
-// NOTCONSOLE
-
-NOTE: If a document doesn't have a value for a vector field on which
-a vector function is executed, an error will be thrown.
-
-You can check if a document has a value for the field `my_vector` by
-`doc['my_vector'].size() == 0`. Your overall script can look like this:
-
-[source,js]
---------------------------------------------------
-"source": "doc['my_vector'].size() == 0 ? 0 : cosineSimilarity(params.queryVector, doc['my_vector'])"
---------------------------------------------------
-// NOTCONSOLE
-
-NOTE: If a document's dense vector field has a number of dimensions
-different from the query's vector, an error will be thrown.
-
-
 [[random-score-function]]
 ===== Random score function
 `random_score` function generates scores that are uniformly distributed
@@ -323,6 +187,9 @@ You can read more about decay functions
 NOTE: Decay functions on dates are limited to dates in the default format
 and default time zone. Also calculations with `now` are not supported.
 
+===== Functions for vector fields
+<<vector-functions, Functions for vector fields>> are accessible through
+`script_score` query.
 
 ==== Faster alternatives
 Script Score Query calculates the score for every hit (matching document).
@@ -422,5 +289,4 @@ through a script:
 Script Score query has equivalent <<decay-functions, decay functions>>
 that can be used in script.
 
-
-
+include::{es-repo-dir}/vectors/vector-functions.asciidoc[]