diff --git a/CHANGELOG.md b/CHANGELOG.md index ec4f97599..06259334b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ Unreleased changes are available as `avenga/couper:edge` container. * `beta_scope_map` attribute for the [JWT Block](./docs/REFERENCE.md#jwt-block) ([#434](https://github.com/avenga/couper/pull/434)) * `saml` [error type](./docs/ERRORS.md#error-types) ([#424](https://github.com/avenga/couper/pull/424)) * `allowed_methods` attribute for the [API](./docs/REFERENCE.md#api-block) or [Endpoint Block](./docs/REFERENCE.md#endpoint-block) ([#444](https://github.com/avenga/couper/pull/444)) + * new HCL functions: `contains()`, `join()`, `keys()`, `length()`, `lookup()`, `set_intersection()`, `to_number()` ([#455](https://github.com/avenga/couper/pull/455)) * **Changed** * Automatically add the `private` directive to the response `Cache-Control` HTTP header field value for all resources protected by [JWT](./docs/REFERENCE.md#jwt-block) ([#418](https://github.com/avenga/couper/pull/418)) diff --git a/docs/REFERENCE.md b/docs/REFERENCE.md index a0bbec805..218e74e6d 100644 --- a/docs/REFERENCE.md +++ b/docs/REFERENCE.md @@ -712,18 +712,25 @@ To access the HTTP status code of the `default` response use `backend_responses. | :----------------------------- | :-------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------------- | :--------------------------------------------------- | | `base64_decode` | string | Decodes Base64 data, as specified in RFC 4648. | `encoded` (string) | `base64_decode("Zm9v")` | | `base64_encode` | string | Encodes Base64 data, as specified in RFC 4648. | `decoded` (string) | `base64_encode("foo")` | +| `contains` | bool | Determines whether a given list contains a given single value as one of its elements. | `list` (tuple or list), `value` (various) | `contains([1,2,3], 2)` | | `default` | string | Returns the first of the given arguments that is not null or an empty string. If no argument matches, the last argument is returned. | `arg...` (various) | `default(request.cookies.foo, "bar")` | +| `join` | string | Concatenates together the string elements of one or more lists with a given separator. | `sep` (string), `lists...` (tuples or lists) | `join("-", [0,1,2,3])` | | `json_decode` | various | Parses the given JSON string and, if it is valid, returns the value it represents. | `encoded` (string) | `json_decode("{\"foo\": 1}")` | | `json_encode` | string | Returns a JSON serialization of the given value. | `val` (various) | `json_encode(request.context.myJWT)` | | `jwt_sign` | string | jwt_sign creates and signs a JSON Web Token (JWT) from information from a referenced [JWT Signing Profile Block](#jwt-signing-profile-block) (or [JWT Block](#jwt-block) with `signing_ttl`) and additional claims provided as a function parameter. | `label` (string), `claims` (object) | `jwt_sign("myJWT")` | +| `keys` | list | Takes a map and returns a sorted list of the map keys. | `inputMap` (object or map) | `keys(request.headers)` | +| `length` | integer | Returns the number of elements in the given collection. | `collection` (tuple, list or map; **no object**) | `length([0,1,2,3])` | +| `lookup` | various | Performs a dynamic lookup into a map. The default (third argument) is returned if the key (second argument) is not found in the inputMap (first argument). | `inputMap` (object or map), `key` (string), `default` (various) | `lookup({a = 1}, "b", "def")` | | `merge` | object or tuple | Deep-merges two or more of either objects or tuples. `null` arguments are ignored. A `null` attribute value in an object removes the previous attribute value. An attribute value with a different type than the current value is set as the new value. `merge()` with no parameters returns `null`. | `arg...` (object or tuple) | `merge(request.headers, { x-additional = "myval" })` | | `oauth2_authorization_url` | string | Creates an OAuth2 authorization URL from a referenced [OAuth2 AC Block](#oauth2-ac-block-beta) or [OIDC Block](#oidc-block). | `label` (string) | `oauth2_authorization_url("myOAuth2")` | | `oauth2_verifier` | string | Creates a cryptographically random key as specified in RFC 7636, applicable for all verifier methods; e.g. to be set as a cookie and read into `verifier_value`. Multiple calls of this function in the same client request context return the same value. | | `oauth2_verifier()` | | `relative_url` | string | Returns a relative URL by retaining `path`, `query` and `fragment` components. The input URL `s` must begin with `/`, `//`, `http://` or `https://`, otherwise an error is thrown. | s (string) | `relative_url("https://httpbin.org/anything?query#fragment") // returns "/anything?query#fragment"` | | `saml_sso_url` | string | Creates a SAML SingleSignOn URL (including the `SAMLRequest` parameter) from a referenced [SAML Block](#saml-block). | `label` (string) | `saml_sso_url("mySAML")` | +| `set_intersection` | list or tuple | Returns a new set containing the elements that exist in all of the given sets. | `sets...` (tuple or list) | `set_intersection(["A", "B", "C"], ["B", D"])` | | `split` | tuple | Divides a given string by a given separator, returning a list of strings containing the characters between the separator sequences. | `sep` (string), `str` (string) | `split(" ", "foo bar qux")` | | `substr` | string | Extracts a sequence of characters from another string and creates a new string. The "`offset`" index may be negative, in which case it is relative to the end of the given string. The "`length`" may be `-1`, in which case the remainder of the string after the given offset will be returned. | `str` (string), `offset` (integer), `length` (integer) | `substr("abcdef", 3, -1)` | | `to_lower` | string | Converts a given string to lowercase. | `s` (string) | `to_lower(request.cookies.name)` | +| `to_number` | number | Converts its argument to a number value. Only numbers, `null`, and strings containing decimal representations of numbers can be converted to number. All other values will produce an error. | `num` (string or number) | `to_number("1,23")`, `to_number(env.PI)` | | `to_upper` | string | Converts a given string to uppercase. | `s` (string) | `to_upper("CamelCase")` | | `unixtime` | integer | Retrieves the current UNIX timestamp in seconds. | | `unixtime()` | | `url_encode` | string | URL-encodes a given string according to RFC 3986. | `s` (string) | `url_encode("abc%&,123")` | diff --git a/eval/context.go b/eval/context.go index 09bbc1d93..8115f42e8 100644 --- a/eval/context.go +++ b/eval/context.go @@ -555,20 +555,27 @@ func newCtyCouperVariablesMap() cty.Value { // Functions func newFunctionsMap() map[string]function.Function { return map[string]function.Function{ - "base64_decode": lib.Base64DecodeFunc, - "base64_encode": lib.Base64EncodeFunc, - "coalesce": lib.DefaultFunc, - "default": lib.DefaultFunc, - "json_decode": stdlib.JSONDecodeFunc, - "json_encode": stdlib.JSONEncodeFunc, - "merge": lib.MergeFunc, - "relative_url": lib.RelativeUrlFunc, - "split": stdlib.SplitFunc, - "substr": stdlib.SubstrFunc, - "to_lower": stdlib.LowerFunc, - "to_upper": stdlib.UpperFunc, - "unixtime": lib.UnixtimeFunc, - "url_encode": lib.UrlEncodeFunc, + "base64_decode": lib.Base64DecodeFunc, + "base64_encode": lib.Base64EncodeFunc, + "coalesce": lib.DefaultFunc, + "contains": stdlib.ContainsFunc, + "default": lib.DefaultFunc, + "join": stdlib.JoinFunc, + "json_decode": stdlib.JSONDecodeFunc, + "json_encode": stdlib.JSONEncodeFunc, + "keys": stdlib.KeysFunc, + "length": stdlib.LengthFunc, + "lookup": stdlib.LookupFunc, + "merge": lib.MergeFunc, + "relative_url": lib.RelativeUrlFunc, + "set_intersection": stdlib.SetIntersectionFunc, + "split": stdlib.SplitFunc, + "substr": stdlib.SubstrFunc, + "to_lower": stdlib.LowerFunc, + "to_number": stdlib.MakeToFunc(cty.Number), + "to_upper": stdlib.UpperFunc, + "unixtime": lib.UnixtimeFunc, + "url_encode": lib.UrlEncodeFunc, } } diff --git a/server/http_integration_test.go b/server/http_integration_test.go index 69cf6cbd3..95c1fef6b 100644 --- a/server/http_integration_test.go +++ b/server/http_integration_test.go @@ -3847,6 +3847,58 @@ func TestFunctions(t *testing.T) { "X-Default-11": "0", "X-Default-12": "", }, http.StatusOK}, + {"contains", "/v1/contains", map[string]string{ + "X-Contains-1": "yes", + "X-Contains-2": "no", + "X-Contains-3": "yes", + "X-Contains-4": "no", + "X-Contains-5": "yes", + "X-Contains-6": "no", + "X-Contains-7": "yes", + "X-Contains-8": "no", + "X-Contains-9": "yes", + "X-Contains-10": "no", + "X-Contains-11": "yes", + }, http.StatusOK}, + {"length", "/v1/length", map[string]string{ + "X-Length-1": "2", + "X-Length-2": "0", + "X-Length-3": "5", + "X-Length-4": "2", + }, http.StatusOK}, + {"join", "/v1/join", map[string]string{ + "X-Join-1": "0-1-a-b-3-c-1.234-true-false", + "X-Join-2": "||", + "X-Join-3": "0-1-2-3-4", + }, http.StatusOK}, + {"keys", "/v1/keys", map[string]string{ + "X-Keys-1": `["a","b","c"]`, + "X-Keys-2": `[]`, + "X-Keys-3": `["couper-request-id","user-agent"]`, + }, http.StatusOK}, + {"set_intersection", "/v1/set_intersection", map[string]string{ + "X-Set_Intersection-1": `[1,3]`, + "X-Set_Intersection-2": `[1,3]`, + "X-Set_Intersection-3": `[1,3]`, + "X-Set_Intersection-4": `[1,3]`, + "X-Set_Intersection-5": `[3]`, + "X-Set_Intersection-6": `[3]`, + "X-Set_Intersection-7": `[]`, + "X-Set_Intersection-8": `[]`, + "X-Set_Intersection-9": `[]`, + "X-Set_Intersection-10": `[]`, + "X-Set_Intersection-11": `[2.2]`, + "X-Set_Intersection-12": `["b","d"]`, + "X-Set_Intersection-13": `[true]`, + "X-Set_Intersection-14": `[{"a":1}]`, + "X-Set_Intersection-15": `[[1,2]]`, + }, http.StatusOK}, + {"lookup", "/v1/lookup", map[string]string{ + "X-Lookup-1": "1", + "X-Lookup-2": "default", + "X-Lookup-3": "Go-http-client/1.1", + "X-Lookup-4": "default", + }, http.StatusOK}, } { t.Run(tc.path[1:], func(subT *testing.T) { helper := test.New(subT) @@ -3870,6 +3922,144 @@ func TestFunctions(t *testing.T) { } } +func TestFunction_to_number(t *testing.T) { + client := newClient() + + shutdown, _ := newCouper("testdata/integration/functions/01_couper.hcl", test.New(t)) + defer shutdown() + + helper := test.New(t) + + req, err := http.NewRequest(http.MethodGet, "http://example.com:8080/v1/to_number", nil) + helper.Must(err) + + res, err := client.Do(req) + helper.Must(err) + + if res.StatusCode != http.StatusOK { + t.Fatalf("expected Status %d, got: %d", http.StatusOK, res.StatusCode) + } + + resBytes, err := io.ReadAll(res.Body) + helper.Must(err) + helper.Must(res.Body.Close()) + + exp := `{"float-2_34":2.34,"float-_3":0.3,"from-env":3.14159,"int":34,"int-3_":3,"int-3_0":3,"null":null}` + if string(resBytes) != exp { + t.Fatalf("Unexpected result\nwant: %s\n got: %s", exp, string(resBytes)) + } +} + +func TestFunction_to_number_errors(t *testing.T) { + client := newClient() + + shutdown, logHook := newCouper("testdata/integration/functions/01_couper.hcl", test.New(t)) + defer shutdown() + + type testCase struct { + name string + path string + expMsg string + } + + for _, tc := range []testCase{ + {"string", "/v1/to_number/string", `expression evaluation error: 01_couper.hcl:62,23-28: Invalid function argument; Invalid value for "v" parameter: cannot convert "two" to number; given string must be a decimal representation of a number.`}, + {"bool", "/v1/to_number/bool", `expression evaluation error: 01_couper.hcl:70,23-27: Invalid function argument; Invalid value for "v" parameter: cannot convert bool to number.`}, + {"tuple", "/v1/to_number/tuple", `expression evaluation error: 01_couper.hcl:78,23-24: Invalid function argument; Invalid value for "v" parameter: cannot convert tuple to number.`}, + {"object", "/v1/to_number/object", `expression evaluation error: 01_couper.hcl:86,23-24: Invalid function argument; Invalid value for "v" parameter: cannot convert object to number.`}, + } { + t.Run(tc.path[1:], func(subT *testing.T) { + helper := test.New(subT) + + req, err := http.NewRequest(http.MethodGet, "http://example.com:8080"+tc.path, nil) + helper.Must(err) + + res, err := client.Do(req) + helper.Must(err) + + if res.StatusCode != http.StatusInternalServerError { + subT.Fatalf("%q: expected Status %d, got: %d", tc.name, http.StatusInternalServerError, res.StatusCode) + } + msg := logHook.LastEntry().Message + if msg != tc.expMsg { + subT.Fatalf("%q: expected log message\nwant: %q\ngot: %q", tc.name, tc.expMsg, msg) + } + }) + } +} + +func TestFunction_length_errors(t *testing.T) { + client := newClient() + + shutdown, logHook := newCouper("testdata/integration/functions/01_couper.hcl", test.New(t)) + defer shutdown() + + type testCase struct { + name string + path string + expMsg string + } + + for _, tc := range []testCase{ + {"object", "/v1/length/object", `expression evaluation error: 01_couper.hcl:123,19-26: Error in function call; Call to function "length" failed: collection must be a list, a map or a tuple.`}, + {"string", "/v1/length/string", `expression evaluation error: 01_couper.hcl:131,19-26: Error in function call; Call to function "length" failed: collection must be a list, a map or a tuple.`}, + {"null", "/v1/length/null", `expression evaluation error: 01_couper.hcl:139,26-30: Invalid function argument; Invalid value for "collection" parameter: argument must not be null.`}, + } { + t.Run(tc.path[1:], func(subT *testing.T) { + helper := test.New(subT) + + req, err := http.NewRequest(http.MethodGet, "http://example.com:8080"+tc.path, nil) + helper.Must(err) + + res, err := client.Do(req) + helper.Must(err) + + if res.StatusCode != http.StatusInternalServerError { + subT.Fatalf("%q: expected Status %d, got: %d", tc.name, http.StatusInternalServerError, res.StatusCode) + } + msg := logHook.LastEntry().Message + if msg != tc.expMsg { + subT.Fatalf("%q: expected log message\nwant: %q\ngot: %q", tc.name, tc.expMsg, msg) + } + }) + } +} + +func TestFunction_lookup_errors(t *testing.T) { + client := newClient() + + shutdown, logHook := newCouper("testdata/integration/functions/01_couper.hcl", test.New(t)) + defer shutdown() + + type testCase struct { + name string + path string + expMsg string + } + + for _, tc := range []testCase{ + {"null inputMap", "/v1/lookup/inputMap-null", `expression evaluation error: 01_couper.hcl:200,26-30: Invalid function argument; Invalid value for "inputMap" parameter: argument must not be null.`}, + } { + t.Run(tc.path[1:], func(subT *testing.T) { + helper := test.New(subT) + + req, err := http.NewRequest(http.MethodGet, "http://example.com:8080"+tc.path, nil) + helper.Must(err) + + res, err := client.Do(req) + helper.Must(err) + + if res.StatusCode != http.StatusInternalServerError { + subT.Fatalf("%q: expected Status %d, got: %d", tc.name, http.StatusInternalServerError, res.StatusCode) + } + msg := logHook.LastEntry().Message + if msg != tc.expMsg { + subT.Fatalf("%q: expected log message\nwant: %q\ngot: %q", tc.name, tc.expMsg, msg) + } + }) + } +} + func TestEndpoint_Response(t *testing.T) { client := newClient() var redirSeen bool diff --git a/server/testdata/integration/functions/01_couper.hcl b/server/testdata/integration/functions/01_couper.hcl index 06b230b94..0d5b42f2e 100644 --- a/server/testdata/integration/functions/01_couper.hcl +++ b/server/testdata/integration/functions/01_couper.hcl @@ -41,5 +41,171 @@ server "api" { } } } + + endpoint "/to_number" { + response { + json_body = { + float-2_34 = to_number("2.34") + float-_3 = to_number(".3") + int = to_number("34") + int-3_ = to_number("3.") + int-3_0 = to_number("3.0") + null = to_number(null) + from-env = to_number(env.PI) + } + } + } + + endpoint "/to_number/string" { + response { + json_body = { + error = to_number("two") + } + } + } + + endpoint "/to_number/bool" { + response { + json_body = { + error = to_number(true) + } + } + } + + endpoint "/to_number/tuple" { + response { + json_body = { + error = to_number([1]) + } + } + } + + endpoint "/to_number/object" { + response { + json_body = { + error = to_number({a = 1}) + } + } + } + + endpoint "/contains" { + response { + headers = { + x-contains-1 = contains(["a", "b"], "a") ? "yes" : "no" + x-contains-2 = contains(["a", "b"], "c") ? "yes" : "no" + x-contains-3 = contains([0, 1], 0) ? "yes" : "no" + x-contains-4 = contains([0, 1], 2) ? "yes" : "no" + x-contains-5 = contains([0.1, 1.1], 0.1) ? "yes" : "no" + x-contains-6 = contains([0.1, 1.1], 0.10000000001) ? "yes" : "no" + x-contains-7 = contains([{a = 1, aa = {aaa = 1}}, {b = 2}], {a = 1, aa = {aaa = 1}}) ? "yes" : "no" + x-contains-8 = contains([{a = 1}, {b = 2}], {c = 3}) ? "yes" : "no" + x-contains-9 = contains([[1,2], [3,4]], [1,2]) ? "yes" : "no" + x-contains-10 = contains([[1,2], [3,4]], [5,6]) ? "yes" : "no" + x-contains-11 = contains(["3.14159", "42"], env.PI) ? "yes" : "no" + } + } + } + + endpoint "/length" { + response { + headers = { + x-length-1 = length([0, 1]) # tuple + x-length-2 = length([]) + x-length-3 = length(split(",", "0,1,2,3,4")) # list + x-length-4 = length(request.headers) # map + } + } + } + + endpoint "/length/object" { + response { + headers = { + error = length({a = 1}) + } + } + } + + endpoint "/length/string" { + response { + headers = { + error = length("abcde") + } + } + } + + endpoint "/length/null" { + response { + headers = { + error = length(null) + } + } + } + + endpoint "/join" { + response { + headers = { + x-join-1 = join("-",[0, 1],["a","b"],[3,"c"],[1.234],[true,false]) + x-join-2 = "|${join("-",[])}|" + x-join-3 = join("-", split(",", "0,1,2,3,4")) + } + } + } + + endpoint "/keys" { + response { + headers = { + x-keys-1 = json_encode(keys({a = 1, c = 2, b = {d = 3}})) + x-keys-2 = json_encode(keys({})) + x-keys-3 = json_encode(keys(request.headers)) + } + } + } + + endpoint "/set_intersection" { + response { + headers = { + x-set_intersection-1 = json_encode((set_intersection([1,3]))) + x-set_intersection-2 = json_encode(set_intersection([0,1,2,3], [1,3])) + x-set_intersection-3 = json_encode(set_intersection([1,3],[0,1,2,3])) + x-set_intersection-4 = json_encode(set_intersection([1,3],[1,3])) + x-set_intersection-5 = json_encode(set_intersection([0,1,2,3], [1,3], [3,5])) + x-set_intersection-6 = json_encode(set_intersection([0,1,2,3], [3,4,5])) + x-set_intersection-7 = json_encode(set_intersection([0,1,2,3],[4,5])) + x-set_intersection-8 = json_encode(set_intersection([0,1,2,3],[])) + x-set_intersection-9 = json_encode(set_intersection([],[1,3])) + x-set_intersection-10 = json_encode(set_intersection([0,1,2,3], [1,4], [3,5])) + x-set_intersection-11 = json_encode(set_intersection([1.1,2.2,3.3], [2.2,4.4])) + x-set_intersection-12 = json_encode(set_intersection(["a","b","c","d"], ["b","d","e"])) + x-set_intersection-13 = json_encode(set_intersection([true,false], [true])) + x-set_intersection-14 = json_encode(set_intersection([{a=1},{b=2}], [{a=1},{c=3}])) + x-set_intersection-15 = json_encode(set_intersection([[1,2],[3,4]], [[1,2],[5,6]])) + } + } + } + + endpoint "/lookup" { + response { + headers = { + x-lookup-1 = lookup({a = "1"}, "a", "default") + x-lookup-2 = lookup({a = "1"}, "b", "default") + x-lookup-3 = lookup(request.headers, "user-agent", "default") + x-lookup-4 = lookup(request.headers, "content-type", "default") + } + } + } + + endpoint "/lookup/inputMap-null" { + response { + headers = { + error = lookup(null, "a", "default") + } + } + } + } +} + +defaults { + environment_variables = { + PI = "3.14159" } }