Implement predicate type conversion.

README.md

@@ -40,14 +40,15 @@ The REST Layer framework is composed of several sub-packages:
   - [Hooks](#hooks)
   - [Sub Resources](#sub-resources)
   - [Dependency](#dependency)
-- [Filtering](#filtering)
-- [Sorting](#sorting)
-- [Field Selection](#field-selection)
-  - [Field Aliasing](#field-aliasing)
-  - [Field Parameters](#field-parameters)
-  - [Embedding](#embedding)
-- [Pagination](#pagination)
-- [Skipping](#skipping)
+- [Quering](#quering)
+  - [Filtering](#filtering)
+  - [Sorting](#sorting)
+  - [Field Selection](#field-selection)
+    - [Field Aliasing](#field-aliasing)
+    - [Field Parameters](#field-parameters)
+    - [Embedding](#embedding)
+  - [Pagination](#pagination)
+  - [Skipping](#skipping)
 - [Authentication & Authorization](#authentication-and-authorization)
 - [Conditional Requests](#conditional-requests)
 - [Data Integrity & Concurrency Control](#data-integrity-and-concurrency-control)
@@ -72,6 +73,10 @@ Below is an overview over recent breaking changes, starting from an arbitrary po
 
 - PR #151: `ValuesValidator FieldValidator` attribute in `schema.Dict` struct replaced by `Values Field`.
 - PR #179: `ValuesValidator FieldValidator` attribute in `schema.Array` struct replaced by `Values Field`.
+- PR #204: 
+  - Storage drivers need to accept pointer to `Expression` implementer in `query.Predicate`.
+  - `filter` parameters in sub-query will be validated for type match.
+  - `filter` parameters will be validated for type match only, instead of type & constrains.
 
 From the next release and onwards (0.2), this list will summarize breaking changes done to master since the last release.
 
@@ -818,7 +823,11 @@ post = schema.Schema{
 }
 ```
 
-## Filtering
+## Quering
+
+When supplying query parameters be sure to honor URL encoding scheme. If you need to include `+` sign, use `%2B`, etc.
+
+### Filtering
 
 To filter resources, you use the `filter` query-string parameter. The format of the parameter is inspired by the [MongoDB query format](http://docs.mongodb.org/manual/tutorial/query-documents/). The `filter` parameter can be used with `GET` and `DELETE` methods on resource URLs.
 
@@ -897,7 +906,7 @@ The same example with flags:
 However, keep in mind that Storers have to support regular expression and depending on the implementation of the storage handler the accepted syntax may vary.
 An error of `ErrNotImplemented` will be returned for those storage backends which do not support the `$regex` operator.
 
-### Filter operators
+#### Filter operators
 
 | Operator  | Usage                           | Description
 | --------- | ------------------------------- | ------------
@@ -914,7 +923,7 @@ An error of `ErrNotImplemented` will be returned for those storage backends whic
 
 *Some storage handlers may not support all operators. Refer to the storage handler's documentation for more info.*
 
-## Sorting
+### Sorting
 
 Sorting of resource items is defined through the `sort` query-string parameter. The `sort` value is a list of resource's fields separated by comas (`,`). To invert a field's sort, you can prefix its name with a minus (`-`) character. The `sort` parameter can be used with `GET` and `DELETE` methods on resource URLs.
 
@@ -924,7 +933,7 @@ Here we sort the result by ascending quantity and descending date:
 
     /posts?sort=quantity,-created
 
-## Field Selection
+### Field Selection
 
 REST APIs tend to grow over time. Resources get more and more fields to fulfill the needs for new features. But each time fields are added, all existing API clients automatically get the additional cost. This tend to lead to huge waste of bandwidth and added latency due to the transfer of unnecessary data. As a workaround, the `field` parameter can be used to minimize and customize the response body from requests with a `GET`, `PATCH` or `PUT` method on resource URLs.
 
@@ -953,7 +962,7 @@ $ http -b :8080/api/users/ar6eimekj5lfktka9mt0/posts fields=='meta{title,body}'
 ]
 ```
 
-### Field Aliasing
+#### Field Aliasing
 
 It's also possible to rename fields in the response using aliasing. To create an alias, prefix the field name by the wanted alias separated by a colon (`:`):
 
@@ -983,7 +992,7 @@ $ http -b :8080/api/users/ar6eimekj5lfktka9mt0/posts fields=='meta{title,b:body}
 ]
 ```
 
-### Field Parameters
+#### Field Parameters
 
 Field parameters are used to apply a transformation on the value of a field using custom logic.
 
@@ -1033,7 +1042,7 @@ schema.Schema{
 
 Only parameters listed in the `Params` field will be accepted. You `Handler` function is called with the current value of the field and parameters sent by the user if any. Your function can apply wanted transformations on the value and return it. If an error is returned, a `422` error will be triggered with your error message associated to the field.
 
-### Embedding
+#### Embedding
 
 With sub-fields notation you can also request referenced resources or connections (sub-resources). REST Layer will recognize them automatically and fetch the associated resources in order embed their data in the response. This can save a lot of unnecessary sequential round-trips:
 
@@ -1071,13 +1080,13 @@ Notice the `sort` and `limit` parameters passed to the `comments` field. Those a
 
 Such request can quickly generate a lot of queries on the storage handler. To ensure a fast response time, REST layer tries to coalesce those storage requests and to execute them concurrently whenever possible.
 
-## Pagination
+### Pagination
 
 Pagination is supported on collection URLs using the `page` and `limit` query-string parameters and can be used for resource list view URLs with request method `GET` and `DELETE`. If you don't define a default pagination limit using `PaginationDefaultLimit` resource configuration parameter, the resource won't be paginated for list `GET` requests until you provide the `limit` query-string parameter. The `PaginationDefaultLimit` does not apply to list `DELETE` requests, but the `limit` and `page` parameters may still be used to delete a subset of items.
 
 If your collections are large enough, failing to define a reasonable `PaginationDefaultLimit` parameter may quickly render your API unusable.
 
-## Skipping
+### Skipping
 
 Skipping of resource items is defined through the `skip` query-string parameter. The `skip` value is a positive integer defining the number of items to skip when querying for items, and can be applied for requests with method `GET` or `DELETE`.
 

examples/auth-jwt/main.go

@@ -103,7 +103,7 @@ func (a AuthResourceHook) OnFind(ctx context.Context, q *query.Query) error {
 		return resource.ErrForbidden
 	}
 	// Add a predicate to the query to restrict to result on objects owned by this user
-	q.Predicate = append(q.Predicate, query.Equal{Field: a.UserField, Value: user.ID})
+	q.Predicate = append(q.Predicate, &query.Equal{Field: a.UserField, Value: user.ID})
 	return nil
 }
 
@@ -187,7 +187,7 @@ func (a AuthResourceHook) OnClear(ctx context.Context, q *query.Query) error {
 		return resource.ErrForbidden
 	}
 	// Add a predicate to the query to restrict to impact of the clear on objects owned by this user
-	q.Predicate = append(q.Predicate, query.Equal{Field: a.UserField, Value: user.ID})
+	q.Predicate = append(q.Predicate, &query.Equal{Field: a.UserField, Value: user.ID})
 	return nil
 }
 

examples/auth/main.go

@@ -85,7 +85,7 @@ func (a AuthResourceHook) OnFind(ctx context.Context, q *query.Query, offset, li
 		return resource.ErrForbidden
 	}
 	// Add a predicate to the query to restrict to result on objects owned by this user
-	q.Predicate = append(q.Predicate, query.Equal{Field: a.UserField, Value: user.ID})
+	q.Predicate = append(q.Predicate, &query.Equal{Field: a.UserField, Value: user.ID})
 	return nil
 }
 
@@ -169,7 +169,7 @@ func (a AuthResourceHook) OnClear(ctx context.Context, q *query.Query) error {
 		return resource.ErrForbidden
 	}
 	// Add a predicate to the query to restrict to impact of the clear on objects owned by this user
-	q.Predicate = append(q.Predicate, query.Equal{Field: a.UserField, Value: user.ID})
+	q.Predicate = append(q.Predicate, &query.Equal{Field: a.UserField, Value: user.ID})
 	return nil
 }
 

graphql/query.go

@@ -113,7 +113,7 @@ func listParamResolver(r *resource.Resource, p graphql.ResolveParams, params url
 	if filter, ok := p.Args["filter"].(string); ok && filter != "" {
 		p, err := query.ParsePredicate(filter)
 		if err == nil {
-			err = p.Validate(r.Validator())
+			err = p.Prepare(r.Validator())
 		}
 		if err != nil {
 			return nil, fmt.Errorf("invalid `filter` parameter: %v", err)
@@ -124,7 +124,7 @@ func listParamResolver(r *resource.Resource, p graphql.ResolveParams, params url
 		if filter := params.Get("filter"); filter != "" {
 			p, err := query.ParsePredicate(filter)
 			if err == nil {
-				err = p.Validate(r.Validator())
+				err = p.Prepare(r.Validator())
 			}
 			if err != nil {
 				return nil, fmt.Errorf("invalid `filter` parameter: %v", err)

graphql/types.go

@@ -96,7 +96,7 @@ func getSubResourceResolver(r *resource.Resource) graphql.FieldResolveFn {
 			return nil, err
 		}
 		// Limit the connection to parent's owned.
-		q.Predicate = append(q.Predicate, query.Equal{Field: r.ParentField(), Value: parent["id"]})
+		q.Predicate = append(q.Predicate, &query.Equal{Field: r.ParentField(), Value: parent["id"]})
 		list, err := r.Find(p.Context, q)
 		if err != nil {
 			return nil, err

resource/resource.go

@@ -143,8 +143,9 @@ func (r *Resource) Bind(name, field string, s schema.Schema, h Storer, c Conf) *
 	r.validator.fallback.Fields[name] = schema.Field{
 		ReadOnly: true,
 		Validator: &schema.Connection{
-			Path:  "." + name,
-			Field: field,
+			Path:      "." + name,
+			Field:     field,
+			Validator: s,
 		},
 		Params: schema.Params{
 			"skip": schema.Param{

resource/resource_test.go

@@ -11,9 +11,10 @@ import (
 )
 
 func TestResourceBind(t *testing.T) {
+	barSchema := schema.Schema{Fields: schema.Fields{"foo": {}}}
 	i := NewIndex()
 	foo := i.Bind("foo", schema.Schema{}, nil, DefaultConf)
-	bar := foo.Bind("bar", "foo", schema.Schema{Fields: schema.Fields{"foo": {}}}, nil, DefaultConf)
+	bar := foo.Bind("bar", "foo", barSchema, nil, DefaultConf)
 	assert.Equal(t, "bar", bar.Name())
 	assert.Equal(t, "foo.bar", bar.Path())
 	assert.Equal(t, "foo", bar.ParentField())
@@ -26,8 +27,9 @@ func TestResourceBind(t *testing.T) {
 			"bar": {
 				ReadOnly: true,
 				Validator: &schema.Connection{
-					Path:  ".bar",
-					Field: "foo",
+					Path:      ".bar",
+					Field:     "foo",
+					Validator: barSchema,
 				},
 				Params: schema.Params{
 					"skip": schema.Param{

resource/storage.go

@@ -150,15 +150,15 @@ func (s storageWrapper) MultiGet(ctx context.Context, ids []interface{}) (items
 		q := &query.Query{}
 		if len(ids) == 1 {
 			q.Predicate = query.Predicate{
-				query.Equal{Field: "id", Value: ids[0]},
+				&query.Equal{Field: "id", Value: ids[0]},
 			}
 		} else {
 			v := make([]query.Value, len(ids))
 			for i, id := range ids {
 				v[i] = query.Value(id)
 			}
 			q.Predicate = query.Predicate{
-				query.In{Field: "id", Values: v},
+				&query.In{Field: "id", Values: v},
 			}
 		}
 		q.Window = &query.Window{Limit: len(ids)}
@@ -193,13 +193,13 @@ func (s storageWrapper) Find(ctx context.Context, q *query.Query) (list *ItemLis
 		// pattern that could be converted to multi get.
 		if len(q.Predicate) == 1 && (q.Window == nil || q.Window.Offset == 0) && len(q.Sort) == 0 {
 			switch op := q.Predicate[0].(type) {
-			case query.Equal:
+			case *query.Equal:
 				// When query pattern is a single document request by its id,
 				// use the multi get API.
 				if id, ok := op.Value.(string); ok && op.Field == "id" && (q.Window == nil || q.Window.Limit == 1) {
 					return wrapMgetList(mg.MultiGet(ctx, []interface{}{id}))
 				}
-			case query.In:
+			case *query.In:
 				// When query pattern is a list of documents request by their
 				// ids, use the multi get API.
 				if op.Field == "id" && (q.Window == nil || q.Window.Limit == len(op.Values)) {

rest/method_item_patch_test.go

@@ -55,7 +55,7 @@ func TestPatchItem(t *testing.T) {
 			var item *resource.Item
 
 			s := vars.Storers[name]
-			q := query.Query{Predicate: query.Predicate{query.Equal{Field: "id", Value: id}}, Window: &query.Window{Limit: 1}}
+			q := query.Query{Predicate: query.Predicate{&query.Equal{Field: "id", Value: id}}, Window: &query.Window{Limit: 1}}
 			if items, err := s.Find(context.Background(), &q); err != nil {
 				t.Errorf("s.Find failed: %s", err)
 				return

rest/method_item_put_test.go

@@ -55,7 +55,7 @@ func TestPutItem(t *testing.T) {
 			var item *resource.Item
 
 			s := vars.Storers[name]
-			q := query.Query{Predicate: query.Predicate{query.Equal{Field: "id", Value: id}}, Window: &query.Window{Limit: 1}}
+			q := query.Query{Predicate: query.Predicate{&query.Equal{Field: "id", Value: id}}, Window: &query.Window{Limit: 1}}
 			if items, err := s.Find(context.Background(), &q); err != nil {
 				t.Errorf("s.Find failed: %s", err)
 				return

rest/method_post_test.go

@@ -34,7 +34,7 @@ func TestHandlerPostList(t *testing.T) {
 			ResponseBody: `{"foo":"bar","id":"1"}`,
 			ExtraTest: func(t *testing.T, vars *requestTestVars) {
 				q := &query.Query{
-					Predicate: query.Predicate{query.Equal{Field: "id", Value: "1"}},
+					Predicate: query.Predicate{&query.Equal{Field: "id", Value: "1"}},
 					Window:    &query.Window{Limit: 1},
 				}
 				s, ok := vars.Storers["foo"]

rest/resource_path.go

@@ -111,7 +111,7 @@ func (p ResourcePath) ParentsExist(ctx context.Context) error {
 		}
 		// Create a query with the parent path fields + the current path id.
 		q := &query.Query{
-			Predicate: append(predicate[:], query.Equal{Field: "id", Value: p[i].Value}),
+			Predicate: append(predicate[:], &query.Equal{Field: "id", Value: p[i].Value}),
 		}
 		// Execute all intermediate checks concurrently
 		wait.Add(1)
@@ -128,7 +128,7 @@ func (p ResourcePath) ParentsExist(ctx context.Context) error {
 			}
 		}(i)
 		// Push the resource field=value for the next hops.
-		predicate = append(predicate, query.Equal{Field: p[i].Field, Value: p[i].Value})
+		predicate = append(predicate, &query.Equal{Field: p[i].Field, Value: p[i].Value})
 	}
 	// Fail on first error.
 	for i := 0; i < parents; i++ {

rest/routing.go

@@ -188,7 +188,7 @@ func (r *RouteMatch) Query() (*query.Query, *Error) {
 	// Append route fields to the query
 	for _, rp := range r.ResourcePath {
 		if rp.Value != nil {
-			qp.q.Predicate = append(qp.q.Predicate, query.Equal{Field: rp.Field, Value: rp.Value})
+			qp.q.Predicate = append(qp.q.Predicate, &query.Equal{Field: rp.Field, Value: rp.Value})
 		}
 	}
 
@@ -260,7 +260,7 @@ func (qp *queryParser) parsePredicate(params url.Values) {
 		for _, filter := range filters {
 			if p, err := query.ParsePredicate(filter); err != nil {
 				qp.addIssue("filter", err.Error())
-			} else if err := p.Validate(qp.rsc.Validator()); err != nil {
+			} else if err := p.Prepare(qp.rsc.Validator()); err != nil {
 				qp.addIssue("filter", err.Error())
 			} else {
 				qp.q.Predicate = append(qp.q.Predicate, p...)

rest/routing_test.go

@@ -209,9 +209,9 @@ func TestRoutePathParentsExists(t *testing.T) {
 		// There's 3 components in the path but only 2 are parents
 		assert.Len(t, h.queries, 2)
 		// query on /foo/1234
-		assert.Contains(t, h.queries, query.Query{Predicate: query.Predicate{query.Equal{Field: "id", Value: "1234"}}})
+		assert.Contains(t, h.queries, query.Query{Predicate: query.Predicate{&query.Equal{Field: "id", Value: "1234"}}})
 		// query on /bar/5678 with foo/1234 context
-		assert.Contains(t, h.queries, query.Query{Predicate: query.Predicate{query.Equal{Field: "f", Value: "1234"}, query.Equal{Field: "id", Value: "5678"}}})
+		assert.Contains(t, h.queries, query.Query{Predicate: query.Predicate{&query.Equal{Field: "f", Value: "1234"}, &query.Equal{Field: "id", Value: "5678"}}})
 	}
 
 	route = newRoute("GET")
@@ -267,7 +267,7 @@ func TestRouteQueryFilter(t *testing.T) {
 		t.Errorf("unexpected error: %v", rErr)
 	}
 	want := &query.Query{
-		Predicate: query.Predicate{query.Equal{Field: "a", Value: "b"}},
+		Predicate: query.Predicate{&query.Equal{Field: "a", Value: "b"}},
 		Window:    query.Page(1, resource.DefaultConf.PaginationDefaultLimit, 0),
 	}
 	if !reflect.DeepEqual(q, want) {

schema/all_test.go

@@ -70,6 +70,51 @@ func (tc fieldValidatorTestCase) Run(t *testing.T) {
 	})
 }
 
+type fieldQueryValidatorTestCase struct {
+	Name             string
+	Validator        schema.FieldValidator
+	ReferenceChecker schema.ReferenceChecker
+	Input, Expect    interface{}
+	Error            string
+}
+
+func (tc fieldQueryValidatorTestCase) Run(t *testing.T) {
+	t.Run(tc.Name, func(t *testing.T) {
+		t.Parallel()
+
+		if cmp, ok := tc.Validator.(schema.Compiler); ok {
+			err := cmp.Compile(tc.ReferenceChecker)
+			if err != nil {
+				t.Errorf("Validator.Compile(%v): unexpected error: %v", tc.ReferenceChecker, err)
+			}
+		}
+
+		var v interface{}
+		var err error
+		if queryValidator, ok := tc.Validator.(schema.FieldQueryValidator); ok {
+			v, err = queryValidator.ValidateQuery(tc.Input)
+		} else {
+			v, err = tc.Validator.Validate(tc.Input)
+		}
+
+		if tc.Error == "" {
+			if err != nil {
+				t.Errorf("Validator.ValidateQuery(%v): unexpected error: %v", tc.ReferenceChecker, err)
+			}
+			if !reflect.DeepEqual(v, tc.Expect) {
+				t.Errorf("Validator.ValidateQuery(%v): expected: %v, got: %v", tc.Input, tc.Expect, v)
+			}
+		} else {
+			if err == nil || err.Error() != tc.Error {
+				t.Errorf("Validator.ValidateQuery(%v): expected error: %v, got: %v", tc.ReferenceChecker, tc.Error, err)
+			}
+			if v != nil {
+				t.Errorf("Validator.ValidateQuery(%v): expected: nil, got: %v", tc.Input, v)
+			}
+		}
+	})
+}
+
 type fieldSerializerTestCase struct {
 	Name             string
 	Serializer       schema.FieldSerializer