Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(dsl): Support source filtering in search requests #196

Merged
merged 11 commits into from
May 4, 2023

Conversation

mvelimir
Copy link
Member

No description provided.

def aggregate(aggregation: ElasticAggregation): SearchAndAggregateRequest

def excludes(fields: String*): SearchRequest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to support excludes(field: String, fields: String*)?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'll have to choose one or the other as having both will probably cause ambiguity as excludes(oneValue) satisfies both. I think the decision should come down to whether we think the list will be used in a dynamic manner, in which case the stricter form can be less convenient.

def highlights(value: Highlights): SearchRequest

def includes(fields: String*): SearchRequest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to support includes(field: String, fields: String*)?


def includes(fields: String*): R

final def includes(schema: Schema.Record[_]): R = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we annotate this with @tailrec?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this implementation it's not possible due to the flatMap call on the list, and I'm not sure it is possible with a different implementation either, because each element might call the recursive function.

def searchAfter(value: Json): SearchRequest
}

private[elasticsearch] final case class Search(
index: IndexName,
query: ElasticQuery[_],
sortBy: Set[Sort],
excluded: Option[List[String]],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a Chunk would be more appropriate here, especially since we rely on concatenation in def excludes, even though most likely excludes will not be used multiple times.

included merge excluded.fold(Obj())(excluded => Obj("excludes" -> Arr(excluded.map(_.toJson): _*)))
}
.orElse(excluded.map(excluded => Obj("excludes" -> Arr(excluded.map(_.toJson): _*))))
.fold(Obj())(filters => Obj("_source" -> filters))
Copy link
Collaborator

@arnoldlacko arnoldlacko Apr 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if something like this could be a bit easier to read:

val sourceJson: Json = 
   (included, excluded) match {
      case (None, None) => Obj()
      case (in, ex) => 
         Obj( "_source" -> {
            val includes = included.fold(Obj())(included => Obj("includes" -> Arr(included.map(_.toJson): _*)))
            val excludes = excluded.fold(Obj())(excluded => Obj("excludes" -> Arr(excluded.map(_.toJson): _*))
            includes merge excludes
         }
      }

Would this work?


def includes(fields: String*): R

final def includes(schema: Schema.Record[_]): R = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense (or be possible) to have something like

includeOnly[A](implicit schema: Schema.Record[A])

?

@@ -671,13 +707,23 @@ object ElasticRequest {
val sortJson: Json =
if (self.sortBy.nonEmpty) Obj("sort" -> Arr(self.sortBy.toList.map(_.paramsToJson): _*)) else Obj()

val sourceJson: Json =
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above. (Arnold's comment)


import zio.schema.Schema

trait HasSourceFiltering[R <: HasSourceFiltering[R]] {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can make this trait private[elasticsearch]?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also we should write scaladoc for methods in this trait.

Comment on lines 201 to 215
.includes()
.excludes() match {
case r: ElasticRequest.Search => r.toJson
}
val expected =
"""
|{
| "_source" : {
| "includes" : [],
| "excludes" : []
| },
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we allow this?

@mvelimir mvelimir force-pushed the feature/include-and-exclude-fields branch from 2fe2f74 to 340ac40 Compare May 4, 2023 15:17
@drmarjanovic drmarjanovic merged commit 7d3316d into main May 4, 2023
@drmarjanovic drmarjanovic deleted the feature/include-and-exclude-fields branch May 4, 2023 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants