Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add generic ES query building utilities #5168

Merged
merged 14 commits into from
Apr 18, 2023
Merged

Add generic ES query building utilities #5168

merged 14 commits into from
Apr 18, 2023

Conversation

mantas-sidlauskas
Copy link
Contributor

@mantas-sidlauskas mantas-sidlauskas commented Mar 20, 2023 •
edited
Loading

What changed?
Adding local ES query building utilities to remove dependency on external packages.

All elasticsearch queries can use local builder instead of versioned package provided one. You can replace ES package specific code

elastic.NewSearchSource().Query(elastic.NewExistsQuery("user")).Size(10).SortBy(elastic.NewFieldSort("runid").Desc())

To this one:

 NewBuilder().Query(NewExistsQuery("user")).Size(10).Sortby(NewFieldSort("runid").Desc())

Why?
Local package is not depending on external/version specific ES package. This will be reused for OpenSearch client also

How did you test it?
Added query builder test to check output against ESv7

Potential risks

Release notes

Documentation Changes

@coveralls
Copy link

coveralls commented Mar 21, 2023 •
edited
Loading

Pull Request Test Coverage Report for Build 0187939f-4078-4cec-a871-8a0cc77d5575

  • 228 of 376 (60.64%) changed or added relevant lines in 8 files are covered.
  • 190 unchanged lines in 17 files lost coverage.
  • Overall coverage decreased (-0.04%) to 57.072%
Changes Missing Coverage Covered Lines Changed/Added Lines %
common/elasticsearch/client_v6.go 7 11 63.64%
common/elasticsearch/query/exists_query.go 19 25 76.0%
common/elasticsearch/client_v7.go 0 10 0.0%
common/elasticsearch/query/sort.go 20 32 62.5%
common/elasticsearch/query/builder.go 50 64 78.13%
common/elasticsearch/query/range_query.go 37 55 67.27%
common/elasticsearch/query/bool_query.go 48 72 66.67%
common/elasticsearch/query/match_query.go 47 107 43.93%
Files with Coverage Reduction New Missed Lines %
service/history/execution/mutable_state_util.go 2 36.04%
service/history/task/transfer_standby_task_executor.go 2 86.4%
service/matching/matcher.go 2 91.46%
common/types/shared.go 4 40.42%
common/task/fifoTaskScheduler.go 5 84.54%
service/history/queue/timer_queue_processor_base.go 5 77.92%
common/persistence/nosql/nosqlplugin/cassandra/workflow.go 6 59.1%
common/persistence/statsComputer.go 6 93.57%
service/history/execution/mutable_state_builder.go 7 68.56%
common/persistence/serialization/parser.go 8 62.41%
Totals Coverage Status
Change from base Build 0187817c-ca37-462d-94a5-6e0826698861: -0.04%
Covered Lines: 85438
Relevant Lines: 149702
💛 - Coveralls

@Groxx
Copy link
Contributor

Groxx commented Mar 27, 2023

While I find this interesting and probably useful...
what exactly does it do / what's the purpose? And how would it be used by e.g. users or other code?

@mantas-sidlauskas mantas-sidlauskas mentioned this pull request Mar 28, 2023
Merged
@mantas-sidlauskas
Copy link
Contributor Author

mantas-sidlauskas commented Mar 29, 2023 •
edited
Loading

Added more context to the description

Shaddoll
Shaddoll reviewed Apr 11, 2023
View reviewed changes
common/elasticsearch/query/builder.go
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this used anywhere?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The next PR will use this, instead of Olivere library builder.

@demirkayaender demirkayaender merged commit b18be27 into uber:master Apr 18, 2023
davidporter-id-au added a commit that referenced this pull request Apr 19, 2023
@davidporter-id-au
Squashed commit of the following:
c405734 
commit b18be27
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Tue Apr 18 14:12:09 2023 +0300

    Add generic ES query building utilities (#5168)

commit 824f0ac
Author: agautam478 <72432016+agautam478@users.noreply.github.com>
Date:   Fri Apr 14 13:37:29 2023 -0700

    Fixed the nil pointer issues in the InactiveDomain Invariant (#5213)

commit c5678dd
Author: Ketsia <115650494+ketsiambaku@users.noreply.github.com>
Date:   Wed Apr 12 15:21:10 2023 -0700

    Fix consistent query metric (#5170)

    * add shardid tag to log

    * remove counter for overall scope

    * fix lint

commit eade936
Author: David Porter <david.porter@uber.com>
Date:   Wed Apr 12 13:54:13 2023 -0700

    Corrects the config-store handling for not-found errors (#5203)

commit 9fc4485
Author: lancezhao-ins <99238165+lancezhao-ins@users.noreply.github.com>
Date:   Tue Apr 11 04:21:15 2023 +1000

    Allow registering search attributes without Advance Visibility enabled (#5185)

    * remove validation & test for add search attribute with no advanced config

    - Remove validation for Advance Visibility Store
    - Add Advance Visibility Config check before update ElasticSearch/OpenSearch mapping
    - Remove co-related test for 'no advanced config'

    * Update CHANGELOG.md

    Update CHANGELOG.md

    * Add warn level message if skip updating OpenSearch/ElasticSearch mapping

    * Add warn level message and add validSearchAttributes in development.yaml

    ---------

    Co-authored-by: Quanzheng Long <prclqz@gmail.com>

commit d165c7b
Author: neil-xie <104041627+neil-xie@users.noreply.github.com>
Date:   Mon Apr 10 10:46:04 2023 -0700

    Update idls version (#5200)

commit b2bc8bf
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Mon Apr 10 20:12:53 2023 +0300

    Add thin ES clients (#5162)

    * Add thin ES clients
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Shaddoll Shaddoll Shaddoll approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@mantas-sidlauskas @coveralls @Groxx @Shaddoll @demirkayaender