Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor ES access with new es client #4900

Merged
merged 11 commits into from
Dec 8, 2022
Merged

Conversation

cangfengzhs
Copy link
Contributor

@cangfengzhs cangfengzhs commented Nov 17, 2022

What type of PR is this?

  • bug
  • feature
  • enhancement

What problem(s) does this PR solve?

Issue(s) number:

Description:

Before that, in #4891 and #4970, I have introduced libcurl and created httpClient and esclient.

In this PR, I will discard the original es calling method and use the ESAdapter and ESClient I created to access ES. ESAdapter modifies the data format written to ES by full-text index.

Note, however, that the PR does not modify the use logic of full-text index, and still needs to rely on the native index. This problem will be solved in the next PR.

How do you solve it?

Special notes for your reviewer, ex. impact of this fix, design document, etc:

Checklist:

Tests:

  • Unit test(positive and negative cases)
  • Function test
  • Performance test
  • N/A

Affects:

  • Documentation affected (Please add the label if documentation needs to be modified.)
  • Incompatibility (If it breaks the compatibility, please describe it and add the label.)
  • If it's needed to cherry-pick (If cherry-pick to some branches is required, please label the destination version(s).)
  • Performance impacted: Consumes more CPU/Memory

Release notes:

Please confirm whether to be reflected in release notes and how to describe:

ex. Fixed the bug .....

@cangfengzhs cangfengzhs marked this pull request as ready for review December 4, 2022 06:08
@codecov-commenter
Copy link

codecov-commenter commented Dec 4, 2022

Codecov Report

Base: 76.86% // Head: 77.20% // Increases project coverage by +0.34% 🎉

Coverage data is based on head (ab1b16d) compared to base (8adec8d).
Patch coverage: 42.90% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4900      +/-   ##
==========================================
+ Coverage   76.86%   77.20%   +0.34%     
==========================================
  Files        1105     1098       -7     
  Lines       81432    81174     -258     
==========================================
+ Hits        62594    62674      +80     
+ Misses      18838    18500     -338     
Impacted Files Coverage Δ
src/common/expression/TextSearchExpression.h 77.58% <ø> (+20.68%) ⬆️
src/common/http/HttpClient.h 100.00% <ø> (ø)
src/graph/executor/admin/SpaceExecutor.cpp 76.19% <0.00%> (-0.74%) ⬇️
src/graph/executor/maintain/FTIndexExecutor.cpp 0.00% <0.00%> (ø)
src/graph/executor/query/TraverseExecutor.h 100.00% <ø> (ø)
src/graph/validator/AdminValidator.h 66.66% <ø> (ø)
src/graph/validator/LookupValidator.cpp 81.11% <0.00%> (+0.85%) ⬆️
src/graph/validator/MaintainValidator.cpp 74.94% <0.00%> (+0.17%) ⬆️
src/kvstore/listener/elasticsearch/ESListener.cpp 0.00% <0.00%> (ø)
src/kvstore/listener/elasticsearch/ESListener.h 0.00% <ø> (ø)
... and 61 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@cangfengzhs cangfengzhs changed the title Refactor es Refactor ES access with new es client Dec 6, 2022
virtual HttpResponse delete_(const std::string& url,
const std::vector<std::string>& headers,
const std::string& username,
const std::string& password);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not store the username and password in HttpClient to avoid to pass them in each function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the beginning, httpclient was designed to be stateless, and it only has some static functions. Later, in order to facilitate the use of gmock in the test, we changed it to non static, but the httpclient is still stateless.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition, username, password and url are bound, but httpclient and url are not bound. Therefore, it is not reasonable to put username and password in httpclient. Maybe you have other better suggestions?

Copy link
Contributor

@critical27 critical27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~ Good job

@@ -6,6 +6,8 @@
#include <gtest/gtest.h>
#include <thrift/lib/cpp/concurrency/ThreadManager.h>

#include <map>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems you don't need to modify this test since derived Listener do not need to overload apply function anymore. Not a big issue, up to you.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to compile and pass the test

src/kvstore/listener/elasticsearch/ESListener.cpp Outdated Show resolved Hide resolved
@critical27 critical27 added the doc affected PR: improvements or additions to documentation label Dec 8, 2022
@critical27
Copy link
Contributor

This PR involves doc updated, @randomJoe211 , including:

  1. Syntax change when create fulltext index
  2. When this version has been released, user need to rebuild fulltext index.
    Ask @cangfengzhs for details.

Copy link
Contributor

@dutor dutor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc affected PR: improvements or additions to documentation ready-for-testing PR: ready for the CI test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants