Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fixes and refactoring #38

Merged
merged 3 commits into from
Dec 19, 2022

Conversation

msfroh
Copy link
Collaborator

@msfroh msfroh commented Dec 2, 2022

  1. Shallow clone SearchResponse in SearchActionFilter. The only properties that we're currently changing are SearchHit and timeTookMillis. This is also less brittle as we don't try to implement deserialization ourselves.
  2. KendraIntelligentRanker doesn't transform if the required client settings (endpoint + execution plan ID) are missing.
  3. A lot of refactoring to prepare for other transformers. Where we previously had logic of the form if kendra_intelligent_ranking then do Kendra intelligent ranking, now the APIs support arbitrary named transformers. This required updates to tests and configuration code.
  4. Added tests and fixes for SearchConfigurationExtBuilder.

Signed-off-by: Michael Froh froh@amazon.com

Description

See commit message above.

Issues Resolved

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@msfroh msfroh requested a review from a team December 2, 2022 19:46
@macohen macohen requested a review from noCharger December 14, 2022 04:15
Copy link
Collaborator

@noCharger noCharger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess setting class is more specific towards that transformer itself, whereas the configuration is the schema of the search workflow. There are two things I am not sure if we should put in this PR:

  1. Can I use multiple same reranker, with different setting and configuration?
  2. Do we want to provide more flexibility, such as one transformer depend on several other transformers.

1. Shallow clone SearchResponse in SearchActionFilter. The only
   properties that we're currently changing are SearchHit and
   timeTookMillis. This is also less brittle as we don't try to
   implement deserialization ourselves.
2. KendraIntelligentRanker doesn't transform if the required client
   settings (endpoint + execution plan ID) are missing.
3. A lot of refactoring to prepare for other transformers. Where we
   previously had logic of the form `if kendra_intelligent_ranking then
   do Kendra intelligent ranking`, now the APIs support arbitrary named
   transformers. This required updates to tests and configuration code.
4. Added tests and fixes for SearchConfigurationExtBuilder.

Signed-off-by: Michael Froh <froh@amazon.com>
Looking at the PR on GitHub, I spotted a few mistakes and possible
improvements.

Signed-off-by: Michael Froh <froh@amazon.com>
Signed-off-by: Michael Froh <froh@amazon.com>
@msfroh msfroh force-pushed the generic-refactoring branch from 2a40bd3 to 698f275 Compare December 17, 2022 01:13
@codecov-commenter
Copy link

codecov-commenter commented Dec 17, 2022

Codecov Report

Merging #38 (698f275) into main (d9b44e7) will increase coverage by 5.09%.
The diff coverage is 59.60%.

@@             Coverage Diff              @@
##               main      #38      +/-   ##
============================================
+ Coverage     60.43%   65.52%   +5.09%     
- Complexity      155      172      +17     
============================================
  Files            29       29              
  Lines           867      879      +12     
  Branches        119      119              
============================================
+ Hits            524      576      +52     
+ Misses          304      254      -50     
- Partials         39       49      +10     
Impacted Files Coverage Δ
...search/search/relevance/SearchRelevancePlugin.java 0.00% <0.00%> (ø)
.../configuration/ResultTransformerConfiguration.java 100.00% <ø> (ø)
...configuration/KendraIntelligentRankerSettings.java 76.66% <ø> (ø)
.../KendraIntelligentRankingConfigurationFactory.java 0.00% <0.00%> (ø)
...draintelligentranking/client/KendraHttpClient.java 21.56% <23.33%> (+21.56%) ⬆️
...uration/KendraIntelligentRankingConfiguration.java 26.53% <50.00%> (-7.82%) ⬇️
...ch/relevance/configuration/ConfigurationUtils.java 66.66% <60.00%> (ø)
...e/configuration/SearchConfigurationExtBuilder.java 67.64% <66.66%> (+53.61%) ⬆️
...draintelligentranking/KendraIntelligentRanker.java 67.61% <91.66%> (+0.29%) ⬆️
...rch/relevance/actionfilter/SearchActionFilter.java 84.76% <96.00%> (+3.72%) ⬆️
... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

serviceEndpoint = clientSettings.getServiceEndpoint();
executionPlanId = clientSettings.getExecutionPlanId();
if (isValid()) {
Copy link
Collaborator

@noCharger noCharger Dec 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking:

This is what I preferred:

    if (!isValid()) {
      amazonHttpClient = null;
      aws4Signer = null;
      awsCredentialsProvider = null;
      errorHandler = null;
      responseHandler = null;
      return;
    }
    amazonHttpClient = AccessController.doPrivileged((PrivilegedAction<AmazonHttpClient>) () -> new AmazonHttpClient(new ClientConfiguration()));
    errorHandler = new SimpleAwsErrorHandler();
    responseHandler = new SimpleResponseHandler();
    aws4Signer = new AWS4Signer();
    aws4Signer.setServiceName(KENDRA_RANKING_SERVICE_NAME);
    aws4Signer.setRegionName(clientSettings.getServiceRegion());
   ...

Copy link
Collaborator

@noCharger noCharger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome! Approved with non-blocking comments.

@macohen
Copy link
Collaborator

macohen commented Dec 19, 2022

We do want to review the full set of metrics and logging events to capture. We'll create another issue for that purpose.

@macohen macohen merged commit d8ba75b into opensearch-project:main Dec 19, 2022
@noCharger noCharger added backport 2.x Backport to 2.x branch enhancement change or upgrade that increases software capabilities beyond original client specifications labels Dec 20, 2022
@noCharger noCharger added backport 2.x Backport to 2.x branch and removed backport 2.x Backport to 2.x branch labels Jan 3, 2023
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 3, 2023
* Bug fixes and refactoring

1. Shallow clone SearchResponse in SearchActionFilter. The only
   properties that we're currently changing are SearchHit and
   timeTookMillis. This is also less brittle as we don't try to
   implement deserialization ourselves.
2. KendraIntelligentRanker doesn't transform if the required client
   settings (endpoint + execution plan ID) are missing.
3. A lot of refactoring to prepare for other transformers. Where we
   previously had logic of the form `if kendra_intelligent_ranking then
   do Kendra intelligent ranking`, now the APIs support arbitrary named
   transformers. This required updates to tests and configuration code.
4. Added tests and fixes for SearchConfigurationExtBuilder.

Signed-off-by: Michael Froh <froh@amazon.com>

* Address PR comments from myself

Looking at the PR on GitHub, I spotted a few mistakes and possible
improvements.

Signed-off-by: Michael Froh <froh@amazon.com>

* Address comments by noCharger

Signed-off-by: Michael Froh <froh@amazon.com>

Signed-off-by: Michael Froh <froh@amazon.com>
(cherry picked from commit d8ba75b)
noCharger pushed a commit that referenced this pull request Jan 3, 2023
* Bug fixes and refactoring

1. Shallow clone SearchResponse in SearchActionFilter. The only
   properties that we're currently changing are SearchHit and
   timeTookMillis. This is also less brittle as we don't try to
   implement deserialization ourselves.
2. KendraIntelligentRanker doesn't transform if the required client
   settings (endpoint + execution plan ID) are missing.
3. A lot of refactoring to prepare for other transformers. Where we
   previously had logic of the form `if kendra_intelligent_ranking then
   do Kendra intelligent ranking`, now the APIs support arbitrary named
   transformers. This required updates to tests and configuration code.
4. Added tests and fixes for SearchConfigurationExtBuilder.

Signed-off-by: Michael Froh <froh@amazon.com>

* Address PR comments from myself

Looking at the PR on GitHub, I spotted a few mistakes and possible
improvements.

Signed-off-by: Michael Froh <froh@amazon.com>

* Address comments by noCharger

Signed-off-by: Michael Froh <froh@amazon.com>

Signed-off-by: Michael Froh <froh@amazon.com>
(cherry picked from commit d8ba75b)

Co-authored-by: msfroh <froh@amazon.com>
@mingshl mingshl added bug Something isn't working and removed enhancement change or upgrade that increases software capabilities beyond original client specifications labels Jan 23, 2023
@mingshl mingshl mentioned this pull request Jan 26, 2023
5 tasks
@mingshl mingshl added maintenance A change to add support for new versions of OpenSearch or OpenSearch Dashboards from upstream. enhancement change or upgrade that increases software capabilities beyond original client specifications and removed bug Something isn't working maintenance A change to add support for new versions of OpenSearch or OpenSearch Dashboards from upstream. labels Jan 26, 2023
@msfroh msfroh deleted the generic-refactoring branch February 7, 2023 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch enhancement change or upgrade that increases software capabilities beyond original client specifications
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants