Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(decide): change to return reasons as a part of tuple in decision hierarchy #415

Merged
merged 10 commits into from
Jan 14, 2021

Conversation

jaeopt
Copy link
Contributor

@jaeopt jaeopt commented Dec 18, 2020

Summary

  • Change to return reasons as a part of tuple in decision hierarchy (from passing down a reference for modification)

Test plan

  • Validate that reasons are collected properly with or without INCLUDE_REASONS option.

@jaeopt jaeopt requested a review from a team as a code owner December 18, 2020 21:53
@jaeopt jaeopt removed their assignment Dec 18, 2020
@coveralls
Copy link

coveralls commented Dec 18, 2020

Pull Request Test Coverage Report for Build 1644

  • 148 of 152 (97.37%) changed or added relevant lines in 12 files are covered.
  • 3 unchanged lines in 3 files lost coverage.
  • Overall coverage increased (+0.2%) to 89.958%

Changes Missing Coverage Covered Lines Changed/Added Lines %
core-api/src/main/java/com/optimizely/ab/config/audience/UserAttribute.java 5 6 83.33%
core-api/src/main/java/com/optimizely/ab/optimizelydecision/DefaultDecisionReasons.java 4 5 80.0%
core-api/src/main/java/com/optimizely/ab/bucketing/DecisionService.java 70 72 97.22%
Files with Coverage Reduction New Missed Lines %
core-api/src/main/java/com/optimizely/ab/config/audience/EmptyCondition.java 1 50.0%
core-api/src/main/java/com/optimizely/ab/config/audience/UserAttribute.java 1 94.34%
core-api/src/main/java/com/optimizely/ab/internal/ExperimentUtils.java 1 92.11%
Totals Coverage Status
Change from base Build 1643: 0.2%
Covered Lines: 4452
Relevant Lines: 4949

💛 - Coveralls

@@ -545,17 +546,18 @@ public boolean setForcedVariation(@Nonnull Experiment experiment,
*
* @param experiment The experiment forced.
* @param userId The user ID to be used for bucketing.
* @param reasons Decision log messages
* @return The variation the user was bucketed into. This value can be null if the
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should update comments to return DecisionResponse

@@ -344,15 +368,14 @@ Variation getWhitelistedVariation(@Nonnull Experiment experiment,
* @param experiment {@link Experiment} in which the user was bucketed.
* @param userProfile {@link UserProfile} of the user.
* @param projectConfig The current projectConfig
* @param reasons Decision log messages
* @return null if the {@link UserProfileService} implementation is null or the user was not previously bucketed.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should update comments with DecisionResponse

Copy link
Contributor

@thomaszurkan-optimizely thomaszurkan-optimizely left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks much cleaner. There were a few comments/corrections. Did we downscope reasons with this change?

@@ -427,7 +423,7 @@ private Boolean isFeatureEnabled(@Nonnull ProjectConfig projectConfig,

Map<String, ?> copiedAttributes = copyAttributes(attributes);
FeatureDecision.DecisionSource decisionSource = FeatureDecision.DecisionSource.ROLLOUT;
FeatureDecision featureDecision = decisionService.getVariationForFeature(featureFlag, userId, copiedAttributes, projectConfig);
FeatureDecision featureDecision = decisionService.getVariationForFeature(featureFlag, userId, copiedAttributes, projectConfig).getResult();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The decision service methods are all marked as nullable return value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A good catch. I'll change them all @nonnull.

@Nonnull Map<String, ?> filteredAttributes,
@Nonnull ProjectConfig projectConfig) {
return getVariation(experiment, userId, filteredAttributes, projectConfig, Collections.emptyList(), DefaultDecisionReasons.newInstance());
public DecisionResponse<Variation> getVariation(@Nonnull Experiment experiment,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These methods are all marked as nullable return value. However, the usage does not test for null:

Variation variation = decisionService.getVariation(experiment, userId, copiedAttributes, projectConfig).getResult();

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll change them all @nonnull.

} catch (NullPointerException e) {
String message = reasons.addInfo("attribute or value null for match %s", match != null ? match : "legacy condition");
logger.error(message, e);
logger.error("attribute or value null for match {}", match != null ? match : "legacy condition", e);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the only reason to have reasons is for debugging, then, not having these doesn't make sense. also, we need to make sure we are doing the same for other sdks. We were including attribute failures before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it's a good to have in reasons, so was added before. Found that this is the only meaningful message from conditions. It's quite expensive to apply the unused reasons to all Conditions, that's why I decided to drop this from reasons. We provide other relevant reasons like "does not meet audience condition", ... and they can take a look at log messages for investigation if needed.

Copy link
Contributor

@thomaszurkan-optimizely thomaszurkan-optimizely left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jaeopt jaeopt merged commit 5bb2588 into master Jan 14, 2021
@jaeopt jaeopt deleted the jae/return-reasons branch January 14, 2021 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants