Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Draft] floors feature implementation #2341

Closed
wants to merge 17 commits into from

Conversation

PubMatic-OpenWrap
Copy link
Contributor

@PubMatic-OpenWrap PubMatic-OpenWrap commented Aug 4, 2022

Started development for floors feature as mentioned in PRD #1162.
This PR contains changes for floors feature and mainly covers basic implementation of Signalling and Enforcement.

Copy link
Contributor

@SyntaxNode SyntaxNode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution. This is a large PR, so I have a large amount of comments. :)

config/config.go Outdated Show resolved Hide resolved
config/config.go Outdated
type PriceFloors struct {
Enabled bool `mapstructure:"enabled"`
UseDynamicData bool `mapstructure:"use_dynamic_data"`
EnforceFloorsRate int `mapstructure:"enforce_floors_rate"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What measurement applies to the rate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This global level config, which defines percent chance that PBS should suppress any bids below the matched floor from entering the auction.
e.g. if EnforceFloorsRate = 100, floors enforcement would be applied to all the bids,
if EnforceFloorsRate = 50, floors enforcement would be applied to 50% of the bids

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha. Can you please add a comment explaining that it's a random chance from 0 to 100.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

enabled: true
use_dynamic_data: false
enforce_floors_rate: 100
enforce_deal_floors: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change all values from their default for this test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default config is tested in TestDefaults

var bidfloor bidFloor
bidfloor.bidFloorCur = bidRequest.Imp[i].BidFloorCur
if bidfloor.bidFloorCur == "" {
bidfloor.bidFloorCur = "USD"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned this is a separate hardcoding of "default to USD". Is there a way we can reuse the existing code or tie it in better? Perhaps a default currency constant shared by both?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case floor currency not provided, can we set it to req.curr[0] to avoid hardcoding ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me. Just please be sure that req.curr[0] is guaranteed to exist at this point.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes we can access req.curr[0]

exchange/floors.go Outdated Show resolved Hide resolved
floors/rule.go Show resolved Hide resolved
floors/rule.go Show resolved Hide resolved
wt2 += 1 << (segNum - comb[j][k])
}
return wt1 < wt2
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider extracting the sort algorithm and testing it separately.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is getting tested in TestPrepareRuleCombinations(),
Let us know if it serves purpose.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bsardo what is your opinion?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it would be preferable to separate the sorting out into a separate function. I see that TestPrepareRuleCombinations is verifying the sorting but I think compartmentalizing the sorting algorithm and writing a small unit test specifically for that makes the code a bit easier to read/maintain and allows us to easily establish confidence that the sorting logic is working independent of the rule generation logic.

I do think though that we first need to solve the issue I described in a different comment regarding the rule generation algorithm complexity because it might affect this code.

openrtb_ext/floors.go Show resolved Hide resolved
openrtb_ext/floors.go Outdated Show resolved Hide resolved
@SyntaxNode SyntaxNode self-assigned this Aug 4, 2022
@bsardo bsardo requested a review from mansinahar August 15, 2022 17:31
config/config.go Outdated
@@ -1217,6 +1218,11 @@ func SetupViper(v *viper.Viper, filename string) {
v.SetDefault("experiment.adscert.inprocess.domain_renewal_interval_seconds", 30)
v.SetDefault("experiment.adscert.remote.url", "")
v.SetDefault("experiment.adscert.remote.signing_timeout_ms", 5)

v.SetDefault("experiment.price_floors.enabled", true)
Copy link
Contributor

@SyntaxNode SyntaxNode Aug 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please set this to false by default. We'll highlight the option to turn it on in the relevant release notes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

config/config.go Outdated
type PriceFloors struct {
Enabled bool `mapstructure:"enabled"`
UseDynamicData bool `mapstructure:"use_dynamic_data"`
EnforceFloorsRate int `mapstructure:"enforce_floors_rate"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha. Can you please add a comment explaining that it's a random chance from 0 to 100.

config/experiment.go Outdated Show resolved Hide resolved
var bidfloor bidFloor
bidfloor.bidFloorCur = bidRequest.Imp[i].BidFloorCur
if bidfloor.bidFloorCur == "" {
bidfloor.bidFloorCur = "USD"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me. Just please be sure that req.curr[0] is guaranteed to exist at this point.

Copy link
Contributor

@SyntaxNode SyntaxNode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank for replying to the feedback so far. Sorry for the delay in a follow up review. There are more PRs submitted than usual this summer.

I focused this review on logic flow. I left out nitpicks and tests for now. I'd like to see alignment of the floorsExt setting with how bidder specific requests are typically created within the codebase. Please see me other comments for details.

// signalFloors function does singanlling of floors,
// Internally validation of floors parameters and validation of rules is done,
// Based on number of modelGroups and modelWeight, one model is selected and imp.bidfloor and imp.bidfloorcur is updated
func signalFloors(r *AuctionRequest, floor floors.Floor, conversions currency.Conversions, responseDebugAllow bool) []error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for adding the description. You may wish to consider changing the name of the method, perhaps selectFloorsAndModifyImp would be a more descriptive choice?

.. or perhaps updateFloorsInImps? That matches the comment on the call to this method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed function to selectFloorsAndModifyImp()

@@ -293,6 +301,12 @@ func (e *exchange) HoldAuction(ctx context.Context, r AuctionRequest, debugLog *
var bidResponseExt *openrtb_ext.ExtBidResponse
if anyBidsReturned {

//If floor enforcement config enabled then filter bids
adapterBids, bidRejections := enforceFloors(&r, adapterBids, e.floor, conversions, responseDebugAllow)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's easier to return a slice of errors instead of bidRejections, feel free to go for it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

returned slice of errors

seatBids, rejections = enforceFloorToBids(r.BidRequestWrapper.BidRequest, seatBids, conversions, enforceDealFloors)
}
requestExt.SetPrebid(prebidExt)
err = r.BidRequestWrapper.RebuildRequest()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

enforceFloors is only called from HoldAuction. It's odd to see a change to the request happening here after it's already been sent to the bidder. I also find it confusing that prebidExt would be modified in floors.ShouldEnforce.

All code which modifies a request for a specific bidder is contained in cleanOpenRTBRequest. Could you please refactor this code to move this logic there? ... except for enforceFloorToBids as since that operates on the bids it needs to stay in this location.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

floors/enforce.go Outdated Show resolved Hide resolved
if floorExt.Enforcement.EnforcePBS == nil {
floorExt.Enforcement.EnforcePBS = new(bool)
}
*floorExt.Enforcement.EnforcePBS = shouldEnforce
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this update the floorExt in addition to returning the value? Can we just use the return value and not modify the request?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This update is done to handle below case:
When Enforcement.EnforcePBS is not defined and based on EnforceRate enforcement needs to be done or skipped,
In that case this value is updated with appropriate value.
And to update prebidExt due to this case requestExt.SetPrebid(prebidExt) is done in enforceFloors()

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No other code in this PR check for EnforcePBS. Is this therefore a signal downstream to bidders so they know what behavior PBS performed? @bretg checking on the spec intention here.

floors/floors.go Outdated
}

floorData.ModelGroups, floorModelErrList = validateFloorModelGroups(floorData.ModelGroups)
if len(floorData.ModelGroups) == 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't we also want to report validation errors for the model groups which were rejected? We could send them down as warnings.

"github.com/prebid/prebid-server/openrtb_ext"
)

func validateFloorRules(Schema openrtb_ext.PriceFloorSchema, delimiter string, RuleValues map[string]float64) []error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rename to something which indicates the rule values are modified. I would normally expect this to just return a list of errors, but it's also modifying the floors and filtering our invalid ones. Better yet, consider returning a new map of rules instead of modifying the original collection - and reassign after.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed function to selectValidFloorModelGroups()

floors/floors.go Outdated
floorData.ModelGroups = selectFloorModelGroup(floorData.ModelGroups, rand.Intn)
}

if floorData.ModelGroups[0].Schema.Delimiter == "" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider assigning a model group variable, since there is only one left at this point in the code. Something like:

modelGroup := floorData.ModelGroups[0]

floors/rule.go Outdated

func updateImpExtWithFloorDetails(matchedRule string, imp *openrtb2.Imp, floorVal float64) {
imp.Ext, _ = jsonparser.Set(imp.Ext, []byte(`"`+matchedRule+`"`), "prebid", "floors", "floorRule")
imp.Ext, _ = jsonparser.Set(imp.Ext, []byte(fmt.Sprintf("%.4f", floorVal)), "prebid", "floors", "floorRuleValue")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may not use jsonparser.Set. it is not production ready code and has caused panics for us in the past.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to change this to use the standard Go json library. We've tried to use jsonparser.Setbefore and it seg faulted on us in certain cases.

return *Floors.Enabled
}
return true
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please reorganized these structs such that the "parent" struct is listed before the "child" struct. I would expect to see PriceFloorRules declared first and for PriceFloorSchema to be declared after PriceFloorModelGroup, etc..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reorganised structures

return errs
}
prebidExt := requestExt.GetPrebid()
if floor != nil && floor.IsFloorEnabled() && prebidExt.Floors.GetEnabled() && prebidExt.Floors != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the prebidExt.Floors != nil check be before prebidExt.Floors.GetEnabled()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

floors/floors.go Outdated
ENFORCE_RATE_MAX int = 100
)

type Config struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the rationale behind having a separate config struct that is similar to the PriceFloors struct in experiment.go? Why can't we just define that struct here and use it in experiment.go as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To implement below interface this structure was introduced

floors/floors.go Outdated
return fc.EnforceDealFloors
}

type Floor interface {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the rationale behind this interface? Do you envision any other implementations of this interface?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no other implementation.
If we do not use Interface, then we might have to return config struct member values

floors/rule.go Outdated
} else {
skipRate = RootSkipRate
}
return skipRate > f(SKIP_RATE_MAX+1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be >= ? Consider a skipRate of 100. In that case we should be skipping enforcing this feature 100% of the time but if the random number selected would be 100 then this would return false

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrected

"github.com/prebid/prebid-server/openrtb_ext"
)

func validateFloorRules(Schema openrtb_ext.PriceFloorSchema, delimiter string, RuleValues map[string]float64) []error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with @SyntaxNode's comments. Can you also please rename the input parameters so that the first letter is in lower case? This is just to follow Go's convention of naming for parameters

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

if len(parsedKey) != len(Schema.Fields) {
// Number of fields in rule and number of schema fields are not matching
errs = append(errs, fmt.Errorf("Invalid Floor Rule = '%s' for Schema Fields = '%v'", key, Schema.Fields))
delete(RuleValues, key)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like you might have missed a continue statement here. If it's an invalid rule, we don't want to execute logic on L20 and L21, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch

floors/rule.go Outdated
return skipRate > f(SKIP_RATE_MAX+1)
}

func findRule(RuleValues map[string]float64, delimiter string, desiredRuleKey []string, numFields int) (string, bool) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: Please rename RuleValues parameter to ruleValues in order to follow Go's naming convention

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -955,6 +969,7 @@ func (e *exchange) makeExtBidResponse(adapterBids map[openrtb_ext.BidderName]*pb
bidResponseExt.Debug = &openrtb_ext.ExtResponseDebug{
HttpCalls: make(map[openrtb_ext.BidderName][]*openrtb_ext.ExtHttpCall),
ResolvedRequest: r.ResolvedBidRequest,
UpdatedRequest: r.UpdatedBidRequest,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @SyntaxNode here. I think of resolved requests as the final request generated by PBS after adding/updating parts of the request based on certain logic. This includes merging stored requests and IMHO should also include changes made by the price floor feature.

Also, in addition to it being an expensive field as @SyntaxNode pointed out, it will affect network bandwidth as well as your response gets bigger.

@bsardo
Copy link
Collaborator

bsardo commented Aug 30, 2022

Hello @pm-jaydeep-mohite, thanks for contributing! Just for future reference, if you don't mind, please merge with master instead of force pushing after your PR has been assigned for review. Force pushing makes it hard to figure out what has changed since the initial review which leads to a lot of extra time spent by our team on the delta reviews, especially for larger PRs. 🙂

Removed interfaces used to get config values and used config structure to get values
@PubMatic-OpenWrap PubMatic-OpenWrap changed the title [WIP]: Changes for floors feature floors feature implementation Sep 5, 2022
@PubMatic-OpenWrap PubMatic-OpenWrap requested review from SyntaxNode and mansinahar and removed request for SyntaxNode and mansinahar September 5, 2022 10:48
@bsardo bsardo assigned bsardo and unassigned mansinahar Sep 13, 2022
@bsardo bsardo requested review from bsardo and removed request for mansinahar September 13, 2022 17:14
Copy link
Contributor

@SyntaxNode SyntaxNode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the updates and your patience for a re-review. Looking a lot better IMHO after the changes. I'm in the middle of reviewing tests and comparing the expected behavior against the spec. I hope to finish that mid next week.

@@ -26,6 +28,17 @@ const (
// Experiment defines if experimental features are available
type Experiment struct {
AdCerts ExperimentAdsCert `mapstructure:"adscert"`
// Floors feature configuration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: This comment does not seem necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed comment

type PriceFloors struct {
Enabled bool `mapstructure:"enabled"`
UseDynamicData bool `mapstructure:"use_dynamic_data"`
// Floors enforcement rate, values can be 0 to 100,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: Please follow Go standards and begin the comment with the type name, ie:
// EnforceFloorsRate values can be 0 ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will update at other places as well

@@ -955,6 +969,7 @@ func (e *exchange) makeExtBidResponse(adapterBids map[openrtb_ext.BidderName]*pb
bidResponseExt.Debug = &openrtb_ext.ExtResponseDebug{
HttpCalls: make(map[openrtb_ext.BidderName][]*openrtb_ext.ExtHttpCall),
ResolvedRequest: r.ResolvedBidRequest,
UpdatedRequest: r.UpdatedBidRequest,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm cool with that.

"github.com/prebid/prebid-server/openrtb_ext"
)

// Check for Floors enforcement for deals,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: Please follow Go standards and begin each comment with the type of the thing the comment applies to. I won't make this any more comments for this nitpick if you could please apply the pattern throughout the PR.

Example in this case would be
``// checkDealsForEnforcement checks for floors enforcement...`

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

}

for bidderName, seatBid := range seatBids {
eligibleBids := make([]*pbsOrtbBid, 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Performance Nitpick: Consider setting the initial capacity to len(seatBid.bids) to try and avoid array resizing. ie:
make([]*pbsOrtbBid, 0, len(seatBid.bids))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

floors/floors.go Outdated
MODEL_WEIGHT_MAX_VALUE int = 100
MODEL_WEIGHT_MIN_VALUE int = 0
ENFORCE_RATE_MIN int = 0
ENFORCE_RATE_MAX int = 100
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: In Go, it's common to use CamelCase for constants instead of CAPS. ie DEFAULT_DELIMITER -> DefaultDelimiter. This is largely due to the capital letter at the start of the variable name indicating if the type is package private or exported. If no other package needs these constants, please start with a lower case letter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated to use CamelCase

floors/rule.go Outdated

func updateImpExtWithFloorDetails(matchedRule string, imp *openrtb2.Imp, floorVal float64) {
imp.Ext, _ = jsonparser.Set(imp.Ext, []byte(`"`+matchedRule+`"`), "prebid", "floors", "floorRule")
imp.Ext, _ = jsonparser.Set(imp.Ext, []byte(fmt.Sprintf("%.4f", floorVal)), "prebid", "floors", "floorRuleValue")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to change this to use the standard Go json library. We've tried to use jsonparser.Setbefore and it seg faulted on us in certain cases.

wt2 += 1 << (segNum - comb[j][k])
}
return wt1 < wt2
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bsardo what is your opinion?

delete(ruleValues, key)
continue
}
delete(ruleValues, key)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since delete will always happen, should it be moved up before the if statement?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

errs = append(errs, fmt.Errorf("Invalid SkipRate at root level = '%v'", floorExt.SkipRate))
}
return errs
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validateFloorSkipRates will only return a single error. I recommend to change the return type to the singular error and the overall flow to:

if ..
  return fmt.Errorf(..))

if ...
  return fmt.Errorf(..))

return nil

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like we're still returning a slice of errors. If the intention is to return a single error I agree with @SyntaxNode's recommendation.
If the intention is to return multiple errors then the first return nested inside the first if block should be deleted.

@SyntaxNode
Copy link
Contributor

We recently fixed a security issue with a dependency. Please merge from master to resolve.

@PubMatic-OpenWrap
Copy link
Contributor Author

Thank you for the updates and your patience for a re-review. Looking a lot better IMHO after the changes. I'm in the middle of reviewing tests and comparing the expected behavior against the spec. I hope to finish that mid next week.

Thank you for reviewing changes,
Let us know once done with test case review so that can push all the changes in single commit

@PubMatic-OpenWrap
Copy link
Contributor Author

PubMatic-OpenWrap commented Sep 26, 2022

Hello @pm-jaydeep-mohite, thanks for contributing! Just for future reference, if you don't mind, please merge with master instead of force pushing after your PR has been assigned for review. Force pushing makes it hard to figure out what has changed since the initial review which leads to a lot of extra time spent by our team on the delta reviews, especially for larger PRs. 🙂

Force pushing is done to commit changes after rebasing feature branch with latest master branch, which is done to view feature changes commit by commit on top of latest master.

If this approach is not feasible then we can merge changes from master.
@bsardo Please let me know your opinion.

@bsardo
Copy link
Collaborator

bsardo commented Sep 26, 2022

Hi @PubMatic-OpenWrap, we're familiar with rebasing and recommend using it when working on a PR in draft mode. However, once the review process begins, we strongly prefer merging with master because it keeps the PR history in tact and makes it much easier for reviewers to identify what has changed since their last review. This is especially important with larger PRs. As the PR get closer to its final state, the last thing we want to do is have to re-review 3k lines of code because of a force push when we may only be expecting a few small changes based on recent comments that can easily be observed with a merge with master. I hope this helps.

Copy link
Collaborator

@bsardo bsardo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @PubMatic-OpenWrap,
I left a bunch of comments mostly around meeting the requirements. I'd also like to direct your attention to one comment in particular that involves the design. Please see my comment about O(2^N) complexity. I believe this should probably be resolved before making some of the other changes to the signaling logic.

Also it seems like you might have made some changes locally and forgot to push them to the PR based on your answers to some of @SyntaxNode's comments. You'll need to merge with master as well to resolve some recently introduced conflicts.

I still need to give the enforcement piece a closer look. I will do that shortly but I wanted to share this other feedback with you in the interim.

Thanks!

return errs
}
prebidExt := requestExt.GetPrebid()
if floor.Enabled && prebidExt.Floors != nil && prebidExt.Floors.GetEnabled() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The requirements indicate that floors config can be specified at the host, account and request level. I don't see any account config changes in this PR. Is that something you were planning to add in a future PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes are done, will be pushed in next commit

// PriceFloorRules defines the contract for bidrequest.ext.prebid.floors
type PriceFloorRules struct {
FloorMin float64 `json:"floormin,omitempty"`
FloorMinCur string `json:"floormincur,omitempty"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this supposed to be float64 according to v2 of the schema?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

according to v2 of schema float data type is mentioned,
should we use float32 or float64 ?

FloorMin float64 `json:"floormin,omitempty"`
FloorMinCur string `json:"floormincur,omitempty"`
SkipRate int `json:"skiprate,omitempty"`
Location *PriceFloorEndpoint `json:"location,omitempty"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is listed as a string with possible values ‘request’, ‘fetch’ or ‘noData’.

Comment on lines 43 to 45
EnforcePBS *bool `json:"enforcepbs,omitempty"`
FloorDeals *bool `json:"floordeals,omitempty"`
BidAdjustment bool `json:"bidadjustment,omitempty"`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is BidAdjustment not a pointer like the others? I see the default value is true while it is false for the pointer fields.

type PriceFloorModelGroup struct {
Currency string `json:"currency,omitempty"`
ModelWeight int `json:"modelweight,omitempty"`
DebugWeight int `json:"debugweight,omitempty"` // Added for Debug purpose, shall be removed
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we removing this field?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

floors/rule.go Outdated
Comment on lines 150 to 156
value := CATCH_ALL
if isMobileDevice(request.Device.UA) {
value = Phone
} else if isTabletDevice(request.Device.UA) {
value = Tablet
}
return value
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the requirements, "*" should only be returned if request.Device.UA does not exist so perhaps there should be an early return statement here:

if request.Device == nil || len(request.Device.UA) == 0 {
    return CATCH_ALL
}

That nil check is probably needed anyway otherwise isMobileDevice and isTabletDevice could potentially try to access a field on a nil value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

} else if isTabletDevice(request.Device.UA) {
value = Tablet
}
return value
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The requirements also indicate that if request.Device.UA is present but it is not mobile or tablet then it should return "desktop":

if device.ua is present, resolve deviceType to:

'phone' if UA matches one of these patterns: "Phone", "iPhone", "Android.*Mobile", "Mobile.*Android"
else 'tablet' if UA matches one of these: "tablet", "iPad", "Windows NT.*touch", "touch.*Windows NT", "Android"
else 'desktop'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added desktop

}

func isMobileDevice(userAgent string) bool {
isMobile, err := regexp.MatchString("(?i)Phone|iPhone|Android|Mobile", userAgent)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This regexp doesn't appear to match the requirements, particularly the android patterns:
'phone' if UA matches one of these patterns: "Phone", "iPhone", "Android.*Mobile", "Mobile.*Android"

}

func isTabletDevice(userAgent string) bool {
isTablet, err := regexp.MatchString("(?i)tablet|iPad|Windows NT", userAgent)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This regexp doesn't appear to match the requirements, particularly the Windows NT and Android patterns:
else 'tablet' if UA matches one of these: "tablet", "iPad", "Windows NT.*touch", "touch.*Windows NT", "Android"

Comment on lines +255 to +257
if gptSlot != "" {
value = gptSlot
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are we supposed to return as the implied else condition here if gptSlot is an empty string? Currently we are returning an empty string but is that correct? I see that getpbsadslot(imp) returns "*" so should we be doing the same here?
I see that most of these functions will default to returning "*" which it appears you're doing so these values would at least match the wildcard in the generated rule combinations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If gptSlot is an empty string then we are returning "*"

@bsardo bsardo assigned AlexBVolcy and unassigned SyntaxNode Feb 1, 2023
@PubMatic-OpenWrap PubMatic-OpenWrap changed the title floors feature implementation [Draft] floors feature implementation Feb 7, 2023
@bsardo bsardo marked this pull request as draft February 7, 2023 18:13
@bsardo
Copy link
Collaborator

bsardo commented Jun 5, 2023

Closing. This PR is no longer needed since the logic is covered by a series of other floors PRs.

@bsardo bsardo closed this Jun 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants