Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for account specific configuration of /cookie_sync endpoint #1115

Merged
merged 6 commits into from
Mar 15, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 28 additions & 3 deletions docs/application-settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ There are two ways to configure application settings: database and file. This do
- `analytics-config.auction-events.<channel>` - defines which channels are supported by analytics for this account
- `bid-validations.banner-creative-max-size` - Overrides creative max size validation for banners.
- `status` - allows to mark account as `active` or `inactive`.
- `cookie-sync.default-limit` - if the "limit" isn't specified in the `/cookie_sync` request, this is what to use
- `cookie-sync.max-limit` - if the "limit" is specified in the `/cookie_sync` request, it can't be greater than this value
- `cookie-sync.default-coop-sync` - if the "coopSync" value isn't specified in the `/cookie_sync` request, use this

Here are the definitions of the "purposes" that can be defined in the GDPR setting configurations:
```
Expand Down Expand Up @@ -77,6 +80,10 @@ accounts:
auction-events:
amp: true
status: active
cookie-sync:
default-limit: 5
max-limit: 8
default-coop-sync: true
gdpr:
enabled: true
integration-enabled:
Expand Down Expand Up @@ -187,7 +194,7 @@ Prebid Server returns expected data in the expected order. Here's an example con
settings:
database:
type: mysql
account-query: SELECT uuid, price_granularity, banner_cache_ttl, video_cache_ttl, events_enabled, enforce_ccpa, tcf_config, analytics_sampling_factor, truncate_target_attr, default_integration, analytics_config, bid_validations, status FROM accounts_account where uuid = ? LIMIT 1
account-query: SELECT uuid, price_granularity, banner_cache_ttl, video_cache_ttl, events_enabled, enforce_ccpa, tcf_config, analytics_sampling_factor, truncate_target_attr, default_integration, analytics_config, bid_validations, status, config FROM accounts_account where uuid = ? LIMIT 1
```

The SQL query for account must:
Expand All @@ -203,7 +210,9 @@ The SQL query for account must:
* maximum targeting attribute size, integer
* default integration value, string
* analytics configuration, JSON string, see below
* bid validations configuration, JSON string, see below
* status, string. Expected values: "active", "inactive", NULL. Only "inactive" has any effect and only when settings.enforce-valid-account is on.
* consolidated configuration, JSON string, see below
* specify a special single `%ACCOUNT_ID%` placeholder in the `WHERE` clause that will be replaced with account ID in
runtime

Expand All @@ -215,7 +224,7 @@ If a host company doesn't support a given field, or they have a different table
settings:
database:
type: mysql
account-query: SELECT uuid, 'med', banner_cache_ttl, video_cache_ttl, events_enabled, enforce_ccpa, tcf_config, 0, null, default_integration, '{}', '{}' FROM myaccountstable where uuid = ? LIMIT 1
account-query: SELECT uuid, 'med', banner_cache_ttl, video_cache_ttl, events_enabled, enforce_ccpa, tcf_config, 0, null, default_integration, '{}', '{}', status, '{}' FROM myaccountstable where uuid = ? LIMIT 1
```
### Configuration Details

Expand Down Expand Up @@ -351,7 +360,7 @@ Valid values are:

#### Analytics Validations configuration JSON

The `analytics_config` configuration column format:
The `analytics_config` configuration column format:

```json
{
Expand All @@ -363,6 +372,21 @@ The `analytics_config` configuration column format:
}
```

#### Consolidated configuration JSON

The `config` column is envisioned as a new home for all configuration values. All new configuration values are added here.
The schema of this JSON document has following format so far:

```json
{
"cookie-sync": {
"default-limit": 5,
"max-limit": 8,
"default-coop-sync": true
}
}
```

#### Creating the accounts table

Traditionally the table name used by Prebid Server is `accounts_account`. No one remembers why. But here's SQL
Expand All @@ -386,6 +410,7 @@ you could use to create your table:
`analytics_config` varchar(512) DEFAULT NULL,
`bid_validations` json DEFAULT NULL,
`status` enum('active','inactive') DEFAULT 'active',
`config` json DEFAULT NULL,
`updated_by` int(11) DEFAULT NULL,
`updated_by_user` varchar(64) DEFAULT NULL,
`updated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
Expand Down
83 changes: 61 additions & 22 deletions src/main/java/org/prebid/server/handler/CookieSyncHandler.java
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
import org.prebid.server.proto.response.UsersyncInfo;
import org.prebid.server.settings.ApplicationSettings;
import org.prebid.server.settings.model.Account;
import org.prebid.server.settings.model.AccountCookieSyncConfig;
import org.prebid.server.settings.model.AccountGdprConfig;
import org.prebid.server.util.HttpUtil;

Expand Down Expand Up @@ -76,7 +77,7 @@ public class CookieSyncHandler implements Handler<RoutingContext> {
private final TcfDefinerService tcfDefinerService;
private final PrivacyEnforcementService privacyEnforcementService;
private final Integer gdprHostVendorId;
private final boolean defaultCoopSync;
private final Boolean defaultCoopSync;
private final List<Collection<String>> listOfCoopSyncBidders;
private final Set<String> setOfCoopSyncBidders;
private final AnalyticsReporterDelegator analyticsDelegator;
Expand Down Expand Up @@ -189,11 +190,12 @@ private Future<Account> accountById(String accountId, Timeout timeout) {
return StringUtils.isBlank(accountId)
? Future.succeededFuture(Account.empty(accountId))
: applicationSettings.getAccountById(accountId, timeout)
.otherwise(Account.empty(accountId));
.otherwise(Account.empty(accountId));
}

private void handleCookieSyncContextResult(AsyncResult<CookieSyncContext> cookieSyncContextResult,
RoutingContext routingContext) {

if (cookieSyncContextResult.succeeded()) {
final CookieSyncContext cookieSyncContext = cookieSyncContextResult.result();

Expand All @@ -204,12 +206,11 @@ private void handleCookieSyncContextResult(AsyncResult<CookieSyncContext> cookie
return;
}

final CookieSyncRequest cookieSyncRequest = cookieSyncContext.getCookieSyncRequest();
final Set<String> biddersToSync = biddersToSync(cookieSyncRequest);

isAllowedForHostVendorId(tcfContext)
.setHandler(hostTcfResponseResult ->
respondByTcfResponse(hostTcfResponseResult, biddersToSync, cookieSyncContext));
.setHandler(hostTcfResponseResult -> respondByTcfResponse(
hostTcfResponseResult,
biddersToSync(cookieSyncContext),
cookieSyncContext));
} else {
final Throwable error = cookieSyncContextResult.cause();
handleErrors(error, routingContext, null);
Expand Down Expand Up @@ -239,33 +240,50 @@ private static boolean isGdprParamsNotConsistent(CookieSyncRequest request) {
* <p>
* If bidder list was omitted in request, that means sync should be done for all bidders.
*/
private Set<String> biddersToSync(CookieSyncRequest cookieSyncRequest) {
private Set<String> biddersToSync(CookieSyncContext cookieSyncContext) {
final CookieSyncRequest cookieSyncRequest = cookieSyncContext.getCookieSyncRequest();

final List<String> requestBidders = cookieSyncRequest.getBidders();

if (CollectionUtils.isEmpty(requestBidders)) {
return activeBidders;
}

final Boolean requestCoopSync = cookieSyncRequest.getCoopSync();
final boolean coop = requestCoopSync != null ? requestCoopSync : defaultCoopSync;
final Account account = cookieSyncContext.getAccount();

if (coopSyncAllowed(cookieSyncRequest, account)) {
final Integer requestLimit = resolveLimit(cookieSyncContext);

if (coop) {
final Integer limit = cookieSyncRequest.getLimit();
return limit == null
return requestLimit == null
? addAllCoopSyncBidders(requestBidders)
: addCoopSyncBidders(requestBidders, limit);
: addCoopSyncBidders(requestBidders, requestLimit);
}

return new HashSet<>(requestBidders);
}

private Boolean coopSyncAllowed(CookieSyncRequest cookieSyncRequest, Account account) {
final Boolean requestCoopSync = cookieSyncRequest.getCoopSync();
if (requestCoopSync != null) {
return requestCoopSync;
}

final AccountCookieSyncConfig accountCookieSyncConfig = account.getCookieSync();
final Boolean accountCoopSync = accountCookieSyncConfig != null
? accountCookieSyncConfig.getDefaultCoopSync()
: null;

return ObjectUtils.firstNonNull(accountCoopSync, defaultCoopSync);
}

/**
* If host vendor id is null, host allowed to sync cookies.
*/
private Future<HostVendorTcfResponse> isAllowedForHostVendorId(TcfContext tcfContext) {
return gdprHostVendorId == null
? Future.succeededFuture(HostVendorTcfResponse.allowedVendor())
: tcfDefinerService.resultForVendorIds(Collections.singleton(gdprHostVendorId), tcfContext)
.map(this::toHostVendorTcfResponse);
.map(this::toHostVendorTcfResponse);
}

private HostVendorTcfResponse toHostVendorTcfResponse(TcfResponse<Integer> tcfResponse) {
Expand Down Expand Up @@ -327,6 +345,7 @@ private String bidderNameFor(String bidder) {
private void respondByTcfResponse(AsyncResult<HostVendorTcfResponse> hostTcfResponseResult,
Set<String> biddersToSync,
CookieSyncContext cookieSyncContext) {

final TcfContext tcfContext = cookieSyncContext.getPrivacyContext().getTcfContext();
if (hostTcfResponseResult.succeeded()) {

Expand Down Expand Up @@ -406,6 +425,7 @@ private Set<String> extractCcpaEnforcedBidders(Account account, Collection<Strin
private void respondWithRejectedBidders(CookieSyncContext cookieSyncContext,
Collection<String> bidders,
RejectedBidders rejectedBidders) {

updateCookieSyncTcfMetrics(bidders, rejectedBidders.getRejectedByTcf());

final RoutingContext routingContext = cookieSyncContext.getRoutingContext();
Expand All @@ -418,10 +438,10 @@ private void respondWithRejectedBidders(CookieSyncContext cookieSyncContext,

updateCookieSyncMatchMetrics(bidders, bidderStatuses);

final CookieSyncRequest cookieSyncRequest = cookieSyncContext.getCookieSyncRequest();
final Integer limit = cookieSyncRequest.getLimit();
final List<BidderUsersyncStatus> updatedBidderStatuses = trimBiddersToLimit(limit, bidderStatuses);
final List<BidderUsersyncStatus> updatedBidderStatuses =
trimBiddersToLimit(bidderStatuses, resolveLimit(cookieSyncContext));
final String status = uidsCookie.hasLiveUids() ? "ok" : "no_cookie";

final CookieSyncResponse response = CookieSyncResponse.of(status, updatedBidderStatuses);

final String body = mapper.encode(response);
Expand Down Expand Up @@ -548,14 +568,33 @@ private void updateCookieSyncMatchMetrics(Collection<String> syncBidders,
.forEach(metrics::updateCookieSyncMatchesMetric);
}

private static List<BidderUsersyncStatus> trimBiddersToLimit(Integer limit,
List<BidderUsersyncStatus> bidderStatuses) {
private static Integer resolveLimit(CookieSyncContext cookieSyncContext) {
final Integer limit = cookieSyncContext.getCookieSyncRequest().getLimit();

final AccountCookieSyncConfig cookieSyncConfig = cookieSyncContext.getAccount().getCookieSync();
if (cookieSyncConfig == null) {
return limit;
}

final Integer resolvedLimit = ObjectUtils.defaultIfNull(limit, cookieSyncConfig.getDefaultLimit());
if (resolvedLimit == null) {
return null;
}

final Integer maxLimit = cookieSyncConfig.getMaxLimit();

return maxLimit == null ? resolvedLimit : Math.min(resolvedLimit, maxLimit);
}

private static List<BidderUsersyncStatus> trimBiddersToLimit(List<BidderUsersyncStatus> bidderStatuses,
Integer limit) {

if (limit != null && limit > 0 && limit < bidderStatuses.size()) {
Collections.shuffle(bidderStatuses);
return bidderStatuses.subList(0, limit);
} else {
return bidderStatuses;
}

return bidderStatuses;
}

private void handleErrors(Throwable error, RoutingContext routingContext, TcfContext tcfContext) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ public Future<Account> getAccountById(String accountId, Timeout timeout) {
return jdbcClient.executeQuery(
selectAccountQuery,
Collections.singletonList(accountId),
result -> mapToModelOrError(result, row -> Account.builder()
result -> mapToModelOrError(result, row -> partialAccountBuilderFrom(row.getString(13))
.id(row.getString(0))
.priceGranularity(row.getString(1))
.bannerCacheTtl(row.getInteger(2))
Expand Down Expand Up @@ -153,6 +153,12 @@ private static <T> Future<T> failedIfNull(T value, String id, String errorPrefix
: Future.failedFuture(new PreBidException(String.format("%s not found: %s", errorPrefix, id)));
}

private Account.AccountBuilder partialAccountBuilderFrom(String config) {
final Account partialAccount = toModel(config, Account.class);

return partialAccount != null ? partialAccount.toBuilder() : Account.builder();
}

private <T> T toModel(String source, Class<T> targetClass) {
try {
return source != null ? mapper.decodeValue(source, targetClass) : null;
Expand Down
7 changes: 6 additions & 1 deletion src/main/java/org/prebid/server/settings/model/Account.java
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
package org.prebid.server.settings.model;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.Builder;
import lombok.Value;
import org.apache.commons.lang3.ObjectUtils;

@Builder
@Builder(toBuilder = true)
@Value
public class Account {

Expand Down Expand Up @@ -34,6 +35,9 @@ public class Account {

AccountStatus status;

@JsonProperty("cookie-sync")
AccountCookieSyncConfig cookieSync;

public Account merge(Account another) {
return Account.builder()
.id(ObjectUtils.defaultIfNull(id, another.id))
Expand All @@ -50,6 +54,7 @@ public Account merge(Account another) {
.analyticsConfig(ObjectUtils.defaultIfNull(analyticsConfig, another.analyticsConfig))
.bidValidations(ObjectUtils.defaultIfNull(bidValidations, another.bidValidations))
.status(ObjectUtils.defaultIfNull(status, another.status))
.cookieSync(ObjectUtils.defaultIfNull(cookieSync, another.cookieSync))
.build();
}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package org.prebid.server.settings.model;

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.Value;

@Value(staticConstructor = "of")
public class AccountCookieSyncConfig {

@JsonProperty("default-limit")
Integer defaultLimit;

@JsonProperty("max-limit")
Integer maxLimit;

@JsonProperty("default-coop-sync")
Boolean defaultCoopSync;
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
import org.prebid.server.json.JacksonMapper;
import org.prebid.server.settings.model.Account;
import org.prebid.server.settings.model.AccountAnalyticsConfig;
import org.prebid.server.settings.model.AccountBidValidationConfig;
import org.prebid.server.settings.model.AccountCookieSyncConfig;
import org.prebid.server.settings.model.AccountGdprConfig;
import org.prebid.server.settings.model.AccountStatus;

Expand Down Expand Up @@ -32,8 +34,12 @@ public class AccountConfigurationProperties {

private String analyticsConfig;

private String bidValidations;

private AccountStatus status;

private String cookieSync;

public Account toAccount(JacksonMapper mapper) {
return Account.builder()
.priceGranularity(getPriceGranularity())
Expand All @@ -46,7 +52,9 @@ public Account toAccount(JacksonMapper mapper) {
.truncateTargetAttr(getTruncateTargetAttr())
.defaultIntegration(getDefaultIntegration())
.analyticsConfig(toModel(mapper, getAnalyticsConfig(), AccountAnalyticsConfig.class))
.bidValidations(toModel(mapper, getBidValidations(), AccountBidValidationConfig.class))
.status(getStatus())
.cookieSync(toModel(mapper, getCookieSync(), AccountCookieSyncConfig.class))
.build();
}

Expand Down
Loading