Merge branch 'master' into update-currency-service-to-accept-bidrequest

# Conflicts: # src/main/java/org/prebid/server/auction/ExchangeService.java # src/test/java/org/prebid/server/auction/CurrencyConversionServiceTest.java # src/test/java/org/prebid/server/auction/ExchangeServiceTest.java
prebid · Jan 15, 2021 · d0da74c · d0da74c
2 parents 2fb0215 + 220481c
commit d0da74c
Show file tree

Hide file tree

Showing 809 changed files with 21,714 additions and 8,136 deletions.
diff --git a/.github/workflows/release-drafter.yml b/.github/workflows/release-drafter.yml
@@ -14,7 +14,7 @@ jobs:
       - name: Extract tag from commit message
         run: |
           target_tag=${COMMIT_MSG#"Prebid Server prepare release "}
-          echo "::set-env name=TARGET_TAG::$target_tag"
+          echo "TARGET_TAG=$target_tag" >> $GITHUB_ENV
         env:
           COMMIT_MSG: ${{ github.event.head_commit.message }}
       - name: Create and publish release

diff --git a/README.md b/README.md
@@ -1,50 +1,48 @@
-### This code is being used in production by multiple Prebid.org members, but is not the "official" version. See https://github.com/prebid/prebid-server/
+### This is the Java version of Prebid Server. See the Prebid Server [Feature List](https://docs.prebid.org/prebid-server/features/pbs-feature-idx.html) and [FAQ entry](https://docs.prebid.org/faq/prebid-server-faq.html#why-are-there-two-versions-of-prebid-server-are-they-kept-in-sync) to understand the differences between PBS-Java and [PBS-Go](https://github.com/prebid/prebid-server).
 
-# Prebid Server
+# Prebid Server (Java)
 
-[![GitHub version](https://badge.fury.io/gh/rubicon-project%2fprebid-server-java.svg)](http://badge.fury.io/gh/rubicon-project%2fprebid-server-java)
-[![Language grade: Java](https://img.shields.io/lgtm/grade/java/g/rubicon-project/prebid-server-java.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/rubicon-project/prebid-server-java/context:java)
-[![Total alerts](https://img.shields.io/lgtm/alerts/g/rubicon-project/prebid-server-java.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/rubicon-project/prebid-server-java/alerts/)
-[![GitHub contributors](https://img.shields.io/github/contributors/rubicon-project/prebid-server-java.svg)](https://GitHub.com/rubicon-project/prebid-server-java/contributors/)
-[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/rubicon-project/prebid-server-java/blob/master/docs/contributing.md) 
-[![GitHub pull-requests closed](https://img.shields.io/github/issues-pr-closed/rubicon-project/prebid-server-java.svg)](https://GitHub.com/rubicon-project/prebid-server-java/pull/)
+[![GitHub version](https://badge.fury.io/gh/prebid%2fprebid-server-java.svg)](http://badge.fury.io/gh/prebid%2fprebid-server-java)
+[![Language grade: Java](https://img.shields.io/lgtm/grade/java/g/prebid/prebid-server-java.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/prebid/prebid-server-java/context:java)
+[![Total alerts](https://img.shields.io/lgtm/alerts/g/prebid/prebid-server-java.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/prebid/prebid-server-java/alerts/)
+[![GitHub contributors](https://img.shields.io/github/contributors/prebid/prebid-server-java.svg)](https://GitHub.com/prebid/prebid-server-java/contributors/)
+[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/prebid/prebid-server-java/blob/master/docs/contributing.md) 
+[![GitHub pull-requests closed](https://img.shields.io/github/issues-pr-closed/prebid/prebid-server-java.svg)](https://GitHub.com/prebid/prebid-server-java/pull/)
 
 Prebid Server is an open source implementation of Server-Side Header Bidding.
-It is managed by [Prebid.org](http://prebid.org/overview/what-is-prebid-org.html),
-and upholds the principles from the [Prebid Code of Conduct](http://prebid.org/wrapper_code_of_conduct.html).
+It is managed by Prebid.org,
+and upholds the principles from the [Prebid Code of Conduct](https://prebid.org/wrapper_code_of_conduct.html).
 
 This project does not support the same set of Bidders as Prebid.js, although there is overlap.
 The current set can be found in the [adapters](./src/main/java/org/prebid/server/bidder) package. If you don't see the one you want, feel free to [contribute it](docs/developers/add-new-bidder.md).
 
 For more information, see:
 
-- [What is Prebid?](http://prebid.org/overview/intro.html)
-- [Getting started with Prebid Server](http://prebid.org/dev-docs/get-started-with-prebid-server.html)
-- [Current Bidders](http://prebid.org/dev-docs/prebid-server-bidders.html)
+- [What is Prebid?](https://prebid.org/why-prebid/)
+- [Getting started with Prebid Server](https://docs.prebid.org/prebid-server/overview/prebid-server-overview.html)
+- [Current Bidders](https://docs.prebid.org/dev-docs/pbs-bidders.html)
 
 # Getting Started
 
 The server makes the following assumptions:
 - No ranking or decisioning is performed by this server. It just proxies requests.
 - No ad quality management (e.g., malware, viruses, deceptive creatives) is performed by this server.
 - This server does no fraud scanning and does nothing to prevent bad traffic.
-- This server does no logging.
-- This server has not user profiling or user data collection capabilities.
+- This server logs errors but not requests.
+- This server has no user profiling or user data collection capabilities.
 
 This project is built upon [Vert.x](http://vertx.io) to achieve high request throughput. 
 We use [Maven](https://maven.apache.org) and attempt to introduce minimal dependencies.
 
 When running, the server responds to several HTTP [endpoints](docs/endpoints).
 
-To start the Prebid Server you need to do the following steps:
-
 ## Building
 
 Follow next steps to create JAR file which can be deployed locally.
 
 - Download or clone a project:
 ```bash
-git clone https://github.com/rubicon-project/prebid-server-java.git
+git clone https://github.com/prebid/prebid-server-java.git
 ```
 
 - Move to project directory:
@@ -84,28 +82,27 @@ and verify response status is `200 OK`.
 # Documentation
 
 ## Development
-- [Differences Between Prebid Server Go and Java](differenceBetweenPBSGo-and-Java.md)
-- [Endpoints](endpoints)
-- [Adding new bidder](developers/add-new-bidder.md)
-- [Adding new analytics module](developers/add-new-analytics-module.md)
-- [Adding viewability support](developers/add-viewability-vendors.md)
-- [Auction result post-processing](developers/auction-result-post-processing.md)
-- [Cookie Syncs](developers/cookie-syncs.md)
-- [Stored Requests](developers/stored-requests.md)
-- [Unit Tests](developers/unit-tests.md)
-- [GDPR](developers/gdpr.md)
+- [Endpoints](docs/endpoints)
+- [Adding new bidder](docs/developers/add-new-bidder.md)
+- [Adding new analytics module](docs/developers/add-new-analytics-module.md)
+- [Adding viewability support](docs/developers/add-viewability-vendors.md)
+- [Auction result post-processing](docs/developers/auction-result-post-processing.md)
+- [Cookie Syncs](docs/developers/cookie-syncs.md)
+- [Stored Requests](docs/developers/stored-requests.md)
+- [Unit Tests](docs/developers/unit-tests.md)
+- [GDPR](docs/developers/gdpr.md)
 
 ## Maintenance
-- [Build for local](build.md)
-- [Build for AWS](build-aws.md)
-- [Configure application](config.md)
-  - [Full list of configuration options](config-app.md)
-  - [Application settings](application-settings.md)
-- [Run with optimizations](run.md)
-- [Metrics](metrics.md)
+- [Build for local](docs/build.md)
+- [Build for AWS](docs/build-aws.md)
+- [Configure application](docs/config.md)
+  - [Full list of configuration options](docs/config-app.md)
+  - [Application settings](docs/application-settings.md)
+- [Run with optimizations](docs/run.md)
+- [Metrics](docs/metrics.md)
 
 ## Contributing
-- [Contributing](contributing.md)
-- [Code Style](code-style.md)
-- [Code Review](code-reviews.md)
-- [Versioning](versioning.md)
+- [Contributing](docs/contributing.md)
+- [Code Style](docs/code-style.md)
+- [Code Review](docs/code-reviews.md)
+- [Versioning](docs/versioning.md)
diff --git a/docs/application-settings.md b/docs/application-settings.md
@@ -23,14 +23,17 @@ There are two ways to configure application settings: database and file. This do
 - `truncate-target-attr` - Maximum targeting attributes size. Values between 1 and 255.
 - `default-integration` - Default integration to assume.
 - `analytics-config.auction-events.<channel>` - defines which channels are supported by analytics for this account
+- `bid-validations.banner-creative-max-size` - Overrides creative max size validation for banners.
+- `status` - allows to mark account as `active` or `inactive`.
 
+Here are the definitions of the "purposes" that can be defined in the GDPR setting configurations:
 ```
 Purpose   | Purpose goal                    | Purpose meaning for PBS (n\a - not affected)  
 ----------|---------------------------------|---------------------------------------------
 p1        | Access device                   | Stops usersync for given vendor and stops settings cookie on `/seuid`
 p2        | Select basic ads                | Verify consent for each vendor as appropriate for the enforcement method before calling a bid adapter. If consent is not granted, log a metric and skip it.
 p3        | Personalized ads profile        | n\a
-p4        | Select personalized ads         | Verify consent for each vendor that passed the Purpose 2. If consent is not granted, remove the bidrequest.userId, user.ext.eids, user.ext.digitrust, device.if attributes and call the adapter.
+p4        | Select personalized ads         | Verify consent for each vendor that passed the Purpose 2. If consent is not granted, remove the bidrequest.userId, user.ext.eids, device.if attributes and call the adapter.
 p5        | Personalized content profile    | n\a
 p6        | Select personalized content     | n\a
 p7        | Measure ad performance          | Verify consent for each analytics module. If consent is not grantet skip it.
@@ -42,22 +45,26 @@ sf1       | Precise geo                     | Verifies user opt-in. If the user
 sf2       | Fingerprinting                  | n\a
 ```
 
-## File application setting
+## Setting Account Configuration in Files
 
 In file based approach all configuration stores in .yaml files, path to which are defined in application properties.
 
 ### Configuration in application.yaml
 
-```
+The general idea is that you'll place all the account-specific settings in a separate YAML file and point to that file.
+
+```yaml
 settings:
   filesystem:
     settings-filename: <directory to yaml file with settings>
 ```
 ### File format
 
-```
+Here's an example YAML file containing account-specific settings:
+
+```yaml
 accounts:
-  - id: 14062
+  - id: 1111
     bannerCacheTtl: 100
     videoCacheTtl: 100
     eventsEnabled: true
@@ -69,6 +76,7 @@ accounts:
     analytics-config:
       auction-events:
         amp: true
+    status: active
     gdpr:
       enabled: true
       integration-enabled:
@@ -151,63 +159,79 @@ accounts:
       purpose-one-treatment-interpretation: ignore
 ```
 
-
-## Database application setting
+## Setting Account Configuration in the Database
+
+In database approach account properties are stored in database table.
 
-In database approach account properties are stored in database table with name accounts_account.
+SQL query for retrieving account is configurable and can be specified in [application configuration](config-app.md). 
+Requirements for the SQL query stated below.
 
 ### Configuration in application.yaml
-```
+
+```yaml
 settings:
   database:
-    type: <mysql or postgres>
     pool-size: 20
-    type: mysql
+    type: <mysql or postgres>
     host: <host>
     port: <port>
+    account-query: <SQL query for account>
 ```
 
-### Table description 
+### Configurable SQL query for account requirements
 
-Query to create accounts_account table:
+The general approach is that each host company can set up their database however they wish, so long as the configurable query run by
+Prebid Server returns expected data in the expected order. Here's an example configuration:
 
+```yaml
+settings:
+  database:
+    type: mysql
+    account-query: SELECT uuid, price_granularity, banner_cache_ttl, video_cache_ttl, events_enabled, enforce_ccpa, tcf_config, analytics_sampling_factor, truncate_target_attr, default_integration, analytics_config, bid_validations FROM accounts_account where uuid = ? LIMIT 1
 ```
-'CREATE TABLE `accounts_account` (
-`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
-`uuid` varchar(40) NOT NULL,
-`price_granularity` enum('low','med','high','auto','dense','unknown') NOT NULL DEFAULT 'unknown',
-`granularityMultiplier` decimal(9,3) DEFAULT NULL,
-`banner_cache_ttl` int(11) DEFAULT NULL,
-`video_cache_ttl` int(11) DEFAULT NULL,
-`events_enabled` bit(1) DEFAULT NULL,
-`enforce_ccpa` bit(1) DEFAULT NULL,
-`enforce_gdpr` bit(1) DEFAULT NULL,
-`tcf_config` json DEFAULT NULL,
-`analytics_sampling_factor` tinyint(4) DEFAULT NULL,
-`truncate_target_attr` tinyint(3) unsigned DEFAULT NULL,
-`default_integration` varchar(64) DEFAULT NULL,
-`analytics_config` varchar(512) DEFAULT NULL,
-`status` enum('active','inactive') DEFAULT 'active',
-`updated_by` int(11) DEFAULT NULL,
-`updated_by_user` varchar(64) DEFAULT NULL,
-`updated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
-PRIMARY KEY (`id`),
-UNIQUE KEY `uuid` (`uuid`))
-ENGINE=InnoDB DEFAULT CHARSET=utf8'
-```
 
-where tcf_config column is json with next format
+The SQL query for account must:
+* return following columns, with specified type, in this order:
+    * account ID, string
+    * price granularity, string
+    * banner cache TTL, integer
+    * video cache TTL, integer
+    * events enabled flag, boolean
+    * enforce CCPA flag, boolean
+    * TCF configuration, JSON string, see below
+    * analytics sampling factor, integer
+    * maximum targeting attribute size, integer
+    * default integration value, string
+    * analytics configuration, JSON string, see below
+* specify a special single `%ACCOUNT_ID%` placeholder in the `WHERE` clause that will be replaced with account ID in 
+runtime
+
+It is recommended to include `LIMIT 1` clause in the query because only the very first result returned will be taken.
+
+If a host company doesn't support a given field, or they have a different table name, they can just update the query with whatever values are needed. e.g.
 
+```yaml
+settings:
+  database:
+    type: mysql
+    account-query: SELECT uuid, 'med', banner_cache_ttl, video_cache_ttl, events_enabled, enforce_ccpa, tcf_config, 0, null, default_integration, '{}', '{}' FROM myaccountstable where uuid = ? LIMIT 1
 ```
+### Configuration Details
+
+#### TCF configuration JSON
+
+Here's an example of the value that the `tcf_config` column can take:
+
+```json
 {
   "enabled": true,
    "integration-enabled": {
       "video": true,
       "web": true,
       "app": true,
       "amp": true
-   }
-  "purpose-one-treatment-interpretation": "ignore"
+   },
+  "purpose-one-treatment-interpretation": "ignore",
   "purposes": {
     "p1": {
       "enforce-purpose": "full",
@@ -309,10 +333,62 @@ where tcf_config column is json with next format
 }
 ```
 
-Query used to get an account:
+#### Bid Validations configuration JSON
+
+The `bid_validations` column is json with this format:
+
+```json
+{
+  "banner-creative-max-size": "enforce"
+}
+```
+
+Valid values are:
+- "skip": don't do anything about creative max size for this publisher
+- "warn": if a bidder returns a creative that's larger in height or width than any of the allowed sizes, log an operational warning.
+- "enforce": if a bidder returns a creative that's larger in height or width than any of the allowed sizes, reject the bid and log an operational warning.
+
+#### Analytics Validations configuration JSON
+
+The `analytics_config`  configuration column format:
+
+```json
+{
+  "auction-events": {
+    "web": true,   // the analytics adapter should log auction events when the channel is web
+    "amp": true,   // the analytics adapter should log auction events when the channel is AMP
+    "app": false   // the analytics adapter should not log auction events when the channel is app
+  }
+}
 ```
-SELECT uuid, price_granularity, banner_cache_ttl, video_cache_ttl, events_enabled, enforce_ccpa, tcf_config, analytics_sampling_factor, truncate_target_attr, default_integration, analytics_config 
-FROM accounts_account where uuid = ?
-LIMIT 1
 
+#### Creating the accounts table
+
+Traditionally the table name used by Prebid Server is `accounts_account`. No one remembers why. But here's SQL 
+you could use to create your table:
+
+```sql
+'CREATE TABLE `accounts_account` (
+    `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
+    `uuid` varchar(40) NOT NULL,
+    `price_granularity` enum('low','med','high','auto','dense','unknown') NOT NULL DEFAULT 'unknown',
+    `granularityMultiplier` decimal(9,3) DEFAULT NULL,
+    `banner_cache_ttl` int(11) DEFAULT NULL,
+    `video_cache_ttl` int(11) DEFAULT NULL,
+    `events_enabled` bit(1) DEFAULT NULL,
+    `enforce_ccpa` bit(1) DEFAULT NULL,
+    `enforce_gdpr` bit(1) DEFAULT NULL,
+    `tcf_config` json DEFAULT NULL,
+    `analytics_sampling_factor` tinyint(4) DEFAULT NULL,
+    `truncate_target_attr` tinyint(3) unsigned DEFAULT NULL,
+    `default_integration` varchar(64) DEFAULT NULL,
+    `analytics_config` varchar(512) DEFAULT NULL,
+    `bid_validations` json DEFAULT NULL,
+    `status` enum('active','inactive') DEFAULT 'active',
+    `updated_by` int(11) DEFAULT NULL,
+    `updated_by_user` varchar(64) DEFAULT NULL,
+    `updated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
+PRIMARY KEY (`id`),
+UNIQUE KEY `uuid` (`uuid`))
+ENGINE=InnoDB DEFAULT CHARSET=utf8'
 ```
diff --git a/docs/build.md b/docs/build.md
@@ -19,7 +19,7 @@ Follow next steps to create JAR which can be deployed locally.
 
 Download or clone a project locally:
 ```bash
-git clone https://github.com/rubicon-project/prebid-server-java.git
+git clone https://github.com/prebid/prebid-server-java.git
 ```
 
 Move to project directory:

diff --git a/docs/code-reviews.md b/docs/code-reviews.md
@@ -1,7 +1,7 @@
 # Code Reviews
 
 ## Standards
-Anyone is free to review and comment on any [open pull requests](https://github.com/rubicon-project/prebid-server-java/pulls).
+Anyone is free to review and comment on any [open pull requests](https://github.com/prebid/prebid-server-java/pulls).
 
 All pull requests must be reviewed and approved by at least one [core member](https://github.com/orgs/prebid/teams/core/members) before merge.
 
@@ -38,7 +38,7 @@ Some examples include:
 - Can we improve the user's experience in any way?
 - Have the relevant [docs]() been added or updated? If not, add the `needs docs` label.
 - Do you believe that the code works by looking at the unit tests? If not, suggest more tests until you do!
-- Is the motivation behind these changes clear? If not, there must be [an issue](https://github.com/rubicon-project/prebid-server-java/issues) 
+- Is the motivation behind these changes clear? If not, there must be [an issue](https://github.com/prebid/prebid-server-java/issues) 
 explaining it. Are there better ways to achieve those goals?
 - Does the code use any global, mutable state? [Inject dependencies](https://en.wikipedia.org/wiki/Dependency_injection) instead!
 - Can the code be organized into smaller, more modular pieces?