Add customized admin endpoints

docs/config-app.md

@@ -100,10 +100,10 @@ There are several typical keys:
 But feel free to add additional bidder's specific options.
 
 ## Admin
-- `logger-level-modifier.enabled` - enable the `/admin` endpoint.
+- `logger-level-modifier.enabled` - enable the `/pbs-admin/admin` endpoint.
 
 ## Currency Converter
-- `currency-converter.external-rates.enabled` - if equals to `true` the currency conversion service will be enabled to fetch updated rates and convert bid currencies from external source. Also enables `/currency-rates` endpoint on admin port.
+- `currency-converter.external-rates.enabled` - if equals to `true` the currency conversion service will be enabled to fetch updated rates and convert bid currencies from external source. Also enables `/pbs-admin/currency-rates` endpoint on admin port.
 - `currency-converter.external-rates.url` - the url for Prebid.org’s currency file. [More details](http://prebid.org/dev-docs/modules/currency.html)
 - `currency-converter.external-rates.default-timeout-ms` - default operation timeout for fetching currency rates.
 - `currency-converter.external-rates.refresh-period-ms` - default refresh period for currency rates updates.
@@ -197,7 +197,7 @@ For caching available next options:
 - `settings.in-memory-cache.ttl-seconds` - how long (in seconds) data will be available in LRU cache.
 - `settings.in-memory-cache.cache-size` - the size of LRU cache.
 - `settings.in-memory-cache.notification-endpoints-enabled` - if equals to `true` two additional endpoints will be
-available: [/storedrequests/openrtb2](endpoints/storedrequests/openrtb2.md) and [/storedrequests/amp](endpoints/storedrequests/amp.md).
+available: [/pbs-admin/storedrequests/openrtb2](endpoints/storedrequests/openrtb2.md) and [/pbs-admin/storedrequests/amp](endpoints/storedrequests/amp.md).
 - `settings.in-memory-cache.http-update.endpoint` - the url to fetch stored request updates.
 - `settings.in-memory-cache.http-update.amp-endpoint` - the url to fetch AMP stored request updates.
 - `settings.in-memory-cache.http-update.refresh-rate` - refresh period in ms for stored request updates.

docs/differenceBetweenPBSGo-and-Java.md

@@ -12,7 +12,7 @@ and not the other for an interim period. This page tracks known differences that
 1) PBS-Java supports Stored Responses [issue 861](https://github.com/prebid/prebid-server/issues/861). PBS-Java [PR 354](https://github.com/rubicon-project/prebid-server-java/pull/354).
 1) PBS-Java supports Currency conversion. PBS-Go has it implemented, but disabled by default(still under dev) [issue 280](https://github.com/prebid/prebid-server/issues/280), [issue 760](https://github.com/prebid/prebid-server/pull/760). PBS-Java [PR 22](https://github.com/rubicon-project/prebid-server-java/pull/22)
 1) PBS-Java Currency conversion supports finding intermediate conversion rate, e.g. if pairs USD : AUD = 1.2 and EUR : AUD = 1.5 are present and EUR to USD conversion is needed, will return (1/1.5) * 1.2 conversion rate.
-1) PBS-Go Currency conversion admin debug endpoint exposes following information: Sync source URL, Internal rates, Update frequency, Last update. PBS-Java `/currency-rates` admin endpoint currently supports checking the latest update time only and is not available if currency conversion is disabled.
+1) PBS-Go Currency conversion admin debug endpoint exposes following information: Sync source URL, Internal rates, Update frequency, Last update. PBS-Java `/pbs-admin/currency-rates` admin endpoint currently supports checking the latest update time only and is not available if currency conversion is disabled.
 1) PBS-Java supports IP-address lookup in certain scenarios around GDPR. See https://github.com/rubicon-project/prebid-server-java/blob/master/docs/developers/PrebidServerJava_GDPR_Requirements.pdf
 1) PBS-Java supports InfluxDB, Graphite and Prometheus, PBS-Go supports InfluxDB and Prometheus as metrics backend.
 1) PBS-Java has Circuit Breaker mechanism for database, http and geolocation requests. This can protect the server in scenarios where an external service becomes unavailable.

docs/endpoints/admin.md

@@ -1,12 +1,14 @@
 # Admin Endpoint
 
-This `/admin` endpoint is bound to `admin.port`.
-
 Unavailable if `logger level modifier` is disabled (`logger-level-modifier.enabled` config property)
 
+This `/pbs-admin/admin` endpoint can be configured:
+ - `admin-endpoints.logger-level-modifier.on-application-port` - when equals to `false` endpoint will be bound to `admin.port`.
+ - `admin-endpoints.logger-level-modifier.protected` - when equals to `true` endpoint will be protected by basic authentication configured in `admin-endpoints.credentials` 
+
 This endpoint will set the desirable logging level and number of logs for `400` responses.
 
 ### Query Params
 
-- `logging`: Desirable logging level: `info`, `warn`, `trace`, `error`, `fatal`, `debug`.
-- `records`: numbers of logs with changed logging level. (0 < n < 100_000)
+- `logging` - Desirable logging level: `info`, `warn`, `trace`, `error`, `fatal`, `debug`.
+- `records` - numbers of logs with changed logging level. (0 < n < 100_000)

docs/endpoints/currency-rates.md

@@ -1,9 +1,11 @@
 # Currency Rates
 
-This /currency-rates endpoint is bound to `admin.port`.
-
 Unavailable if currency conversion is disabled (`currency-converter.external-rates.enabled` config property)
 
+This `/pbs-admin/currency-rates` endpoint can be configured:
+ - `admin-endpoints.currency-rates.on-application-port` - when equals to `false` endpoint will be bound to `admin.port`.
+ - `admin-endpoints.currency-rates.protected` - when equals to `true` endpoint will be protected by basic authentication configured in `admin-endpoints.credentials` 
+
 This endpoint will return a json with the latest update timestamp.
 
 The timestamp will be in the ISO-8601 format, using UTC.

docs/endpoints/storedrequests/amp.md

@@ -1,6 +1,10 @@
 # Amp
 
-This `/storedrequests/amp` endpoint is bound to `admin.port`.
+Unavailable if notification is disabled (`settings.in-memory-cache.notification-endpoints-enabled` config property)
+
+This `/pbs-admin/storedrequests/amp` endpoint can be configured:
+ - `admin-endpoints.storedrequest-amp.on-application-port` - when equals to `false` endpoint will be bound to `admin.port`.
+ - `admin-endpoints.storedrequest-amp.protected` - when equals to `true` endpoint will be protected by basic authentication configured in `admin-endpoints.credentials` 
 
 The goal is to update/invalidate AMP stored request/impression in-memory caches.
 

docs/endpoints/storedrequests/openrtb2.md

@@ -1,6 +1,10 @@
 # Openrtb2
 
-This `/storedrequests/openrtb2` endpoint is bound to `admin.port`.
+Unavailable if notification is disabled (`settings.in-memory-cache.notification-endpoints-enabled` config property)
+
+This `/pbs-admin/storedrequests/openrtb2` endpoint can be configured:
+ - `admin-endpoints.storedrequest.on-application-port` - when equals to `false` endpoint will be bound to `admin.port`.
+ - `admin-endpoints.storedrequest.protected` - when equals to `true` endpoint will be protected by basic authentication configured in `admin-endpoints.credentials` 
 
 The goal is to update/invalidate stored request/impression in-memory caches.
 
@@ -10,7 +14,7 @@ Possible HTTP requests examples described below.
 
 1. Update in-memory cache for specified stored request and stored impression:
 
-`POST /storedrequests/openrtb2`
+`POST /pbs-admin/storedrequests/openrtb2`
 
 ```json
 {
@@ -25,7 +29,7 @@ Possible HTTP requests examples described below.
 
 2. Invalidate in-memory cache for specified stored request and stored impression:
 
-`DELETE /storedrequests/openrtb2`
+`DELETE /pbs-admin/storedrequests/openrtb2`
 
 ```json
 {

docs/pbs-java-and-go-features-review.md

@@ -18,7 +18,7 @@ User ID Module |+|+
 Bid Categories |-|+
 Apps/Accounts Blacklist  |+|+
 Event Notification endpoint |+|-
-Video Auction endpoint |-|+
+Video Auction endpoint |+|+
 Video Impression Tracking endpoint |+|-
 GDPR |+|+
 COPPA |+|+
@@ -27,7 +27,7 @@ Cooperative Cookie Syncing |+|-
 All adapters ported to OpenRTB |+|-
 Remote File Downloader |+|-
 Bidder Generator |+|-
-
+Customized video endpoints |+|-
 
 **
 * PBS-Java Currency conversion supports finding intermediate conversion rate;

pom.xml

@@ -130,6 +130,11 @@
             <artifactId>vertx-dropwizard-metrics</artifactId>
             <version>${vertx.version}</version>
         </dependency>
+        <dependency>
+            <groupId>io.vertx</groupId>
+            <artifactId>vertx-auth-common</artifactId>
+            <version>${vertx.version}</version>
+        </dependency>
         <dependency>
             <groupId>org.projectlombok</groupId>
             <artifactId>lombok</artifactId>

src/main/java/org/prebid/server/handler/CustomizedAdminEndpoint.java

@@ -0,0 +1,80 @@
+package org.prebid.server.handler;
+
+import io.vertx.core.Future;
+import io.vertx.core.Handler;
+import io.vertx.ext.auth.AuthProvider;
+import io.vertx.ext.web.Router;
+import io.vertx.ext.web.RoutingContext;
+import io.vertx.ext.web.handler.BasicAuthHandler;
+import org.apache.commons.collections4.MapUtils;
+import org.apache.commons.lang3.StringUtils;
+
+import java.util.Map;
+import java.util.Objects;
+
+public class CustomizedAdminEndpoint {
+
+    private final Handler<RoutingContext> handler;
+    private final boolean isOnApplicationPort;
+    private final boolean isProtected;
+    private final String path;
+    private Map<String, String> adminEndpointCredentials;
+
+    public CustomizedAdminEndpoint(String path, Handler<RoutingContext> handler, boolean isOnApplicationPort,
+                                   boolean isProtected) {
+        this.path = path;
+        this.handler = handler;
+        this.isOnApplicationPort = isOnApplicationPort;
+        this.isProtected = isProtected;
+    }
+
+    public CustomizedAdminEndpoint(String path, Handler<RoutingContext> handler, boolean isOnApplicationPort,
+                                   boolean isProtected, Map<String, String> adminEndpointCredentials) {
+        this.path = path;
+        this.handler = handler;
+        this.isOnApplicationPort = isOnApplicationPort;
+        this.isProtected = isProtected;
+        this.adminEndpointCredentials = adminEndpointCredentials;
+    }
+
+    public boolean isOnApplicationPort() {
+        return isOnApplicationPort;
+    }
+
+    public void router(Router router) {
+        if (isProtected) {
+            routeSecureToHandler(router);
+        } else {
+            routeToHandler(router);
+        }
+    }
+
+    private void routeToHandler(Router router) {
+        router.route(path).handler(handler);
+    }
+
+    private void routeSecureToHandler(Router router) {
+        if (adminEndpointCredentials == null) {
+            throw new IllegalArgumentException("Credentials for admin endpoint is empty.");
+        }
+        final AuthProvider authProvider = createAuthProvider(adminEndpointCredentials);
+        router.route(path).handler(BasicAuthHandler.create(authProvider)).handler(handler);
+    }
+
+    private AuthProvider createAuthProvider(Map<String, String> credentials) {
+        return (authInfo, resultHandler) -> {
+            if (MapUtils.isEmpty(credentials)) {
+                resultHandler.handle(Future.failedFuture("Credentials not set in configuration."));
+                return;
+            }
+            final String requestUsername = authInfo.getString("username");
+            final String requestPassword = StringUtils.chomp(authInfo.getString("password"));
+            final String storedPassword = credentials.get(requestUsername);
+            if (StringUtils.isNotBlank(requestPassword) && Objects.equals(storedPassword, requestPassword)) {
+                resultHandler.handle(Future.succeededFuture());
+            } else {
+                resultHandler.handle(Future.failedFuture("No such user, or password incorrect."));
+            }
+        };
+    }
+}

src/main/java/org/prebid/server/spring/config/AdminEndpointsConfiguration.java

@@ -0,0 +1,105 @@
+package org.prebid.server.spring.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.prebid.server.currency.CurrencyConversionService;
+import org.prebid.server.execution.LogModifier;
+import org.prebid.server.handler.AdminHandler;
+import org.prebid.server.handler.CurrencyRatesHandler;
+import org.prebid.server.handler.CustomizedAdminEndpoint;
+import org.prebid.server.handler.SettingsCacheNotificationHandler;
+import org.prebid.server.json.JacksonMapper;
+import org.prebid.server.settings.SettingsCache;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.context.annotation.Bean;
+import org.springframework.context.annotation.Configuration;
+import org.springframework.stereotype.Component;
+
+import java.util.Map;
+
+@Configuration
+public class AdminEndpointsConfiguration {
+
+    private static final String LOGGER_LEVEL_ENDPOINT = "/pbs-admin/admin";
+    private static final String CURRENCY_RATES_ENDPOINT = "/pbs-admin/currency-rates";
+    private static final String STOREDREQUESTS_OPENRTB_ENDPOINT = "/pbs-admin/storedrequests/openrtb2";
+    private static final String STOREDREQUESTS_AMP_ENDPOINT = "/pbs-admin/storedrequests/amp";
+
+    @Bean
+    @ConditionalOnProperty(prefix = "logger-level-modifier", name = "enabled", havingValue = "true")
+    CustomizedAdminEndpoint loggerLevelModifierEndpoint(
+            LogModifier logModifier,
+            @Autowired(required = false) AdminEndpointCredentials adminEndpointCredentials,
+            @Value("${admin-endpoints.logger-level-modifier.on-application-port}") boolean isOnApplicationPort,
+            @Value("${admin-endpoints.logger-level-modifier.protected}") boolean isProtected) {
+        final AdminHandler adminHandler = new AdminHandler(logModifier);
+        final Map<String, String> adminEndpointCredentialsMap = adminEndpointCredentials == null
+                ? null
+                : adminEndpointCredentials.getCredentials();
+        return new CustomizedAdminEndpoint(LOGGER_LEVEL_ENDPOINT, adminHandler, isOnApplicationPort,
+                isProtected, adminEndpointCredentialsMap);
+    }
+
+    @Bean
+    @ConditionalOnProperty(prefix = "currency-converter.external-rates", name = "enabled", havingValue = "true")
+    CustomizedAdminEndpoint currencyConversionRatesEndpoint(
+            CurrencyConversionService currencyConversionRates,
+            JacksonMapper mapper,
+            @Autowired(required = false) AdminEndpointCredentials adminEndpointCredentials,
+            @Value("${admin-endpoints.currency-rates.on-application-port}") boolean isOnApplicationPort,
+            @Value("${admin-endpoints.currency-rates.protected}") boolean isProtected) {
+        final CurrencyRatesHandler currencyRatesHandler = new CurrencyRatesHandler(currencyConversionRates, mapper);
+        final Map<String, String> adminEndpointCredentialsMap = adminEndpointCredentials == null
+                ? null
+                : adminEndpointCredentials.getCredentials();
+        return new CustomizedAdminEndpoint(CURRENCY_RATES_ENDPOINT, currencyRatesHandler, isOnApplicationPort,
+                isProtected, adminEndpointCredentialsMap);
+    }
+
+    @Bean
+    @ConditionalOnProperty(prefix = "settings.in-memory-cache", name = "notification-endpoints-enabled",
+            havingValue = "true")
+    CustomizedAdminEndpoint cacheNotificationEndpoint(
+            SettingsCache settingsCache,
+            JacksonMapper mapper,
+            @Autowired(required = false) AdminEndpointCredentials adminEndpointCredentials,
+            @Value("${admin-endpoints.storedrequest.on-application-port}") boolean isOnApplicationPort,
+            @Value("${admin-endpoints.storedrequest.protected}") boolean isProtected) {
+        final SettingsCacheNotificationHandler cacheNotificationHandler =
+                new SettingsCacheNotificationHandler(settingsCache, mapper);
+        final Map<String, String> adminEndpointCredentialsMap = adminEndpointCredentials == null
+                ? null
+                : adminEndpointCredentials.getCredentials();
+        return new CustomizedAdminEndpoint(STOREDREQUESTS_OPENRTB_ENDPOINT, cacheNotificationHandler,
+                isOnApplicationPort, isProtected, adminEndpointCredentialsMap);
+    }
+
+    @Bean
+    @ConditionalOnProperty(prefix = "settings.in-memory-cache", name = "notification-endpoints-enabled",
+            havingValue = "true")
+    CustomizedAdminEndpoint ampCacheNotificationEndpoint(
+            SettingsCache ampSettingsCache,
+            JacksonMapper mapper,
+            @Autowired(required = false) AdminEndpointCredentials adminEndpointCredentials,
+            @Value("${admin-endpoints.storedrequest-amp.on-application-port}") boolean isOnApplicationPort,
+            @Value("${admin-endpoints.storedrequest-amp.protected}") boolean isProtected) {
+        final SettingsCacheNotificationHandler settingsCacheNotificationHandler =
+                new SettingsCacheNotificationHandler(ampSettingsCache, mapper);
+        final Map<String, String> adminEndpointCredentialsMap = adminEndpointCredentials == null
+                ? null
+                : adminEndpointCredentials.getCredentials();
+        return new CustomizedAdminEndpoint(STOREDREQUESTS_AMP_ENDPOINT, settingsCacheNotificationHandler,
+                isOnApplicationPort, isProtected, adminEndpointCredentialsMap);
+    }
+
+    @Component
+    @ConfigurationProperties(prefix = "admin-endpoints")
+    @Data
+    @NoArgsConstructor
+    public static class AdminEndpointCredentials {
+        private Map<String, String> credentials;
+    }
+}

src/main/java/org/prebid/server/spring/config/AdminServerConfiguration.java

@@ -0,0 +1,70 @@
+package org.prebid.server.spring.config;
+
+import io.vertx.core.Vertx;
+import io.vertx.core.http.HttpServer;
+import io.vertx.core.logging.Logger;
+import io.vertx.core.logging.LoggerFactory;
+import io.vertx.ext.web.Router;
+import io.vertx.ext.web.handler.BodyHandler;
+import org.prebid.server.handler.CustomizedAdminEndpoint;
+import org.prebid.server.handler.VersionHandler;
+import org.prebid.server.json.JacksonMapper;
+import org.prebid.server.vertx.ContextRunner;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.beans.factory.annotation.Qualifier;
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
+import org.springframework.context.annotation.Bean;
+import org.springframework.context.annotation.Configuration;
+
+import javax.annotation.PostConstruct;
+import java.util.List;
+
+@Configuration
+@ConditionalOnProperty(prefix = "admin", name = "port")
+public class AdminServerConfiguration {
+
+    private static final Logger logger = LoggerFactory.getLogger(AdminServerConfiguration.class);
+
+    @Autowired
+    private ContextRunner contextRunner;
+
+    @Autowired
+    private Vertx vertx;
+
+    @Autowired
+    @Qualifier("adminRouter")
+    private Router adminRouter;
+
+    @Value("${admin.port}")
+    private int adminPort;
+
+    @Bean
+    VersionHandler versionHandler(JacksonMapper mapper) {
+        return VersionHandler.create("git-revision.json", mapper);
+    }
+
+    @Bean(name = "adminRouter")
+    Router adminRouter(BodyHandler bodyHandler, VersionHandler versionHandler,
+                       List<CustomizedAdminEndpoint> customizedAdminEndpoints) {
+        final Router router = Router.router(vertx);
+        router.route().handler(bodyHandler);
+        router.route("/version").handler(versionHandler);
+
+        customizedAdminEndpoints.stream()
+                .filter(customizedAdminEndpoint -> !customizedAdminEndpoint.isOnApplicationPort())
+                .forEach(customizedAdminEndpoint -> customizedAdminEndpoint.router(router));
+
+        return router;
+    }
+
+    @PostConstruct
+    public void startAdminServer() {
+        logger.info("Starting Admin Server to serve requests on port {0,number,#}", adminPort);
+
+        contextRunner.<HttpServer>runOnServiceContext(future ->
+                vertx.createHttpServer().requestHandler(adminRouter).listen(adminPort, future));
+
+        logger.info("Successfully started Admin Server");
+    }
+}