Add an ability to temporarily enable verbose HTTP interaction logging…

… for auctions endpoints (#775)

* Add component for HTTP interaction logging

* Implement endpoint handler that controls HTTP interaction logging

* Add tests for HttpInteractionLogHandler

* Add documentation for HTTP interaction logging

* Remove obsolete AdminManager

* Move HTTP interaction logging into a separate logback logger

* Adjust header in documentation

* Fix doc

* Fix docs

Co-authored-by: rpanchyk <rpanchyk@rubiconproject.com>

docs/config-app.md

@@ -114,8 +114,8 @@ There are several typical keys:
 
 But feel free to add additional bidder's specific options.
 
-## Logger Level Modifier
-- `logger-level-modifier.enabled` - enable the `/admin` endpoint.
+## Logging
+- `logging.http-interaction.max-limit` - maximum value for the number of interactions to log in one take.
 
 ## Currency Converter
 - `currency-converter.external-rates.enabled` - if equals to `true` the currency conversion service will be enabled to fetch updated rates and convert bid currencies from external source. Also enables `/currency-rates` endpoint on admin port.
@@ -149,10 +149,10 @@ But feel free to add additional bidder's specific options.
 - `admin-endpoints.cache-invalidation.on-application-port` - when equals to `false` endpoint will be bound to `admin.port`.
 - `admin-endpoints.cache-invalidation.protected` - when equals to `true` endpoint will be protected by basic authentication configured in `admin-endpoints.credentials` 
 
-- `admin-endpoints.logger-level-modifier.enabled` - if equals to `true` the endpoint will be available.
-- `admin-endpoints.logger-level-modifier.path` - the server context path where the endpoint will be accessible.
-- `admin-endpoints.logger-level-modifier.on-application-port` - when equals to `false` endpoint will be bound to `admin.port`.
-- `admin-endpoints.logger-level-modifier.protected` - when equals to `true` endpoint will be protected by basic authentication configured in `admin-endpoints.credentials` 
+- `admin-endpoints.logging-httpinteraction.enabled` - if equals to `true` the endpoint will be available.
+- `admin-endpoints.logging-httpinteraction.path` - the server context path where the endpoint will be accessible.
+- `admin-endpoints.logging-httpinteraction.on-application-port` - when equals to `false` endpoint will be bound to `admin.port`.
+- `admin-endpoints.logging-httpinteraction.protected` - when equals to `true` endpoint will be protected by basic authentication configured in `admin-endpoints.credentials` 
 
 - `admin-endpoints.credentials` - user and password for access to admin endpoints if `admin-endpoints.[NAME].protected` is true`.
 

docs/endpoints/admin.md

@@ -1,10 +0,0 @@
-# Admin Endpoint
-
-Unavailable if `logger level modifier` is disabled (`logger-level-modifier.enabled` config property).
-
-This endpoint will set the desirable logging level and number of logs for `400` responses.
-
-### Query Params
-
-- `logging` - Desirable logging level: `info`, `warn`, `trace`, `error`, `fatal`, `debug`.
-- `records` - numbers of logs with changed logging level. (0 < n < 100_000).

docs/endpoints/logging/httpinteraction.md

@@ -0,0 +1,18 @@
+# Enable HTTP interaction logging endpoint
+
+This endpoint has a path `/logging/httpinteraction` by default (can be configured).
+
+This endpoint turns on temporary logging of raw HTTP requests and responses, mainly for troubleshooting production issues. 
+
+Interaction is logged at `INFO` level using `http-interaction` logback logger so make sure this logger has at least 
+`INFO` or more verbose level set ([logback configuration](../../../src/main/resources/logback-spring.xml) bundled in JAR 
+file sets this logger to `INFO` level).
+
+### Query Params
+- `endpoint` - endpoint to be affected; valid values: [auction](../openrtb2/auction.md), [amp](../openrtb2/amp.md); 
+if omitted all valid endpoints will be affected
+- `statusCode` - specifies that only interactions resulting in this response status code should be logged; 
+valid values: >=200 and <=500
+- `account` - specifies that only interactions involving this account should be logged
+- `limit` - number of interactions to log; there is an upper threshold for this value set in 
+[configuration](../../config-app.md) 

src/main/java/org/prebid/server/exception/InvalidRequestException.java

@@ -5,7 +5,6 @@
 import java.util.Collections;
 import java.util.List;
 
-@SuppressWarnings("serial")
 public class InvalidRequestException extends RuntimeException {
 
     @Getter

src/main/java/org/prebid/server/handler/HttpInteractionLogHandler.java

@@ -0,0 +1,97 @@
+package org.prebid.server.handler;
+
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.vertx.core.Handler;
+import io.vertx.core.MultiMap;
+import io.vertx.ext.web.RoutingContext;
+import org.prebid.server.exception.InvalidRequestException;
+import org.prebid.server.log.HttpInteractionLogger;
+import org.prebid.server.log.model.HttpLogSpec;
+
+import java.util.Arrays;
+import java.util.Objects;
+
+public class HttpInteractionLogHandler implements Handler<RoutingContext> {
+
+    private static final String ENDPOINT_PARAMETER = "endpoint";
+    private static final String STATUS_CODE_PARAMETER = "statusCode";
+    private static final String ACCOUNT_PARAMETER = "account";
+    private static final String LIMIT_PARAMETER = "limit";
+
+    private final int maxLimit;
+    private final HttpInteractionLogger httpInteractionLogger;
+
+    public HttpInteractionLogHandler(int maxLimit, HttpInteractionLogger httpInteractionLogger) {
+        this.maxLimit = maxLimit;
+        this.httpInteractionLogger = Objects.requireNonNull(httpInteractionLogger);
+    }
+
+    @Override
+    public void handle(RoutingContext context) {
+        final MultiMap parameters = context.request().params();
+
+        try {
+            httpInteractionLogger.setSpec(HttpLogSpec.of(
+                    readEndpoint(parameters),
+                    readStatusCode(parameters),
+                    readAccount(parameters),
+                    readLimit(parameters)));
+        } catch (InvalidRequestException e) {
+            context.response().setStatusCode(HttpResponseStatus.BAD_REQUEST.code()).end(e.getMessage());
+            return;
+        }
+
+        context.response().end();
+    }
+
+    private HttpLogSpec.Endpoint readEndpoint(MultiMap parameters) {
+        final String endpoint = parameters.get(ENDPOINT_PARAMETER);
+        try {
+            return endpoint != null ? HttpLogSpec.Endpoint.valueOf(endpoint) : null;
+        } catch (IllegalArgumentException e) {
+            throw new InvalidRequestException(String.format(
+                    "Invalid '%s' parameter value, allowed values '%s'",
+                    ENDPOINT_PARAMETER,
+                    Arrays.toString(HttpLogSpec.Endpoint.values())));
+        }
+    }
+
+    private Integer readStatusCode(MultiMap parameters) {
+        final Integer statusCode = getIntParameter(STATUS_CODE_PARAMETER, parameters);
+
+        if (statusCode != null && (statusCode < 200 || statusCode > 500)) {
+            throw new InvalidRequestException(String.format(
+                    "Parameter '%s' must be between %d and %d", STATUS_CODE_PARAMETER, 200, 500));
+        }
+
+        return statusCode;
+    }
+
+    private String readAccount(MultiMap parameters) {
+        return parameters.get(ACCOUNT_PARAMETER);
+    }
+
+    private int readLimit(MultiMap parameters) {
+        final Integer limit = getIntParameter(LIMIT_PARAMETER, parameters);
+
+        if (limit == null) {
+            throw new InvalidRequestException(String.format("Missing required parameter '%s'", LIMIT_PARAMETER));
+        }
+
+        if (limit < 1 || limit > maxLimit) {
+            throw new InvalidRequestException(String.format(
+                    "Parameter '%s' must be between %d and %d", LIMIT_PARAMETER, 1, maxLimit));
+        }
+
+        return limit;
+    }
+
+    private Integer getIntParameter(String parameterName, MultiMap parameters) {
+        final String value = parameters.get(parameterName);
+        try {
+            return value != null ? Integer.parseInt(value) : null;
+        } catch (NumberFormatException e) {
+            throw new InvalidRequestException(String.format("Invalid '%s' parameter value", parameterName));
+        }
+    }
+}