SIANXKE-178: enhanced configuration options

Author: anx-abruckner

README.md

@@ -44,7 +44,7 @@ LOGGING = {
 
 ## Details
 
-Most of times you don't have to care about these details. But in case you need to dig deep:
+Most of the times you don't have to care about these details. But in case you need to dig deep:
 
 * All logs are configured using logger name "django.request".
 * If HTTP status code is between 400 - 599, URIs are logged at ERROR level, otherwise they are logged at INFO level.
@@ -53,7 +53,40 @@ Most of times you don't have to care about these details. But in case you need t
 See `REQUEST_LOGGING_HTTP_4XX_LOG_LEVEL` setting to override this.
 
 
-A `no_logging` decorator is included for views with sensitive data.
+A `no_logging` decorator is included for views with sensitive data. This decorator allows control over logging behaviour of single views via the following parameters:
+```
+* value
+    * False: the view does NOT log any activity at all (overrules settings of log_headers, log_body, log_response and automatically sets them to False).
+    * True: the view logs incoming requests (potentially log headers, body and response, depending on their specific settings)
+    * None: NO_LOGGING_DEFAULT_VALUE is used (can be defined in settings file as DJANGO_REQUEST_LOGGING_NO_LOGGING_DEFAULT_VALUE)
+* msg
+    * Reason for deactivation of logging gets logged instead of request itself (only if silent=True and value=False)
+    * NO_LOGGING_MSG is used by default
+* log_headers
+    * False: request headers will not get logged
+    * True: request headers will get logged (if value is True)
+    * None: LOG_HEADERS_DEFAULT_VALUE is used (can be defined in settings file as DJANGO_REQUEST_LOGGING_LOG_HEADERS_DEFAULT_VALUE)
+* no_header_logging_msg
+    * Reason for deactivation of header logging gets logged instead of headers (only if silent=True and log_headers=False)
+    * NO_HEADER_LOGGING_MSG is used by default
+* log_body
+    * False: request body will not get logged
+    * True: request headers will get logged (if value is True)
+    * None: LOG_BODY_DEFAULT_VALUE is used (can be defined in settings file as DJANGO_REQUEST_LOGGING_LOG_BODY_DEFAULT_VALUE)
+* no_body_logging_msg
+    * Reason for deactivation of body logging gets logged instead of body (only if silent=True and log_body=False)
+    * NO_BODY_LOGGING_MSG is used by default
+* log_response
+    * False: response will not get logged
+    * True: response will get logged (if value is True)
+    * None: LOG_RESPONSE_DEFAULT_VALUE is used (can be defined in settings file as DJANGO_REQUEST_LOGGING_LOG_RESPONSE_DEFAULT_VALUE)
+* no_response_logging_msg
+    * Reason for deactivation of body logging gets logged instead of body (only if silent=True and log_body=False)
+    * NO_RESPONSE_LOGGING_MSG is used by default
+* silent
+    * True: deactivate logging of alternative messages case parts of the logging are deactivated (request/header/body/response)
+    * False: alternative messages for deactivated parts of logging (request/header/body/response) are logged instead
+```
 
 By default, value of Http headers `HTTP_AUTHORIZATION` and `HTTP_PROXY_AUTHORIZATION` are replaced wih `*****`. You can use `REQUEST_LOGGING_SENSITIVE_HEADERS` setting to override this default behaviour with your list of sensitive headers.
 
@@ -73,13 +106,23 @@ By default, HTTP status codes between 400 - 499 are logged at ERROR level.  You
 If you set `REQUEST_LOGGING_HTTP_4XX_LOG_LEVEL=logging.INFO` they will be logged the same as normal requests.
 ### REQUEST_LOGGING_SENSITIVE_HEADERS
 The value of the headers defined in this settings will be replaced with `'*****'` to hide the sensitive information while logging. By default it is set as `REQUEST_LOGGING_SENSITIVE_HEADERS = ["HTTP_AUTHORIZATION", "HTTP_PROXY_AUTHORIZATION"]`
+### DJANGO_REQUEST_LOGGING_LOGGER_NAME
+Name of the logger that is used to log django.request occurrances with the new LoggingMiddleware. Defaults to "django.request".
+### DJANGO_REQUEST_LOGGING_NO_LOGGING_DEFAULT_VALUE
+Global default to activate/deactivate logging of all views. Can be overruled for each individual view by using the @no_logging decator's "value" parameter.
+### DJANGO_REQUEST_LOGGING_LOG_HEADERS_DEFAULT_VALUE = True
+Global default to activate/deactivate logging of request headers for all views. Can be overruled for each individual view by using the @no_logging decator's "log_headers" parameter.
+### DJANGO_REQUEST_LOGGING_LOG_BODY_DEFAULT_VALUE = True
+Global default to activate/deactivate logging of request bodys for all views. Can be overruled for each individual view by using the @no_logging decator's "log_body" parameter.
+### DJANGO_REQUEST_LOGGING_LOG_RESPONSE_DEFAULT_VALUE = True
+Global default to activate/deactivate logging of responses for all views. Can be overruled for each individual view by using the @no_logging decator's "log_response" parameter.
 
 
 ## Deploying, Etc.
 
 ### Maintenance
 
-Use `pyenv` to maintain a set of virtualenvs for 2.7 and a couple versions of Python 3. 
+Use `pyenv` to maintain a set of virtualenvs for 2.7 and a couple versions of Python 3.
 Make sure the `requirements-dev.txt` installs for all of them, at least until we give up on 2.7.
 At that point, update this README to let users know the last version they can use with 2.7.
 
@@ -93,17 +136,17 @@ At that point, update this README to let users know the last version they can us
     index-servers=
         testpypi
         pypi
-    
+
     [testpypi]
     username = rhumbix
     password = password for dev@rhumbix.com at Pypi
-    
+
     [pypi]
     username = rhumbix
     password = password for dev@rhumbix.com at Pypi
 ```
 
-### Publishing  
+### Publishing
 
 - Bump the version value in `request_logging/__init__.py`
 - Run `python setup.py publish`

request_logging/decorators.py

@@ -1,10 +1,29 @@
-from .middleware import NO_LOGGING_ATTR, NO_LOGGING_MSG_ATTR, NO_LOGGING_MSG
+from .middleware import NO_LOGGING_ATTR, NO_LOGGING_MSG_ATTR, NO_LOGGING_MSG, NO_LOGGING_DEFAULT_VALUE, \
+    LOG_HEADERS_ATTR, LOG_HEADERS_DEFAULT_VALUE, LOG_BODY_ATTR, LOG_BODY_DEFAULT_VALUE, LOG_RESPONSE_ATTR, \
+    LOG_RESPONSE_DEFAULT_VALUE, NO_RESPONSE_LOGGING_MSG_ATTR, NO_RESPONSE_LOGGING_MSG, NO_HEADER_LOGGING_MSG_ATTR, \
+    NO_HEADER_LOGGING_MSG, NO_BODY_LOGGING_MSG_ATTR, NO_BODY_LOGGING_MSG
 
 
-def no_logging(msg=None, silent=False):
+def no_logging(msg=None, silent=False, value=None, log_headers=None, no_header_logging_msg=None, log_body=None,
+               no_body_logging_msg=None, log_response=None, no_response_logging_msg=None):
+    def _set_attr(func, attr_name, value, default_value=None):
+        setattr(func, attr_name, value if value is not None else default_value)
+
+    def _set_attr_msg(func, silent_value, msg_attr_name, message, default_message=None):
+        setattr(func, msg_attr_name, (message if message else default_message) if not silent_value else None)
+
     def wrapper(func):
-        setattr(func, NO_LOGGING_ATTR, True)
-        setattr(func, NO_LOGGING_MSG_ATTR, (msg if msg else NO_LOGGING_MSG) if not silent else None)
+        _set_attr(func, NO_LOGGING_ATTR, value, NO_LOGGING_DEFAULT_VALUE)
+        _set_attr_msg(func, silent, NO_LOGGING_MSG_ATTR, msg, NO_LOGGING_MSG)
+
+        _set_attr(func, LOG_HEADERS_ATTR, log_headers, LOG_HEADERS_DEFAULT_VALUE)
+        _set_attr_msg(func, silent, NO_HEADER_LOGGING_MSG_ATTR, no_header_logging_msg, NO_HEADER_LOGGING_MSG)
+
+        _set_attr(func, LOG_BODY_ATTR, log_body, LOG_BODY_DEFAULT_VALUE)
+        _set_attr_msg(func, silent, NO_BODY_LOGGING_MSG_ATTR, no_body_logging_msg, NO_BODY_LOGGING_MSG)
+
+        _set_attr(func, LOG_RESPONSE_ATTR, log_response, LOG_RESPONSE_DEFAULT_VALUE)
+        _set_attr_msg(func, silent, NO_RESPONSE_LOGGING_MSG_ATTR, no_response_logging_msg, NO_RESPONSE_LOGGING_MSG)
         return func
 
     return wrapper

request_logging/middleware.py

@@ -35,7 +35,42 @@
 NO_LOGGING_ATTR = "no_logging"
 NO_LOGGING_MSG_ATTR = "no_logging_msg"
 NO_LOGGING_MSG = "No logging for this endpoint"
-request_logger = logging.getLogger("django.request")
+NO_LOGGING_DEFAULT_VALUE = getattr(
+    settings,
+    "DJANGO_REQUEST_LOGGING_NO_LOGGING_DEFAULT_VALUE",
+    False
+)
+LOG_HEADERS_ATTR = "log_headers"
+LOG_HEADERS_DEFAULT_VALUE = getattr(
+    settings,
+    "DJANGO_REQUEST_LOGGING_LOG_HEADERS_DEFAULT_VALUE",
+    True
+)
+NO_HEADER_LOGGING_MSG_ATTR = "no_header_logging_msg"
+NO_HEADER_LOGGING_MSG = "No header logging for this endpoint"
+LOG_BODY_ATTR = "log_body"
+LOG_BODY_DEFAULT_VALUE = getattr(
+    settings,
+    "DJANGO_REQUEST_LOGGING_LOG_BODY_DEFAULT_VALUE",
+    True
+)
+NO_BODY_LOGGING_MSG_ATTR = "no_body_logging_msg"
+NO_BODY_LOGGING_MSG = "No body logging for this endpoint"
+LOG_RESPONSE_ATTR = "response_logging"
+LOG_RESPONSE_DEFAULT_VALUE = getattr(
+    settings,
+    "DJANGO_REQUEST_LOGGING_LOG_RESPONSE_DEFAULT_VALUE",
+    True
+)
+NO_RESPONSE_LOGGING_MSG_ATTR = "no_response_logging_msg"
+NO_RESPONSE_LOGGING_MSG = "No response logging for this endpoint"
+
+LOGGER_NAME = getattr(
+    settings,
+    "DJANGO_REQUEST_LOGGING_LOGGER_NAME",
+    "django.request"
+)
+request_logger = logging.getLogger(LOGGER_NAME)
 
 
 class Logger:
@@ -118,8 +153,8 @@ def __init__(self, get_response=None):
         self.logger = ColourLogger("cyan", "magenta") if enable_colorize else Logger()
 
     def __call__(self, request):
-        # cache in a local reference (instead of a member reference) and then pass in as argument 
-        # in order to avoid other threads overwriting the original self.cached_request_body reference, 
+        # cache in a local reference (instead of a member reference) and then pass in as argument
+        # in order to avoid other threads overwriting the original self.cached_request_body reference,
         # is this done to preserve the original value in case it is mutated during the get_response invocation?
         cached_request_body = request.body
         response = self.get_response(request)
@@ -135,7 +170,7 @@ def process_request(self, request, response, cached_request_body):
         else:
             return self._log_request(request, response, cached_request_body)
 
-    def _should_log_route(self, request):
+    def _get_func(self, request):
         # request.urlconf may be set by middleware or application level code.
         # Use this urlconf if present or default to None.
         # https://docs.djangoproject.com/en/2.1/topics/http/urls/#how-django-processes-a-request
@@ -162,10 +197,33 @@ def _should_log_route(self, request):
         elif hasattr(view, "view_class"):
             # This is for django class-based views
             func = getattr(view.view_class, method, None)
-        no_logging = getattr(func, NO_LOGGING_ATTR, False)
+
+        return func
+
+    def _should_log_route(self, request):
+        func = self._get_func(request)
+        no_logging = getattr(func, NO_LOGGING_ATTR, NO_LOGGING_DEFAULT_VALUE)
         no_logging_msg = getattr(func, NO_LOGGING_MSG_ATTR, None)
         return no_logging, no_logging_msg
 
+    def _should_log_headers(self, request):
+        func = self._get_func(request)
+        header_logging = getattr(func, LOG_HEADERS_ATTR, LOG_HEADERS_DEFAULT_VALUE)
+        no_header_logging_msg = getattr(func, NO_HEADER_LOGGING_MSG_ATTR, None)
+        return header_logging, no_header_logging_msg
+
+    def _should_log_body(self, request):
+        func = self._get_func(request)
+        body_logging = getattr(func, LOG_BODY_ATTR, LOG_BODY_DEFAULT_VALUE)
+        no_body_logging_msg = getattr(func, NO_BODY_LOGGING_MSG_ATTR, None)
+        return body_logging, no_body_logging_msg
+
+    def _should_log_response(self, request):
+        func = self._get_func(request)
+        response_logging = getattr(func, LOG_RESPONSE_ATTR, LOG_RESPONSE_DEFAULT_VALUE)
+        no_response_logging_msg = getattr(func, NO_RESPONSE_LOGGING_MSG_ATTR, None)
+        return response_logging, no_response_logging_msg
+
     def _skip_logging_request(self, request, reason):
         method_path = "{} {}".format(request.method, request.get_full_path())
         no_log_context = {
@@ -191,6 +249,14 @@ def _log_request(self, request, response, cached_request_body):
         self._log_request_body(request, logging_context, log_level, cached_request_body)
 
     def _log_request_headers(self, request, logging_context, log_level):
+        log_headers, because = self._should_log_headers(request)
+        if not log_headers:
+            if because is not None:
+                self.logger.log_error(
+                    logging.INFO, "no headers logged", {"args": {}, "kwargs": {"extra": {"no_header_logging": because}}}
+                )
+            return None
+
         if IS_DJANGO_VERSION_GTE_3_2_0:
             headers = {k: v if k not in self.sensitive_headers else "*****" for k, v in request.headers.items()}
         else:
@@ -204,6 +270,14 @@ def _log_request_headers(self, request, logging_context, log_level):
             self.logger.log(log_level, headers, logging_context)
 
     def _log_request_body(self, request, logging_context, log_level, cached_request_body):
+        log_body, because = self._should_log_body(request)
+        if not log_body:
+            if because is not None:
+                self.logger.log_error(
+                    logging.INFO, "no body logged", {"args": {}, "kwargs": {"extra": {"log_body": because}}}
+                )
+            return None
+
         if cached_request_body is not None:
             content_type = request.META.get("CONTENT_TYPE", "")
             is_multipart = content_type.startswith("multipart/form-data")
@@ -222,6 +296,13 @@ def process_response(self, request, response):
                     logging.INFO, resp_log, {"args": {}, "kwargs": {"extra": {"no_logging": because}}}
                 )
             return response
+        log_response, because = self._should_log_response(request)
+        if not log_response:
+            if because is not None:
+                self.logger.log_error(
+                    logging.INFO, resp_log, {"args": {}, "kwargs": {"extra": {"log_response": because}}}
+                )
+            return response
         logging_context = self._get_logging_context(request, response)
 
         if response.status_code in range(400, 500):