Compositional metrics

CHANGELOG.md

@@ -57,6 +57,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 - `Recall` and `Precision` metrics (and their functional counterparts `recall` and `precision`) can now be generalized to Recall@K and Precision@K with the use of `top_k` parameter ([#4842](https://github.com/PyTorchLightning/pytorch-lightning/pull/4842))
 
 
+- Added compositional metrics ([#5464](https://github.com/PyTorchLightning/pytorch-lightning/pull/5464))
+
 
 ### Changed
 

docs/source/metrics.rst

@@ -255,6 +255,51 @@ In practise this means that:
     val = metric(pred, target) # this value can be backpropagated
     val = metric.compute() # this value cannot be backpropagated
 
+******************
+Metric Arithmetics
+******************
+
+Metrics support most of python built-in operators for arithmetic, logic and bitwise operations.
+
+For example for a metric that should return the sum of two different metrics, implementing a new metric is an overhead that is not necessary. 
+It can now be done with:
+
+.. code-block:: python
+
+    first_metric = MyFirstMetric()
+    second_metric = MySecondMetric()
+
+    new_metric = first_metric + second_metric
+
+``new_metric.update(*args, **kwargs)`` now calls update of ``first_metric`` and ``second_metric``. It forwards all positional arguments but 
+forwards only the keyword arguments that are available in respective metric's update declaration.
+
+Similarly ``new_metric.compute()`` now calls compute of ``first_metric`` and ``second_metric`` and adds the results up.
+
+This pattern is implemented for the following operators (with ``a`` being metrics and ``b`` being metrics, tensors, integer or floats):
+
+* Addition (``a + b``)
+* Bitwise AND (``a & b``)
+* Equality (``a == b``)
+* Floordivision (``a // b``)
+* Greater Equal (``a >= b``)
+* Greater (``a > b``)
+* Less Equal (``a <= b``)
+* Less (``a < b``)
+* Matrix Multiplication (``a @b ``)
+* Modulo (``a % b``)
+* Multiplication (``a * b``)
+* Inequality (``a != b``)
+* Bitwise OR (``a | b``)
+* Power (``a ** b``)
+* Substraction (``a - b``)
+* True Division (``a / b``)
+* Bitwise XOR (``a ^ b``)
+* Absolute Value (``abs(a)``)
+* Inversion (``~a``)
+* Negative Value (``neg(a)``)
+* Positive Value (``pos(a)``)
+
 ****************
 MetricCollection
 ****************

pytorch_lightning/metrics/compositional.py

@@ -0,0 +1,92 @@
+from typing import Callable, Union
+
+import torch
+
+from pytorch_lightning.metrics.metric import Metric
+
+
+class CompositionalMetric(Metric):
+    """Composition of two metrics with a specific operator
+    which will be executed upon metric's compute
+
+    """
+
+    def __init__(
+        self,
+        operator: Callable,
+        metric_a: Union[Metric, int, float, torch.Tensor],
+        metric_b: Union[Metric, int, float, torch.Tensor, None],
+    ):
+        """
+
+        Args:
+            operator: the operator taking in one (if metric_b is None)
+                or two arguments. Will be applied to outputs of metric_a.compute()
+                and (optionally if metric_b is not None) metric_b.compute()
+            metric_a: first metric whose compute() result is the first argument of operator
+            metric_b: second metric whose compute() result is the second argument of operator.
+                For operators taking in only one input, this should be None
+        """
+        super().__init__()
+
+        self.op = operator
+
+        if isinstance(metric_a, torch.Tensor):
+            self.register_buffer("metric_a", metric_a)
+        else:
+            self.metric_a = metric_a
+
+        if isinstance(metric_b, torch.Tensor):
+            self.register_buffer("metric_b", metric_b)
+        else:
+            self.metric_b = metric_b
+
+    def _sync_dist(self, dist_sync_fn=None):
+        # No syncing required here. syncing will be done in metric_a and metric_b
+        pass
+
+    def update(self, *args, **kwargs):
+        if isinstance(self.metric_a, Metric):
+            self.metric_a.update(*args, **self.metric_a._filter_kwargs(**kwargs))
+
+        if isinstance(self.metric_b, Metric):
+            self.metric_b.update(*args, **self.metric_b._filter_kwargs(**kwargs))
+
+    def compute(self):
+
+        # also some parsing for kwargs?
+        if isinstance(self.metric_a, Metric):
+            val_a = self.metric_a.compute()
+        else:
+            val_a = self.metric_a
+
+        if isinstance(self.metric_b, Metric):
+            val_b = self.metric_b.compute()
+        else:
+            val_b = self.metric_b
+
+        if val_b is None:
+            return self.op(val_a)
+
+        return self.op(val_a, val_b)
+
+    def reset(self):
+        if isinstance(self.metric_a, Metric):
+            self.metric_a.reset()
+
+        if isinstance(self.metric_b, Metric):
+            self.metric_b.reset()
+
+    def persistent(self, mode: bool = False):
+        if isinstance(self.metric_a, Metric):
+            self.metric_a.persistent(mode=mode)
+        if isinstance(self.metric_b, Metric):
+            self.metric_b.persistent(mode=mode)
+
+    def __repr__(self):
+        repr_str = (
+            self.__class__.__name__
+            + f"(\n  {self.op.__name__}(\n    {repr(self.metric_a)},\n    {repr(self.metric_b)}\n  )\n)"
+        )
+
+        return repr_str