ZQS-721 Implement initial PoC of cost function logger

[dexter2206]

This PR implements a versatile framework for augmenting cost functions and includes a function logger implemented in this framework.

API description:

• Wrappers for cost functions (like logger, and others in the future) are called augmentations.
• Augmentations are done via augment_cost_function, which accepts a function to augment, and optional lists of augmentations for augmenting the function and its gradient. For instance

# Augments only cost function, gradient is left unchanged
augmented_func = augment_cost_function(func, [function_logger(level=level)])

# Augments both cost function and gradient, in different ways
augmented_func = augment_cost_function(
     func,
     cost_function_augmentations=[function_logger(level=logging.INFO)],
     gradient_augmentations=[function_logger(level=logging.DEBUG)]
)

• Augmentations preserve attributes of cost functions, e.g. if in the above example func has number_of_layers attribute then augmented_func has it as well. Accessing such attributes (both querying and setting) are forwarded to the underlying function.
• Implementers of augmentations don't have to handle this forwarding themselves, the internal machinery of augment_cost_function already ensures that.
• If more than one augmentation is provided, they are applied in left-to-right order.

Main architectural decisions:

• Augmentations are just higher-order functions transforming functions into functions.
• A specific type of augmentation, called ConditionalSideEffect is implemented as an ABC for convenience. This specific type of augmentations passes through arguments to the wrapped function and propagates the return value, while triggering some side effect (e.g. logging) when the predicate is met. The predicates are the same as save_conditions in our history recorders (which is why they are reused in the code proposed in this PR).
• FunctionLogger is implemented as a subclass of ConditionalSideEffect.
• ConditionalSideEffects need to be wrapped into a higher order function. In case of FunctionLogger it is function_logger.
• Attribute forwarding prioritizes augmentation attributes. For instance, if both a function and its augmentation have bar attribute then accessing bar of the augmented function accesses augmentation's attribute (see tests for examples of this behavior)

Some questions that are still open:

• What should be the message logged by FunctionLogger? How configurable should it be?
• What if some use-cases require applying same augmentation to both cost function and gradient? I think we should leave to the user's to ensure that such constraints are met. Another way would be to introduce additional common_augmentations which would contain all augmentations that would be applied to both cost function and gradient prior to applying specific augmentations.

[codecov]

Codecov Report

Merging #417 (8edc45a) into dev (f0dd6e4) will increase coverage by 0.17%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##              dev     #417      +/-   ##
==========================================
+ Coverage   90.16%   90.33%   +0.17%     
==========================================
  Files          59       60       +1     
  Lines        3354     3413      +59     
  Branches      553      556       +3     
==========================================
+ Hits         3024     3083      +59     
  Misses        222      222              
  Partials      108      108              

Impacted Files	Coverage Δ
...thon/zquantum/core/cost_function_augmenntations.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f0dd6e4...8edc45a. Read the comment docs.

[pcarinhas]

Its worth commenting (for all magic methods) in code why you need to modify them, so that future developers understand and don't break the functionality you are implementing.

[dexter2206]

I added some documentation in the docstrings and several comments. I allowed myself to skip explaining why init and call are needed since those are rather obvious (but feel free to disagree). Let me know if the explanations provided now are sufficient.

[pcarinhas]

Shouldn't this raise a NotImplemtedError ?

[dexter2206]

It doesn't have to, because you are not able to call this method unless you implement some concrete subclass of ConditionalSideEffect. However, after adding docstring the pass statement is no longer needed too, so I removed it.