MAC-calculation

[snimu]

Currently, the number of MACs is calculated in LayerInfo.calculate_macs. However, as both issue#77 and PR#193 show, this is more complex than assumed in LayerInfo.

Theoretically, the code for this might grow pretty large if we want it to be accurate for all pre-defined torch.Modules. For example, there is a difference between a nn.Linear-layer and a nn.Bilinear-layer, not to mention convolutions and other operations.

Moving the calculation of MACs to its own class that is then used by LayerInfo would, I think, make it easier and more readable to implement accurate estimations of different layers over time. It would also allow whomever implements the estimation for a layer to add detailed comments explaining what they are doing. For potentially mathematically complex code, that seems like an advantage.

Here's a rough idea of how something like this might look like:

# macs.py

class MACEst:
    func_map = {
        "Linear": MACEst.linear,
        "LazyLinear": MACEst.linear,
        "Bilinear": MACEst.bilinear,
        ...
    }
 
    def __init__(self, class_name: str, output_size: list[int]) -> None:
        ...
   
    def __call__(self, cur_params: int, name: str) -> int:
        estimator = self.func_map.get(class_name)
        if estimator is None: 
            estimator = self.default_est
        return estimator(cur_params, name)

    @staticmethod
    def default_est(cur_params: int, name: str) -> int:
        # Comment explaining logic
        ...

    @staticmethod
    def linear(cur_params: int, name: str) -> int:
        # Comment explaining logic
        ...

    @staticmethod
    def bilinear(cur_params: int, name: str) -> int:
        # Comment explaining logic
        ...

    ...


# layer_info.py

class LayerInfo:

    ...

    def calculate_macs(self) -> None:
        mac_est = MACEst(self.class_name, self.output_size)

        for name, param in self.module.named_parameters():
            cur_params, name = self.get_param_count(self.module, name, param)
            self.macs += self.mac_est(cur_params)

    ... 

Hope I didn't leave a bug in there.

Of course, the prod-function from layer_info.py could simply be moved to macs.py.

I'm not sure if this is a good idea or not, so I would welcome feedback :)