[Feat] Add Accuracy

[ice-tong]

Motivation

Adding accuracy metric and test case.

Modification

• mmeval/classification/accuracy.py
• tests/unittest/classification/test_accuracy.py

MMCls

• mmcls PR: [Feature] Use mmeval.Accuracy mmpretrain#1046

[mzr1996]

Better to only reserve the first top-k results only in the intermediate results.

[mzr1996]

The intermediate results can be very large if the number of classes is large.

[ice-tong]

Updated, use corrects as the intermediate results.

[mzr1996]

Why calculate corrects one by one instead of a batch?

[ice-tong]

Updated, calculate corrects in batch now.

[zhouzaida]

How fast does the pytorch compared with numpy?

[ice-tong]

This depends on the size of the input. The larger the size of the input, the greater the speedup.

The following is a simple benchmark.

import torch
import numpy as np
import time


def numpy_topk(inputs, k, axis=None, use_torch=False):
    if use_torch:
        values, indices = torch.from_numpy(inputs).topk(k, dim=axis)
        return values.numpy(), indices.numpy()

    indices = np.argsort(inputs, axis=axis)
    indices = np.take(indices, np.arange(k), axis=axis)
    values = np.take_along_axis(inputs, indices, axis=axis)
    return values, indices


def test(shape, k, axis):
    print('Test setting: ', shape, k, axis)
    arr = np.random.rand(*shape)
    t1 = time.time()
    numpy_topk(arr, k, axis, use_torch=False)
    t2 = time.time()
    numpy_topk(arr, k, axis, use_torch=True)
    t3 = time.time()
    print(f'custom impl numpy topk cost: {t2-t1}')
    print(f'torch impl numpy topk cost:  {t3-t2}')


if __name__ == "__main__":
    test((100, 100), k=4, axis=1)
    test((100, 1000), k=4, axis=1)
    test((100, 1000), k=10, axis=1)
    test((1000, 1000), k=4, axis=1)
    test((10000, 1000), k=4, axis=1)

Got the following outputs:

Test setting:  (100, 100) 4 1
custom impl numpy topk cost: 0.0005044937133789062
torch impl numpy topk cost:  0.0046596527099609375
Test setting:  (100, 1000) 4 1
custom impl numpy topk cost: 0.005578756332397461
torch impl numpy topk cost:  0.004395961761474609
Test setting:  (100, 1000) 10 1
custom impl numpy topk cost: 0.005606651306152344
torch impl numpy topk cost:  0.0003619194030761719
Test setting:  (1000, 1000) 4 1
custom impl numpy topk cost: 0.05330657958984375
torch impl numpy topk cost:  0.001957416534423828
Test setting:  (10000, 1000) 4 1
custom impl numpy topk cost: 0.5171260833740234
torch impl numpy topk cost:  0.0032396316528320312

[mzr1996]

Maybe a private method if the method won't be used by users.

[ice-tong]

Done