Pretrained model for HMDB51

[wwdok]

The Action Recognition Models seems doesn't contain a pretrained model for HMDB51, will you add this model in the future ? Or do you now where i can get the pretrained model for HMDB51, i want to run demo with its action classes ~
BTW, i saw you said model trained on AVA dataset is in the plan, when is the release date likely ?
#19 (comment)

[wwdok]

@SuX97 Hi, i see the ckpt log json link are still blank, when will they be add in ?

[dreamerlin]

Hold on, we will update it soon

[SuX97]

@wwdok , checkpoints and logs updated. Check 'em out.

[wwdok]

Great, thanks for your excellent work ！I have just try to run demo with it,
This is the demo script :

import torch
from mmaction.apis import init_recognizer, inference_recognizer
config_file = 'configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb.py'
device = 'cuda:0' # or 'cpu'
device = torch.device(device)
checkpoint = 'checkpoints/tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb_20201123-7f84701b.pth'
model = init_recognizer(config_file,checkpoint, device=device)
# inference the demo video
results = inference_recognizer(model, '/media/user/DATA/dataset/smoking.mp4', 'demo/label_map_hmdb51.txt')
for result in results:
    print(result)

it reports error :

(base) user@user-TUF-Gaming-FX506LU-FX506LU:~/Repo/mmaction2$ python verification_hmdb51.py
Traceback (most recent call last):
  File "verification_hmdb51.py", line 10, in <module>
    model = init_recognizer(config_file,checkpoint, device=device)
  File "/home/user/Repo/mmaction2/mmaction/apis/inference.py", line 40, in init_recognizer
    raise RuntimeError('input data type should be consist with the '
RuntimeError: input data type should be consist with the dataset type in config, but got input type 'video' and dataset type 'RawframeDataset'

Then i modified the dataset_type in tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb.py from 'RawframeDataset' to 'VideoDataset', but it still has error :

(base) userg@user-TUF-Gaming-FX506LU-FX506LU:~/Repo/mmaction2$ python verification_hmdb51.py
Traceback (most recent call last):
  File "verification_hmdb51.py", line 13, in <module>
    results = inference_recognizer(model, '/media/user/DATA/dataset/smoking.mp4', 'demo/label_map_hmdb51.txt')
  File "/home/user/Repo/mmaction2/mmaction/apis/inference.py", line 107, in inference_recognizer
    data = test_pipeline(data)
  File "/home/user/Repo/mmaction2/mmaction/datasets/pipelines/compose.py", line 41, in __call__
    data = t(data)
  File "/home/user/Repo/mmaction2/mmaction/datasets/pipelines/loading.py", line 208, in __call__
    total_frames = results['total_frames']
KeyError: 'total_frames'

So i have a question, does the HMDB51 checkpoint and config support demo with video inference inherently ? If yes, how to solve above error ? Thanks !

[dreamerlin]

Great, thanks for your excellent work ！I have just try to run demo with it,
This is the demo script :

import torch
from mmaction.apis import init_recognizer, inference_recognizer
config_file = 'configs/recognition/tsn/tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb.py'
device = 'cuda:0' # or 'cpu'
device = torch.device(device)
checkpoint = 'checkpoints/tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb_20201123-7f84701b.pth'
model = init_recognizer(config_file,checkpoint, device=device)
# inference the demo video
results = inference_recognizer(model, '/media/weidawang/DATA/dataset/smoking.mp4', 'demo/label_map_hmdb51.txt')
for result in results:
    print(result)

it reports error :

(base) weidawang@weidawang-TUF-Gaming-FX506LU-FX506LU:~/Repo/mmaction2$ python verification_hmdb51.py
Traceback (most recent call last):
  File "verification_hmdb51.py", line 10, in <module>
    model = init_recognizer(config_file,checkpoint, device=device)
  File "/home/weidawang/Repo/mmaction2/mmaction/apis/inference.py", line 40, in init_recognizer
    raise RuntimeError('input data type should be consist with the '
RuntimeError: input data type should be consist with the dataset type in config, but got input type 'video' and dataset type 'RawframeDataset'

Then i modified the dataset_type in tsn_r50_1x1x8_50e_hmdb51_kinetics400_rgb.py from 'RawframeDataset' to 'VideoDataset', but it still has error :

(base) weidawang@weidawang-TUF-Gaming-FX506LU-FX506LU:~/Repo/mmaction2$ python verification_hmdb51.py
Traceback (most recent call last):
  File "verification_hmdb51.py", line 13, in <module>
    results = inference_recognizer(model, '/media/weidawang/DATA/dataset/smoking.mp4', 'demo/label_map_hmdb51.txt')
  File "/home/weidawang/Repo/mmaction2/mmaction/apis/inference.py", line 107, in inference_recognizer
    data = test_pipeline(data)
  File "/home/weidawang/Repo/mmaction2/mmaction/datasets/pipelines/compose.py", line 41, in __call__
    data = t(data)
  File "/home/weidawang/Repo/mmaction2/mmaction/datasets/pipelines/loading.py", line 208, in __call__
    total_frames = results['total_frames']
KeyError: 'total_frames'

So i have a question, does the HMDB51 checkpoint and config support demo with video inference inherently ? If yes, how to solve above error ? Thanks !

For Video. you should also change the pipeline like this as well as setting dataset_type = 'VideoDataset'

[wwdok]

@dreamerlin Thanks for your tip ! I found i copy the content of tsn_r50_video_inference_1x1x3_100e_kinetics400_rgb.py and change num_classes from 400 to 51 is more quick and usable, now the video inference demo with HMDB51 checkpoint works !

[wwdok]

The following table does not display normally

[innerlee]

@wwdok would you mind making a pull request fixing the formatting? The file locates at https://github.com/open-mmlab/mmaction2/tree/master/configs/recognition/tsn/README.md

[wwdok]

@innerlee OK, no problem

[yanxinzju]

@wwdok hello, do you know how to get the file "label_map_hmdb51.txt"

[bit-scientist]

@dreamerlin I am using configs/skeleton/posec3d/slowonly_r50_u48_240e_ntu120_xsub_keypoint.py, checkpoints/slowonly_r50_u48_240e_ntu120_xsub_keypoint-6736b03f.pth and tools/data/skeleton/label_map_ntu120.txt for demo and having the KeyError: 'total_frames' issue as @wwdok once had. I modified the lines 86-89 of .py file above as following:

data = dict(
    videos_per_gpu=1, 
    workers_per_gpu=2,

But still the error persists. What else should be modified? Thanks