[Feature] Support DanceTrack dataset for MOT #543

noahcao · 2022-05-03T15:16:48Z

add dataset support for DanceTrack.
add data conversion tool from DanceTrack label to COCO format.
add an example config file by QDtrack on DanceTrack.
the implementation is verified by going through the training and evaluation pipelines.
edit corresponding documents, English and Chinese.

…to dancetrack

GT9505 · 2022-05-04T12:27:41Z

configs/mot/qdtrack/qdtrack_faster-rcnn_r50_fpn_4e_dancetrack.py

@@ -0,0 +1,130 @@
+_base_ = [


inherit from https://github.com/open-mmlab/mmtracking/blob/master/configs/mot/qdtrack/qdtrack_faster-rcnn_r50_fpn_4e_mot17-private-half.py

GT9505 · 2022-05-04T12:31:57Z

tools/convert_datasets/dancetrack/dancetrack2coco.py

+#
+# This file is adapted from the data label conversion file for MOT
+# But as Dancetrack does not provide public detections and provides
+# official train/val/test splitting, we make necessary adaptation.


Does Dancetrack dataset share a similar structure of annotations with MOT challenge dataset?

Yes. They have the same label format.

GT9505 · 2022-05-04T12:32:21Z

tools/convert_datasets/dancetrack/dancetrack2coco.py

+    return outputs
+
+
+def parse_dets(dets):


delete this function since it is not used.

Deleted in the new commit.

GT9505 · 2022-05-04T12:33:15Z

mmtrack/datasets/dancetrack_dataset.py

+
+
+@DATASETS.register_module()
+class DanceTrackDataset(CocoVideoDataset):


Please inherit from MOTChanllengeDataset, if they share a similar structure of annotations.

I have made this change.

…lenge

GT9505 · 2022-05-05T02:27:01Z

configs/mot/qdtrack/qdtrack_faster-rcnn_r50_fpn_4e_dancetrack.py

+_base_ = [
+    '../../_base_/models/faster_rcnn_r50_fpn.py',
+    '../../_base_/datasets/dancetrack.py', '../../_base_/default_runtime.py'
+]


Use base config of https://github.com/open-mmlab/mmtracking/blob/master/configs/mot/qdtrack/qdtrack_faster-rcnn_r50_fpn_4e_mot17-private-half.py

GT9505 · 2022-05-05T02:28:34Z

mmtrack/datasets/dancetrack_dataset.py

+
+
+@DATASETS.register_module()
+class DanceTrackDataset(MOTChallengeDataset):


Please remove the duplicated codes since we have already inherited from MOTChallengeDataset.

Removed in the new commit.

codecov · 2022-05-05T02:43:02Z

Codecov Report

Merging #543 (d17cc17) into master (88f92dd) will increase coverage by 0.15%.
The diff coverage is 76.92%.

❗ Current head d17cc17 differs from pull request most recent head 04e9b38. Consider uploading reports for the commit 04e9b38 to get more accurate results

@@            Coverage Diff             @@
##           master     #543      +/-   ##
==========================================
+ Coverage   73.08%   73.24%   +0.15%     
==========================================
  Files         129      130       +1     
  Lines        8019     8074      +55     
  Branches     1562     1564       +2     
==========================================
+ Hits         5861     5914      +53     
- Misses       1664     1668       +4     
+ Partials      494      492       -2

Flag	Coverage Δ
unittests	`73.18% <76.92%> (+0.15%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmtrack/datasets/mot_challenge_dataset.py	`73.93% <66.66%> (+0.22%)`	⬆️
mmtrack/datasets/dancetrack_dataset.py	`83.33% <83.33%> (ø)`
mmtrack/datasets/__init__.py	`100.00% <100.00%> (ø)`
mmtrack/datasets/pipelines/transforms.py	`87.61% <0.00%> (-1.15%)`	⬇️
mmtrack/models/track_heads/siamese_rpn_head.py	`93.15% <0.00%> (+0.10%)`	⬆️
mmtrack/models/reid/linear_reid_head.py	`89.85% <0.00%> (+0.14%)`	⬆️
...track/models/trackers/quasi_dense_embed_tracker.py	`75.71% <0.00%> (+0.17%)`	⬆️
mmtrack/datasets/youtube_vis_dataset.py	`77.23% <0.00%> (+0.18%)`	⬆️
mmtrack/models/track_heads/roi_track_head.py	`87.93% <0.00%> (+0.21%)`	⬆️
mmtrack/models/track_heads/roi_embed_head.py	`85.34% <0.00%> (+0.25%)`	⬆️
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 88f92dd...04e9b38. Read the comment docs.

GT9505 · 2022-05-07T04:34:53Z

mmtrack/datasets/dancetrack_dataset.py

+            BENCHMARK='DanceTrack',
+            # Option Values: 'train', 'val', 'test'
+            SPLIT_TO_EVAL='val',


The only differences are BENCHMARK and SPLIT_TO_EVAL.
We can override the get_benchmark() in MOTChanllengeDataset to return both DanceTrack and val in order to reuse most codes as far as possiable.

I'm not sure your intention, because in this function from mot_challenge_dataset.py, there is also an operation by

if 'half-train' in self.ann_file: dataset_config[ 'GT_LOC_FORMAT'] = '{gt_folder}/{seq}/gt/gt_half-train.txt' elif 'half-val' in self.ann_file: dataset_config[ 'GT_LOC_FORMAT'] = '{gt_folder}/{seq}/gt/gt_half-val.txt'

I have made BENCHMARK returned by an override get_benchmark() in DanceTrackDataset in the new commit.

I create a commit to show the intention.
The if and elif will be skipped, since dancetrack only have train and val split

GT9505 · 2022-05-07T04:35:47Z

tools/convert_datasets/dancetrack/dancetrack2coco.py

+    return parser.parse_args()
+
+
+def parse_gts(gts, is_mot15):


remove is_mot15 for dancetrack dataset

GT9505 · 2022-05-07T04:36:38Z

configs/mot/qdtrack/qdtrack_faster-rcnn_r50_fpn_4e_dancetrack.py

+_base_ = [
+    '../../_base_/models/faster_rcnn_r50_fpn.py',
+    '../../_base_/datasets/dancetrack.py', '../../_base_/default_runtime.py'
+]


Forgot to use base config of https://github.com/open-mmlab/mmtracking/blob/master/configs/mot/qdtrack/qdtrack_faster-rcnn_r50_fpn_4e_mot17-private-half.py

GT9505 · 2022-05-10T03:58:30Z

configs/mot/qdtrack/qdtrack_faster-rcnn_r50_fpn_4e_dancetrack.py

+    type='QDTrack',
+    detector=dict(
+        backbone=dict(
+            norm_cfg=dict(requires_grad=False),
+            style='caffe',
+            init_cfg=dict(
+                type='Pretrained', checkpoint='torchvision://resnet50')),
+        rpn_head=dict(bbox_coder=dict(clip_border=False)),
+        roi_head=dict(
+            bbox_head=dict(
+                loss_bbox=dict(type='L1Loss', loss_weight=1.0),
+                bbox_coder=dict(clip_border=False),
+                num_classes=1)),
+        init_cfg=dict(
+            type='Pretrained',
+            checkpoint=  # noqa: E251
+            'https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco-person/faster_rcnn_r50_fpn_1x_coco-person_20201216_175929-d022e227.pth'  # noqa: E501
+        )),
+    track_head=dict(
+        type='QuasiDenseTrackHead',


After using base config of ./qdtrack_faster-rcnn_r50_fpn_4e_crowdhuman_mot17-private-half.py, some duplicated keys can be removedd

I realized that it was a fault to set ./qdtrack_faster-rcnn_r50_fpn_4e_crowdhuman_mot17-private-half.py as a base because it is based on '../../_base_/datasets/mot_challenge.py' while the dancetrack qdtrack config shoule be based on '../../_base_/datasets/dancetrack.py'. I think the right way should be creating a new config based on:

_base_ = [ '../../_base_/models/faster_rcnn_r50_fpn.py', '../../_base_/datasets/dancetrack.py', '../../_base_/default_runtime.py' ]

Given the rule that different base config files should have no key conflict, the dancetrack qdtrack config file should inherit no config file using '../../_base_/datasets/mot_challenge.py'. To be precise, if it inherits '../../_base_/datasets/dancetrack.py' and ./qdtrack_faster-rcnn_r50_fpn_4e_mot17-private-half.py' at the same time, it raises error

KeyError: "Duplicate key is not allowed among bases. Duplicate keys: {'data_root', 'train_pipeline', 'img_norm_cfg', 'test_pipeline', 'data', 'dataset_type'}"

GT9505 · 2022-05-10T04:01:37Z

mmtrack/datasets/dancetrack_dataset.py

+            BENCHMARK='DanceTrack',
+            # Option Values: 'train', 'val', 'test'
+            SPLIT_TO_EVAL='val',


I create a commit to show the intention.
The if and elif will be skipped, since dancetrack only have train and val split

noahcao

Reviewed the code. MOT17 config file is verified with given checkpoint to achieve the same HOTA scores.

I refactored the config files to disentangle the config for dataset and the config of model.

* fix format * support dancetrack dataset * delete unused function; inherit dancetrack dataset class from MOTChallenge * remove duplicated content in DancetrackDataset * remove legacy code * refactor DanceTrackDataset class * fix error in dancetrack qdtrack config file * refactor qdtrack config files. * Update dancetrack_dataset.py Co-authored-by: Tao Gong <gt950513@mail.ustc.edu.cn>

noahcao added 3 commits May 3, 2022 11:13

fix format

8b69aaf

support dancetrack dataset

f7777e1

Merge branch 'dancetrack' of https://github.com/noahcao/mmtracking in…

71d0582

…to dancetrack

OceanPang requested a review from GT9505 May 3, 2022 15:29

GT9505 reviewed May 4, 2022

View reviewed changes

delete unused function; inherit dancetrack dataset class from MOTChal…

5058e88

…lenge

GT9505 reviewed May 5, 2022

View reviewed changes

remove duplicated content in DancetrackDataset

a29e15b

noahcao requested a review from GT9505 May 6, 2022 01:52

noahcao mentioned this pull request May 6, 2022

Custom MOT data #530

Open

GT9505 reviewed May 7, 2022

View reviewed changes

remove legacy code

5d21063

noahcao requested a review from GT9505 May 9, 2022 05:28

refactor DanceTrackDataset class

ae1c3c5

GT9505 reviewed May 10, 2022

View reviewed changes

noahcao added 2 commits May 10, 2022 18:04

fix error in dancetrack qdtrack config file

f4caf15

refactor qdtrack config files.

d17cc17

noahcao commented May 11, 2022

View reviewed changes

Update dancetrack_dataset.py

04e9b38

GT9505 approved these changes May 12, 2022

View reviewed changes

GT9505 merged commit 4e11840 into open-mmlab:master May 12, 2022

noahcao mentioned this pull request May 12, 2022

Roadmap of MMTracking #73

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Support DanceTrack dataset for MOT #543

[Feature] Support DanceTrack dataset for MOT #543

noahcao commented May 3, 2022 •

edited

Loading

GT9505 May 4, 2022

GT9505 May 4, 2022

noahcao May 4, 2022

GT9505 May 4, 2022

noahcao May 4, 2022

GT9505 May 4, 2022

noahcao May 4, 2022

GT9505 May 5, 2022

GT9505 May 5, 2022

noahcao May 5, 2022

codecov bot commented May 5, 2022 •

edited

Loading

GT9505 May 7, 2022

noahcao May 7, 2022

GT9505 May 10, 2022

GT9505 May 7, 2022

noahcao May 7, 2022

GT9505 May 7, 2022

noahcao May 7, 2022

GT9505 May 10, 2022

noahcao May 10, 2022

GT9505 May 10, 2022

noahcao left a comment



		@DATASETS.register_module()
		class DanceTrackDataset(CocoVideoDataset):



		@DATASETS.register_module()
		class DanceTrackDataset(MOTChallengeDataset):

[Feature] Support DanceTrack dataset for MOT #543

[Feature] Support DanceTrack dataset for MOT #543

Conversation

noahcao commented May 3, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented May 5, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

noahcao left a comment

Choose a reason for hiding this comment

noahcao commented May 3, 2022 •

edited

Loading

codecov bot commented May 5, 2022 •

edited

Loading