CenterNet(Objects as points)

[kellenf]

model	AP / FPS	Flip AP / FPS	Note
ResNet18-DCN	28.1 / 142	30.0 / 71	official code record
ResNet18-DCN	28.0	30.0	official code run with DCNv2 latest
ResNet18-DCN	29.5	31.0	mmdet
ResNet18	23.2	24.7	Official code training
ResNet18	26.0	27.4	mmdet
DLANet34	37.4 / 52	39.2 / 28	official code record
DLANet34	38.6		mmdet

Compared to the source code, we refer to CenterNet-Better, and make the following changes：

• fix wrong image mean and variance in image normalization to be compatible with the pre-trained backbone.
• Use SGD rather than ADAM optimizer and add warmup and grad clip. If I use the original Adam optimizer, the ResNet18-DCN mAP is 29.1.
• Use DistributedDataParallel as other models in MMDetection rather than using DataParallel.

Due to complex data enhancement, we find that the performance is unstable and may fluctuate by about 0.4 mAP. mAP 29.4 ~ 29.8 is acceptable in ResNet18-DCN.

DLANet source code will be released in other PRs.

[jshilong]

Thanks for your contribution!
We would give some general advice on code structure first, and give some detailed comments after you reproduce all the performance and refactor the code.

[ZwwWayne]

The general structure of the code looks good to me.

[igo312]

@kellenf I don't know if it's appropriate to ask question here.

Really exceting about this work!

But I think there is may a bug here.

I notice when generating the box target in centernet_head.py it should calculate the width_ratio and height_ratio. However there is a paramter named img_shape in function get_targets you use img_metas[0]['pad_shape'] directly. It will occur index error when some x or y is close to the border and the actual pad_shape is bigger than img_metas[0][pad_shape] result in the width_ratio or height_ratio is smaller than correct value so that the index error raiesd.

And when the dataset get multi scale and set the resuze:keep_ratio=True. It seems that target box will be not corret.

What I think it should feed img_metas instead of img_shape and get the img_shape in the loop such like:

for batch_id in range(bs):
    img_h, img_w = img_metas[batch_id]['pad_shape'][:2]

[hhaAndroid]

@kellenf I don't know if it's appropriate to ask question here.

Really exceting about this work!

But I think there is may a bug here.

I notice when generating the box target in centernet_head.py it should calculate the width_ratio and height_ratio. However there is a paramter named img_shape in function get_targets you use img_metas[0]['pad_shape'] directly. It will occur index error when some x or y is close to the border and the actual pad_shape is bigger than img_metas[0][pad_shape] result in the width_ratio or height_ratio is smaller than correct value so that the index error raiesd.

And when the dataset get multi scale and set the resuze:keep_ratio=True. It seems that target box will be not corret.

What I think it should feed img_metas instead of img_shape and get the img_shape in the loop such like:

for batch_id in range(bs):
    img_h, img_w = img_metas[batch_id]['pad_shape'][:2]

@igo312 Thanks for your attention. In the default configuration, width_ratio or height_ratio is always equal to 4, and multi-scale training is not supported by default, so there is no problem. But if you consider multi-scale training, there are hidden dangers. The best way is to replace img_metas[0]['pad_shape'] with img_metas[0]['batch_input_shape']. What do you think？

[igo312]

@igo312 Thanks for your attention. In the default configuration, width_ratio or height_ratio is always equal to 4...

@hhaAndroid Well I don't really understand what's the difference between batch_input_shape and pad_shape with same index.It still point to same image.

The problem should be when a batch get difference scale of image, using the index 0 of img_meta represent all img_meta should not be appropriate.

[hhaAndroid]

@igo312 Thanks for your attention. In the default configuration, width_ratio or height_ratio is always equal to 4...

@hhaAndroid Well I don't really understand what's the difference between batch_input_shape and pad_shape with same index.It still point to same image.

The problem should be when a batch get difference scale of image, using the index 0 of img_meta represent all img_meta should not be appropriate.

batch_input_shape refers to the size of the image input to the backbone, batch images are always the same.

[igo312]

batch_input_shape refers to the size of the image input to the backbone, batch images are always the same.

yeah...You are right, batch_input_shape will work and sorry for my unclear aboud these things.

Anyway, really thanks for your response!