resnet backbone dont converge

[victor00070]

I changed the backbone from vgg to resnet50. First, I have to use very low learning rate otherwise training diverges. Second, mAP is usually lower than vgg backbone adaptive teacher even if I train long. How can I achieve good performance with resnet50 backbone?
I am using gradient clipping and learning rate of 0.001 to stop diverging.

[yujheli]

By VGG you mean you train the model on the cityscapes dataset? The config file for clipart is using Resnet-101 and it is working. Replacing with ResNet50 should not be an issue with a similar config file.