feature space adaptation alone

[jianingwangind]

Hi @wasidennis , before trying to combination feature and output space adaptation, i want to exploit the power of feature space adaptation and was trying to implement feature space alone. The feature map i was using for both domain is the output from layer3 in DeepMulti model, which has a depth 1024. I just set lambda-adv-target1(here the feature map adaptation takes place) as 0.001 and lambda-adv-target2 as 0. But please have a look at the loss during trainig. The loss_D1 decreases to 0 while the loss_adv1 keep increasing. Do you have any ideas? Is it because the segmentation network is shared over domains and isn't updated effectively so the feature maps coming from the same generator of the two domains are easy to distinguish?
Looking forward to your reply!

[jianingwangind]

iter = 1/ 250000, loss_seg1 = 0.000 loss_seg2 = 4.504 loss_adv1 = 0.688, loss_adv2 = 0.701 loss_D1 = 0.691 loss_D2 = 0.693
exp = ./snapshots/GTA2Cityscapes_single
iter = 2/ 250000, loss_seg1 = 0.000 loss_seg2 = 5.427 loss_adv1 = 0.635, loss_adv2 = 0.690 loss_D1 = 0.675 loss_D2 = 0.693
exp = ./snapshots/GTA2Cityscapes_single
iter = 3/ 250000, loss_seg1 = 0.000 loss_seg2 = 3.574 loss_adv1 = 0.590, loss_adv2 = 0.683 loss_D1 = 0.655 loss_D2 = 0.693
exp = ./snapshots/GTA2Cityscapes_single
iter = 4/ 250000, loss_seg1 = 0.000 loss_seg2 = 4.939 loss_adv1 = 0.642, loss_adv2 = 0.679 loss_D1 = 0.625 loss_D2 = 0.693
exp = ./snapshots/GTA2Cityscapes_single
iter = 5/ 250000, loss_seg1 = 0.000 loss_seg2 = 5.252 loss_adv1 = 0.817, loss_adv2 = 0.681 loss_D1 = 0.571 loss_D2 = 0.693
exp = ./snapshots/GTA2Cityscapes_single
iter = 6/ 250000, loss_seg1 = 0.000 loss_seg2 = 2.932 loss_adv1 = 1.118, loss_adv2 = 0.669 loss_D1 = 0.536 loss_D2 = 0.694
exp = ./snapshots/GTA2Cityscapes_single
iter = 7/ 250000, loss_seg1 = 0.000 loss_seg2 = 2.585 loss_adv1 = 1.429, loss_adv2 = 0.667 loss_D1 = 0.440 loss_D2 = 0.694
exp = ./snapshots/GTA2Cityscapes_single
iter = 8/ 250000, loss_seg1 = 0.000 loss_seg2 = 2.548 loss_adv1 = 1.504, loss_adv2 = 0.672 loss_D1 = 0.397 loss_D2 = 0.692
exp = ./snapshots/GTA2Cityscapes_single
iter = 9/ 250000, loss_seg1 = 0.000 loss_seg2 = 7.192 loss_adv1 = 1.974, loss_adv2 = 0.679 loss_D1 = 0.281 loss_D2 = 0.692
exp = ./snapshots/GTA2Cityscapes_single
iter = 10/ 250000, loss_seg1 = 0.000 loss_seg2 = 3.202 loss_adv1 = 2.110, loss_adv2 = 0.686 loss_D1 = 0.294 loss_D2 = 0.691
exp = ./snapshots/GTA2Cityscapes_single
iter = 11/ 250000, loss_seg1 = 0.000 loss_seg2 = 1.929 loss_adv1 = 2.131, loss_adv2 = 0.690 loss_D1 = 0.153 loss_D2 = 0.692
exp = ./snapshots/GTA2Cityscapes_single
iter = 12/ 250000, loss_seg1 = 0.000 loss_seg2 = 3.053 loss_adv1 = 3.365, loss_adv2 = 0.693 loss_D1 = 0.175 loss_D2 = 0.690
exp = ./snapshots/GTA2Cityscapes_single
iter = 13/ 250000, loss_seg1 = 0.000 loss_seg2 = 3.893 loss_adv1 = 3.286, loss_adv2 = 0.696 loss_D1 = 0.157 loss_D2 = 0.688
exp = ./snapshots/GTA2Cityscapes_single
iter = 14/ 250000, loss_seg1 = 0.000 loss_seg2 = 2.733 loss_adv1 = 3.633, loss_adv2 = 0.702 loss_D1 = 0.072 loss_D2 = 0.687
exp = ./snapshots/GTA2Cityscapes_single
iter = 15/ 250000, loss_seg1 = 0.000 loss_seg2 = 3.252 loss_adv1 = 5.770, loss_adv2 = 0.689 loss_D1 = 0.019 loss_D2 = 0.696
exp = ./snapshots/GTA2Cityscapes_single
iter = 16/ 250000, loss_seg1 = 0.000 loss_seg2 = 2.261 loss_adv1 = 7.522, loss_adv2 = 0.687 loss_D1 = 0.060 loss_D2 = 0.693
exp = ./snapshots/GTA2Cityscapes_single
iter = 17/ 250000, loss_seg1 = 0.000 loss_seg2 = 1.139 loss_adv1 = 9.581, loss_adv2 = 0.698 loss_D1 = 0.016 loss_D2 = 0.682
exp = ./snapshots/GTA2Cityscapes_single
iter = 18/ 250000, loss_seg1 = 0.000 loss_seg2 = 1.615 loss_adv1 = 7.422, loss_adv2 = 0.684 loss_D1 = 0.013 loss_D2 = 0.683
exp = ./snapshots/GTA2Cityscapes_single
iter = 19/ 250000, loss_seg1 = 0.000 loss_seg2 = 1.902 loss_adv1 = 6.613, loss_adv2 = 0.696 loss_D1 = 0.032 loss_D2 = 0.686
exp = ./snapshots/GTA2Cityscapes_single
iter = 20/ 250000, loss_seg1 = 0.000 loss_seg2 = 2.271 loss_adv1 = 9.327, loss_adv2 = 0.678 loss_D1 = 0.132 loss_D2 = 0.686
exp = ./snapshots/GTA2Cityscapes_single
iter = 21/ 250000, loss_seg1 = 0.000 loss_seg2 = 4.740 loss_adv1 = 7.290, loss_adv2 = 0.685 loss_D1 = 0.020 loss_D2 = 0.680
exp = ./snapshots/GTA2Cityscapes_single
iter = 22/ 250000, loss_seg1 = 0.000 loss_seg2 = 1.068 loss_adv1 = 6.424, loss_adv2 = 0.662 loss_D1 = 0.102 loss_D2 = 0.665
exp = ./snapshots/GTA2Cityscapes_single
iter = 23/ 250000, loss_seg1 = 0.000 loss_seg2 = 1.491 loss_adv1 = 5.819, loss_adv2 = 0.642 loss_D1 = 0.113 loss_D2 = 0.673
exp = ./snapshots/GTA2Cityscapes_single
iter = 24/ 250000, loss_seg1 = 0.000 loss_seg2 = 1.819 loss_adv1 = 6.994, loss_adv2 = 0.621 loss_D1 = 0.005 loss_D2 = 0.687
exp = ./snapshots/GTA2Cityscapes_single
iter = 25/ 250000, loss_seg1 = 0.000 loss_seg2 = 0.990 loss_adv1 = 8.553, loss_adv2 = 0.711 loss_D1 = 0.001 loss_D2 = 0.675
exp = ./snapshots/GTA2Cityscapes_single
iter = 26/ 250000, loss_seg1 = 0.000 loss_seg2 = 2.314 loss_adv1 = 8.346, loss_adv2 = 0.770 loss_D1 = 0.018 loss_D2 = 0.640
exp = ./snapshots/GTA2Cityscapes_single
iter = 27/ 250000, loss_seg1 = 0.000 loss_seg2 = 0.809 loss_adv1 = 11.396, loss_adv2 = 0.730 loss_D1 = 0.000 loss_D2 = 0.664
exp = ./snapshots/GTA2Cityscapes_single
iter = 28/ 250000, loss_seg1 = 0.000 loss_seg2 = 2.640 loss_adv1 = 9.855, loss_adv2 = 0.738 loss_D1 = 0.041 loss_D2 = 0.671
exp = ./snapshots/GTA2Cityscapes_single
iter = 29/ 250000, loss_seg1 = 0.000 loss_seg2 = 1.536 loss_adv1 = 4.906, loss_adv2 = 0.796 loss_D1 = 0.050 loss_D2 = 0.673
exp = ./snapshots/GTA2Cityscapes_single
iter = 30/ 250000, loss_seg1 = 0.000 loss_seg2 = 1.098 loss_adv1 = 9.137, loss_adv2 = 0.733 loss_D1 = 0.008 loss_D2 = 0.700
exp = ./snapshots/GTA2Cityscapes_single
iter = 31/ 250000, loss_seg1 = 0.000 loss_seg2 = 2.407 loss_adv1 = 11.512, loss_adv2 = 0.833 loss_D1 = 1.986 loss_D2 = 0.697

[wasidennis]

Based on experiments we tried before, feature space adaptation is more sensitive and would need a much smaller weight for the adversarial loss.

[tarun005]

I am facing the same trouble. The adversarial loss is diverging and loss_D is going to 0 very quickly. could you mention the hyper-parameter setting to reproduce the feature space adaptation setting? Specifically, did you pre-train your model on synthetic data before performing the adaptation?

[WangKK1996]

I am facing the same trouble. The adversarial loss is diverging and loss_D is going to 0 very quickly. could you mention the hyper-parameter setting to reproduce the feature space adaptation setting? Specifically, did you pre-train your model on synthetic data before performing the adaptation?

I met the some trouble!I'm a beginner at segmentation and GAN but I guess when the source and the target domain are too far away from each other, the Discriminator can tell the difference between them very easily so the gradient tend to be 0; I tried to add weight for the adversarial loss but the Generator starts to produce white noise. So is there any traning tricks to avoid this problem?