why the train and val dice are Less than 0

[yml-bit]

we only change the input but still CT and trained, the train and val dices are less than 0 in the training, what's wrong?

[AlexYouXin]

For the standard calculating way, Dice scores range in [0, 1]. And the weight for Dice loss in our code is also normalized. So a negative Dice is strictly avoided. Please check the Dice score for each class. Maybe a detailed check for the input/prediction/ground truth is a good way to solve your problem.

[yml-bit]

For the standard calculating way, Dice scores range in [0, 1]. And the weight for Dice loss in our code is also normalized. So a negative Dice is strictly avoided. Please check the Dice score for each class. Maybe a detailed check for the input/prediction/ground truth is a good way to solve your problem.

Thank you for your prompt response. Can you share your training log to compare to see the training process correctly. Thank you!

[AlexYouXin]

Here I give the training log information of the first 300 epochs. Hope that it can help!

[21:54:39.443] epoch : 0, iteration : 8, train loss : 0.869225, train loss_ce: 0.834584, train loss_dice: 0.903867, train dice : 0.096133
[21:55:16.902] epoch : 0, iteration : 16, train loss : 0.699132, train loss_ce: 0.655983, train loss_dice: 0.742281, train dice : 0.257719
[21:55:40.858] epoch : 0, mean train loss : 1.047659, mean train ce loss: 1.307570, mean train dice : 0.212252
[21:56:02.068] epoch : 0, iteration : 6, val loss : 1.130263, val loss_ce: 1.510375, val loss_dice: 0.750152, val dice : 0.249848
[21:56:09.346] epoch : 0, mean val loss : 1.097060, mean val ce loss: 1.418509, mean val dice : 0.224389
[21:56:09.694] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 1, val dice: 0.22438941895961761
[21:56:56.647] epoch : 1, iteration : 24, train loss : 1.150042, train loss_ce: 1.553062, train loss_dice: 0.747022, train dice : 0.252978
[21:57:34.089] epoch : 1, iteration : 32, train loss : 0.828518, train loss_ce: 0.969924, train loss_dice: 0.687112, train dice : 0.312888
[21:57:50.797] epoch : 1, mean train loss : 0.976200, mean train ce loss: 1.211021, mean train dice : 0.258620
[21:58:06.292] epoch : 1, iteration : 12, val loss : 1.030873, val loss_ce: 1.352672, val loss_dice: 0.709074, val dice : 0.290926
[21:58:17.897] epoch : 1, mean val loss : 0.893180, mean val ce loss: 1.056200, mean val dice : 0.269840
[21:58:18.194] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 2, val dice: 0.2698398232460022
[21:58:55.801] epoch : 2, iteration : 40, train loss : 1.056669, train loss_ce: 1.425964, train loss_dice: 0.687374, train dice : 0.312626
[21:59:33.218] epoch : 2, iteration : 48, train loss : 1.118496, train loss_ce: 1.498088, train loss_dice: 0.738904, train dice : 0.261096
[21:59:59.285] epoch : 2, mean train loss : 0.911438, mean train ce loss: 1.121580, mean train dice : 0.298704
[22:00:15.880] epoch : 2, iteration : 18, val loss : 0.921765, val loss_ce: 1.186695, val loss_dice: 0.656834, val dice : 0.343166
[22:00:25.739] epoch : 2, iteration : 24, val loss : 0.887424, val loss_ce: 1.098053, val loss_dice: 0.676794, val dice : 0.323206
[22:00:25.885] epoch : 2, mean val loss : 0.883200, mean val ce loss: 1.025231, mean val dice : 0.258831
[22:00:46.130] epoch : 3, iteration : 56, train loss : 0.904238, train loss_ce: 1.138815, train loss_dice: 0.669661, train dice : 0.330339
[22:01:23.612] epoch : 3, iteration : 64, train loss : 0.769415, train loss_ce: 0.860580, train loss_dice: 0.678250, train dice : 0.321750
[22:01:58.906] epoch : 3, iteration : 72, train loss : 1.098064, train loss_ce: 1.420604, train loss_dice: 0.775525, train dice : 0.224475
[22:01:59.100] epoch : 3, mean train loss : 0.861178, mean train ce loss: 1.021090, mean train dice : 0.298733
[22:02:23.448] epoch : 3, iteration : 30, val loss : 0.778885, val loss_ce: 0.785826, val loss_dice: 0.771943, val dice : 0.228057
[22:02:32.895] epoch : 3, mean val loss : 0.831884, mean val ce loss: 0.938015, mean val dice : 0.274248
[22:02:33.318] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 4, val dice: 0.2742476463317871
[22:03:18.324] epoch : 4, iteration : 80, train loss : 0.780883, train loss_ce: 0.842174, train loss_dice: 0.719592, train dice : 0.280408
[22:03:55.709] epoch : 4, iteration : 88, train loss : 0.812381, train loss_ce: 0.979153, train loss_dice: 0.645610, train dice : 0.354390
[22:04:03.070] epoch : 4, mean train loss : 0.848014, mean train ce loss: 0.998142, mean train dice : 0.302113
[22:04:23.547] epoch : 4, iteration : 36, val loss : 1.148447, val loss_ce: 1.580969, val loss_dice: 0.715925, val dice : 0.284075
[22:04:31.121] epoch : 4, mean val loss : 0.820177, mean val ce loss: 0.937127, mean val dice : 0.296774
[22:04:31.412] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 5, val dice: 0.2967742681503296
[22:05:17.757] epoch : 5, iteration : 96, train loss : 0.884117, train loss_ce: 1.078014, train loss_dice: 0.690220, train dice : 0.309780
[22:05:55.151] epoch : 5, iteration : 104, train loss : 0.745009, train loss_ce: 0.827596, train loss_dice: 0.662422, train dice : 0.337578
[22:06:11.915] epoch : 5, mean train loss : 0.798879, mean train ce loss: 0.917040, mean train dice : 0.319283
[22:06:27.077] epoch : 5, iteration : 42, val loss : 0.855839, val loss_ce: 1.005996, val loss_dice: 0.705682, val dice : 0.294318
[22:06:45.794] epoch : 5, iteration : 48, val loss : 0.746089, val loss_ce: 0.835729, val loss_dice: 0.656449, val dice : 0.343551
[22:06:45.968] epoch : 5, mean val loss : 0.814362, mean val ce loss: 0.938181, mean val dice : 0.309457
[22:06:46.253] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 6, val dice: 0.30945709347724915
[22:07:17.194] epoch : 6, iteration : 112, train loss : 0.794405, train loss_ce: 0.924933, train loss_dice: 0.663876, train dice : 0.336124
[22:07:54.565] epoch : 6, iteration : 120, train loss : 0.737065, train loss_ce: 0.721781, train loss_dice: 0.752350, train dice : 0.247650
[22:08:20.635] epoch : 6, mean train loss : 0.808317, mean train ce loss: 0.943394, mean train dice : 0.326759
[22:08:43.496] epoch : 6, iteration : 54, val loss : 0.722643, val loss_ce: 0.661506, val loss_dice: 0.783780, val dice : 0.216220
[22:08:46.918] epoch : 6, mean val loss : 0.793956, mean val ce loss: 0.863453, mean val dice : 0.275542
[22:09:06.276] epoch : 7, iteration : 128, train loss : 0.746620, train loss_ce: 0.815337, train loss_dice: 0.677904, train dice : 0.322096
[22:09:43.683] epoch : 7, iteration : 136, train loss : 0.724296, train loss_ce: 0.817595, train loss_dice: 0.630997, train dice : 0.369003
[22:10:19.065] epoch : 7, iteration : 144, train loss : 0.735721, train loss_ce: 0.683437, train loss_dice: 0.788004, train dice : 0.211996
[22:10:19.252] epoch : 7, mean train loss : 0.764191, mean train ce loss: 0.844483, mean train dice : 0.316101
[22:10:37.751] epoch : 7, iteration : 60, val loss : 0.844693, val loss_ce: 1.056224, val loss_dice: 0.633161, val dice : 0.366839
[22:10:48.228] epoch : 7, mean val loss : 0.860243, mean val ce loss: 1.035255, mean val dice : 0.314769
[22:10:48.515] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 8, val dice: 0.31476891040802
[22:11:43.310] epoch : 8, iteration : 152, train loss : 0.665330, train loss_ce: 0.616288, train loss_dice: 0.714372, train dice : 0.285628
[22:12:20.727] epoch : 8, iteration : 160, train loss : 0.649157, train loss_ce: 0.639160, train loss_dice: 0.659154, train dice : 0.340846
[22:12:28.195] epoch : 8, mean train loss : 0.751400, mean train ce loss: 0.843735, mean train dice : 0.340935
[22:12:42.590] epoch : 8, iteration : 66, val loss : 0.834157, val loss_ce: 1.018154, val loss_dice: 0.650161, val dice : 0.349839
[22:12:54.983] epoch : 8, iteration : 72, val loss : 0.819119, val loss_ce: 0.954884, val loss_dice: 0.683354, val dice : 0.316646
[22:12:55.123] epoch : 8, mean val loss : 0.785649, mean val ce loss: 0.891650, mean val dice : 0.320351
[22:12:55.419] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 9, val dice: 0.32035133242607117
[22:13:31.191] epoch : 9, iteration : 168, train loss : 0.736341, train loss_ce: 0.784374, train loss_dice: 0.688307, train dice : 0.311693
[22:14:08.580] epoch : 9, iteration : 176, train loss : 0.685926, train loss_ce: 0.751423, train loss_dice: 0.620428, train dice : 0.379572
[22:14:25.311] epoch : 9, mean train loss : 0.779302, mean train ce loss: 0.879572, mean train dice : 0.320968
[22:14:50.636] epoch : 9, iteration : 78, val loss : 0.964528, val loss_ce: 1.270776, val loss_dice: 0.658280, val dice : 0.341720
[22:14:56.569] epoch : 9, mean val loss : 0.850345, mean val ce loss: 1.014281, mean val dice : 0.313592
[22:15:36.855] epoch : 10, iteration : 184, train loss : 1.155556, train loss_ce: 1.597749, train loss_dice: 0.713362, train dice : 0.286638
[22:16:14.269] epoch : 10, iteration : 192, train loss : 0.950676, train loss_ce: 1.181821, train loss_dice: 0.719532, train dice : 0.280468
[22:16:40.400] epoch : 10, mean train loss : 0.789545, mean train ce loss: 0.891549, mean train dice : 0.312458
[22:17:01.342] epoch : 10, iteration : 84, val loss : 1.211069, val loss_ce: 1.681532, val loss_dice: 0.740605, val dice : 0.259395
[22:17:03.547] epoch : 10, mean val loss : 0.840884, mean val ce loss: 0.981701, mean val dice : 0.299934
[22:17:27.281] epoch : 11, iteration : 200, train loss : 1.031020, train loss_ce: 1.316101, train loss_dice: 0.745938, train dice : 0.254062
[22:18:04.666] epoch : 11, iteration : 208, train loss : 0.718673, train loss_ce: 0.821668, train loss_dice: 0.615678, train dice : 0.384322
[22:18:39.906] epoch : 11, iteration : 216, train loss : 0.856844, train loss_ce: 0.787192, train loss_dice: 0.926496, train dice : 0.073504
[22:18:40.101] epoch : 11, mean train loss : 0.801771, mean train ce loss: 0.922278, mean train dice : 0.318737
[22:19:03.407] epoch : 11, iteration : 90, val loss : 0.834885, val loss_ce: 1.044943, val loss_dice: 0.624827, val dice : 0.375173
[22:19:09.031] epoch : 11, iteration : 96, val loss : 0.814149, val loss_ce: 0.956253, val loss_dice: 0.672045, val dice : 0.327955
[22:19:09.274] epoch : 11, mean val loss : 0.787421, mean val ce loss: 0.884081, mean val dice : 0.309239
[22:20:05.936] epoch : 12, iteration : 224, train loss : 0.717951, train loss_ce: 0.756220, train loss_dice: 0.679681, train dice : 0.320319
[22:20:43.323] epoch : 12, iteration : 232, train loss : 0.780146, train loss_ce: 0.921669, train loss_dice: 0.638622, train dice : 0.361378
[22:20:50.673] epoch : 12, mean train loss : 0.768053, mean train ce loss: 0.862302, mean train dice : 0.326196
[22:21:18.830] epoch : 12, iteration : 102, val loss : 0.829342, val loss_ce: 0.892784, val loss_dice: 0.765899, val dice : 0.234101
[22:21:26.073] epoch : 12, mean val loss : 0.779017, mean val ce loss: 0.878275, mean val dice : 0.320240
[22:22:19.693] epoch : 13, iteration : 240, train loss : 0.835404, train loss_ce: 0.983925, train loss_dice: 0.686883, train dice : 0.313117
[22:22:57.098] epoch : 13, iteration : 248, train loss : 0.757820, train loss_ce: 0.856643, train loss_dice: 0.658998, train dice : 0.341002
[22:23:13.991] epoch : 13, mean train loss : 0.811027, mean train ce loss: 0.925361, mean train dice : 0.303308
[22:23:33.970] epoch : 13, iteration : 108, val loss : 0.706748, val loss_ce: 0.775755, val loss_dice: 0.637741, val dice : 0.362259
[22:23:45.134] epoch : 13, mean val loss : 0.725688, mean val ce loss: 0.783616, mean val dice : 0.332240
[22:23:45.424] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 14, val dice: 0.3322397470474243
[22:24:21.696] epoch : 14, iteration : 256, train loss : 0.703147, train loss_ce: 0.756461, train loss_dice: 0.649833, train dice : 0.350167
[22:24:59.120] epoch : 14, iteration : 264, train loss : 0.666289, train loss_ce: 0.720716, train loss_dice: 0.611861, train dice : 0.388139
[22:25:25.173] epoch : 14, mean train loss : 0.750013, mean train ce loss: 0.843703, mean train dice : 0.343676
[22:25:50.352] epoch : 14, iteration : 114, val loss : 0.898095, val loss_ce: 1.167680, val loss_dice: 0.628510, val dice : 0.371490
[22:26:08.031] epoch : 14, iteration : 120, val loss : 0.861428, val loss_ce: 1.079169, val loss_dice: 0.643688, val dice : 0.356312
[22:26:08.211] epoch : 14, mean val loss : 0.795065, mean val ce loss: 0.925445, mean val dice : 0.335314
[22:26:08.505] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 15, val dice: 0.335314005613327
[22:26:30.488] epoch : 15, iteration : 272, train loss : 0.674784, train loss_ce: 0.692065, train loss_dice: 0.657503, train dice : 0.342497
[22:27:07.948] epoch : 15, iteration : 280, train loss : 0.799062, train loss_ce: 0.968411, train loss_dice: 0.629714, train dice : 0.370286
[22:27:43.298] epoch : 15, iteration : 288, train loss : 0.689523, train loss_ce: 0.633670, train loss_dice: 0.745377, train dice : 0.254623
[22:27:43.417] epoch : 15, mean train loss : 0.742684, mean train ce loss: 0.814667, mean train dice : 0.329298
[22:28:02.900] epoch : 15, iteration : 126, val loss : 0.714988, val loss_ce: 0.789960, val loss_dice: 0.640017, val dice : 0.359983
[22:28:12.529] epoch : 15, mean val loss : 0.763122, mean val ce loss: 0.851987, mean val dice : 0.325743
[22:29:01.871] epoch : 16, iteration : 296, train loss : 0.706762, train loss_ce: 0.786618, train loss_dice: 0.626906, train dice : 0.373094
[22:29:39.310] epoch : 16, iteration : 304, train loss : 0.734380, train loss_ce: 0.781604, train loss_dice: 0.687156, train dice : 0.312844
[22:29:46.824] epoch : 16, mean train loss : 0.729513, mean train ce loss: 0.804735, mean train dice : 0.345710
[22:30:06.616] epoch : 16, iteration : 132, val loss : 0.829495, val loss_ce: 1.026878, val loss_dice: 0.632112, val dice : 0.367888
[22:30:13.388] epoch : 16, mean val loss : 0.779473, mean val ce loss: 0.905508, mean val dice : 0.346561
[22:30:13.667] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 17, val dice: 0.3465611934661865
[22:31:00.996] epoch : 17, iteration : 312, train loss : 0.754819, train loss_ce: 0.900697, train loss_dice: 0.608941, train dice : 0.391059
[22:31:38.381] epoch : 17, iteration : 320, train loss : 0.611313, train loss_ce: 0.582115, train loss_dice: 0.640511, train dice : 0.359489
[22:31:55.075] epoch : 17, mean train loss : 0.755602, mean train ce loss: 0.848312, mean train dice : 0.337108
[22:32:19.514] epoch : 17, iteration : 138, val loss : 0.764128, val loss_ce: 0.912494, val loss_dice: 0.615762, val dice : 0.384238
[22:32:31.295] epoch : 17, iteration : 144, val loss : 0.839172, val loss_ce: 1.055722, val loss_dice: 0.622622, val dice : 0.377378
[22:32:31.406] epoch : 17, mean val loss : 0.831542, mean val ce loss: 1.000959, mean val dice : 0.337876
[22:33:05.157] epoch : 18, iteration : 328, train loss : 0.875867, train loss_ce: 1.040144, train loss_dice: 0.711589, train dice : 0.288411
[22:33:42.574] epoch : 18, iteration : 336, train loss : 0.798277, train loss_ce: 0.912970, train loss_dice: 0.683585, train dice : 0.316415
[22:34:08.673] epoch : 18, mean train loss : 0.750934, mean train ce loss: 0.823005, mean train dice : 0.321138
[22:34:37.234] epoch : 18, iteration : 150, val loss : 0.857548, val loss_ce: 1.083787, val loss_dice: 0.631309, val dice : 0.368691
[22:34:38.492] epoch : 18, mean val loss : 0.797056, mean val ce loss: 0.925420, mean val dice : 0.331308
[22:35:04.172] epoch : 19, iteration : 344, train loss : 0.817432, train loss_ce: 0.997171, train loss_dice: 0.637693, train dice : 0.362307
[22:35:41.562] epoch : 19, iteration : 352, train loss : 0.839893, train loss_ce: 0.954118, train loss_dice: 0.725669, train dice : 0.274331
[22:36:16.838] epoch : 19, iteration : 360, train loss : 0.674304, train loss_ce: 0.588403, train loss_dice: 0.760205, train dice : 0.239795
[22:36:17.055] epoch : 19, mean train loss : 0.757885, mean train ce loss: 0.843522, mean train dice : 0.327751
[22:36:38.741] epoch : 19, iteration : 156, val loss : 0.774002, val loss_ce: 0.933789, val loss_dice: 0.614214, val dice : 0.385786
[22:36:44.519] epoch : 19, mean val loss : 0.799526, mean val ce loss: 0.900395, mean val dice : 0.301343
[22:37:32.406] epoch : 20, iteration : 368, train loss : 0.638811, train loss_ce: 0.655353, train loss_dice: 0.622269, train dice : 0.377731
[22:38:09.791] epoch : 20, iteration : 376, train loss : 0.780107, train loss_ce: 0.873343, train loss_dice: 0.686872, train dice : 0.313128
[22:38:17.242] epoch : 20, mean train loss : 0.706427, mean train ce loss: 0.747661, mean train dice : 0.334806
[22:38:32.292] epoch : 20, iteration : 162, val loss : 0.742215, val loss_ce: 0.877340, val loss_dice: 0.607089, val dice : 0.392911
[22:38:46.344] epoch : 20, iteration : 168, val loss : 0.795514, val loss_ce: 0.913677, val loss_dice: 0.677350, val dice : 0.322650
[22:38:46.548] epoch : 20, mean val loss : 0.766647, mean val ce loss: 0.868550, mean val dice : 0.335256
[22:39:36.955] epoch : 21, iteration : 384, train loss : 0.839838, train loss_ce: 1.024194, train loss_dice: 0.655481, train dice : 0.344519
[22:40:14.352] epoch : 21, iteration : 392, train loss : 0.635557, train loss_ce: 0.559283, train loss_dice: 0.711831, train dice : 0.288169
[22:40:31.100] epoch : 21, mean train loss : 0.722913, mean train ce loss: 0.754015, mean train dice : 0.308188
[22:40:52.878] epoch : 21, iteration : 174, val loss : 0.800207, val loss_ce: 0.963179, val loss_dice: 0.637236, val dice : 0.362764
[22:41:01.200] epoch : 21, mean val loss : 0.732496, mean val ce loss: 0.805239, mean val dice : 0.340248
[22:41:34.033] epoch : 22, iteration : 400, train loss : 0.807741, train loss_ce: 0.997622, train loss_dice: 0.617860, train dice : 0.382140
[22:42:11.420] epoch : 22, iteration : 408, train loss : 0.685578, train loss_ce: 0.763697, train loss_dice: 0.607459, train dice : 0.392541
[22:42:37.534] epoch : 22, mean train loss : 0.714200, mean train ce loss: 0.777036, mean train dice : 0.348635
[22:42:57.597] epoch : 22, iteration : 180, val loss : 0.721204, val loss_ce: 0.823080, val loss_dice: 0.619328, val dice : 0.380672
[22:43:01.157] epoch : 22, mean val loss : 0.781763, mean val ce loss: 0.891122, mean val dice : 0.327595
[22:43:23.406] epoch : 23, iteration : 416, train loss : 0.719448, train loss_ce: 0.730077, train loss_dice: 0.708819, train dice : 0.291181
[22:44:00.806] epoch : 23, iteration : 424, train loss : 0.689497, train loss_ce: 0.698646, train loss_dice: 0.680347, train dice : 0.319653
[22:44:36.094] epoch : 23, iteration : 432, train loss : 0.706845, train loss_ce: 0.730777, train loss_dice: 0.682914, train dice : 0.317086
[22:44:36.312] epoch : 23, mean train loss : 0.741030, mean train ce loss: 0.824181, mean train dice : 0.342120
[22:44:50.214] epoch : 23, iteration : 186, val loss : 0.730801, val loss_ce: 0.804260, val loss_dice: 0.657343, val dice : 0.342657
[22:45:03.489] epoch : 23, iteration : 192, val loss : 0.562354, val loss_ce: 0.443248, val loss_dice: 0.681460, val dice : 0.318540
[22:45:03.699] epoch : 23, mean val loss : 0.743634, mean val ce loss: 0.818725, mean val dice : 0.331457
[22:45:51.077] epoch : 24, iteration : 440, train loss : 0.603264, train loss_ce: 0.568050, train loss_dice: 0.638478, train dice : 0.361522
[22:46:28.488] epoch : 24, iteration : 448, train loss : 0.922105, train loss_ce: 1.188061, train loss_dice: 0.656149, train dice : 0.343851
[22:46:35.888] epoch : 24, mean train loss : 0.747269, mean train ce loss: 0.836085, mean train dice : 0.341547
[22:46:53.920] epoch : 24, iteration : 198, val loss : 0.774346, val loss_ce: 0.908816, val loss_dice: 0.639875, val dice : 0.360125
[22:47:03.038] epoch : 24, mean val loss : 0.742682, mean val ce loss: 0.839493, mean val dice : 0.354130
[22:47:03.337] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 25, val dice: 0.35412997007369995
[22:47:49.655] epoch : 25, iteration : 456, train loss : 0.741067, train loss_ce: 0.857687, train loss_dice: 0.624447, train dice : 0.375553
[22:48:27.087] epoch : 25, iteration : 464, train loss : 0.820594, train loss_ce: 0.958506, train loss_dice: 0.682681, train dice : 0.317319
[22:48:43.946] epoch : 25, mean train loss : 0.747065, mean train ce loss: 0.837213, mean train dice : 0.343083
[22:49:08.887] epoch : 25, iteration : 204, val loss : 0.890973, val loss_ce: 1.152699, val loss_dice: 0.629248, val dice : 0.370752
[22:49:16.496] epoch : 25, mean val loss : 0.825051, mean val ce loss: 0.969219, mean val dice : 0.319117
[22:49:51.019] epoch : 26, iteration : 472, train loss : 0.724584, train loss_ce: 0.818806, train loss_dice: 0.630363, train dice : 0.369637
[22:50:28.445] epoch : 26, iteration : 480, train loss : 0.749052, train loss_ce: 0.820486, train loss_dice: 0.677618, train dice : 0.322382
[22:50:54.535] epoch : 26, mean train loss : 0.770318, mean train ce loss: 0.856481, mean train dice : 0.315845
[22:51:14.903] epoch : 26, iteration : 210, val loss : 0.805072, val loss_ce: 0.994768, val loss_dice: 0.615376, val dice : 0.384624
[22:51:20.751] epoch : 26, iteration : 216, val loss : 0.834211, val loss_ce: 1.044819, val loss_dice: 0.623604, val dice : 0.376396
[22:51:20.924] epoch : 26, mean val loss : 0.803129, mean val ce loss: 0.963758, mean val dice : 0.357500
[22:51:21.218] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 27, val dice: 0.35750025510787964
[22:51:43.163] epoch : 27, iteration : 488, train loss : 0.888180, train loss_ce: 1.131350, train loss_dice: 0.645010, train dice : 0.354990
[22:52:20.558] epoch : 27, iteration : 496, train loss : 0.799093, train loss_ce: 0.866787, train loss_dice: 0.731400, train dice : 0.268600
[22:52:55.874] epoch : 27, iteration : 504, train loss : 0.588016, train loss_ce: 0.513361, train loss_dice: 0.662672, train dice : 0.337328
[22:52:55.996] epoch : 27, mean train loss : 0.771884, mean train ce loss: 0.885965, mean train dice : 0.342197
[22:53:16.516] epoch : 27, iteration : 222, val loss : 0.707414, val loss_ce: 0.766902, val loss_dice: 0.647925, val dice : 0.352075
[22:53:27.509] epoch : 27, mean val loss : 0.775234, mean val ce loss: 0.889138, mean val dice : 0.338670
[22:54:20.724] epoch : 28, iteration : 512, train loss : 0.667073, train loss_ce: 0.655178, train loss_dice: 0.678968, train dice : 0.321032
[22:54:58.125] epoch : 28, iteration : 520, train loss : 0.680386, train loss_ce: 0.722823, train loss_dice: 0.637950, train dice : 0.362050
[22:55:05.439] epoch : 28, mean train loss : 0.751986, mean train ce loss: 0.864148, mean train dice : 0.360176
[22:55:29.995] epoch : 28, iteration : 228, val loss : 0.729198, val loss_ce: 0.804476, val loss_dice: 0.653920, val dice : 0.346080
[22:55:36.332] epoch : 28, mean val loss : 0.874393, mean val ce loss: 1.060014, mean val dice : 0.311228
[22:56:19.395] epoch : 29, iteration : 528, train loss : 0.933815, train loss_ce: 1.233782, train loss_dice: 0.633848, train dice : 0.366152
[22:56:56.788] epoch : 29, iteration : 536, train loss : 0.749246, train loss_ce: 0.873616, train loss_dice: 0.624877, train dice : 0.375123
[22:57:13.532] epoch : 29, mean train loss : 0.773496, mean train ce loss: 0.886150, mean train dice : 0.339159
[22:57:30.134] epoch : 29, iteration : 234, val loss : 0.703511, val loss_ce: 0.798036, val loss_dice: 0.608985, val dice : 0.391015
[22:57:40.444] epoch : 29, iteration : 240, val loss : 0.763202, val loss_ce: 0.910117, val loss_dice: 0.616288, val dice : 0.383712
[22:57:40.668] epoch : 29, mean val loss : 0.780842, mean val ce loss: 0.882657, mean val dice : 0.320974
[22:58:07.777] epoch : 30, iteration : 544, train loss : 0.688043, train loss_ce: 0.749672, train loss_dice: 0.626414, train dice : 0.373586
[22:58:45.170] epoch : 30, iteration : 552, train loss : 0.838397, train loss_ce: 1.039744, train loss_dice: 0.637050, train dice : 0.362950
[22:59:11.326] epoch : 30, mean train loss : 0.770011, mean train ce loss: 0.890492, mean train dice : 0.350470
[22:59:30.738] epoch : 30, iteration : 246, val loss : 0.808159, val loss_ce: 0.997448, val loss_dice: 0.618870, val dice : 0.381130
[22:59:38.730] epoch : 30, mean val loss : 0.775078, mean val ce loss: 0.899797, mean val dice : 0.349641
[23:00:14.721] epoch : 31, iteration : 560, train loss : 0.706430, train loss_ce: 0.768456, train loss_dice: 0.644404, train dice : 0.355596
[23:00:52.051] epoch : 31, iteration : 568, train loss : 1.025743, train loss_ce: 1.388176, train loss_dice: 0.663310, train dice : 0.336690
[23:01:27.303] epoch : 31, iteration : 576, train loss : 1.034871, train loss_ce: 1.365879, train loss_dice: 0.703864, train dice : 0.296136
[23:01:27.555] epoch : 31, mean train loss : 0.846099, mean train ce loss: 1.035858, mean train dice : 0.343659
[23:01:46.755] epoch : 31, iteration : 252, val loss : 0.919681, val loss_ce: 1.200024, val loss_dice: 0.639339, val dice : 0.360661
[23:01:54.727] epoch : 31, mean val loss : 0.811472, mean val ce loss: 0.974639, mean val dice : 0.351696
[23:02:50.705] epoch : 32, iteration : 584, train loss : 0.774294, train loss_ce: 0.823778, train loss_dice: 0.724809, train dice : 0.275191
[23:03:28.103] epoch : 32, iteration : 592, train loss : 0.654233, train loss_ce: 0.688985, train loss_dice: 0.619480, train dice : 0.380520
[23:03:35.807] epoch : 32, mean train loss : 0.746708, mean train ce loss: 0.842126, mean train dice : 0.348710
[23:03:54.845] epoch : 32, iteration : 258, val loss : 0.790141, val loss_ce: 0.979720, val loss_dice: 0.600562, val dice : 0.399438
[23:04:05.418] epoch : 32, iteration : 264, val loss : 0.877446, val loss_ce: 1.113688, val loss_dice: 0.641204, val dice : 0.358796
[23:04:05.721] epoch : 32, mean val loss : 0.800700, mean val ce loss: 0.946810, mean val dice : 0.345410
[23:04:46.719] epoch : 33, iteration : 600, train loss : 0.782136, train loss_ce: 0.906112, train loss_dice: 0.658159, train dice : 0.341841
[23:05:24.116] epoch : 33, iteration : 608, train loss : 0.674426, train loss_ce: 0.641064, train loss_dice: 0.707788, train dice : 0.292212
[23:05:40.941] epoch : 33, mean train loss : 0.760076, mean train ce loss: 0.871087, mean train dice : 0.350936
[23:05:59.371] epoch : 33, iteration : 270, val loss : 0.855105, val loss_ce: 1.085192, val loss_dice: 0.625018, val dice : 0.374982
[23:06:11.375] epoch : 33, mean val loss : 0.775065, mean val ce loss: 0.895492, mean val dice : 0.345361
[23:06:46.481] epoch : 34, iteration : 616, train loss : 0.624465, train loss_ce: 0.628657, train loss_dice: 0.620272, train dice : 0.379728
[23:07:23.888] epoch : 34, iteration : 624, train loss : 0.693414, train loss_ce: 0.740387, train loss_dice: 0.646440, train dice : 0.353560
[23:07:50.057] epoch : 34, mean train loss : 0.776928, mean train ce loss: 0.879335, mean train dice : 0.325478
[23:08:08.112] epoch : 34, iteration : 276, val loss : 0.949159, val loss_ce: 1.249380, val loss_dice: 0.648939, val dice : 0.351061
[23:08:18.532] epoch : 34, mean val loss : 0.751220, mean val ce loss: 0.843734, mean val dice : 0.341295
[23:08:40.639] epoch : 35, iteration : 632, train loss : 0.656140, train loss_ce: 0.693552, train loss_dice: 0.618729, train dice : 0.381271
[23:09:18.027] epoch : 35, iteration : 640, train loss : 0.643501, train loss_ce: 0.691798, train loss_dice: 0.595204, train dice : 0.404796
[23:09:53.277] epoch : 35, iteration : 648, train loss : 0.687611, train loss_ce: 0.595184, train loss_dice: 0.780039, train dice : 0.219961
[23:09:53.408] epoch : 35, mean train loss : 0.695831, mean train ce loss: 0.758243, mean train dice : 0.366581
[23:10:14.549] epoch : 35, iteration : 282, val loss : 0.825825, val loss_ce: 1.036740, val loss_dice: 0.614909, val dice : 0.385091
[23:10:31.022] epoch : 35, iteration : 288, val loss : 0.548343, val loss_ce: 0.486192, val loss_dice: 0.610494, val dice : 0.389506
[23:10:31.222] epoch : 35, mean val loss : 0.765748, mean val ce loss: 0.873919, mean val dice : 0.342423
[23:11:22.430] epoch : 36, iteration : 656, train loss : 0.773844, train loss_ce: 0.819788, train loss_dice: 0.727901, train dice : 0.272099
[23:11:59.824] epoch : 36, iteration : 664, train loss : 0.781090, train loss_ce: 0.931051, train loss_dice: 0.631128, train dice : 0.368872
[23:12:07.217] epoch : 36, mean train loss : 0.704518, mean train ce loss: 0.769267, mean train dice : 0.360231
[23:12:29.999] epoch : 36, iteration : 294, val loss : 0.840269, val loss_ce: 1.073847, val loss_dice: 0.606691, val dice : 0.393309
[23:12:34.558] epoch : 36, mean val loss : 0.761210, mean val ce loss: 0.885586, mean val dice : 0.363166
[23:12:34.822] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 37, val dice: 0.3631663918495178
[23:13:13.730] epoch : 37, iteration : 672, train loss : 0.642966, train loss_ce: 0.669667, train loss_dice: 0.616265, train dice : 0.383735
[23:13:51.122] epoch : 37, iteration : 680, train loss : 0.713633, train loss_ce: 0.817014, train loss_dice: 0.610253, train dice : 0.389747
[23:14:08.104] epoch : 37, mean train loss : 0.727489, mean train ce loss: 0.796642, mean train dice : 0.341663
[23:14:27.906] epoch : 37, iteration : 300, val loss : 0.761560, val loss_ce: 0.906184, val loss_dice: 0.616936, val dice : 0.383064
[23:14:33.759] epoch : 37, mean val loss : 0.682422, mean val ce loss: 0.735647, mean val dice : 0.370803
[23:14:34.036] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 38, val dice: 0.3708033859729767
[23:15:10.505] epoch : 38, iteration : 688, train loss : 0.753757, train loss_ce: 0.912431, train loss_dice: 0.595083, train dice : 0.404917
[23:15:47.911] epoch : 38, iteration : 696, train loss : 0.601419, train loss_ce: 0.591530, train loss_dice: 0.611308, train dice : 0.388692
[23:16:13.973] epoch : 38, mean train loss : 0.699455, mean train ce loss: 0.729598, mean train dice : 0.330687
[23:16:29.893] epoch : 38, iteration : 306, val loss : 0.660934, val loss_ce: 0.707010, val loss_dice: 0.614857, val dice : 0.385143
[23:16:36.688] epoch : 38, iteration : 312, val loss : 0.718188, val loss_ce: 0.813364, val loss_dice: 0.623011, val dice : 0.376989
[23:16:36.825] epoch : 38, mean val loss : 0.743286, mean val ce loss: 0.837121, mean val dice : 0.350548
[23:17:01.933] epoch : 39, iteration : 704, train loss : 0.707415, train loss_ce: 0.752648, train loss_dice: 0.662183, train dice : 0.337817
[23:17:39.311] epoch : 39, iteration : 712, train loss : 0.749110, train loss_ce: 0.800903, train loss_dice: 0.697317, train dice : 0.302683
[23:18:14.581] epoch : 39, iteration : 720, train loss : 0.603255, train loss_ce: 0.448651, train loss_dice: 0.757858, train dice : 0.242142
[23:18:14.828] epoch : 39, mean train loss : 0.697358, mean train ce loss: 0.759039, mean train dice : 0.364323
[23:18:33.598] epoch : 39, iteration : 318, val loss : 0.858391, val loss_ce: 1.094834, val loss_dice: 0.621949, val dice : 0.378051
[23:18:41.095] epoch : 39, mean val loss : 0.757809, mean val ce loss: 0.885769, mean val dice : 0.370151
[23:19:37.406] epoch : 40, iteration : 728, train loss : 0.739397, train loss_ce: 0.887450, train loss_dice: 0.591343, train dice : 0.408657
[23:20:14.802] epoch : 40, iteration : 736, train loss : 0.626665, train loss_ce: 0.668449, train loss_dice: 0.584882, train dice : 0.415118
[23:20:22.150] epoch : 40, mean train loss : 0.692471, mean train ce loss: 0.762270, mean train dice : 0.377328
[23:20:41.348] epoch : 40, iteration : 324, val loss : 0.678367, val loss_ce: 0.766200, val loss_dice: 0.590534, val dice : 0.409466
[23:20:48.323] epoch : 40, mean val loss : 0.773229, mean val ce loss: 0.909570, mean val dice : 0.363112
[23:21:32.674] epoch : 41, iteration : 744, train loss : 0.710348, train loss_ce: 0.793980, train loss_dice: 0.626715, train dice : 0.373285
[23:22:10.070] epoch : 41, iteration : 752, train loss : 0.776000, train loss_ce: 0.912216, train loss_dice: 0.639784, train dice : 0.360216
[23:22:26.979] epoch : 41, mean train loss : 0.710398, mean train ce loss: 0.784195, mean train dice : 0.363398
[23:22:41.218] epoch : 41, iteration : 330, val loss : 0.681584, val loss_ce: 0.788726, val loss_dice: 0.574442, val dice : 0.425558
[23:22:54.293] epoch : 41, iteration : 336, val loss : 0.846850, val loss_ce: 1.097669, val loss_dice: 0.596031, val dice : 0.403969
[23:22:54.573] epoch : 41, mean val loss : 0.709298, mean val ce loss: 0.824197, mean val dice : 0.405602
[23:22:54.860] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 42, val dice: 0.40560173988342285
[23:23:25.443] epoch : 42, iteration : 760, train loss : 0.764939, train loss_ce: 0.869865, train loss_dice: 0.660014, train dice : 0.339986
[23:24:02.845] epoch : 42, iteration : 768, train loss : 0.772615, train loss_ce: 0.888933, train loss_dice: 0.656298, train dice : 0.343702
[23:24:29.083] epoch : 42, mean train loss : 0.685975, mean train ce loss: 0.741801, mean train dice : 0.369850
[23:24:50.088] epoch : 42, iteration : 342, val loss : 0.736817, val loss_ce: 0.703450, val loss_dice: 0.770183, val dice : 0.229817
[23:24:54.517] epoch : 42, mean val loss : 0.701584, mean val ce loss: 0.756453, mean val dice : 0.353285
[23:25:21.966] epoch : 43, iteration : 776, train loss : 0.707184, train loss_ce: 0.761167, train loss_dice: 0.653201, train dice : 0.346799
[23:25:59.342] epoch : 43, iteration : 784, train loss : 0.812999, train loss_ce: 0.961130, train loss_dice: 0.664867, train dice : 0.335133
[23:26:34.606] epoch : 43, iteration : 792, train loss : 0.963898, train loss_ce: 1.209723, train loss_dice: 0.718073, train dice : 0.281927
[23:26:34.859] epoch : 43, mean train loss : 0.661104, mean train ce loss: 0.678832, mean train dice : 0.356625
[23:26:50.481] epoch : 43, iteration : 348, val loss : 0.657668, val loss_ce: 0.750255, val loss_dice: 0.565081, val dice : 0.434919
[23:27:09.369] epoch : 43, mean val loss : 0.714989, mean val ce loss: 0.799570, mean val dice : 0.369591
[23:28:02.568] epoch : 44, iteration : 800, train loss : 0.551954, train loss_ce: 0.507805, train loss_dice: 0.596103, train dice : 0.403897
[23:28:39.970] epoch : 44, iteration : 808, train loss : 0.717129, train loss_ce: 0.845783, train loss_dice: 0.588474, train dice : 0.411526
[23:28:47.508] epoch : 44, mean train loss : 0.675735, mean train ce loss: 0.729360, mean train dice : 0.377891
[23:29:12.638] epoch : 44, iteration : 354, val loss : 0.712429, val loss_ce: 0.850353, val loss_dice: 0.574504, val dice : 0.425496
[23:29:18.340] epoch : 44, iteration : 360, val loss : 0.546505, val loss_ce: 0.492250, val loss_dice: 0.600760, val dice : 0.399240
[23:29:18.446] epoch : 44, mean val loss : 0.681759, mean val ce loss: 0.733881, mean val dice : 0.370364
[23:29:59.854] epoch : 45, iteration : 816, train loss : 0.870776, train loss_ce: 1.137848, train loss_dice: 0.603703, train dice : 0.396297
[23:30:37.256] epoch : 45, iteration : 824, train loss : 0.517320, train loss_ce: 0.498633, train loss_dice: 0.536008, train dice : 0.463992
[23:30:54.085] epoch : 45, mean train loss : 0.709150, mean train ce loss: 0.796600, mean train dice : 0.378301
[23:31:17.404] epoch : 45, iteration : 366, val loss : 0.694109, val loss_ce: 0.806194, val loss_dice: 0.582025, val dice : 0.417975
[23:31:22.869] epoch : 45, mean val loss : 0.728616, mean val ce loss: 0.831326, mean val dice : 0.374094
[23:31:53.760] epoch : 46, iteration : 832, train loss : 0.716593, train loss_ce: 0.806650, train loss_dice: 0.626536, train dice : 0.373464
[23:32:31.149] epoch : 46, iteration : 840, train loss : 0.585058, train loss_ce: 0.586345, train loss_dice: 0.583771, train dice : 0.416229
[23:32:57.521] epoch : 46, mean train loss : 0.669393, mean train ce loss: 0.724450, mean train dice : 0.385664
[23:33:16.110] epoch : 46, iteration : 372, val loss : 0.707199, val loss_ce: 0.828056, val loss_dice: 0.586342, val dice : 0.413658
[23:33:18.772] epoch : 46, mean val loss : 0.764585, mean val ce loss: 0.880030, mean val dice : 0.350861
[23:33:39.865] epoch : 47, iteration : 848, train loss : 0.648169, train loss_ce: 0.659248, train loss_dice: 0.637090, train dice : 0.362910
[23:34:17.265] epoch : 47, iteration : 856, train loss : 0.728145, train loss_ce: 0.867399, train loss_dice: 0.588891, train dice : 0.411109
[23:34:52.513] epoch : 47, iteration : 864, train loss : 0.536778, train loss_ce: 0.386269, train loss_dice: 0.687287, train dice : 0.312713
[23:34:52.880] epoch : 47, mean train loss : 0.671622, mean train ce loss: 0.729156, mean train dice : 0.385912
[23:35:05.502] epoch : 47, iteration : 378, val loss : 0.625626, val loss_ce: 0.672490, val loss_dice: 0.578763, val dice : 0.421237
[23:35:19.570] epoch : 47, iteration : 384, val loss : 0.695063, val loss_ce: 0.786880, val loss_dice: 0.603245, val dice : 0.396755
[23:35:19.981] epoch : 47, mean val loss : 0.725190, mean val ce loss: 0.826707, mean val dice : 0.376328
[23:36:11.189] epoch : 48, iteration : 872, train loss : 0.634558, train loss_ce: 0.484126, train loss_dice: 0.784990, train dice : 0.215010
[23:36:48.572] epoch : 48, iteration : 880, train loss : 0.563251, train loss_ce: 0.575370, train loss_dice: 0.551131, train dice : 0.448869
[23:36:55.976] epoch : 48, mean train loss : 0.675102, mean train ce loss: 0.711475, mean train dice : 0.361271
[23:37:14.490] epoch : 48, iteration : 390, val loss : 0.776039, val loss_ce: 0.953086, val loss_dice: 0.598992, val dice : 0.401008
[23:37:23.289] epoch : 48, mean val loss : 0.753167, mean val ce loss: 0.889442, mean val dice : 0.383108
[23:38:12.812] epoch : 49, iteration : 888, train loss : 0.512065, train loss_ce: 0.439976, train loss_dice: 0.584153, train dice : 0.415847
[23:38:50.219] epoch : 49, iteration : 896, train loss : 0.789864, train loss_ce: 0.982904, train loss_dice: 0.596824, train dice : 0.403176
[23:39:07.030] epoch : 49, mean train loss : 0.674491, mean train ce loss: 0.744522, mean train dice : 0.395541
[23:39:25.088] epoch : 49, iteration : 396, val loss : 0.809606, val loss_ce: 1.039575, val loss_dice: 0.579636, val dice : 0.420364
[23:39:37.680] epoch : 49, mean val loss : 0.729681, mean val ce loss: 0.834907, mean val dice : 0.375545
[23:40:12.681] epoch : 50, iteration : 904, train loss : 0.585848, train loss_ce: 0.604299, train loss_dice: 0.567397, train dice : 0.432603
[23:40:50.082] epoch : 50, iteration : 912, train loss : 0.588621, train loss_ce: 0.496789, train loss_dice: 0.680454, train dice : 0.319546
[23:41:16.194] epoch : 50, mean train loss : 0.678290, mean train ce loss: 0.731040, mean train dice : 0.374460
[23:41:30.172] epoch : 50, iteration : 402, val loss : 0.604629, val loss_ce: 0.585728, val loss_dice: 0.623530, val dice : 0.376470
[23:41:48.157] epoch : 50, iteration : 408, val loss : 0.963130, val loss_ce: 1.252189, val loss_dice: 0.674072, val dice : 0.325928
[23:41:48.456] epoch : 50, mean val loss : 0.740618, mean val ce loss: 0.852041, mean val dice : 0.370804
[23:42:05.106] epoch : 51, iteration : 920, train loss : 0.750511, train loss_ce: 0.861739, train loss_dice: 0.639284, train dice : 0.360716
[23:42:42.508] epoch : 51, iteration : 928, train loss : 0.632421, train loss_ce: 0.593252, train loss_dice: 0.671591, train dice : 0.328409
[23:43:17.857] epoch : 51, iteration : 936, train loss : 1.781626, train loss_ce: 2.771393, train loss_dice: 0.791860, train dice : 0.208140
[23:43:18.130] epoch : 51, mean train loss : 0.752939, mean train ce loss: 0.866182, mean train dice : 0.360304
[23:43:39.200] epoch : 51, iteration : 414, val loss : 0.793981, val loss_ce: 0.939194, val loss_dice: 0.648767, val dice : 0.351233
[23:43:51.074] epoch : 51, mean val loss : 0.764071, mean val ce loss: 0.894432, mean val dice : 0.366291
[23:44:51.234] epoch : 52, iteration : 944, train loss : 0.764192, train loss_ce: 0.818452, train loss_dice: 0.709933, train dice : 0.290067
[23:45:28.615] epoch : 52, iteration : 952, train loss : 0.804175, train loss_ce: 1.008166, train loss_dice: 0.600185, train dice : 0.399815
[23:45:35.940] epoch : 52, mean train loss : 0.745169, mean train ce loss: 0.848824, mean train dice : 0.358485
[23:46:01.246] epoch : 52, iteration : 420, val loss : 0.746618, val loss_ce: 0.904649, val loss_dice: 0.588587, val dice : 0.411413
[23:46:16.290] epoch : 52, mean val loss : 0.773320, mean val ce loss: 0.893302, mean val dice : 0.346662
[23:46:57.238] epoch : 53, iteration : 960, train loss : 0.738651, train loss_ce: 0.878831, train loss_dice: 0.598471, train dice : 0.401529
[23:47:34.637] epoch : 53, iteration : 968, train loss : 0.671233, train loss_ce: 0.640759, train loss_dice: 0.701707, train dice : 0.298293
[23:47:51.392] epoch : 53, mean train loss : 0.703319, mean train ce loss: 0.761844, mean train dice : 0.355205
[23:48:10.381] epoch : 53, iteration : 426, val loss : 0.746924, val loss_ce: 0.920711, val loss_dice: 0.573136, val dice : 0.426864
[23:48:21.639] epoch : 53, iteration : 432, val loss : 0.809268, val loss_ce: 1.042579, val loss_dice: 0.575957, val dice : 0.424043
[23:48:21.831] epoch : 53, mean val loss : 0.692559, mean val ce loss: 0.759372, mean val dice : 0.374255
[23:48:55.272] epoch : 54, iteration : 976, train loss : 0.823833, train loss_ce: 1.040368, train loss_dice: 0.607299, train dice : 0.392701
[23:49:32.674] epoch : 54, iteration : 984, train loss : 0.578284, train loss_ce: 0.551505, train loss_dice: 0.605063, train dice : 0.394937
[23:49:58.696] epoch : 54, mean train loss : 0.707441, mean train ce loss: 0.786445, mean train dice : 0.371563
[23:50:19.572] epoch : 54, iteration : 438, val loss : 0.787719, val loss_ce: 0.984225, val loss_dice: 0.591212, val dice : 0.408788
[23:50:30.847] epoch : 54, mean val loss : 0.716435, mean val ce loss: 0.791108, mean val dice : 0.358237
[23:50:51.192] epoch : 55, iteration : 992, train loss : 0.693214, train loss_ce: 0.763356, train loss_dice: 0.623072, train dice : 0.376928
[23:51:28.611] epoch : 55, iteration : 1000, train loss : 0.686911, train loss_ce: 0.726521, train loss_dice: 0.647300, train dice : 0.352700
[23:52:03.847] epoch : 55, iteration : 1008, train loss : 0.618841, train loss_ce: 0.498899, train loss_dice: 0.738784, train dice : 0.261216
[23:52:03.998] epoch : 55, mean train loss : 0.713490, mean train ce loss: 0.789367, mean train dice : 0.362387
[23:52:20.801] epoch : 55, iteration : 444, val loss : 1.033415, val loss_ce: 1.367087, val loss_dice: 0.699742, val dice : 0.300258
[23:52:34.154] epoch : 55, mean val loss : 0.757406, mean val ce loss: 0.864422, mean val dice : 0.349610
[23:53:29.997] epoch : 56, iteration : 1016, train loss : 0.574941, train loss_ce: 0.502097, train loss_dice: 0.647785, train dice : 0.352215
[23:54:07.411] epoch : 56, iteration : 1024, train loss : 0.834433, train loss_ce: 1.072856, train loss_dice: 0.596011, train dice : 0.403989
[23:54:15.046] epoch : 56, mean train loss : 0.668862, mean train ce loss: 0.714408, mean train dice : 0.376684
[23:54:27.450] epoch : 56, iteration : 450, val loss : 0.847576, val loss_ce: 1.106658, val loss_dice: 0.588494, val dice : 0.411506
[23:54:45.914] epoch : 56, iteration : 456, val loss : 0.732857, val loss_ce: 0.793485, val loss_dice: 0.672228, val dice : 0.327772
[23:54:46.081] epoch : 56, mean val loss : 0.751897, mean val ce loss: 0.852551, mean val dice : 0.348757
[23:55:31.513] epoch : 57, iteration : 1032, train loss : 0.772133, train loss_ce: 0.875030, train loss_dice: 0.669236, train dice : 0.330764
[23:56:08.921] epoch : 57, iteration : 1040, train loss : 0.593801, train loss_ce: 0.558090, train loss_dice: 0.629511, train dice : 0.370489
[23:56:25.595] epoch : 57, mean train loss : 0.700904, mean train ce loss: 0.759848, mean train dice : 0.358040
[23:56:50.843] epoch : 57, iteration : 462, val loss : 0.655787, val loss_ce: 0.578071, val loss_dice: 0.733503, val dice : 0.266497
[23:56:59.129] epoch : 57, mean val loss : 0.719148, mean val ce loss: 0.796481, mean val dice : 0.358186
[23:57:39.775] epoch : 58, iteration : 1048, train loss : 0.618644, train loss_ce: 0.644971, train loss_dice: 0.592316, train dice : 0.407684
[23:58:17.170] epoch : 58, iteration : 1056, train loss : 0.631435, train loss_ce: 0.644278, train loss_dice: 0.618592, train dice : 0.381408
[23:58:43.264] epoch : 58, mean train loss : 0.664322, mean train ce loss: 0.707978, mean train dice : 0.379333
[23:58:59.170] epoch : 58, iteration : 468, val loss : 0.655329, val loss_ce: 0.730327, val loss_dice: 0.580332, val dice : 0.419668
[23:59:09.109] epoch : 58, mean val loss : 0.662742, mean val ce loss: 0.702736, mean val dice : 0.377251
[23:59:27.195] epoch : 59, iteration : 1064, train loss : 0.603250, train loss_ce: 0.644736, train loss_dice: 0.561763, train dice : 0.438237
[00:00:04.598] epoch : 59, iteration : 1072, train loss : 0.750803, train loss_ce: 0.746260, train loss_dice: 0.755347, train dice : 0.244653
[00:00:39.829] epoch : 59, iteration : 1080, train loss : 0.636678, train loss_ce: 0.578338, train loss_dice: 0.695017, train dice : 0.304983
[00:00:39.976] epoch : 59, mean train loss : 0.693837, mean train ce loss: 0.735465, mean train dice : 0.347790
[00:00:57.414] epoch : 59, iteration : 474, val loss : 0.743137, val loss_ce: 0.866373, val loss_dice: 0.619900, val dice : 0.380100
[00:01:04.854] epoch : 59, iteration : 480, val loss : 0.933203, val loss_ce: 1.202349, val loss_dice: 0.664058, val dice : 0.335942
[00:01:04.970] epoch : 59, mean val loss : 0.770325, mean val ce loss: 0.921221, mean val dice : 0.380572
[00:01:59.575] epoch : 60, iteration : 1088, train loss : 0.698231, train loss_ce: 0.758119, train loss_dice: 0.638342, train dice : 0.361658
[00:02:37.019] epoch : 60, iteration : 1096, train loss : 0.648019, train loss_ce: 0.743259, train loss_dice: 0.552780, train dice : 0.447220
[00:02:44.527] epoch : 60, mean train loss : 0.679866, mean train ce loss: 0.753628, mean train dice : 0.393896
[00:03:09.598] epoch : 60, iteration : 486, val loss : 0.592132, val loss_ce: 0.425839, val loss_dice: 0.758425, val dice : 0.241575
[00:03:20.325] epoch : 60, mean val loss : 0.682704, mean val ce loss: 0.741864, mean val dice : 0.376457
[00:04:10.396] epoch : 61, iteration : 1104, train loss : 0.586870, train loss_ce: 0.633005, train loss_dice: 0.540735, train dice : 0.459265
[00:04:47.776] epoch : 61, iteration : 1112, train loss : 0.789493, train loss_ce: 0.951639, train loss_dice: 0.627347, train dice : 0.372653
[00:05:04.489] epoch : 61, mean train loss : 0.683806, mean train ce loss: 0.755318, mean train dice : 0.387707
[00:05:21.935] epoch : 61, iteration : 492, val loss : 0.664825, val loss_ce: 0.763460, val loss_dice: 0.566189, val dice : 0.433811
[00:05:33.026] epoch : 61, mean val loss : 0.673240, mean val ce loss: 0.732278, mean val dice : 0.385797
[00:06:03.674] epoch : 62, iteration : 1120, train loss : 0.752903, train loss_ce: 0.933258, train loss_dice: 0.572547, train dice : 0.427453
[00:06:41.364] epoch : 62, iteration : 1128, train loss : 0.613802, train loss_ce: 0.656779, train loss_dice: 0.570824, train dice : 0.429176
[00:07:07.387] epoch : 62, mean train loss : 0.675320, mean train ce loss: 0.761078, mean train dice : 0.410439
[00:07:27.669] epoch : 62, iteration : 498, val loss : 0.735492, val loss_ce: 0.886353, val loss_dice: 0.584631, val dice : 0.415369
[00:07:36.398] epoch : 62, iteration : 504, val loss : 0.833041, val loss_ce: 1.097909, val loss_dice: 0.568172, val dice : 0.431828
[00:07:36.518] epoch : 62, mean val loss : 0.728130, mean val ce loss: 0.867886, mean val dice : 0.411625
[00:07:36.842] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 63, val dice: 0.4116247892379761
[00:08:02.209] epoch : 63, iteration : 1136, train loss : 0.674762, train loss_ce: 0.734693, train loss_dice: 0.614831, train dice : 0.385169
[00:08:39.597] epoch : 63, iteration : 1144, train loss : 0.674982, train loss_ce: 0.622725, train loss_dice: 0.727238, train dice : 0.272762
[00:09:14.866] epoch : 63, iteration : 1152, train loss : 0.863579, train loss_ce: 1.059754, train loss_dice: 0.667405, train dice : 0.332595
[00:09:15.061] epoch : 63, mean train loss : 0.682442, mean train ce loss: 0.761033, mean train dice : 0.396150
[00:09:33.863] epoch : 63, iteration : 510, val loss : 0.734483, val loss_ce: 0.703455, val loss_dice: 0.765510, val dice : 0.234490
[00:09:45.701] epoch : 63, mean val loss : 0.692391, mean val ce loss: 0.744482, mean val dice : 0.359700
[00:10:38.577] epoch : 64, iteration : 1160, train loss : 0.576005, train loss_ce: 0.526722, train loss_dice: 0.625287, train dice : 0.374713
[00:11:15.961] epoch : 64, iteration : 1168, train loss : 0.705541, train loss_ce: 0.847926, train loss_dice: 0.563156, train dice : 0.436844
[00:11:23.319] epoch : 64, mean train loss : 0.657317, mean train ce loss: 0.713845, mean train dice : 0.399211
[00:11:39.385] epoch : 64, iteration : 516, val loss : 0.804648, val loss_ce: 1.024010, val loss_dice: 0.585285, val dice : 0.414715
[00:11:51.924] epoch : 64, mean val loss : 0.724266, mean val ce loss: 0.830503, mean val dice : 0.381971
[00:12:29.817] epoch : 65, iteration : 1176, train loss : 0.628830, train loss_ce: 0.706358, train loss_dice: 0.551302, train dice : 0.448698
[00:13:07.221] epoch : 65, iteration : 1184, train loss : 0.510019, train loss_ce: 0.371617, train loss_dice: 0.648420, train dice : 0.351580
[00:13:23.960] epoch : 65, mean train loss : 0.650624, mean train ce loss: 0.704559, mean train dice : 0.403312
[00:13:41.274] epoch : 65, iteration : 522, val loss : 0.660927, val loss_ce: 0.787761, val loss_dice: 0.534093, val dice : 0.465907
[00:13:53.254] epoch : 65, iteration : 528, val loss : 0.791714, val loss_ce: 1.004589, val loss_dice: 0.578838, val dice : 0.421162
[00:13:53.478] epoch : 65, mean val loss : 0.718470, mean val ce loss: 0.850783, mean val dice : 0.413843
[00:13:53.774] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 66, val dice: 0.413843035697937
[00:14:32.948] epoch : 66, iteration : 1192, train loss : 0.696214, train loss_ce: 0.832007, train loss_dice: 0.560420, train dice : 0.439580
[00:15:10.313] epoch : 66, iteration : 1200, train loss : 0.655506, train loss_ce: 0.741454, train loss_dice: 0.569558, train dice : 0.430442
[00:15:36.409] epoch : 66, mean train loss : 0.662113, mean train ce loss: 0.746604, mean train dice : 0.422379
[00:15:56.651] epoch : 66, iteration : 534, val loss : 0.727561, val loss_ce: 0.904651, val loss_dice: 0.550471, val dice : 0.449529
[00:16:07.242] epoch : 66, mean val loss : 0.663818, mean val ce loss: 0.748779, mean val dice : 0.421143
[00:16:07.540] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 67, val dice: 0.4211430549621582
[00:16:34.361] epoch : 67, iteration : 1208, train loss : 0.569805, train loss_ce: 0.611734, train loss_dice: 0.527876, train dice : 0.472124
[00:17:11.767] epoch : 67, iteration : 1216, train loss : 0.694381, train loss_ce: 0.714764, train loss_dice: 0.673998, train dice : 0.326002
[00:17:46.998] epoch : 67, iteration : 1224, train loss : 0.745188, train loss_ce: 0.766803, train loss_dice: 0.723573, train dice : 0.276427
[00:17:47.179] epoch : 67, mean train loss : 0.669325, mean train ce loss: 0.743595, mean train dice : 0.404945
[00:18:06.593] epoch : 67, iteration : 540, val loss : 0.696539, val loss_ce: 0.862079, val loss_dice: 0.531000, val dice : 0.469000
[00:18:12.687] epoch : 67, mean val loss : 0.706607, mean val ce loss: 0.792158, mean val dice : 0.378944
[00:19:12.844] epoch : 68, iteration : 1232, train loss : 0.539388, train loss_ce: 0.367373, train loss_dice: 0.711403, train dice : 0.288597
[00:19:50.243] epoch : 68, iteration : 1240, train loss : 0.602944, train loss_ce: 0.461849, train loss_dice: 0.744038, train dice : 0.255962
[00:19:57.586] epoch : 68, mean train loss : 0.631809, mean train ce loss: 0.671215, mean train dice : 0.407597
[00:20:15.954] epoch : 68, iteration : 546, val loss : 0.703217, val loss_ce: 0.870534, val loss_dice: 0.535901, val dice : 0.464099
[00:20:23.674] epoch : 68, iteration : 552, val loss : 0.578393, val loss_ce: 0.584531, val loss_dice: 0.572255, val dice : 0.427745
[00:20:23.794] epoch : 68, mean val loss : 0.655769, mean val ce loss: 0.723385, mean val dice : 0.411847
[00:20:59.615] epoch : 69, iteration : 1248, train loss : 0.723608, train loss_ce: 0.849912, train loss_dice: 0.597305, train dice : 0.402695
[00:21:37.016] epoch : 69, iteration : 1256, train loss : 0.617909, train loss_ce: 0.710515, train loss_dice: 0.525302, train dice : 0.474698
[00:21:53.708] epoch : 69, mean train loss : 0.640146, mean train ce loss: 0.690401, mean train dice : 0.410108
[00:22:11.216] epoch : 69, iteration : 558, val loss : 0.671391, val loss_ce: 0.806645, val loss_dice: 0.536137, val dice : 0.463863
[00:22:23.875] epoch : 69, mean val loss : 0.654932, mean val ce loss: 0.747650, mean val dice : 0.437786
[00:22:24.167] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 70, val dice: 0.4377860724925995
[00:23:02.363] epoch : 70, iteration : 1264, train loss : 0.591985, train loss_ce: 0.606579, train loss_dice: 0.577391, train dice : 0.422609
[00:23:39.768] epoch : 70, iteration : 1272, train loss : 0.547620, train loss_ce: 0.518683, train loss_dice: 0.576557, train dice : 0.423443
[00:24:05.960] epoch : 70, mean train loss : 0.635635, mean train ce loss: 0.696630, mean train dice : 0.425361
[00:24:27.060] epoch : 70, iteration : 564, val loss : 0.722492, val loss_ce: 0.932488, val loss_dice: 0.512496, val dice : 0.487504
[00:24:35.835] epoch : 70, mean val loss : 0.691567, mean val ce loss: 0.830032, mean val dice : 0.446898
[00:24:36.117] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 71, val dice: 0.4468979835510254
[00:24:59.912] epoch : 71, iteration : 1280, train loss : 0.651711, train loss_ce: 0.757333, train loss_dice: 0.546089, train dice : 0.453911
[00:25:37.300] epoch : 71, iteration : 1288, train loss : 0.590436, train loss_ce: 0.622487, train loss_dice: 0.558385, train dice : 0.441615
[00:26:12.710] epoch : 71, iteration : 1296, train loss : 0.892112, train loss_ce: 1.122406, train loss_dice: 0.661818, train dice : 0.338182
[00:26:13.028] epoch : 71, mean train loss : 0.649212, mean train ce loss: 0.706415, mean train dice : 0.407991
[00:26:28.758] epoch : 71, iteration : 570, val loss : 0.593198, val loss_ce: 0.677092, val loss_dice: 0.509303, val dice : 0.490697
[00:26:44.906] epoch : 71, iteration : 576, val loss : 0.628696, val loss_ce: 0.771622, val loss_dice: 0.485770, val dice : 0.514230
[00:26:45.125] epoch : 71, mean val loss : 0.628310, mean val ce loss: 0.705555, mean val dice : 0.448935
[00:26:45.420] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 72, val dice: 0.44893547892570496
[00:27:32.638] epoch : 72, iteration : 1304, train loss : 0.590063, train loss_ce: 0.552779, train loss_dice: 0.627348, train dice : 0.372652
[00:28:10.019] epoch : 72, iteration : 1312, train loss : 0.690014, train loss_ce: 0.865366, train loss_dice: 0.514661, train dice : 0.485339
[00:28:17.450] epoch : 72, mean train loss : 0.612384, mean train ce loss: 0.662560, mean train dice : 0.437791
[00:28:37.039] epoch : 72, iteration : 582, val loss : 0.650661, val loss_ce: 0.767627, val loss_dice: 0.533696, val dice : 0.466304
[00:28:41.511] epoch : 72, mean val loss : 0.636818, mean val ce loss: 0.744777, mean val dice : 0.471141
[00:28:41.814] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 73, val dice: 0.47114133834838867
[00:29:25.265] epoch : 73, iteration : 1320, train loss : 0.747542, train loss_ce: 0.969474, train loss_dice: 0.525611, train dice : 0.474389
[00:30:02.698] epoch : 73, iteration : 1328, train loss : 0.525436, train loss_ce: 0.543903, train loss_dice: 0.506969, train dice : 0.493031
[00:30:19.453] epoch : 73, mean train loss : 0.599848, mean train ce loss: 0.644340, mean train dice : 0.444645
[00:30:36.267] epoch : 73, iteration : 588, val loss : 0.666261, val loss_ce: 0.819192, val loss_dice: 0.513331, val dice : 0.486669
[00:30:47.460] epoch : 73, mean val loss : 0.583674, mean val ce loss: 0.574555, mean val dice : 0.407207
[00:31:24.688] epoch : 74, iteration : 1336, train loss : 0.570342, train loss_ce: 0.562876, train loss_dice: 0.577808, train dice : 0.422192
[00:32:02.070] epoch : 74, iteration : 1344, train loss : 0.702418, train loss_ce: 0.808757, train loss_dice: 0.596080, train dice : 0.403920
[00:32:28.276] epoch : 74, mean train loss : 0.614221, mean train ce loss: 0.673336, mean train dice : 0.444894
[00:32:43.093] epoch : 74, iteration : 594, val loss : 0.725133, val loss_ce: 0.903712, val loss_dice: 0.546553, val dice : 0.453447
[00:32:55.584] epoch : 74, iteration : 600, val loss : 0.555197, val loss_ce: 0.547049, val loss_dice: 0.563345, val dice : 0.436655
[00:32:55.814] epoch : 74, mean val loss : 0.650437, mean val ce loss: 0.723808, mean val dice : 0.422935
[00:33:20.694] epoch : 75, iteration : 1352, train loss : 0.673881, train loss_ce: 0.687602, train loss_dice: 0.660160, train dice : 0.339840
[00:33:58.076] epoch : 75, iteration : 1360, train loss : 0.567471, train loss_ce: 0.623673, train loss_dice: 0.511270, train dice : 0.488730
[00:34:33.357] epoch : 75, iteration : 1368, train loss : 0.703271, train loss_ce: 0.744537, train loss_dice: 0.662005, train dice : 0.337995
[00:34:33.776] epoch : 75, mean train loss : 0.620045, mean train ce loss: 0.677151, mean train dice : 0.437061
[00:34:53.338] epoch : 75, iteration : 606, val loss : 0.772028, val loss_ce: 1.001647, val loss_dice: 0.542409, val dice : 0.457591
[00:34:56.882] epoch : 75, mean val loss : 0.623217, mean val ce loss: 0.691718, mean val dice : 0.445284
[00:35:52.255] epoch : 76, iteration : 1376, train loss : 0.570103, train loss_ce: 0.619682, train loss_dice: 0.520523, train dice : 0.479477
[00:36:29.672] epoch : 76, iteration : 1384, train loss : 0.535325, train loss_ce: 0.616244, train loss_dice: 0.454405, train dice : 0.545595
[00:36:37.053] epoch : 76, mean train loss : 0.646958, mean train ce loss: 0.743469, mean train dice : 0.449553
[00:36:55.984] epoch : 76, iteration : 612, val loss : 1.005293, val loss_ce: 1.364962, val loss_dice: 0.645624, val dice : 0.354376
[00:37:02.789] epoch : 76, mean val loss : 0.703763, mean val ce loss: 0.793049, mean val dice : 0.385522
[00:37:48.883] epoch : 77, iteration : 1392, train loss : 0.666273, train loss_ce: 0.776086, train loss_dice: 0.556460, train dice : 0.443540
[00:38:26.300] epoch : 77, iteration : 1400, train loss : 0.509911, train loss_ce: 0.476042, train loss_dice: 0.543780, train dice : 0.456220
[00:38:42.999] epoch : 77, mean train loss : 0.646149, mean train ce loss: 0.693756, mean train dice : 0.401459
[00:39:00.232] epoch : 77, iteration : 618, val loss : 0.575387, val loss_ce: 0.639571, val loss_dice: 0.511203, val dice : 0.488797
[00:39:11.867] epoch : 77, iteration : 624, val loss : 0.725344, val loss_ce: 0.936871, val loss_dice: 0.513816, val dice : 0.486184
[00:39:12.031] epoch : 77, mean val loss : 0.722494, mean val ce loss: 0.848666, mean val dice : 0.403677
[00:39:51.019] epoch : 78, iteration : 1408, train loss : 0.904097, train loss_ce: 1.145990, train loss_dice: 0.662204, train dice : 0.337796
[00:40:28.399] epoch : 78, iteration : 1416, train loss : 0.648330, train loss_ce: 0.585442, train loss_dice: 0.711218, train dice : 0.288782
[00:40:54.484] epoch : 78, mean train loss : 0.637764, mean train ce loss: 0.703713, mean train dice : 0.428186
[00:41:19.330] epoch : 78, iteration : 630, val loss : 0.616228, val loss_ce: 0.740054, val loss_dice: 0.492402, val dice : 0.507598
[00:41:22.849] epoch : 78, mean val loss : 0.617553, mean val ce loss: 0.701821, mean val dice : 0.466715
[00:41:42.555] epoch : 79, iteration : 1424, train loss : 0.669236, train loss_ce: 0.825357, train loss_dice: 0.513116, train dice : 0.486884
[00:42:19.929] epoch : 79, iteration : 1432, train loss : 0.533841, train loss_ce: 0.607866, train loss_dice: 0.459816, train dice : 0.540184
[00:42:55.195] epoch : 79, iteration : 1440, train loss : 0.417527, train loss_ce: 0.302752, train loss_dice: 0.532302, train dice : 0.467698
[00:42:55.466] epoch : 79, mean train loss : 0.566035, mean train ce loss: 0.597323, mean train dice : 0.465252
[00:43:15.195] epoch : 79, iteration : 636, val loss : 0.756772, val loss_ce: 1.034642, val loss_dice: 0.478902, val dice : 0.521098
[00:43:18.376] epoch : 79, mean val loss : 0.580309, mean val ce loss: 0.622943, mean val dice : 0.462325
[00:44:07.653] epoch : 80, iteration : 1448, train loss : 0.451192, train loss_ce: 0.508530, train loss_dice: 0.393853, train dice : 0.606147
[00:44:45.046] epoch : 80, iteration : 1456, train loss : 0.520856, train loss_ce: 0.592595, train loss_dice: 0.449117, train dice : 0.550883
[00:44:52.435] epoch : 80, mean train loss : 0.539173, mean train ce loss: 0.576853, mean train dice : 0.498507
[00:45:09.206] epoch : 80, iteration : 642, val loss : 0.501214, val loss_ce: 0.614405, val loss_dice: 0.388023, val dice : 0.611977
[00:45:16.072] epoch : 80, iteration : 648, val loss : 0.518209, val loss_ce: 0.615154, val loss_dice: 0.421263, val dice : 0.578737
[00:45:16.207] epoch : 80, mean val loss : 0.528175, mean val ce loss: 0.570529, mean val dice : 0.514178
[00:45:16.465] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 81, val dice: 0.5141782164573669
[00:45:55.719] epoch : 81, iteration : 1464, train loss : 0.679591, train loss_ce: 0.711382, train loss_dice: 0.647800, train dice : 0.352200
[00:46:33.122] epoch : 81, iteration : 1472, train loss : 0.476252, train loss_ce: 0.510556, train loss_dice: 0.441948, train dice : 0.558052
[00:46:50.110] epoch : 81, mean train loss : 0.538922, mean train ce loss: 0.569239, mean train dice : 0.491395
[00:47:13.300] epoch : 81, iteration : 654, val loss : 0.453651, val loss_ce: 0.310290, val loss_dice: 0.597011, val dice : 0.402989
[00:47:23.829] epoch : 81, mean val loss : 0.616215, mean val ce loss: 0.699859, mean val dice : 0.467428
[00:47:53.118] epoch : 82, iteration : 1480, train loss : 0.599100, train loss_ce: 0.643722, train loss_dice: 0.554478, train dice : 0.445522
[00:48:30.532] epoch : 82, iteration : 1488, train loss : 0.483193, train loss_ce: 0.556891, train loss_dice: 0.409494, train dice : 0.590506
[00:48:56.572] epoch : 82, mean train loss : 0.530206, mean train ce loss: 0.539095, mean train dice : 0.478683
[00:49:13.377] epoch : 82, iteration : 660, val loss : 0.466867, val loss_ce: 0.554316, val loss_dice: 0.379418, val dice : 0.620582
[00:49:26.051] epoch : 82, mean val loss : 0.510350, mean val ce loss: 0.571604, mean val dice : 0.550905
[00:49:26.333] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 83, val dice: 0.5509045124053955
[00:49:50.859] epoch : 83, iteration : 1496, train loss : 0.599848, train loss_ce: 0.742389, train loss_dice: 0.457307, train dice : 0.542693
[00:50:28.278] epoch : 83, iteration : 1504, train loss : 0.515688, train loss_ce: 0.626777, train loss_dice: 0.404598, train dice : 0.595402
[00:51:03.552] epoch : 83, iteration : 1512, train loss : 0.581630, train loss_ce: 0.467597, train loss_dice: 0.695663, train dice : 0.304337
[00:51:03.741] epoch : 83, mean train loss : 0.483502, mean train ce loss: 0.520957, mean train dice : 0.553953
[00:51:25.049] epoch : 83, iteration : 666, val loss : 0.405207, val loss_ce: 0.437821, val loss_dice: 0.372592, val dice : 0.627408
[00:51:31.681] epoch : 83, iteration : 672, val loss : 0.526940, val loss_ce: 0.640828, val loss_dice: 0.413052, val dice : 0.586948
[00:51:31.801] epoch : 83, mean val loss : 0.448481, mean val ce loss: 0.477660, mean val dice : 0.580699
[00:51:32.081] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 84, val dice: 0.5806989073753357
[00:52:25.485] epoch : 84, iteration : 1520, train loss : 0.588202, train loss_ce: 0.753985, train loss_dice: 0.422419, train dice : 0.577581
[00:53:02.887] epoch : 84, iteration : 1528, train loss : 0.534034, train loss_ce: 0.609688, train loss_dice: 0.458379, train dice : 0.541621
[00:53:10.365] epoch : 84, mean train loss : 0.552344, mean train ce loss: 0.613919, mean train dice : 0.509230
[00:53:33.220] epoch : 84, iteration : 678, val loss : 0.564774, val loss_ce: 0.699366, val loss_dice: 0.430182, val dice : 0.569818
[00:53:43.622] epoch : 84, mean val loss : 0.578623, mean val ce loss: 0.632242, mean val dice : 0.474997
[00:54:24.983] epoch : 85, iteration : 1536, train loss : 0.517026, train loss_ce: 0.630201, train loss_dice: 0.403851, train dice : 0.596149
[00:55:02.381] epoch : 85, iteration : 1544, train loss : 0.589734, train loss_ce: 0.748578, train loss_dice: 0.430889, train dice : 0.569111
[00:55:19.075] epoch : 85, mean train loss : 0.489227, mean train ce loss: 0.519971, mean train dice : 0.541518
[00:55:38.104] epoch : 85, iteration : 684, val loss : 0.584808, val loss_ce: 0.742555, val loss_dice: 0.427061, val dice : 0.572939
[00:55:51.543] epoch : 85, mean val loss : 0.508954, mean val ce loss: 0.559313, mean val dice : 0.541405
[00:56:26.742] epoch : 86, iteration : 1552, train loss : 0.493995, train loss_ce: 0.544118, train loss_dice: 0.443872, train dice : 0.556128
[00:57:04.180] epoch : 86, iteration : 1560, train loss : 0.412183, train loss_ce: 0.235044, train loss_dice: 0.589322, train dice : 0.410678
[00:57:30.262] epoch : 86, mean train loss : 0.456359, mean train ce loss: 0.464552, mean train dice : 0.551834
[00:57:50.663] epoch : 86, iteration : 690, val loss : 0.388807, val loss_ce: 0.413397, val loss_dice: 0.364216, val dice : 0.635784
[00:58:01.852] epoch : 86, iteration : 696, val loss : 0.492199, val loss_ce: 0.561402, val loss_dice: 0.422996, val dice : 0.577004
[00:58:01.971] epoch : 86, mean val loss : 0.532682, mean val ce loss: 0.555878, mean val dice : 0.490513
[00:58:25.713] epoch : 87, iteration : 1568, train loss : 0.360055, train loss_ce: 0.308822, train loss_dice: 0.411289, train dice : 0.588711
[00:59:03.125] epoch : 87, iteration : 1576, train loss : 0.555344, train loss_ce: 0.706539, train loss_dice: 0.404148, train dice : 0.595852
[00:59:38.437] epoch : 87, iteration : 1584, train loss : 0.630173, train loss_ce: 0.754623, train loss_dice: 0.505723, train dice : 0.494277
[00:59:38.581] epoch : 87, mean train loss : 0.521236, mean train ce loss: 0.581104, mean train dice : 0.538631
[01:00:00.140] epoch : 87, iteration : 702, val loss : 0.570623, val loss_ce: 0.660573, val loss_dice: 0.480672, val dice : 0.519328
[01:00:09.808] epoch : 87, mean val loss : 0.523088, mean val ce loss: 0.592461, mean val dice : 0.546285
[01:01:05.536] epoch : 88, iteration : 1592, train loss : 0.412978, train loss_ce: 0.280973, train loss_dice: 0.544984, train dice : 0.455016
[01:01:42.971] epoch : 88, iteration : 1600, train loss : 0.409591, train loss_ce: 0.470110, train loss_dice: 0.349071, train dice : 0.650928
[01:01:50.331] epoch : 88, mean train loss : 0.472461, mean train ce loss: 0.488849, mean train dice : 0.543927
[01:02:17.128] epoch : 88, iteration : 708, val loss : 0.471386, val loss_ce: 0.569984, val loss_dice: 0.372788, val dice : 0.627212
[01:02:23.981] epoch : 88, mean val loss : 0.514083, mean val ce loss: 0.586275, mean val dice : 0.558109
[01:03:07.115] epoch : 89, iteration : 1608, train loss : 0.322000, train loss_ce: 0.290873, train loss_dice: 0.353127, train dice : 0.646873
[01:03:44.529] epoch : 89, iteration : 1616, train loss : 0.440405, train loss_ce: 0.531539, train loss_dice: 0.349270, train dice : 0.650730
[01:04:01.369] epoch : 89, mean train loss : 0.472412, mean train ce loss: 0.489510, mean train dice : 0.544687
[01:04:17.452] epoch : 89, iteration : 714, val loss : 0.469854, val loss_ce: 0.518067, val loss_dice: 0.421640, val dice : 0.578360
[01:04:25.685] epoch : 89, iteration : 720, val loss : 0.443883, val loss_ce: 0.557368, val loss_dice: 0.330397, val dice : 0.669603
[01:04:25.802] epoch : 89, mean val loss : 0.471567, mean val ce loss: 0.503024, mean val dice : 0.559891
[01:04:53.415] epoch : 90, iteration : 1624, train loss : 0.451292, train loss_ce: 0.460538, train loss_dice: 0.442046, train dice : 0.557954
[01:05:30.857] epoch : 90, iteration : 1632, train loss : 0.384538, train loss_ce: 0.395677, train loss_dice: 0.373399, train dice : 0.626601
[01:05:57.105] epoch : 90, mean train loss : 0.451934, mean train ce loss: 0.485539, mean train dice : 0.581671
[01:06:17.716] epoch : 90, iteration : 726, val loss : 0.560757, val loss_ce: 0.644843, val loss_dice: 0.476670, val dice : 0.523330
[01:06:25.665] epoch : 90, mean val loss : 0.499807, mean val ce loss: 0.528518, mean val dice : 0.528903
[01:06:48.835] epoch : 91, iteration : 1640, train loss : 0.369762, train loss_ce: 0.317291, train loss_dice: 0.422234, train dice : 0.577766
[01:07:26.251] epoch : 91, iteration : 1648, train loss : 0.415263, train loss_ce: 0.490249, train loss_dice: 0.340277, train dice : 0.659723
[01:08:01.507] epoch : 91, iteration : 1656, train loss : 0.410709, train loss_ce: 0.232877, train loss_dice: 0.588541, train dice : 0.411459
[01:08:01.781] epoch : 91, mean train loss : 0.473442, mean train ce loss: 0.498790, mean train dice : 0.551906
[01:08:20.740] epoch : 91, iteration : 732, val loss : 0.511081, val loss_ce: 0.659384, val loss_dice: 0.362777, val dice : 0.637223
[01:08:23.878] epoch : 91, mean val loss : 0.449774, mean val ce loss: 0.444197, mean val dice : 0.544649
[01:09:13.079] epoch : 92, iteration : 1664, train loss : 0.429755, train loss_ce: 0.484018, train loss_dice: 0.375493, train dice : 0.624507
[01:09:50.472] epoch : 92, iteration : 1672, train loss : 0.330503, train loss_ce: 0.343462, train loss_dice: 0.317544, train dice : 0.682456
[01:09:57.866] epoch : 92, mean train loss : 0.429150, mean train ce loss: 0.460767, mean train dice : 0.602467
[01:10:12.813] epoch : 92, iteration : 738, val loss : 0.439278, val loss_ce: 0.355708, val loss_dice: 0.522847, val dice : 0.477153
[01:10:31.287] epoch : 92, iteration : 744, val loss : 0.512746, val loss_ce: 0.600153, val loss_dice: 0.425339, val dice : 0.574661
[01:10:31.494] epoch : 92, mean val loss : 0.460695, mean val ce loss: 0.462808, mean val dice : 0.541418
[01:11:19.776] epoch : 93, iteration : 1680, train loss : 0.403917, train loss_ce: 0.485574, train loss_dice: 0.322259, train dice : 0.677741
[01:11:57.160] epoch : 93, iteration : 1688, train loss : 0.432945, train loss_ce: 0.376331, train loss_dice: 0.489559, train dice : 0.510441
[01:12:13.824] epoch : 93, mean train loss : 0.450350, mean train ce loss: 0.472179, mean train dice : 0.571479
[01:12:41.753] epoch : 93, iteration : 750, val loss : 0.435933, val loss_ce: 0.554818, val loss_dice: 0.317049, val dice : 0.682951
[01:12:47.108] epoch : 93, mean val loss : 0.457576, mean val ce loss: 0.494902, mean val dice : 0.579750
[01:13:15.028] epoch : 94, iteration : 1696, train loss : 0.501571, train loss_ce: 0.550422, train loss_dice: 0.452719, train dice : 0.547281
[01:13:52.421] epoch : 94, iteration : 1704, train loss : 0.432683, train loss_ce: 0.253934, train loss_dice: 0.611433, train dice : 0.388567
[01:14:18.450] epoch : 94, mean train loss : 0.435135, mean train ce loss: 0.423777, mean train dice : 0.553507
[01:14:34.208] epoch : 94, iteration : 756, val loss : 0.404321, val loss_ce: 0.407328, val loss_dice: 0.401314, val dice : 0.598686
[01:14:49.612] epoch : 94, mean val loss : 0.467491, mean val ce loss: 0.464384, mean val dice : 0.529403
[01:15:18.736] epoch : 95, iteration : 1712, train loss : 0.346171, train loss_ce: 0.347517, train loss_dice: 0.344825, train dice : 0.655175
[01:15:56.154] epoch : 95, iteration : 1720, train loss : 0.311698, train loss_ce: 0.292868, train loss_dice: 0.330529, train dice : 0.669471
[01:16:31.422] epoch : 95, iteration : 1728, train loss : 0.455081, train loss_ce: 0.383288, train loss_dice: 0.526875, train dice : 0.473125
[01:16:31.537] epoch : 95, mean train loss : 0.394985, mean train ce loss: 0.384637, mean train dice : 0.594667
[01:16:52.867] epoch : 95, iteration : 762, val loss : 0.402092, val loss_ce: 0.431790, val loss_dice: 0.372395, val dice : 0.627605
[01:17:02.992] epoch : 95, iteration : 768, val loss : 0.350370, val loss_ce: 0.407466, val loss_dice: 0.293274, val dice : 0.706726
[01:17:03.204] epoch : 95, mean val loss : 0.444649, mean val ce loss: 0.421014, mean val dice : 0.531715
[01:17:59.570] epoch : 96, iteration : 1736, train loss : 0.385002, train loss_ce: 0.211348, train loss_dice: 0.558656, train dice : 0.441344
[01:18:36.956] epoch : 96, iteration : 1744, train loss : 0.407052, train loss_ce: 0.408620, train loss_dice: 0.405484, train dice : 0.594516
[01:18:44.348] epoch : 96, mean train loss : 0.418346, mean train ce loss: 0.381219, mean train dice : 0.544527
[01:19:03.320] epoch : 96, iteration : 774, val loss : 0.508379, val loss_ce: 0.611084, val loss_dice: 0.405673, val dice : 0.594327
[01:19:10.484] epoch : 96, mean val loss : 0.426007, mean val ce loss: 0.454774, mean val dice : 0.602760
[01:19:10.821] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 97, val dice: 0.6027601957321167
[01:19:52.496] epoch : 97, iteration : 1752, train loss : 0.433564, train loss_ce: 0.533528, train loss_dice: 0.333600, train dice : 0.666400
[01:20:29.905] epoch : 97, iteration : 1760, train loss : 0.463055, train loss_ce: 0.497026, train loss_dice: 0.429084, train dice : 0.570916
[01:20:46.807] epoch : 97, mean train loss : 0.440364, mean train ce loss: 0.473044, mean train dice : 0.592316
[01:21:08.821] epoch : 97, iteration : 780, val loss : 0.589705, val loss_ce: 0.718444, val loss_dice: 0.460967, val dice : 0.539033
[01:21:16.385] epoch : 97, mean val loss : 0.448584, mean val ce loss: 0.476793, mean val dice : 0.579626
[01:21:51.918] epoch : 98, iteration : 1768, train loss : 0.394909, train loss_ce: 0.467698, train loss_dice: 0.322121, train dice : 0.677879
[01:22:29.287] epoch : 98, iteration : 1776, train loss : 0.339790, train loss_ce: 0.200368, train loss_dice: 0.479212, train dice : 0.520788
[01:22:55.330] epoch : 98, mean train loss : 0.411051, mean train ce loss: 0.418460, mean train dice : 0.596357
[01:23:14.397] epoch : 98, iteration : 786, val loss : 0.388071, val loss_ce: 0.433214, val loss_dice: 0.342927, val dice : 0.657073
[01:23:24.682] epoch : 98, iteration : 792, val loss : 0.497170, val loss_ce: 0.628209, val loss_dice: 0.366130, val dice : 0.633870
[01:23:24.841] epoch : 98, mean val loss : 0.443980, mean val ce loss: 0.430783, mean val dice : 0.542824
[01:23:48.591] epoch : 99, iteration : 1784, train loss : 0.406220, train loss_ce: 0.364147, train loss_dice: 0.448292, train dice : 0.551708
[01:24:26.009] epoch : 99, iteration : 1792, train loss : 0.482657, train loss_ce: 0.589789, train loss_dice: 0.375525, train dice : 0.624475
[01:25:01.259] epoch : 99, iteration : 1800, train loss : 0.424662, train loss_ce: 0.394349, train loss_dice: 0.454974, train dice : 0.545026
[01:25:01.440] epoch : 99, mean train loss : 0.396647, mean train ce loss: 0.401549, mean train dice : 0.608255
[01:25:21.740] epoch : 99, iteration : 798, val loss : 0.412586, val loss_ce: 0.486052, val loss_dice: 0.339119, val dice : 0.660881
[01:25:30.613] epoch : 99, mean val loss : 0.438084, mean val ce loss: 0.453536, mean val dice : 0.577368
[01:26:27.426] epoch : 100, iteration : 1808, train loss : 0.536633, train loss_ce: 0.640087, train loss_dice: 0.433179, train dice : 0.566821
[01:27:04.814] epoch : 100, iteration : 1816, train loss : 0.279216, train loss_ce: 0.216236, train loss_dice: 0.342196, train dice : 0.657804
[01:27:12.201] epoch : 100, mean train loss : 0.443071, mean train ce loss: 0.452348, mean train dice : 0.566206
[01:27:24.996] epoch : 100, iteration : 804, val loss : 0.360285, val loss_ce: 0.365623, val loss_dice: 0.354947, val dice : 0.645053
[01:27:41.273] epoch : 100, mean val loss : 0.480112, mean val ce loss: 0.528893, mean val dice : 0.568669
[01:28:16.601] epoch : 101, iteration : 1824, train loss : 0.584989, train loss_ce: 0.673723, train loss_dice: 0.496255, train dice : 0.503745
[01:28:53.996] epoch : 101, iteration : 1832, train loss : 0.486751, train loss_ce: 0.555600, train loss_dice: 0.417902, train dice : 0.582098
[01:29:10.752] epoch : 101, mean train loss : 0.463985, mean train ce loss: 0.495214, mean train dice : 0.567244
[01:29:29.494] epoch : 101, iteration : 810, val loss : 0.643833, val loss_ce: 0.832683, val loss_dice: 0.454983, val dice : 0.545017
[01:29:40.974] epoch : 101, iteration : 816, val loss : 0.902511, val loss_ce: 1.134805, val loss_dice: 0.670217, val dice : 0.329783
[01:29:41.092] epoch : 101, mean val loss : 0.580683, mean val ce loss: 0.658983, mean val dice : 0.497618
[01:30:17.141] epoch : 102, iteration : 1840, train loss : 0.922449, train loss_ce: 1.280192, train loss_dice: 0.564706, train dice : 0.435294
[01:30:54.548] epoch : 102, iteration : 1848, train loss : 0.624248, train loss_ce: 0.634508, train loss_dice: 0.613988, train dice : 0.386012
[01:31:20.826] epoch : 102, mean train loss : 0.536314, mean train ce loss: 0.606695, mean train dice : 0.534066
[01:31:43.648] epoch : 102, iteration : 822, val loss : 0.648256, val loss_ce: 0.643892, val loss_dice: 0.652621, val dice : 0.347379
[01:31:55.816] epoch : 102, mean val loss : 0.561630, mean val ce loss: 0.635383, mean val dice : 0.512124
[01:32:19.003] epoch : 103, iteration : 1856, train loss : 0.367428, train loss_ce: 0.402622, train loss_dice: 0.332234, train dice : 0.667766
[01:32:56.345] epoch : 103, iteration : 1864, train loss : 0.499736, train loss_ce: 0.460517, train loss_dice: 0.538954, train dice : 0.461046
[01:33:31.678] epoch : 103, iteration : 1872, train loss : 0.728895, train loss_ce: 0.883839, train loss_dice: 0.573951, train dice : 0.426049
[01:33:31.865] epoch : 103, mean train loss : 0.491643, mean train ce loss: 0.555778, mean train dice : 0.572492
[01:33:53.504] epoch : 103, iteration : 828, val loss : 0.551841, val loss_ce: 0.717052, val loss_dice: 0.386631, val dice : 0.613369
[01:33:55.743] epoch : 103, mean val loss : 0.546541, mean val ce loss: 0.618442, mean val dice : 0.525360
[01:34:41.871] epoch : 104, iteration : 1880, train loss : 0.540122, train loss_ce: 0.390500, train loss_dice: 0.689744, train dice : 0.310256
[01:35:19.273] epoch : 104, iteration : 1888, train loss : 0.343280, train loss_ce: 0.372514, train loss_dice: 0.314045, train dice : 0.685955
[01:35:26.607] epoch : 104, mean train loss : 0.496085, mean train ce loss: 0.545436, mean train dice : 0.553265
[01:35:39.470] epoch : 104, iteration : 834, val loss : 0.441150, val loss_ce: 0.515513, val loss_dice: 0.366788, val dice : 0.633212
[01:35:59.208] epoch : 104, iteration : 840, val loss : 0.317462, val loss_ce: 0.254825, val loss_dice: 0.380099, val dice : 0.619901
[01:35:59.467] epoch : 104, mean val loss : 0.483546, mean val ce loss: 0.527039, mean val dice : 0.559947
[01:36:48.001] epoch : 105, iteration : 1896, train loss : 0.524802, train loss_ce: 0.450315, train loss_dice: 0.599288, train dice : 0.400712
[01:37:25.376] epoch : 105, iteration : 1904, train loss : 0.378412, train loss_ce: 0.390435, train loss_dice: 0.366389, train dice : 0.633611
[01:37:42.144] epoch : 105, mean train loss : 0.445553, mean train ce loss: 0.453865, mean train dice : 0.562759
[01:38:13.336] epoch : 105, iteration : 846, val loss : 0.411378, val loss_ce: 0.186938, val loss_dice: 0.635818, val dice : 0.364182
[01:38:14.513] epoch : 105, mean val loss : 0.442959, mean val ce loss: 0.406978, mean val dice : 0.521061
[01:38:43.226] epoch : 106, iteration : 1912, train loss : 0.387795, train loss_ce: 0.392709, train loss_dice: 0.382882, train dice : 0.617118
[01:39:20.613] epoch : 106, iteration : 1920, train loss : 0.708090, train loss_ce: 0.878877, train loss_dice: 0.537303, train dice : 0.462697
[01:39:46.705] epoch : 106, mean train loss : 0.445839, mean train ce loss: 0.489765, mean train dice : 0.598086
[01:40:03.026] epoch : 106, iteration : 852, val loss : 0.514433, val loss_ce: 0.613696, val loss_dice: 0.415170, val dice : 0.584830
[01:40:14.703] epoch : 106, mean val loss : 0.467005, mean val ce loss: 0.473545, mean val dice : 0.539535
[01:40:31.423] epoch : 107, iteration : 1928, train loss : 0.430008, train loss_ce: 0.471997, train loss_dice: 0.388019, train dice : 0.611981
[01:41:12.594] epoch : 107, iteration : 1936, train loss : 0.450810, train loss_ce: 0.434093, train loss_dice: 0.467527, train dice : 0.532473
[01:41:47.868] epoch : 107, iteration : 1944, train loss : 0.450623, train loss_ce: 0.358464, train loss_dice: 0.542781, train dice : 0.457219
[01:41:48.063] epoch : 107, mean train loss : 0.409587, mean train ce loss: 0.437703, mean train dice : 0.618529
[01:42:05.991] epoch : 107, iteration : 858, val loss : 0.453776, val loss_ce: 0.463426, val loss_dice: 0.444126, val dice : 0.555874
[01:42:16.767] epoch : 107, iteration : 864, val loss : 0.356586, val loss_ce: 0.342741, val loss_dice: 0.370431, val dice : 0.629569
[01:42:16.903] epoch : 107, mean val loss : 0.419041, mean val ce loss: 0.432753, mean val dice : 0.594672
[01:43:10.951] epoch : 108, iteration : 1952, train loss : 0.327850, train loss_ce: 0.238885, train loss_dice: 0.416814, train dice : 0.583186
[01:43:48.347] epoch : 108, iteration : 1960, train loss : 0.397508, train loss_ce: 0.454020, train loss_dice: 0.340996, train dice : 0.659004
[01:43:55.709] epoch : 108, mean train loss : 0.383523, mean train ce loss: 0.392266, mean train dice : 0.625220
[01:44:15.073] epoch : 108, iteration : 870, val loss : 0.369223, val loss_ce: 0.441882, val loss_dice: 0.296564, val dice : 0.703436
[01:44:23.675] epoch : 108, mean val loss : 0.420763, mean val ce loss: 0.442587, mean val dice : 0.601061
[01:45:14.923] epoch : 109, iteration : 1968, train loss : 0.383048, train loss_ce: 0.386027, train loss_dice: 0.380070, train dice : 0.619930
[01:45:52.320] epoch : 109, iteration : 1976, train loss : 0.477375, train loss_ce: 0.508128, train loss_dice: 0.446622, train dice : 0.553378
[01:46:09.106] epoch : 109, mean train loss : 0.459497, mean train ce loss: 0.484127, mean train dice : 0.565133
[01:46:30.164] epoch : 109, iteration : 876, val loss : 0.349386, val loss_ce: 0.436468, val loss_dice: 0.262305, val dice : 0.737695
[01:46:43.401] epoch : 109, mean val loss : 0.412016, mean val ce loss: 0.456210, mean val dice : 0.632178
[01:46:43.667] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 110, val dice: 0.6321776509284973
[01:47:19.068] epoch : 110, iteration : 1984, train loss : 0.341675, train loss_ce: 0.319709, train loss_dice: 0.363641, train dice : 0.636359
[01:47:56.466] epoch : 110, iteration : 1992, train loss : 0.355479, train loss_ce: 0.368504, train loss_dice: 0.342454, train dice : 0.657546
[01:48:22.556] epoch : 110, mean train loss : 0.437734, mean train ce loss: 0.460724, mean train dice : 0.585256
[01:48:43.066] epoch : 110, iteration : 882, val loss : 0.533836, val loss_ce: 0.691334, val loss_dice: 0.376339, val dice : 0.623661
[01:48:51.081] epoch : 110, iteration : 888, val loss : 0.321281, val loss_ce: 0.235655, val loss_dice: 0.406908, val dice : 0.593092
[01:48:51.290] epoch : 110, mean val loss : 0.399508, mean val ce loss: 0.409903, mean val dice : 0.610886
[01:49:15.332] epoch : 111, iteration : 2000, train loss : 0.380901, train loss_ce: 0.432635, train loss_dice: 0.329168, train dice : 0.670832
[01:49:52.709] epoch : 111, iteration : 2008, train loss : 0.443704, train loss_ce: 0.391526, train loss_dice: 0.495883, train dice : 0.504117
[01:50:27.993] epoch : 111, iteration : 2016, train loss : 0.430039, train loss_ce: 0.403965, train loss_dice: 0.456113, train dice : 0.543887
[01:50:28.194] epoch : 111, mean train loss : 0.383686, mean train ce loss: 0.390045, mean train dice : 0.622673
[01:50:47.742] epoch : 111, iteration : 894, val loss : 0.449101, val loss_ce: 0.536871, val loss_dice: 0.361332, val dice : 0.638668
[01:50:55.153] epoch : 111, mean val loss : 0.391696, mean val ce loss: 0.406665, mean val dice : 0.623272
[01:51:39.468] epoch : 112, iteration : 2024, train loss : 0.497682, train loss_ce: 0.472637, train loss_dice: 0.522727, train dice : 0.477273
[01:52:16.832] epoch : 112, iteration : 2032, train loss : 0.249437, train loss_ce: 0.206991, train loss_dice: 0.291883, train dice : 0.708117
[01:52:24.230] epoch : 112, mean train loss : 0.390582, mean train ce loss: 0.372748, mean train dice : 0.591583
[01:52:44.214] epoch : 112, iteration : 900, val loss : 0.538861, val loss_ce: 0.712214, val loss_dice: 0.365509, val dice : 0.634491
[01:52:50.424] epoch : 112, mean val loss : 0.416556, mean val ce loss: 0.447679, mean val dice : 0.614567
[01:53:34.699] epoch : 113, iteration : 2040, train loss : 0.311840, train loss_ce: 0.339562, train loss_dice: 0.284117, train dice : 0.715883
[01:54:12.090] epoch : 113, iteration : 2048, train loss : 0.526803, train loss_ce: 0.525391, train loss_dice: 0.528215, train dice : 0.471785
[01:54:28.864] epoch : 113, mean train loss : 0.354963, mean train ce loss: 0.370594, mean train dice : 0.660668
[01:54:42.551] epoch : 113, iteration : 906, val loss : 0.409472, val loss_ce: 0.518250, val loss_dice: 0.300694, val dice : 0.699306
[01:54:54.727] epoch : 113, iteration : 912, val loss : 0.402881, val loss_ce: 0.498945, val loss_dice: 0.306817, val dice : 0.693183
[01:54:54.882] epoch : 113, mean val loss : 0.356795, mean val ce loss: 0.388520, mean val dice : 0.674931
[01:54:55.170] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 114, val dice: 0.6749305725097656
[01:55:35.312] epoch : 114, iteration : 2056, train loss : 0.341060, train loss_ce: 0.421682, train loss_dice: 0.260438, train dice : 0.739562
[01:56:12.721] epoch : 114, iteration : 2064, train loss : 0.352342, train loss_ce: 0.356905, train loss_dice: 0.347779, train dice : 0.652221
[01:56:38.842] epoch : 114, mean train loss : 0.426069, mean train ce loss: 0.425874, mean train dice : 0.573736
[01:57:00.599] epoch : 114, iteration : 918, val loss : 0.345105, val loss_ce: 0.380386, val loss_dice: 0.309824, val dice : 0.690176
[01:57:11.769] epoch : 114, mean val loss : 0.409909, mean val ce loss: 0.424172, mean val dice : 0.604354
[01:57:37.218] epoch : 115, iteration : 2072, train loss : 0.441259, train loss_ce: 0.540697, train loss_dice: 0.341821, train dice : 0.658179
[01:58:14.628] epoch : 115, iteration : 2080, train loss : 0.460861, train loss_ce: 0.349675, train loss_dice: 0.572048, train dice : 0.427952
[01:58:49.916] epoch : 115, iteration : 2088, train loss : 0.368104, train loss_ce: 0.361799, train loss_dice: 0.374408, train dice : 0.625592
[01:58:50.087] epoch : 115, mean train loss : 0.366967, mean train ce loss: 0.388312, mean train dice : 0.654378
[01:59:09.317] epoch : 115, iteration : 924, val loss : 0.303325, val loss_ce: 0.384699, val loss_dice: 0.221951, val dice : 0.778049
[01:59:16.061] epoch : 115, mean val loss : 0.392534, mean val ce loss: 0.444866, mean val dice : 0.659797
[02:00:20.109] epoch : 116, iteration : 2096, train loss : 0.358737, train loss_ce: 0.344359, train loss_dice: 0.373114, train dice : 0.626886
[02:00:57.483] epoch : 116, iteration : 2104, train loss : 0.363202, train loss_ce: 0.432413, train loss_dice: 0.293991, train dice : 0.706009
[02:01:04.808] epoch : 116, mean train loss : 0.354065, mean train ce loss: 0.354876, mean train dice : 0.646746
[02:01:26.196] epoch : 116, iteration : 930, val loss : 0.373242, val loss_ce: 0.459196, val loss_dice: 0.287287, val dice : 0.712713
[02:01:32.161] epoch : 116, iteration : 936, val loss : 0.345543, val loss_ce: 0.436709, val loss_dice: 0.254377, val dice : 0.745623
[02:01:32.356] epoch : 116, mean val loss : 0.414278, mean val ce loss: 0.438501, mean val dice : 0.609945
[02:02:12.873] epoch : 117, iteration : 2112, train loss : 0.365404, train loss_ce: 0.440669, train loss_dice: 0.290139, train dice : 0.709861
[02:02:50.270] epoch : 117, iteration : 2120, train loss : 0.340791, train loss_ce: 0.235941, train loss_dice: 0.445640, train dice : 0.554360
[02:03:07.137] epoch : 117, mean train loss : 0.352846, mean train ce loss: 0.344496, mean train dice : 0.638805
[02:03:24.636] epoch : 117, iteration : 942, val loss : 0.632374, val loss_ce: 0.548715, val loss_dice: 0.716033, val dice : 0.283967
[02:03:34.619] epoch : 117, mean val loss : 0.395036, mean val ce loss: 0.405137, mean val dice : 0.615064
[02:04:08.514] epoch : 118, iteration : 2128, train loss : 0.353737, train loss_ce: 0.405478, train loss_dice: 0.301997, train dice : 0.698003
[02:04:45.897] epoch : 118, iteration : 2136, train loss : 0.426324, train loss_ce: 0.260098, train loss_dice: 0.592550, train dice : 0.407450
[02:05:12.001] epoch : 118, mean train loss : 0.358091, mean train ce loss: 0.335361, mean train dice : 0.619179
[02:05:31.343] epoch : 118, iteration : 948, val loss : 0.315117, val loss_ce: 0.371351, val loss_dice: 0.258883, val dice : 0.741117
[02:05:38.421] epoch : 118, mean val loss : 0.376699, mean val ce loss: 0.364406, mean val dice : 0.611009
[02:06:08.594] epoch : 119, iteration : 2144, train loss : 0.321890, train loss_ce: 0.364820, train loss_dice: 0.278959, train dice : 0.721041
[02:06:45.937] epoch : 119, iteration : 2152, train loss : 0.473123, train loss_ce: 0.458982, train loss_dice: 0.487264, train dice : 0.512736
[02:07:21.154] epoch : 119, iteration : 2160, train loss : 0.428213, train loss_ce: 0.380127, train loss_dice: 0.476300, train dice : 0.523700
[02:07:21.355] epoch : 119, mean train loss : 0.330296, mean train ce loss: 0.333343, mean train dice : 0.672750
[02:08:34.262] epoch : 119, iteration : 954, val loss : 0.345026, val loss_ce: 0.370904, val loss_dice: 0.319147, val dice : 0.680853
[02:08:46.766] epoch : 119, iteration : 960, val loss : 0.274925, val loss_ce: 0.220582, val loss_dice: 0.329268, val dice : 0.670732
[02:08:46.877] epoch : 119, mean val loss : 0.366751, mean val ce loss: 0.380342, mean val dice : 0.646841
[02:09:34.324] epoch : 120, iteration : 2168, train loss : 0.224561, train loss_ce: 0.236845, train loss_dice: 0.212277, train dice : 0.787723
[02:10:11.716] epoch : 120, iteration : 2176, train loss : 0.333111, train loss_ce: 0.398823, train loss_dice: 0.267400, train dice : 0.732600
[02:10:19.127] epoch : 120, mean train loss : 0.344712, mean train ce loss: 0.329004, mean train dice : 0.639580
[02:10:39.880] epoch : 120, iteration : 966, val loss : 0.317995, val loss_ce: 0.377284, val loss_dice: 0.258706, val dice : 0.741294
[02:10:51.138] epoch : 120, mean val loss : 0.303294, mean val ce loss: 0.319196, mean val dice : 0.712607
[02:10:51.408] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 121, val dice: 0.7126073241233826
[02:11:34.172] epoch : 121, iteration : 2184, train loss : 0.268325, train loss_ce: 0.302682, train loss_dice: 0.233968, train dice : 0.766032
[02:12:11.581] epoch : 121, iteration : 2192, train loss : 0.302783, train loss_ce: 0.311395, train loss_dice: 0.294172, train dice : 0.705828
[02:12:28.244] epoch : 121, mean train loss : 0.322411, mean train ce loss: 0.337724, mean train dice : 0.692902
[02:12:52.485] epoch : 121, iteration : 972, val loss : 0.330386, val loss_ce: 0.413941, val loss_dice: 0.246831, val dice : 0.753169
[02:12:54.654] epoch : 121, mean val loss : 0.318717, mean val ce loss: 0.329632, mean val dice : 0.692199
[02:13:28.055] epoch : 122, iteration : 2200, train loss : 0.383023, train loss_ce: 0.384697, train loss_dice: 0.381349, train dice : 0.618651
[02:14:05.448] epoch : 122, iteration : 2208, train loss : 0.343328, train loss_ce: 0.347586, train loss_dice: 0.339071, train dice : 0.660929
[02:14:31.511] epoch : 122, mean train loss : 0.340796, mean train ce loss: 0.322683, mean train dice : 0.641091
[02:14:46.897] epoch : 122, iteration : 978, val loss : 0.267231, val loss_ce: 0.171801, val loss_dice: 0.362661, val dice : 0.637339
[02:14:58.270] epoch : 122, iteration : 984, val loss : 0.267232, val loss_ce: 0.214948, val loss_dice: 0.319517, val dice : 0.680483
[02:14:58.440] epoch : 122, mean val loss : 0.348943, mean val ce loss: 0.354436, mean val dice : 0.656550
[02:15:20.714] epoch : 123, iteration : 2216, train loss : 0.310613, train loss_ce: 0.279408, train loss_dice: 0.341819, train dice : 0.658181
[02:15:58.110] epoch : 123, iteration : 2224, train loss : 0.447575, train loss_ce: 0.472587, train loss_dice: 0.422562, train dice : 0.577438
[02:16:33.365] epoch : 123, iteration : 2232, train loss : 0.428260, train loss_ce: 0.533125, train loss_dice: 0.323395, train dice : 0.676605
[02:16:33.565] epoch : 123, mean train loss : 0.334750, mean train ce loss: 0.320506, mean train dice : 0.651006
[02:16:56.804] epoch : 123, iteration : 990, val loss : 0.293254, val loss_ce: 0.293157, val loss_dice: 0.293352, val dice : 0.706648
[02:17:02.945] epoch : 123, mean val loss : 0.340058, mean val ce loss: 0.312277, mean val dice : 0.632160
[02:18:00.660] epoch : 124, iteration : 2240, train loss : 0.419382, train loss_ce: 0.419402, train loss_dice: 0.419361, train dice : 0.580639
[02:18:38.071] epoch : 124, iteration : 2248, train loss : 0.285688, train loss_ce: 0.352474, train loss_dice: 0.218903, train dice : 0.781097
[02:18:45.503] epoch : 124, mean train loss : 0.339332, mean train ce loss: 0.357030, mean train dice : 0.678365
[02:19:06.804] epoch : 124, iteration : 996, val loss : 0.255698, val loss_ce: 0.279350, val loss_dice: 0.232047, val dice : 0.767953
[02:19:23.465] epoch : 124, mean val loss : 0.297312, mean val ce loss: 0.274624, mean val dice : 0.679999
[02:20:01.207] epoch : 125, iteration : 2256, train loss : 0.597474, train loss_ce: 0.729082, train loss_dice: 0.465866, train dice : 0.534134
[02:20:38.595] epoch : 125, iteration : 2264, train loss : 0.362712, train loss_ce: 0.394703, train loss_dice: 0.330721, train dice : 0.669279
[02:20:55.366] epoch : 125, mean train loss : 0.439112, mean train ce loss: 0.467299, mean train dice : 0.589076
[02:21:04.900] epoch : 125, iteration : 1002, val loss : 0.750456, val loss_ce: 1.008377, val loss_dice: 0.492535, val dice : 0.507465
[02:21:21.511] epoch : 125, iteration : 1008, val loss : 0.601446, val loss_ce: 0.797700, val loss_dice: 0.405191, val dice : 0.594809
[02:21:21.609] epoch : 125, mean val loss : 0.541835, mean val ce loss: 0.627703, mean val dice : 0.544032
[02:21:57.832] epoch : 126, iteration : 2272, train loss : 0.717381, train loss_ce: 0.944577, train loss_dice: 0.490185, train dice : 0.509815
[02:22:35.237] epoch : 126, iteration : 2280, train loss : 0.294308, train loss_ce: 0.211710, train loss_dice: 0.376906, train dice : 0.623094
[02:23:01.366] epoch : 126, mean train loss : 0.447760, mean train ce loss: 0.480056, mean train dice : 0.584536
[02:23:20.697] epoch : 126, iteration : 1014, val loss : 0.492794, val loss_ce: 0.579285, val loss_dice: 0.406303, val dice : 0.593697
[02:23:28.097] epoch : 126, mean val loss : 0.557238, mean val ce loss: 0.632328, mean val dice : 0.517852
[02:23:45.181] epoch : 127, iteration : 2288, train loss : 0.434489, train loss_ce: 0.532914, train loss_dice: 0.336064, train dice : 0.663936
[02:24:22.538] epoch : 127, iteration : 2296, train loss : 0.428570, train loss_ce: 0.358334, train loss_dice: 0.498807, train dice : 0.501193
[02:24:57.806] epoch : 127, iteration : 2304, train loss : 0.902682, train loss_ce: 1.096131, train loss_dice: 0.709234, train dice : 0.290766
[02:24:57.929] epoch : 127, mean train loss : 0.518810, mean train ce loss: 0.551363, mean train dice : 0.513743
[02:25:19.428] epoch : 127, iteration : 1020, val loss : 0.507671, val loss_ce: 0.646050, val loss_dice: 0.369291, val dice : 0.630709
[02:25:25.124] epoch : 127, mean val loss : 0.431701, mean val ce loss: 0.466612, mean val dice : 0.603211
[02:26:19.182] epoch : 128, iteration : 2312, train loss : 0.707932, train loss_ce: 0.697802, train loss_dice: 0.718063, train dice : 0.281937
[02:26:56.567] epoch : 128, iteration : 2320, train loss : 0.566725, train loss_ce: 0.542844, train loss_dice: 0.590605, train dice : 0.409395
[02:27:03.973] epoch : 128, mean train loss : 0.455398, mean train ce loss: 0.466051, mean train dice : 0.555256
[02:27:29.338] epoch : 128, iteration : 1026, val loss : 0.605210, val loss_ce: 0.778150, val loss_dice: 0.432271, val dice : 0.567729
[02:27:42.856] epoch : 128, iteration : 1032, val loss : 0.326992, val loss_ce: 0.288095, val loss_dice: 0.365888, val dice : 0.634112
[02:27:42.995] epoch : 128, mean val loss : 0.501745, mean val ce loss: 0.563862, mean val dice : 0.560373
[02:28:31.411] epoch : 129, iteration : 2328, train loss : 0.350430, train loss_ce: 0.424664, train loss_dice: 0.276195, train dice : 0.723805
[02:29:08.804] epoch : 129, iteration : 2336, train loss : 0.451919, train loss_ce: 0.377770, train loss_dice: 0.526069, train dice : 0.473931
[02:29:25.564] epoch : 129, mean train loss : 0.418504, mean train ce loss: 0.424772, mean train dice : 0.587764
[02:29:48.959] epoch : 129, iteration : 1038, val loss : 0.304657, val loss_ce: 0.276498, val loss_dice: 0.332816, val dice : 0.667184
[02:30:02.859] epoch : 129, mean val loss : 0.411929, mean val ce loss: 0.477069, mean val dice : 0.653210
[02:30:33.291] epoch : 130, iteration : 2344, train loss : 0.250407, train loss_ce: 0.235843, train loss_dice: 0.264971, train dice : 0.735029
[02:31:10.670] epoch : 130, iteration : 2352, train loss : 0.413287, train loss_ce: 0.395013, train loss_dice: 0.431561, train dice : 0.568439
[02:31:36.852] epoch : 130, mean train loss : 0.388833, mean train ce loss: 0.397871, mean train dice : 0.620206
[02:31:56.095] epoch : 130, iteration : 1044, val loss : 0.388410, val loss_ce: 0.473509, val loss_dice: 0.303312, val dice : 0.696688
[02:32:03.837] epoch : 130, mean val loss : 0.466158, mean val ce loss: 0.502237, mean val dice : 0.569920
[02:32:30.419] epoch : 131, iteration : 2360, train loss : 0.301193, train loss_ce: 0.322731, train loss_dice: 0.279654, train dice : 0.720346
[02:33:07.838] epoch : 131, iteration : 2368, train loss : 0.412989, train loss_ce: 0.323062, train loss_dice: 0.502916, train dice : 0.497084
[02:33:43.100] epoch : 131, iteration : 2376, train loss : 0.501804, train loss_ce: 0.576306, train loss_dice: 0.427302, train dice : 0.572698
[02:33:43.232] epoch : 131, mean train loss : 0.406483, mean train ce loss: 0.405117, mean train dice : 0.592151
[02:33:59.908] epoch : 131, iteration : 1050, val loss : 0.359328, val loss_ce: 0.425420, val loss_dice: 0.293235, val dice : 0.706765
[02:34:09.672] epoch : 131, iteration : 1056, val loss : 0.480313, val loss_ce: 0.574840, val loss_dice: 0.385786, val dice : 0.614214
[02:34:09.799] epoch : 131, mean val loss : 0.417901, mean val ce loss: 0.392167, mean val dice : 0.556365
[02:35:03.765] epoch : 132, iteration : 2384, train loss : 0.466145, train loss_ce: 0.317000, train loss_dice: 0.615289, train dice : 0.384711
[02:35:41.160] epoch : 132, iteration : 2392, train loss : 0.281249, train loss_ce: 0.288720, train loss_dice: 0.273779, train dice : 0.726221
[02:35:48.517] epoch : 132, mean train loss : 0.407834, mean train ce loss: 0.381243, mean train dice : 0.565576
[02:36:06.111] epoch : 132, iteration : 1062, val loss : 0.387638, val loss_ce: 0.399495, val loss_dice: 0.375781, val dice : 0.624219
[02:36:19.555] epoch : 132, mean val loss : 0.392144, mean val ce loss: 0.410917, mean val dice : 0.626629
[02:37:09.196] epoch : 133, iteration : 2400, train loss : 0.397923, train loss_ce: 0.498686, train loss_dice: 0.297160, train dice : 0.702840
[02:37:46.599] epoch : 133, iteration : 2408, train loss : 0.367022, train loss_ce: 0.192849, train loss_dice: 0.541196, train dice : 0.458804
[02:38:03.338] epoch : 133, mean train loss : 0.376908, mean train ce loss: 0.371827, mean train dice : 0.618011
[02:38:22.500] epoch : 133, iteration : 1068, val loss : 0.497671, val loss_ce: 0.600754, val loss_dice: 0.394588, val dice : 0.605412
[02:38:29.832] epoch : 133, mean val loss : 0.456245, mean val ce loss: 0.519710, mean val dice : 0.607220
[02:38:59.485] epoch : 134, iteration : 2416, train loss : 0.654155, train loss_ce: 0.820035, train loss_dice: 0.488275, train dice : 0.511725
[02:39:36.855] epoch : 134, iteration : 2424, train loss : 0.456528, train loss_ce: 0.502996, train loss_dice: 0.410059, train dice : 0.589941
[02:40:02.903] epoch : 134, mean train loss : 0.369313, mean train ce loss: 0.324120, mean train dice : 0.585495
[02:40:19.509] epoch : 134, iteration : 1074, val loss : 0.363855, val loss_ce: 0.419647, val loss_dice: 0.308063, val dice : 0.691937
[02:40:36.264] epoch : 134, iteration : 1080, val loss : 0.402562, val loss_ce: 0.361002, val loss_dice: 0.444123, val dice : 0.555877
[02:40:36.486] epoch : 134, mean val loss : 0.377495, mean val ce loss: 0.405944, mean val dice : 0.650954
[02:40:59.611] epoch : 135, iteration : 2432, train loss : 0.398387, train loss_ce: 0.378969, train loss_dice: 0.417806, train dice : 0.582194
[02:41:37.021] epoch : 135, iteration : 2440, train loss : 0.240434, train loss_ce: 0.292388, train loss_dice: 0.188479, train dice : 0.811521
[02:42:12.380] epoch : 135, iteration : 2448, train loss : 0.580773, train loss_ce: 0.661693, train loss_dice: 0.499853, train dice : 0.500147
[02:42:12.564] epoch : 135, mean train loss : 0.333250, mean train ce loss: 0.336552, mean train dice : 0.670052
[02:42:38.124] epoch : 135, iteration : 1086, val loss : 0.349363, val loss_ce: 0.414483, val loss_dice: 0.284242, val dice : 0.715758
[02:42:44.987] epoch : 135, mean val loss : 0.345512, mean val ce loss: 0.340397, mean val dice : 0.649373
[02:43:42.952] epoch : 136, iteration : 2456, train loss : 0.383809, train loss_ce: 0.412214, train loss_dice: 0.355404, train dice : 0.644596
[02:44:20.340] epoch : 136, iteration : 2464, train loss : 0.337818, train loss_ce: 0.419952, train loss_dice: 0.255685, train dice : 0.744315
[02:44:27.781] epoch : 136, mean train loss : 0.347548, mean train ce loss: 0.323826, mean train dice : 0.628729
[02:44:44.601] epoch : 136, iteration : 1092, val loss : 0.305462, val loss_ce: 0.346299, val loss_dice: 0.264625, val dice : 0.735375
[02:44:59.963] epoch : 136, mean val loss : 0.320944, mean val ce loss: 0.301727, mean val dice : 0.659838
[02:45:41.834] epoch : 137, iteration : 2472, train loss : 0.255686, train loss_ce: 0.282892, train loss_dice: 0.228479, train dice : 0.771521
[02:46:19.233] epoch : 137, iteration : 2480, train loss : 0.463570, train loss_ce: 0.553153, train loss_dice: 0.373988, train dice : 0.626012
[02:46:36.130] epoch : 137, mean train loss : 0.339440, mean train ce loss: 0.291734, mean train dice : 0.612855
[02:46:54.448] epoch : 137, iteration : 1098, val loss : 0.386995, val loss_ce: 0.493620, val loss_dice: 0.280371, val dice : 0.719629
[02:47:06.303] epoch : 137, iteration : 1104, val loss : 0.307239, val loss_ce: 0.163267, val loss_dice: 0.451211, val dice : 0.548789
[02:47:06.421] epoch : 137, mean val loss : 0.341637, mean val ce loss: 0.347709, mean val dice : 0.664435
[02:47:38.158] epoch : 138, iteration : 2488, train loss : 0.344737, train loss_ce: 0.355054, train loss_dice: 0.334419, train dice : 0.665581
[02:48:15.554] epoch : 138, iteration : 2496, train loss : 0.383497, train loss_ce: 0.271628, train loss_dice: 0.495366, train dice : 0.504634
[02:48:41.658] epoch : 138, mean train loss : 0.352888, mean train ce loss: 0.342889, mean train dice : 0.637113
[02:49:05.467] epoch : 138, iteration : 1110, val loss : 0.397716, val loss_ce: 0.500608, val loss_dice: 0.294824, val dice : 0.705176
[02:49:17.824] epoch : 138, mean val loss : 0.360158, mean val ce loss: 0.393788, mean val dice : 0.673472
[02:49:48.396] epoch : 139, iteration : 2504, train loss : 0.310488, train loss_ce: 0.363380, train loss_dice: 0.257596, train dice : 0.742404
[02:50:25.767] epoch : 139, iteration : 2512, train loss : 0.346433, train loss_ce: 0.225261, train loss_dice: 0.467605, train dice : 0.532395
[02:51:01.024] epoch : 139, iteration : 2520, train loss : 0.332190, train loss_ce: 0.324416, train loss_dice: 0.339965, train dice : 0.660035
[02:51:01.217] epoch : 139, mean train loss : 0.317912, mean train ce loss: 0.320289, mean train dice : 0.684465
[02:51:18.957] epoch : 139, iteration : 1116, val loss : 0.314909, val loss_ce: 0.312005, val loss_dice: 0.317813, val dice : 0.682187
[02:51:30.766] epoch : 139, mean val loss : 0.327612, mean val ce loss: 0.348927, mean val dice : 0.693703
[02:52:23.038] epoch : 140, iteration : 2528, train loss : 0.276070, train loss_ce: 0.192869, train loss_dice: 0.359271, train dice : 0.640729
[02:53:00.418] epoch : 140, iteration : 2536, train loss : 0.429897, train loss_ce: 0.258848, train loss_dice: 0.600945, train dice : 0.399055
[02:53:07.910] epoch : 140, mean train loss : 0.300849, mean train ce loss: 0.277036, mean train dice : 0.675338
[02:53:27.381] epoch : 140, iteration : 1122, val loss : 0.264727, val loss_ce: 0.328609, val loss_dice: 0.200845, val dice : 0.799155
[02:53:41.102] epoch : 140, iteration : 1128, val loss : 0.300376, val loss_ce: 0.332104, val loss_dice: 0.268648, val dice : 0.731352
[02:53:41.466] epoch : 140, mean val loss : 0.349622, mean val ce loss: 0.361376, mean val dice : 0.662133
[02:54:20.211] epoch : 141, iteration : 2544, train loss : 0.342405, train loss_ce: 0.353949, train loss_dice: 0.330862, train dice : 0.669138
[02:54:57.625] epoch : 141, iteration : 2552, train loss : 0.217403, train loss_ce: 0.256181, train loss_dice: 0.178626, train dice : 0.821374
[02:55:14.427] epoch : 141, mean train loss : 0.338771, mean train ce loss: 0.315645, mean train dice : 0.638104
[02:55:37.493] epoch : 141, iteration : 1134, val loss : 0.427722, val loss_ce: 0.286521, val loss_dice: 0.568922, val dice : 0.431078
[02:55:43.460] epoch : 141, mean val loss : 0.339660, mean val ce loss: 0.326604, mean val dice : 0.647283
[02:56:22.576] epoch : 142, iteration : 2560, train loss : 0.307900, train loss_ce: 0.366979, train loss_dice: 0.248820, train dice : 0.751180
[02:56:59.976] epoch : 142, iteration : 2568, train loss : 0.346137, train loss_ce: 0.382877, train loss_dice: 0.309397, train dice : 0.690603
[02:57:26.121] epoch : 142, mean train loss : 0.327169, mean train ce loss: 0.331405, mean train dice : 0.677067
[02:57:42.839] epoch : 142, iteration : 1140, val loss : 0.308397, val loss_ce: 0.378850, val loss_dice: 0.237944, val dice : 0.762056
[02:57:52.386] epoch : 142, mean val loss : 0.338412, mean val ce loss: 0.295756, mean val dice : 0.618932
[02:58:08.840] epoch : 143, iteration : 2576, train loss : 0.381453, train loss_ce: 0.418647, train loss_dice: 0.344259, train dice : 0.655741
[02:58:46.244] epoch : 143, iteration : 2584, train loss : 0.382119, train loss_ce: 0.246365, train loss_dice: 0.517874, train dice : 0.482126
[02:59:21.499] epoch : 143, iteration : 2592, train loss : 0.326845, train loss_ce: 0.239538, train loss_dice: 0.414152, train dice : 0.585848
[02:59:21.808] epoch : 143, mean train loss : 0.342667, mean train ce loss: 0.335552, mean train dice : 0.650218
[02:59:41.873] epoch : 143, iteration : 1146, val loss : 0.238160, val loss_ce: 0.245226, val loss_dice: 0.231093, val dice : 0.768907
[02:59:59.516] epoch : 143, iteration : 1152, val loss : 0.308565, val loss_ce: 0.367005, val loss_dice: 0.250126, val dice : 0.749874
[02:59:59.657] epoch : 143, mean val loss : 0.301345, mean val ce loss: 0.323590, mean val dice : 0.720900
[02:59:59.920] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 144, val dice: 0.7209004163742065
[03:00:59.463] epoch : 144, iteration : 2600, train loss : 0.201502, train loss_ce: 0.199174, train loss_dice: 0.203830, train dice : 0.796170
[03:01:36.872] epoch : 144, iteration : 2608, train loss : 0.225702, train loss_ce: 0.234980, train loss_dice: 0.216424, train dice : 0.783576
[03:01:44.272] epoch : 144, mean train loss : 0.283566, mean train ce loss: 0.274610, mean train dice : 0.707478
[03:02:10.134] epoch : 144, iteration : 1158, val loss : 0.267813, val loss_ce: 0.262583, val loss_dice: 0.273044, val dice : 0.726956
[03:02:16.779] epoch : 144, mean val loss : 0.288312, mean val ce loss: 0.249494, mean val dice : 0.672869
[03:02:58.519] epoch : 145, iteration : 2616, train loss : 0.271868, train loss_ce: 0.267475, train loss_dice: 0.276261, train dice : 0.723739
[03:03:35.917] epoch : 145, iteration : 2624, train loss : 0.295386, train loss_ce: 0.275012, train loss_dice: 0.315759, train dice : 0.684241
[03:03:52.679] epoch : 145, mean train loss : 0.299424, mean train ce loss: 0.291488, mean train dice : 0.692641
[03:04:10.609] epoch : 145, iteration : 1164, val loss : 0.207854, val loss_ce: 0.250267, val loss_dice: 0.165441, val dice : 0.834559
[03:04:17.091] epoch : 145, mean val loss : 0.288292, mean val ce loss: 0.318246, mean val dice : 0.741662
[03:04:17.380] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 146, val dice: 0.7416620850563049
[03:04:48.204] epoch : 146, iteration : 2632, train loss : 0.241149, train loss_ce: 0.293148, train loss_dice: 0.189150, train dice : 0.810850
[03:05:25.603] epoch : 146, iteration : 2640, train loss : 0.238153, train loss_ce: 0.224576, train loss_dice: 0.251730, train dice : 0.748270
[03:05:51.699] epoch : 146, mean train loss : 0.283841, mean train ce loss: 0.283100, mean train dice : 0.715418
[03:06:10.962] epoch : 146, iteration : 1170, val loss : 0.196574, val loss_ce: 0.219551, val loss_dice: 0.173597, val dice : 0.826403
[03:06:18.135] epoch : 146, iteration : 1176, val loss : 0.372076, val loss_ce: 0.419585, val loss_dice: 0.324566, val dice : 0.675434
[03:06:18.489] epoch : 146, mean val loss : 0.287145, mean val ce loss: 0.252361, mean val dice : 0.678071
[03:06:35.386] epoch : 147, iteration : 2648, train loss : 0.328638, train loss_ce: 0.329316, train loss_dice: 0.327960, train dice : 0.672040
[03:07:12.809] epoch : 147, iteration : 2656, train loss : 0.318537, train loss_ce: 0.184606, train loss_dice: 0.452468, train dice : 0.547531
[03:07:48.105] epoch : 147, iteration : 2664, train loss : 0.430581, train loss_ce: 0.475340, train loss_dice: 0.385821, train dice : 0.614179
[03:07:48.301] epoch : 147, mean train loss : 0.296756, mean train ce loss: 0.296763, mean train dice : 0.703250
[03:08:09.014] epoch : 147, iteration : 1182, val loss : 0.173393, val loss_ce: 0.201192, val loss_dice: 0.145595, val dice : 0.854405
[03:08:22.263] epoch : 147, mean val loss : 0.267109, mean val ce loss: 0.253236, mean val dice : 0.719018
[03:09:14.048] epoch : 148, iteration : 2672, train loss : 0.418579, train loss_ce: 0.515932, train loss_dice: 0.321226, train dice : 0.678774
[03:09:51.462] epoch : 148, iteration : 2680, train loss : 0.307075, train loss_ce: 0.404401, train loss_dice: 0.209750, train dice : 0.790250
[03:09:58.869] epoch : 148, mean train loss : 0.278057, mean train ce loss: 0.288164, mean train dice : 0.732051
[03:10:14.047] epoch : 148, iteration : 1188, val loss : 0.202514, val loss_ce: 0.210854, val loss_dice: 0.194173, val dice : 0.805827
[03:10:23.945] epoch : 148, mean val loss : 0.280330, mean val ce loss: 0.297765, mean val dice : 0.737105
[03:11:06.553] epoch : 149, iteration : 2688, train loss : 0.329939, train loss_ce: 0.409281, train loss_dice: 0.250598, train dice : 0.749402
[03:11:43.947] epoch : 149, iteration : 2696, train loss : 0.185398, train loss_ce: 0.178388, train loss_dice: 0.192409, train dice : 0.807591
[03:12:00.677] epoch : 149, mean train loss : 0.279538, mean train ce loss: 0.236460, mean train dice : 0.677384
[03:12:14.278] epoch : 149, iteration : 1194, val loss : 0.217510, val loss_ce: 0.209336, val loss_dice: 0.225685, val dice : 0.774315
[03:12:30.778] epoch : 149, iteration : 1200, val loss : 0.212512, val loss_ce: 0.138037, val loss_dice: 0.286987, val dice : 0.713013
[03:12:30.883] epoch : 149, mean val loss : 0.264570, mean val ce loss: 0.228040, mean val dice : 0.698900
[03:13:05.269] epoch : 150, iteration : 2704, train loss : 0.248672, train loss_ce: 0.294927, train loss_dice: 0.202416, train dice : 0.797584
[03:13:42.684] epoch : 150, iteration : 2712, train loss : 0.460795, train loss_ce: 0.548138, train loss_dice: 0.373452, train dice : 0.626548
[03:14:08.765] epoch : 150, mean train loss : 0.349061, mean train ce loss: 0.362917, mean train dice : 0.664795
[03:14:25.540] epoch : 150, iteration : 1206, val loss : 0.600755, val loss_ce: 0.733090, val loss_dice: 0.468421, val dice : 0.531579
[03:14:36.240] epoch : 150, mean val loss : 0.384102, mean val ce loss: 0.380099, mean val dice : 0.611896
[03:15:00.958] epoch : 151, iteration : 2720, train loss : 0.325797, train loss_ce: 0.218133, train loss_dice: 0.433460, train dice : 0.566540
[03:15:38.351] epoch : 151, iteration : 2728, train loss : 0.400699, train loss_ce: 0.397937, train loss_dice: 0.403462, train dice : 0.596538
[03:16:13.711] epoch : 151, iteration : 2736, train loss : 0.477333, train loss_ce: 0.551990, train loss_dice: 0.402676, train dice : 0.597324
[03:16:13.823] epoch : 151, mean train loss : 0.410665, mean train ce loss: 0.412924, mean train dice : 0.591594
[03:16:29.197] epoch : 151, iteration : 1212, val loss : 0.352141, val loss_ce: 0.396206, val loss_dice: 0.308077, val dice : 0.691923
[03:16:45.371] epoch : 151, mean val loss : 0.460314, mean val ce loss: 0.501570, mean val dice : 0.580942
[03:17:37.998] epoch : 152, iteration : 2744, train loss : 0.559899, train loss_ce: 0.488554, train loss_dice: 0.631244, train dice : 0.368756
[03:18:15.392] epoch : 152, iteration : 2752, train loss : 0.291288, train loss_ce: 0.317063, train loss_dice: 0.265513, train dice : 0.734487
[03:18:22.714] epoch : 152, mean train loss : 0.356540, mean train ce loss: 0.373615, mean train dice : 0.660535
[03:18:46.357] epoch : 152, iteration : 1218, val loss : 0.247104, val loss_ce: 0.142998, val loss_dice: 0.351210, val dice : 0.648790
[03:18:56.665] epoch : 152, iteration : 1224, val loss : 0.250418, val loss_ce: 0.146593, val loss_dice: 0.354243, val dice : 0.645757
[03:18:56.843] epoch : 152, mean val loss : 0.354354, mean val ce loss: 0.363491, mean val dice : 0.654783
[03:19:41.228] epoch : 153, iteration : 2760, train loss : 0.382446, train loss_ce: 0.425957, train loss_dice: 0.338936, train dice : 0.661064
[03:20:18.628] epoch : 153, iteration : 2768, train loss : 0.470478, train loss_ce: 0.499167, train loss_dice: 0.441788, train dice : 0.558212
[03:20:35.486] epoch : 153, mean train loss : 0.353816, mean train ce loss: 0.336827, mean train dice : 0.629194
[03:20:56.025] epoch : 153, iteration : 1230, val loss : 0.335895, val loss_ce: 0.096782, val loss_dice: 0.575008, val dice : 0.424992
[03:20:57.374] epoch : 153, mean val loss : 0.305975, mean val ce loss: 0.250558, mean val dice : 0.638607
[03:21:30.253] epoch : 154, iteration : 2776, train loss : 0.377701, train loss_ce: 0.435609, train loss_dice: 0.319793, train dice : 0.680207
[03:22:07.655] epoch : 154, iteration : 2784, train loss : 0.378386, train loss_ce: 0.387370, train loss_dice: 0.369402, train dice : 0.630598
[03:22:33.812] epoch : 154, mean train loss : 0.325863, mean train ce loss: 0.325988, mean train dice : 0.674262
[03:22:51.524] epoch : 154, iteration : 1236, val loss : 0.269016, val loss_ce: 0.340053, val loss_dice: 0.197980, val dice : 0.802020
[03:22:57.063] epoch : 154, mean val loss : 0.355690, mean val ce loss: 0.369274, mean val dice : 0.657895
[03:23:15.356] epoch : 155, iteration : 2792, train loss : 0.380633, train loss_ce: 0.340154, train loss_dice: 0.421113, train dice : 0.578887
[03:23:52.742] epoch : 155, iteration : 2800, train loss : 0.397026, train loss_ce: 0.291994, train loss_dice: 0.502058, train dice : 0.497942
[03:24:28.095] epoch : 155, iteration : 2808, train loss : 0.279197, train loss_ce: 0.171858, train loss_dice: 0.386537, train dice : 0.613463
[03:24:28.276] epoch : 155, mean train loss : 0.369312, mean train ce loss: 0.355041, mean train dice : 0.616418
[03:24:48.955] epoch : 155, iteration : 1242, val loss : 0.338989, val loss_ce: 0.432380, val loss_dice: 0.245598, val dice : 0.754402
[03:24:57.766] epoch : 155, iteration : 1248, val loss : 0.425059, val loss_ce: 0.502177, val loss_dice: 0.347940, val dice : 0.652060
[03:24:57.948] epoch : 155, mean val loss : 0.395156, mean val ce loss: 0.372909, mean val dice : 0.582596
[03:25:53.878] epoch : 156, iteration : 2816, train loss : 0.355589, train loss_ce: 0.301615, train loss_dice: 0.409564, train dice : 0.590436
[03:26:31.260] epoch : 156, iteration : 2824, train loss : 0.235772, train loss_ce: 0.253645, train loss_dice: 0.217898, train dice : 0.782102
[03:26:38.699] epoch : 156, mean train loss : 0.320758, mean train ce loss: 0.312461, mean train dice : 0.670944
[03:26:59.454] epoch : 156, iteration : 1254, val loss : 0.350654, val loss_ce: 0.424747, val loss_dice: 0.276561, val dice : 0.723439
[03:27:02.403] epoch : 156, mean val loss : 0.383590, mean val ce loss: 0.410018, mean val dice : 0.642837
[03:27:45.135] epoch : 157, iteration : 2832, train loss : 0.278951, train loss_ce: 0.341452, train loss_dice: 0.216450, train dice : 0.783550
[03:28:22.532] epoch : 157, iteration : 2840, train loss : 0.332643, train loss_ce: 0.325708, train loss_dice: 0.339577, train dice : 0.660423
[03:28:39.355] epoch : 157, mean train loss : 0.322341, mean train ce loss: 0.334929, mean train dice : 0.690247
[03:29:02.209] epoch : 157, iteration : 1260, val loss : 0.268750, val loss_ce: 0.331703, val loss_dice: 0.205796, val dice : 0.794204
[03:29:06.891] epoch : 157, mean val loss : 0.334781, mean val ce loss: 0.342100, mean val dice : 0.672537
[03:29:37.150] epoch : 158, iteration : 2848, train loss : 0.350834, train loss_ce: 0.379978, train loss_dice: 0.321690, train dice : 0.678310
[03:30:14.549] epoch : 158, iteration : 2856, train loss : 0.332923, train loss_ce: 0.380061, train loss_dice: 0.285785, train dice : 0.714215
[03:30:40.715] epoch : 158, mean train loss : 0.329880, mean train ce loss: 0.315454, mean train dice : 0.655695
[03:31:00.294] epoch : 158, iteration : 1266, val loss : 0.257769, val loss_ce: 0.299270, val loss_dice: 0.216268, val dice : 0.783732
[03:31:15.156] epoch : 158, iteration : 1272, val loss : 0.245670, val loss_ce: 0.194815, val loss_dice: 0.296526, val dice : 0.703474
[03:31:15.348] epoch : 158, mean val loss : 0.344172, mean val ce loss: 0.359724, mean val dice : 0.671379
[03:31:46.245] epoch : 159, iteration : 2864, train loss : 0.426968, train loss_ce: 0.514352, train loss_dice: 0.339585, train dice : 0.660415
[03:32:23.597] epoch : 159, iteration : 2872, train loss : 0.342553, train loss_ce: 0.286378, train loss_dice: 0.398729, train dice : 0.601271
[03:32:58.841] epoch : 159, iteration : 2880, train loss : 0.362132, train loss_ce: 0.370391, train loss_dice: 0.353874, train dice : 0.646126
[03:32:59.046] epoch : 159, mean train loss : 0.326989, mean train ce loss: 0.314498, mean train dice : 0.660521
[03:33:15.245] epoch : 159, iteration : 1278, val loss : 0.501932, val loss_ce: 0.326469, val loss_dice: 0.677394, val dice : 0.322606
[03:33:33.392] epoch : 159, mean val loss : 0.336091, mean val ce loss: 0.264658, mean val dice : 0.592476
[03:34:21.867] epoch : 160, iteration : 2888, train loss : 0.266029, train loss_ce: 0.301472, train loss_dice: 0.230585, train dice : 0.769415
[03:34:59.258] epoch : 160, iteration : 2896, train loss : 0.242758, train loss_ce: 0.259536, train loss_dice: 0.225979, train dice : 0.774021
[03:35:06.624] epoch : 160, mean train loss : 0.300237, mean train ce loss: 0.317382, mean train dice : 0.716907
[03:35:26.650] epoch : 160, iteration : 1284, val loss : 0.217356, val loss_ce: 0.219165, val loss_dice: 0.215547, val dice : 0.784453
[03:35:33.607] epoch : 160, mean val loss : 0.298143, mean val ce loss: 0.271070, mean val dice : 0.674785
[03:36:11.888] epoch : 161, iteration : 2904, train loss : 0.296413, train loss_ce: 0.291675, train loss_dice: 0.301150, train dice : 0.698850
[03:36:49.294] epoch : 161, iteration : 2912, train loss : 0.250472, train loss_ce: 0.223841, train loss_dice: 0.277102, train dice : 0.722898
[03:37:06.045] epoch : 161, mean train loss : 0.280577, mean train ce loss: 0.265709, mean train dice : 0.704555
[03:37:18.528] epoch : 161, iteration : 1290, val loss : 0.246703, val loss_ce: 0.301496, val loss_dice: 0.191909, val dice : 0.808091
[03:37:34.860] epoch : 161, iteration : 1296, val loss : 0.237401, val loss_ce: 0.121875, val loss_dice: 0.352927, val dice : 0.647073
[03:37:35.049] epoch : 161, mean val loss : 0.321379, mean val ce loss: 0.321745, mean val dice : 0.678987
[03:38:04.443] epoch : 162, iteration : 2920, train loss : 0.305573, train loss_ce: 0.364199, train loss_dice: 0.246946, train dice : 0.753054
[03:38:41.833] epoch : 162, iteration : 2928, train loss : 0.271949, train loss_ce: 0.232442, train loss_dice: 0.311456, train dice : 0.688544
[03:39:07.913] epoch : 162, mean train loss : 0.295007, mean train ce loss: 0.244940, mean train dice : 0.654925
[03:39:24.680] epoch : 162, iteration : 1302, val loss : 0.253024, val loss_ce: 0.242555, val loss_dice: 0.263493, val dice : 0.736507
[03:39:32.540] epoch : 162, mean val loss : 0.303985, mean val ce loss: 0.313196, mean val dice : 0.705226
[03:39:58.235] epoch : 163, iteration : 2936, train loss : 0.274072, train loss_ce: 0.255707, train loss_dice: 0.292437, train dice : 0.707563
[03:40:35.616] epoch : 163, iteration : 2944, train loss : 0.297845, train loss_ce: 0.221496, train loss_dice: 0.374193, train dice : 0.625807
[03:41:10.841] epoch : 163, iteration : 2952, train loss : 0.228469, train loss_ce: 0.138586, train loss_dice: 0.318353, train dice : 0.681647
[03:41:10.977] epoch : 163, mean train loss : 0.270476, mean train ce loss: 0.244855, mean train dice : 0.703902
[03:41:33.984] epoch : 163, iteration : 1308, val loss : 0.413251, val loss_ce: 0.535188, val loss_dice: 0.291314, val dice : 0.708686
[03:41:42.416] epoch : 163, mean val loss : 0.334356, mean val ce loss: 0.350338, mean val dice : 0.681627
[03:42:39.341] epoch : 164, iteration : 2960, train loss : 0.360125, train loss_ce: 0.152726, train loss_dice: 0.567523, train dice : 0.432477
[03:43:16.743] epoch : 164, iteration : 2968, train loss : 0.241036, train loss_ce: 0.240614, train loss_dice: 0.241457, train dice : 0.758543
[03:43:24.265] epoch : 164, mean train loss : 0.279481, mean train ce loss: 0.244807, mean train dice : 0.685845
[03:43:40.920] epoch : 164, iteration : 1314, val loss : 0.189966, val loss_ce: 0.211463, val loss_dice: 0.168469, val dice : 0.831531
[03:43:47.158] epoch : 164, iteration : 1320, val loss : 0.193196, val loss_ce: 0.192371, val loss_dice: 0.194021, val dice : 0.805979
[03:43:47.364] epoch : 164, mean val loss : 0.295379, mean val ce loss: 0.276210, mean val dice : 0.685452
[03:44:28.135] epoch : 165, iteration : 2976, train loss : 0.197799, train loss_ce: 0.217622, train loss_dice: 0.177975, train dice : 0.822025
[03:45:05.532] epoch : 165, iteration : 2984, train loss : 0.240756, train loss_ce: 0.174404, train loss_dice: 0.307107, train dice : 0.692893
[03:45:22.335] epoch : 165, mean train loss : 0.259542, mean train ce loss: 0.254222, mean train dice : 0.735138
[03:45:45.065] epoch : 165, iteration : 1326, val loss : 0.338559, val loss_ce: 0.367092, val loss_dice: 0.310026, val dice : 0.689974
[03:45:54.085] epoch : 165, mean val loss : 0.279399, mean val ce loss: 0.281370, mean val dice : 0.722572
[03:46:30.016] epoch : 166, iteration : 2992, train loss : 0.276018, train loss_ce: 0.350840, train loss_dice: 0.201196, train dice : 0.798804
[03:47:07.408] epoch : 166, iteration : 3000, train loss : 0.268828, train loss_ce: 0.190869, train loss_dice: 0.346786, train dice : 0.653214
[03:47:33.567] epoch : 166, mean train loss : 0.290301, mean train ce loss: 0.278487, mean train dice : 0.697885
[03:47:52.977] epoch : 166, iteration : 1332, val loss : 0.189620, val loss_ce: 0.224466, val loss_dice: 0.154774, val dice : 0.845226
[03:48:04.740] epoch : 166, mean val loss : 0.261516, mean val ce loss: 0.210642, mean val dice : 0.687611
[03:48:31.597] epoch : 167, iteration : 3008, train loss : 0.336812, train loss_ce: 0.414593, train loss_dice: 0.259031, train dice : 0.740969
[03:49:08.975] epoch : 167, iteration : 3016, train loss : 0.381816, train loss_ce: 0.386724, train loss_dice: 0.376908, train dice : 0.623092
[03:49:44.219] epoch : 167, iteration : 3024, train loss : 0.331337, train loss_ce: 0.305272, train loss_dice: 0.357402, train dice : 0.642598
[03:49:44.348] epoch : 167, mean train loss : 0.260231, mean train ce loss: 0.254445, mean train dice : 0.733982
[03:50:02.981] epoch : 167, iteration : 1338, val loss : 0.280982, val loss_ce: 0.345706, val loss_dice: 0.216258, val dice : 0.783742
[03:50:14.479] epoch : 167, iteration : 1344, val loss : 0.195700, val loss_ce: 0.235491, val loss_dice: 0.155910, val dice : 0.844090
[03:50:14.732] epoch : 167, mean val loss : 0.246151, mean val ce loss: 0.228312, mean val dice : 0.736011
[03:51:08.821] epoch : 168, iteration : 3032, train loss : 0.240977, train loss_ce: 0.174699, train loss_dice: 0.307255, train dice : 0.692744
[03:51:46.221] epoch : 168, iteration : 3040, train loss : 0.166988, train loss_ce: 0.117208, train loss_dice: 0.216767, train dice : 0.783233
[03:51:53.564] epoch : 168, mean train loss : 0.279903, mean train ce loss: 0.223736, mean train dice : 0.663930
[03:52:10.286] epoch : 168, iteration : 1350, val loss : 0.271806, val loss_ce: 0.283018, val loss_dice: 0.260593, val dice : 0.739407
[03:52:21.592] epoch : 168, mean val loss : 0.248349, mean val ce loss: 0.214456, mean val dice : 0.717757
[03:52:59.201] epoch : 169, iteration : 3048, train loss : 0.274396, train loss_ce: 0.314607, train loss_dice: 0.234184, train dice : 0.765816
[03:53:36.595] epoch : 169, iteration : 3056, train loss : 0.255373, train loss_ce: 0.303357, train loss_dice: 0.207388, train dice : 0.792612
[03:53:53.267] epoch : 169, mean train loss : 0.227682, mean train ce loss: 0.240717, mean train dice : 0.785352
[03:54:13.800] epoch : 169, iteration : 1356, val loss : 0.260751, val loss_ce: 0.261961, val loss_dice: 0.259541, val dice : 0.740459
[03:54:22.487] epoch : 169, mean val loss : 0.269347, mean val ce loss: 0.232684, mean val dice : 0.693989
[03:54:51.522] epoch : 170, iteration : 3064, train loss : 0.223659, train loss_ce: 0.241753, train loss_dice: 0.205565, train dice : 0.794435
[03:55:28.936] epoch : 170, iteration : 3072, train loss : 0.160616, train loss_ce: 0.168912, train loss_dice: 0.152320, train dice : 0.847680
[03:55:55.021] epoch : 170, mean train loss : 0.246402, mean train ce loss: 0.248901, mean train dice : 0.756096
[03:56:11.473] epoch : 170, iteration : 1362, val loss : 0.222496, val loss_ce: 0.218161, val loss_dice: 0.226831, val dice : 0.773169
[03:56:22.683] epoch : 170, iteration : 1368, val loss : 0.255057, val loss_ce: 0.346128, val loss_dice: 0.163987, val dice : 0.836013
[03:56:22.927] epoch : 170, mean val loss : 0.287122, mean val ce loss: 0.297112, mean val dice : 0.722868
[03:56:43.148] epoch : 171, iteration : 3080, train loss : 0.266899, train loss_ce: 0.203037, train loss_dice: 0.330761, train dice : 0.669239
[03:57:20.560] epoch : 171, iteration : 3088, train loss : 0.245030, train loss_ce: 0.217639, train loss_dice: 0.272421, train dice : 0.727579
[03:57:55.807] epoch : 171, iteration : 3096, train loss : 0.326002, train loss_ce: 0.303809, train loss_dice: 0.348196, train dice : 0.651804
[03:57:56.054] epoch : 171, mean train loss : 0.250475, mean train ce loss: 0.257545, mean train dice : 0.756594
[03:58:11.118] epoch : 171, iteration : 1374, val loss : 0.199569, val loss_ce: 0.160094, val loss_dice: 0.239043, val dice : 0.760957
[03:58:22.646] epoch : 171, mean val loss : 0.210614, mean val ce loss: 0.225606, mean val dice : 0.804377
[03:58:22.913] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 172, val dice: 0.804377019405365
[03:59:07.701] epoch : 172, iteration : 3104, train loss : 0.164773, train loss_ce: 0.095997, train loss_dice: 0.233548, train dice : 0.766452
[03:59:45.095] epoch : 172, iteration : 3112, train loss : 0.179824, train loss_ce: 0.123423, train loss_dice: 0.236225, train dice : 0.763775
[03:59:52.406] epoch : 172, mean train loss : 0.245324, mean train ce loss: 0.200292, mean train dice : 0.709645
[04:00:08.880] epoch : 172, iteration : 1380, val loss : 0.293560, val loss_ce: 0.359887, val loss_dice: 0.227234, val dice : 0.772766
[04:00:18.336] epoch : 172, mean val loss : 0.253045, mean val ce loss: 0.226757, mean val dice : 0.720666
[04:01:02.132] epoch : 173, iteration : 3120, train loss : 0.182497, train loss_ce: 0.215539, train loss_dice: 0.149455, train dice : 0.850545
[04:01:39.534] epoch : 173, iteration : 3128, train loss : 0.171029, train loss_ce: 0.148762, train loss_dice: 0.193296, train dice : 0.806704
[04:01:56.270] epoch : 173, mean train loss : 0.264563, mean train ce loss: 0.225285, mean train dice : 0.696158
[04:02:17.239] epoch : 173, iteration : 1386, val loss : 0.201004, val loss_ce: 0.199509, val loss_dice: 0.202499, val dice : 0.797501
[04:02:35.653] epoch : 173, iteration : 1392, val loss : 0.300029, val loss_ce: 0.367912, val loss_dice: 0.232147, val dice : 0.767853
[04:02:35.770] epoch : 173, mean val loss : 0.259373, mean val ce loss: 0.266639, mean val dice : 0.747893
[04:03:06.774] epoch : 174, iteration : 3136, train loss : 0.264393, train loss_ce: 0.335873, train loss_dice: 0.192913, train dice : 0.807087
[04:03:44.188] epoch : 174, iteration : 3144, train loss : 0.195226, train loss_ce: 0.225122, train loss_dice: 0.165330, train dice : 0.834670
[04:04:10.224] epoch : 174, mean train loss : 0.239772, mean train ce loss: 0.224847, mean train dice : 0.745303
[04:04:34.769] epoch : 174, iteration : 1398, val loss : 0.347080, val loss_ce: 0.129770, val loss_dice: 0.564390, val dice : 0.435610
[04:04:35.967] epoch : 174, mean val loss : 0.274864, mean val ce loss: 0.261867, mean val dice : 0.712139
[04:04:57.387] epoch : 175, iteration : 3152, train loss : 0.352208, train loss_ce: 0.265326, train loss_dice: 0.439090, train dice : 0.560910
[04:05:34.771] epoch : 175, iteration : 3160, train loss : 0.283151, train loss_ce: 0.315033, train loss_dice: 0.251269, train dice : 0.748731
[04:06:10.049] epoch : 175, iteration : 3168, train loss : 0.288246, train loss_ce: 0.287858, train loss_dice: 0.288635, train dice : 0.711365
[04:06:10.393] epoch : 175, mean train loss : 0.269974, mean train ce loss: 0.290795, mean train dice : 0.750847
[04:06:28.462] epoch : 175, iteration : 1404, val loss : 0.217887, val loss_ce: 0.180923, val loss_dice: 0.254850, val dice : 0.745150
[04:06:41.705] epoch : 175, mean val loss : 0.266383, mean val ce loss: 0.219125, mean val dice : 0.686359
[04:07:32.054] epoch : 176, iteration : 3176, train loss : 0.232456, train loss_ce: 0.257236, train loss_dice: 0.207675, train dice : 0.792325
[04:08:09.462] epoch : 176, iteration : 3184, train loss : 0.190785, train loss_ce: 0.202347, train loss_dice: 0.179223, train dice : 0.820777
[04:08:16.865] epoch : 176, mean train loss : 0.292306, mean train ce loss: 0.311843, mean train dice : 0.727230
[04:08:34.893] epoch : 176, iteration : 1410, val loss : 0.310182, val loss_ce: 0.380498, val loss_dice: 0.239867, val dice : 0.760133
[04:08:47.874] epoch : 176, iteration : 1416, val loss : 0.285901, val loss_ce: 0.355080, val loss_dice: 0.216722, val dice : 0.783278
[04:08:48.060] epoch : 176, mean val loss : 0.408501, mean val ce loss: 0.446635, mean val dice : 0.629634
[04:09:26.605] epoch : 177, iteration : 3192, train loss : 0.279313, train loss_ce: 0.288514, train loss_dice: 0.270112, train dice : 0.729888
[04:10:04.002] epoch : 177, iteration : 3200, train loss : 0.525615, train loss_ce: 0.665612, train loss_dice: 0.385618, train dice : 0.614382
[04:10:20.761] epoch : 177, mean train loss : 0.387525, mean train ce loss: 0.406388, mean train dice : 0.631338
[04:10:41.661] epoch : 177, iteration : 1422, val loss : 0.415519, val loss_ce: 0.488853, val loss_dice: 0.342185, val dice : 0.657815
[04:10:46.042] epoch : 177, mean val loss : 0.363624, mean val ce loss: 0.394880, mean val dice : 0.667631
[04:11:20.235] epoch : 178, iteration : 3208, train loss : 0.258598, train loss_ce: 0.277340, train loss_dice: 0.239856, train dice : 0.760144
[04:11:57.633] epoch : 178, iteration : 3216, train loss : 0.250754, train loss_ce: 0.252202, train loss_dice: 0.249307, train dice : 0.750693
[04:12:23.704] epoch : 178, mean train loss : 0.372069, mean train ce loss: 0.396876, mean train dice : 0.652738
[04:12:38.973] epoch : 178, iteration : 1428, val loss : 0.391417, val loss_ce: 0.525812, val loss_dice: 0.257022, val dice : 0.742978
[04:12:44.068] epoch : 178, mean val loss : 0.405003, mean val ce loss: 0.461791, mean val dice : 0.651785
[04:13:03.524] epoch : 179, iteration : 3224, train loss : 0.473217, train loss_ce: 0.466583, train loss_dice: 0.479852, train dice : 0.520148
[04:13:40.950] epoch : 179, iteration : 3232, train loss : 0.346161, train loss_ce: 0.246592, train loss_dice: 0.445729, train dice : 0.554271
[04:14:16.317] epoch : 179, iteration : 3240, train loss : 0.330669, train loss_ce: 0.359914, train loss_dice: 0.301425, train dice : 0.698575
[04:14:16.453] epoch : 179, mean train loss : 0.325846, mean train ce loss: 0.316894, mean train dice : 0.665201
[04:14:33.333] epoch : 179, iteration : 1434, val loss : 0.261468, val loss_ce: 0.287019, val loss_dice: 0.235917, val dice : 0.764083
[04:14:45.179] epoch : 179, iteration : 1440, val loss : 0.278905, val loss_ce: 0.316123, val loss_dice: 0.241687, val dice : 0.758313
[04:14:45.280] epoch : 179, mean val loss : 0.332568, mean val ce loss: 0.341914, mean val dice : 0.676777
[04:15:42.288] epoch : 180, iteration : 3248, train loss : 0.163135, train loss_ce: 0.164013, train loss_dice: 0.162257, train dice : 0.837743
[04:16:19.690] epoch : 180, iteration : 3256, train loss : 0.347718, train loss_ce: 0.371370, train loss_dice: 0.324067, train dice : 0.675933
[04:16:27.044] epoch : 180, mean train loss : 0.297417, mean train ce loss: 0.307369, mean train dice : 0.712535
[04:16:48.963] epoch : 180, iteration : 1446, val loss : 0.403853, val loss_ce: 0.473430, val loss_dice: 0.334277, val dice : 0.665723
[04:16:57.133] epoch : 180, mean val loss : 0.332840, mean val ce loss: 0.329418, mean val dice : 0.663739
[04:17:39.053] epoch : 181, iteration : 3264, train loss : 0.283898, train loss_ce: 0.308936, train loss_dice: 0.258860, train dice : 0.741140
[04:18:16.453] epoch : 181, iteration : 3272, train loss : 0.370333, train loss_ce: 0.357404, train loss_dice: 0.383263, train dice : 0.616737
[04:18:33.345] epoch : 181, mean train loss : 0.290468, mean train ce loss: 0.272042, mean train dice : 0.691106
[04:18:50.733] epoch : 181, iteration : 1452, val loss : 0.296593, val loss_ce: 0.366471, val loss_dice: 0.226715, val dice : 0.773285
[04:19:03.789] epoch : 181, mean val loss : 0.336871, mean val ce loss: 0.351559, mean val dice : 0.677818
[04:19:32.642] epoch : 182, iteration : 3280, train loss : 0.242321, train loss_ce: 0.299513, train loss_dice: 0.185128, train dice : 0.814872
[04:20:10.056] epoch : 182, iteration : 3288, train loss : 0.256433, train loss_ce: 0.307120, train loss_dice: 0.205747, train dice : 0.794253
[04:20:36.108] epoch : 182, mean train loss : 0.309336, mean train ce loss: 0.279371, mean train dice : 0.660700
[04:20:49.653] epoch : 182, iteration : 1458, val loss : 0.360602, val loss_ce: 0.468979, val loss_dice: 0.252225, val dice : 0.747775
[04:21:01.917] epoch : 182, iteration : 1464, val loss : 0.344064, val loss_ce: 0.301206, val loss_dice: 0.386921, val dice : 0.613079
[04:21:02.096] epoch : 182, mean val loss : 0.318976, mean val ce loss: 0.327658, mean val dice : 0.689706
[04:21:26.116] epoch : 183, iteration : 3296, train loss : 0.290787, train loss_ce: 0.326563, train loss_dice: 0.255010, train dice : 0.744990
[04:22:03.542] epoch : 183, iteration : 3304, train loss : 0.290901, train loss_ce: 0.158241, train loss_dice: 0.423560, train dice : 0.576440
[04:22:38.955] epoch : 183, iteration : 3312, train loss : 0.380955, train loss_ce: 0.437177, train loss_dice: 0.324734, train dice : 0.675266
[04:22:39.139] epoch : 183, mean train loss : 0.304437, mean train ce loss: 0.315442, mean train dice : 0.706568
[04:22:56.503] epoch : 183, iteration : 1470, val loss : 0.282977, val loss_ce: 0.325936, val loss_dice: 0.240018, val dice : 0.759982
[04:23:11.664] epoch : 183, mean val loss : 0.280439, mean val ce loss: 0.286124, mean val dice : 0.725247
[04:24:03.456] epoch : 184, iteration : 3320, train loss : 0.606199, train loss_ce: 0.765048, train loss_dice: 0.447350, train dice : 0.552650
[04:24:40.853] epoch : 184, iteration : 3328, train loss : 0.259824, train loss_ce: 0.119289, train loss_dice: 0.400359, train dice : 0.599641
[04:24:48.284] epoch : 184, mean train loss : 0.335009, mean train ce loss: 0.348742, mean train dice : 0.678723
[04:25:12.271] epoch : 184, iteration : 1476, val loss : 0.286398, val loss_ce: 0.370707, val loss_dice: 0.202089, val dice : 0.797911
[04:25:16.419] epoch : 184, mean val loss : 0.281004, mean val ce loss: 0.307089, mean val dice : 0.745082
[04:26:02.335] epoch : 185, iteration : 3336, train loss : 0.184298, train loss_ce: 0.225360, train loss_dice: 0.143236, train dice : 0.856764
[04:26:39.738] epoch : 185, iteration : 3344, train loss : 0.217627, train loss_ce: 0.276235, train loss_dice: 0.159019, train dice : 0.840981
[04:26:56.513] epoch : 185, mean train loss : 0.254374, mean train ce loss: 0.265815, mean train dice : 0.757068
[04:27:07.927] epoch : 185, iteration : 1482, val loss : 0.151865, val loss_ce: 0.176909, val loss_dice: 0.126820, val dice : 0.873180
[04:27:17.820] epoch : 185, iteration : 1488, val loss : 0.167459, val loss_ce: 0.194171, val loss_dice: 0.140747, val dice : 0.859253
[04:27:17.947] epoch : 185, mean val loss : 0.263797, mean val ce loss: 0.246936, mean val dice : 0.719342
[04:27:53.186] epoch : 186, iteration : 3352, train loss : 0.286827, train loss_ce: 0.350986, train loss_dice: 0.222667, train dice : 0.777333
[04:28:30.564] epoch : 186, iteration : 3360, train loss : 0.196361, train loss_ce: 0.163679, train loss_dice: 0.229042, train dice : 0.770958
[04:28:56.677] epoch : 186, mean train loss : 0.313280, mean train ce loss: 0.313827, mean train dice : 0.687268
[04:29:21.821] epoch : 186, iteration : 1494, val loss : 0.295727, val loss_ce: 0.330502, val loss_dice: 0.260952, val dice : 0.739048
[04:29:28.434] epoch : 186, mean val loss : 0.282794, mean val ce loss: 0.283143, mean val dice : 0.717556
[04:29:54.122] epoch : 187, iteration : 3368, train loss : 0.415651, train loss_ce: 0.508249, train loss_dice: 0.323053, train dice : 0.676947
[04:30:31.506] epoch : 187, iteration : 3376, train loss : 0.316530, train loss_ce: 0.325585, train loss_dice: 0.307475, train dice : 0.692525
[04:31:06.757] epoch : 187, iteration : 3384, train loss : 0.432259, train loss_ce: 0.430686, train loss_dice: 0.433832, train dice : 0.566168
[04:31:06.897] epoch : 187, mean train loss : 0.283789, mean train ce loss: 0.274297, mean train dice : 0.706719
[04:31:23.055] epoch : 187, iteration : 1500, val loss : 0.249255, val loss_ce: 0.276508, val loss_dice: 0.222002, val dice : 0.777998
[04:31:32.363] epoch : 187, mean val loss : 0.240571, mean val ce loss: 0.221598, mean val dice : 0.740457
[04:32:21.376] epoch : 188, iteration : 3392, train loss : 0.335643, train loss_ce: 0.095605, train loss_dice: 0.575681, train dice : 0.424319
[04:32:58.802] epoch : 188, iteration : 3400, train loss : 0.127136, train loss_ce: 0.165817, train loss_dice: 0.088455, train dice : 0.911545
[04:33:06.183] epoch : 188, mean train loss : 0.302208, mean train ce loss: 0.256080, mean train dice : 0.651664
[04:33:20.240] epoch : 188, iteration : 1506, val loss : 0.261410, val loss_ce: 0.213067, val loss_dice: 0.309752, val dice : 0.690248
[04:33:30.905] epoch : 188, iteration : 1512, val loss : 0.180941, val loss_ce: 0.228488, val loss_dice: 0.133395, val dice : 0.866605
[04:33:31.015] epoch : 188, mean val loss : 0.280226, mean val ce loss: 0.286113, mean val dice : 0.725661
[04:34:10.680] epoch : 189, iteration : 3408, train loss : 0.202525, train loss_ce: 0.255172, train loss_dice: 0.149878, train dice : 0.850122
[04:34:48.098] epoch : 189, iteration : 3416, train loss : 0.179320, train loss_ce: 0.137479, train loss_dice: 0.221161, train dice : 0.778839
[04:35:05.098] epoch : 189, mean train loss : 0.250763, mean train ce loss: 0.241951, mean train dice : 0.740426
[04:35:27.023] epoch : 189, iteration : 1518, val loss : 0.223249, val loss_ce: 0.224727, val loss_dice: 0.221770, val dice : 0.778230
[04:35:35.941] epoch : 189, mean val loss : 0.266245, mean val ce loss: 0.242170, mean val dice : 0.709680
[04:36:04.027] epoch : 190, iteration : 3424, train loss : 0.229513, train loss_ce: 0.295235, train loss_dice: 0.163791, train dice : 0.836209
[04:36:41.410] epoch : 190, iteration : 3432, train loss : 0.243682, train loss_ce: 0.238466, train loss_dice: 0.248898, train dice : 0.751102
[04:37:07.571] epoch : 190, mean train loss : 0.252576, mean train ce loss: 0.244751, mean train dice : 0.739599
[04:37:24.789] epoch : 190, iteration : 1524, val loss : 0.192411, val loss_ce: 0.194696, val loss_dice: 0.190125, val dice : 0.809875
[04:37:39.380] epoch : 190, mean val loss : 0.252185, mean val ce loss: 0.230096, mean val dice : 0.725727
[04:38:04.564] epoch : 191, iteration : 3440, train loss : 0.199030, train loss_ce: 0.129122, train loss_dice: 0.268938, train dice : 0.731062
[04:38:41.959] epoch : 191, iteration : 3448, train loss : 0.161892, train loss_ce: 0.198203, train loss_dice: 0.125580, train dice : 0.874420
[04:39:17.198] epoch : 191, iteration : 3456, train loss : 0.434047, train loss_ce: 0.151666, train loss_dice: 0.716429, train dice : 0.283571
[04:39:17.316] epoch : 191, mean train loss : 0.247832, mean train ce loss: 0.203627, mean train dice : 0.707964
[04:39:29.529] epoch : 191, iteration : 1530, val loss : 0.135847, val loss_ce: 0.173114, val loss_dice: 0.098579, val dice : 0.901421
[04:39:44.022] epoch : 191, iteration : 1536, val loss : 0.277318, val loss_ce: 0.244423, val loss_dice: 0.310213, val dice : 0.689787
[04:39:44.166] epoch : 191, mean val loss : 0.250428, mean val ce loss: 0.235740, mean val dice : 0.734885
[04:40:37.137] epoch : 192, iteration : 3464, train loss : 0.211570, train loss_ce: 0.129076, train loss_dice: 0.294065, train dice : 0.705935
[04:41:14.535] epoch : 192, iteration : 3472, train loss : 0.184923, train loss_ce: 0.234146, train loss_dice: 0.135700, train dice : 0.864300
[04:41:22.019] epoch : 192, mean train loss : 0.251009, mean train ce loss: 0.244668, mean train dice : 0.742649
[04:41:41.349] epoch : 192, iteration : 1542, val loss : 0.338543, val loss_ce: 0.095427, val loss_dice: 0.581660, val dice : 0.418340
[04:41:49.696] epoch : 192, mean val loss : 0.270195, mean val ce loss: 0.241981, mean val dice : 0.701592
[04:42:37.731] epoch : 193, iteration : 3480, train loss : 0.280431, train loss_ce: 0.321508, train loss_dice: 0.239354, train dice : 0.760646
[04:43:15.114] epoch : 193, iteration : 3488, train loss : 0.163790, train loss_ce: 0.199416, train loss_dice: 0.128163, train dice : 0.871837
[04:43:31.841] epoch : 193, mean train loss : 0.253665, mean train ce loss: 0.251282, mean train dice : 0.743953
[04:43:57.216] epoch : 193, iteration : 1548, val loss : 0.224748, val loss_ce: 0.249337, val loss_dice: 0.200158, val dice : 0.799842
[04:44:10.884] epoch : 193, mean val loss : 0.222245, mean val ce loss: 0.183063, mean val dice : 0.738573
[04:44:46.820] epoch : 194, iteration : 3496, train loss : 0.270699, train loss_ce: 0.320259, train loss_dice: 0.221139, train dice : 0.778861
[04:45:24.170] epoch : 194, iteration : 3504, train loss : 0.320365, train loss_ce: 0.361491, train loss_dice: 0.279239, train dice : 0.720761
[04:45:50.260] epoch : 194, mean train loss : 0.215229, mean train ce loss: 0.190937, mean train dice : 0.760480
[04:46:03.758] epoch : 194, iteration : 1554, val loss : 0.243110, val loss_ce: 0.260314, val loss_dice: 0.225906, val dice : 0.774094
[04:46:18.395] epoch : 194, iteration : 1560, val loss : 0.252519, val loss_ce: 0.240588, val loss_dice: 0.264451, val dice : 0.735549
[04:46:18.570] epoch : 194, mean val loss : 0.217680, mean val ce loss: 0.221853, mean val dice : 0.786494
[04:46:48.609] epoch : 195, iteration : 3512, train loss : 0.280570, train loss_ce: 0.240247, train loss_dice: 0.320892, train dice : 0.679108
[04:47:25.980] epoch : 195, iteration : 3520, train loss : 0.245723, train loss_ce: 0.196511, train loss_dice: 0.294935, train dice : 0.705065
[04:48:01.204] epoch : 195, iteration : 3528, train loss : 0.262007, train loss_ce: 0.075581, train loss_dice: 0.448433, train dice : 0.551567
[04:48:01.484] epoch : 195, mean train loss : 0.228516, mean train ce loss: 0.199879, mean train dice : 0.742847
[04:48:20.399] epoch : 195, iteration : 1566, val loss : 0.217619, val loss_ce: 0.257462, val loss_dice: 0.177777, val dice : 0.822223
[04:48:27.194] epoch : 195, mean val loss : 0.224588, mean val ce loss: 0.197712, mean val dice : 0.748535
[04:49:16.736] epoch : 196, iteration : 3536, train loss : 0.107664, train loss_ce: 0.115215, train loss_dice: 0.100112, train dice : 0.899888
[04:49:54.111] epoch : 196, iteration : 3544, train loss : 0.403012, train loss_ce: 0.285415, train loss_dice: 0.520610, train dice : 0.479390
[04:50:01.644] epoch : 196, mean train loss : 0.238854, mean train ce loss: 0.227387, mean train dice : 0.749680
[04:50:14.843] epoch : 196, iteration : 1572, val loss : 0.196385, val loss_ce: 0.229319, val loss_dice: 0.163450, val dice : 0.836550
[04:50:26.464] epoch : 196, mean val loss : 0.215826, mean val ce loss: 0.205918, mean val dice : 0.774266
[04:51:13.572] epoch : 197, iteration : 3552, train loss : 0.216854, train loss_ce: 0.267588, train loss_dice: 0.166120, train dice : 0.833880
[04:51:50.974] epoch : 197, iteration : 3560, train loss : 0.202194, train loss_ce: 0.097795, train loss_dice: 0.306594, train dice : 0.693406
[04:52:07.761] epoch : 197, mean train loss : 0.274943, mean train ce loss: 0.228981, mean train dice : 0.679096
[04:52:28.506] epoch : 197, iteration : 1578, val loss : 0.113538, val loss_ce: 0.135199, val loss_dice: 0.091877, val dice : 0.908123
[04:52:36.984] epoch : 197, iteration : 1584, val loss : 0.166282, val loss_ce: 0.186646, val loss_dice: 0.145917, val dice : 0.854083
[04:52:37.199] epoch : 197, mean val loss : 0.230434, mean val ce loss: 0.220940, mean val dice : 0.760072
[04:53:08.299] epoch : 198, iteration : 3568, train loss : 0.163373, train loss_ce: 0.218181, train loss_dice: 0.108564, train dice : 0.891436
[04:53:45.677] epoch : 198, iteration : 3576, train loss : 0.320501, train loss_ce: 0.178758, train loss_dice: 0.462244, train dice : 0.537756
[04:54:11.837] epoch : 198, mean train loss : 0.226423, mean train ce loss: 0.208653, mean train dice : 0.755807
[04:54:32.218] epoch : 198, iteration : 1590, val loss : 0.184592, val loss_ce: 0.142739, val loss_dice: 0.226446, val dice : 0.773554
[04:54:36.750] epoch : 198, mean val loss : 0.235629, mean val ce loss: 0.205554, mean val dice : 0.734297
[04:55:03.743] epoch : 199, iteration : 3584, train loss : 0.252863, train loss_ce: 0.245326, train loss_dice: 0.260399, train dice : 0.739601
[04:55:41.114] epoch : 199, iteration : 3592, train loss : 0.089913, train loss_ce: 0.117646, train loss_dice: 0.062180, train dice : 0.937820
[04:56:16.334] epoch : 199, iteration : 3600, train loss : 0.446384, train loss_ce: 0.330350, train loss_dice: 0.562417, train dice : 0.437583
[04:56:16.528] epoch : 199, mean train loss : 0.225459, mean train ce loss: 0.209633, mean train dice : 0.758715
[04:56:33.386] epoch : 199, iteration : 1596, val loss : 0.131409, val loss_ce: 0.177499, val loss_dice: 0.085318, val dice : 0.914682
[04:56:50.655] epoch : 199, mean val loss : 0.240637, mean val ce loss: 0.187864, mean val dice : 0.706590
[04:57:37.432] epoch : 200, iteration : 3608, train loss : 0.225900, train loss_ce: 0.046925, train loss_dice: 0.404875, train dice : 0.595125
[04:58:14.822] epoch : 200, iteration : 3616, train loss : 0.207530, train loss_ce: 0.186878, train loss_dice: 0.228181, train dice : 0.771819
[04:58:22.503] epoch : 200, mean train loss : 0.273400, mean train ce loss: 0.256248, mean train dice : 0.709449
[04:58:39.725] epoch : 200, iteration : 1602, val loss : 0.134703, val loss_ce: 0.156257, val loss_dice: 0.113148, val dice : 0.886852
[04:58:49.375] epoch : 200, iteration : 1608, val loss : 0.282005, val loss_ce: 0.319692, val loss_dice: 0.244317, val dice : 0.755683
[04:58:49.476] epoch : 200, mean val loss : 0.252105, mean val ce loss: 0.263414, mean val dice : 0.759204
[04:59:35.625] epoch : 201, iteration : 3624, train loss : 0.265188, train loss_ce: 0.316651, train loss_dice: 0.213724, train dice : 0.786276
[05:00:12.991] epoch : 201, iteration : 3632, train loss : 0.229051, train loss_ce: 0.253883, train loss_dice: 0.204219, train dice : 0.795781
[05:00:29.768] epoch : 201, mean train loss : 0.253110, mean train ce loss: 0.258512, mean train dice : 0.752291
[05:00:48.733] epoch : 201, iteration : 1614, val loss : 0.301645, val loss_ce: 0.366886, val loss_dice: 0.236405, val dice : 0.763595
[05:00:55.689] epoch : 201, mean val loss : 0.335265, mean val ce loss: 0.360790, mean val dice : 0.690260
[05:01:26.740] epoch : 202, iteration : 3640, train loss : 0.255322, train loss_ce: 0.219913, train loss_dice: 0.290732, train dice : 0.709268
[05:02:04.118] epoch : 202, iteration : 3648, train loss : 0.296417, train loss_ce: 0.328439, train loss_dice: 0.264394, train dice : 0.735606
[05:02:30.298] epoch : 202, mean train loss : 0.260738, mean train ce loss: 0.245778, mean train dice : 0.724301
[05:02:56.178] epoch : 202, iteration : 1620, val loss : 0.272465, val loss_ce: 0.236140, val loss_dice: 0.308790, val dice : 0.691210
[05:03:03.413] epoch : 202, mean val loss : 0.352002, mean val ce loss: 0.391093, mean val dice : 0.687089
[05:03:20.348] epoch : 203, iteration : 3656, train loss : 0.259383, train loss_ce: 0.177885, train loss_dice: 0.340882, train dice : 0.659118
[05:03:59.484] epoch : 203, iteration : 3664, train loss : 0.187693, train loss_ce: 0.236428, train loss_dice: 0.138959, train dice : 0.861041
[05:04:34.770] epoch : 203, iteration : 3672, train loss : 0.315855, train loss_ce: 0.240895, train loss_dice: 0.390815, train dice : 0.609185
[05:04:34.893] epoch : 203, mean train loss : 0.257280, mean train ce loss: 0.242838, mean train dice : 0.728278
[05:04:49.511] epoch : 203, iteration : 1626, val loss : 0.323569, val loss_ce: 0.397913, val loss_dice: 0.249225, val dice : 0.750775
[05:04:58.609] epoch : 203, iteration : 1632, val loss : 0.400384, val loss_ce: 0.503740, val loss_dice: 0.297028, val dice : 0.702972
[05:04:58.732] epoch : 203, mean val loss : 0.292852, mean val ce loss: 0.288448, mean val dice : 0.702743
[05:05:54.414] epoch : 204, iteration : 3680, train loss : 0.289441, train loss_ce: 0.231775, train loss_dice: 0.347107, train dice : 0.652893
[05:06:31.808] epoch : 204, iteration : 3688, train loss : 0.339569, train loss_ce: 0.327576, train loss_dice: 0.351561, train dice : 0.648439
[05:06:39.223] epoch : 204, mean train loss : 0.309861, mean train ce loss: 0.311333, mean train dice : 0.691612
[05:06:57.035] epoch : 204, iteration : 1638, val loss : 0.675400, val loss_ce: 0.949924, val loss_dice: 0.400876, val dice : 0.599124
[05:07:06.331] epoch : 204, mean val loss : 0.428678, mean val ce loss: 0.524066, mean val dice : 0.666710
[05:07:52.241] epoch : 205, iteration : 3696, train loss : 0.232085, train loss_ce: 0.269324, train loss_dice: 0.194846, train dice : 0.805154
[05:08:29.614] epoch : 205, iteration : 3704, train loss : 0.654869, train loss_ce: 0.951733, train loss_dice: 0.358005, train dice : 0.641995
[05:08:46.305] epoch : 205, mean train loss : 0.406860, mean train ce loss: 0.435293, mean train dice : 0.621573
[05:09:06.565] epoch : 205, iteration : 1644, val loss : 0.365750, val loss_ce: 0.423801, val loss_dice: 0.307699, val dice : 0.692301
[05:09:18.473] epoch : 205, mean val loss : 0.399265, mean val ce loss: 0.388316, mean val dice : 0.589786
[05:09:45.708] epoch : 206, iteration : 3712, train loss : 0.406839, train loss_ce: 0.531851, train loss_dice: 0.281828, train dice : 0.718172
[05:10:23.117] epoch : 206, iteration : 3720, train loss : 0.291895, train loss_ce: 0.313277, train loss_dice: 0.270512, train dice : 0.729488
[05:10:49.210] epoch : 206, mean train loss : 0.390696, mean train ce loss: 0.412085, mean train dice : 0.630694
[05:11:02.285] epoch : 206, iteration : 1650, val loss : 0.270742, val loss_ce: 0.313569, val loss_dice: 0.227915, val dice : 0.772085
[05:11:18.094] epoch : 206, iteration : 1656, val loss : 0.291766, val loss_ce: 0.362268, val loss_dice: 0.221265, val dice : 0.778735
[05:11:18.279] epoch : 206, mean val loss : 0.305041, mean val ce loss: 0.306533, mean val dice : 0.696450
[05:11:49.863] epoch : 207, iteration : 3728, train loss : 0.251134, train loss_ce: 0.267421, train loss_dice: 0.234848, train dice : 0.765152
[05:12:27.246] epoch : 207, iteration : 3736, train loss : 0.265338, train loss_ce: 0.189167, train loss_dice: 0.341510, train dice : 0.658490
[05:13:02.499] epoch : 207, iteration : 3744, train loss : 0.543969, train loss_ce: 0.625876, train loss_dice: 0.462062, train dice : 0.537938
[05:13:02.638] epoch : 207, mean train loss : 0.378783, mean train ce loss: 0.395722, mean train dice : 0.638155
[05:13:22.624] epoch : 207, iteration : 1662, val loss : 0.359681, val loss_ce: 0.134859, val loss_dice: 0.584503, val dice : 0.415497
[05:13:29.371] epoch : 207, mean val loss : 0.333392, mean val ce loss: 0.280790, mean val dice : 0.614006
[05:14:15.733] epoch : 208, iteration : 3752, train loss : 0.306555, train loss_ce: 0.328369, train loss_dice: 0.284741, train dice : 0.715259
[05:14:53.109] epoch : 208, iteration : 3760, train loss : 0.244672, train loss_ce: 0.191555, train loss_dice: 0.297789, train dice : 0.702211
[05:15:00.462] epoch : 208, mean train loss : 0.324809, mean train ce loss: 0.345752, mean train dice : 0.696133
[05:15:16.958] epoch : 208, iteration : 1668, val loss : 0.299488, val loss_ce: 0.367629, val loss_dice: 0.231346, val dice : 0.768654
[05:15:26.498] epoch : 208, mean val loss : 0.327114, mean val ce loss: 0.339382, mean val dice : 0.685154
[05:16:06.914] epoch : 209, iteration : 3768, train loss : 0.170685, train loss_ce: 0.174190, train loss_dice: 0.167181, train dice : 0.832819
[05:16:44.312] epoch : 209, iteration : 3776, train loss : 0.421969, train loss_ce: 0.476515, train loss_dice: 0.367424, train dice : 0.632576
[05:17:01.056] epoch : 209, mean train loss : 0.280027, mean train ce loss: 0.277705, mean train dice : 0.717650
[05:17:12.676] epoch : 209, iteration : 1674, val loss : 0.258237, val loss_ce: 0.309413, val loss_dice: 0.207062, val dice : 0.792938
[05:17:25.657] epoch : 209, iteration : 1680, val loss : 0.368457, val loss_ce: 0.389671, val loss_dice: 0.347243, val dice : 0.652757
[05:17:25.891] epoch : 209, mean val loss : 0.280129, mean val ce loss: 0.289871, mean val dice : 0.729614
[05:17:58.796] epoch : 210, iteration : 3784, train loss : 0.295800, train loss_ce: 0.334379, train loss_dice: 0.257222, train dice : 0.742778
[05:18:36.197] epoch : 210, iteration : 3792, train loss : 0.246712, train loss_ce: 0.212622, train loss_dice: 0.280802, train dice : 0.719198
[05:19:02.357] epoch : 210, mean train loss : 0.260149, mean train ce loss: 0.232225, mean train dice : 0.711926
[05:19:21.426] epoch : 210, iteration : 1686, val loss : 0.237484, val loss_ce: 0.276998, val loss_dice: 0.197971, val dice : 0.802029
[05:19:28.974] epoch : 210, mean val loss : 0.274825, mean val ce loss: 0.280273, mean val dice : 0.730624
[05:19:48.630] epoch : 211, iteration : 3800, train loss : 0.301062, train loss_ce: 0.260237, train loss_dice: 0.341886, train dice : 0.658114
[05:20:25.983] epoch : 211, iteration : 3808, train loss : 0.212422, train loss_ce: 0.264484, train loss_dice: 0.160360, train dice : 0.839640
[05:21:01.222] epoch : 211, iteration : 3816, train loss : 0.382018, train loss_ce: 0.387700, train loss_dice: 0.376337, train dice : 0.623663
[05:21:01.394] epoch : 211, mean train loss : 0.260581, mean train ce loss: 0.260877, mean train dice : 0.739715
[05:21:16.392] epoch : 211, iteration : 1692, val loss : 0.227113, val loss_ce: 0.280434, val loss_dice: 0.173791, val dice : 0.826209
[05:21:23.835] epoch : 211, mean val loss : 0.229881, mean val ce loss: 0.204890, mean val dice : 0.745127
[05:22:27.023] epoch : 212, iteration : 3824, train loss : 0.375656, train loss_ce: 0.345278, train loss_dice: 0.406034, train dice : 0.593966
[05:23:04.404] epoch : 212, iteration : 3832, train loss : 0.208079, train loss_ce: 0.271358, train loss_dice: 0.144801, train dice : 0.855199
[05:23:11.862] epoch : 212, mean train loss : 0.262829, mean train ce loss: 0.239140, mean train dice : 0.713482
[05:23:33.567] epoch : 212, iteration : 1698, val loss : 0.206795, val loss_ce: 0.179324, val loss_dice: 0.234267, val dice : 0.765733
[05:23:51.303] epoch : 212, iteration : 1704, val loss : 0.149353, val loss_ce: 0.159083, val loss_dice: 0.139623, val dice : 0.860377
[05:23:51.401] epoch : 212, mean val loss : 0.212407, mean val ce loss: 0.178982, mean val dice : 0.754168
[05:24:35.314] epoch : 213, iteration : 3840, train loss : 0.193858, train loss_ce: 0.227981, train loss_dice: 0.159735, train dice : 0.840265
[05:25:12.696] epoch : 213, iteration : 3848, train loss : 0.344705, train loss_ce: 0.211023, train loss_dice: 0.478387, train dice : 0.521613
[05:25:29.542] epoch : 213, mean train loss : 0.247738, mean train ce loss: 0.253054, mean train dice : 0.757579
[05:25:56.956] epoch : 213, iteration : 1710, val loss : 0.403676, val loss_ce: 0.468814, val loss_dice: 0.338538, val dice : 0.661462
[05:26:08.557] epoch : 213, mean val loss : 0.341447, mean val ce loss: 0.328211, mean val dice : 0.645317
[05:26:37.134] epoch : 214, iteration : 3856, train loss : 0.202561, train loss_ce: 0.239460, train loss_dice: 0.165662, train dice : 0.834338
[05:27:14.525] epoch : 214, iteration : 3864, train loss : 0.319815, train loss_ce: 0.175830, train loss_dice: 0.463801, train dice : 0.536199
[05:27:40.658] epoch : 214, mean train loss : 0.249526, mean train ce loss: 0.255996, mean train dice : 0.756944
[05:28:01.404] epoch : 214, iteration : 1716, val loss : 0.211415, val loss_ce: 0.222135, val loss_dice: 0.200695, val dice : 0.799305
[05:28:08.226] epoch : 214, mean val loss : 0.263313, mean val ce loss: 0.237196, mean val dice : 0.710569
[05:28:35.438] epoch : 215, iteration : 3872, train loss : 0.181702, train loss_ce: 0.215187, train loss_dice: 0.148217, train dice : 0.851783
[05:29:12.908] epoch : 215, iteration : 3880, train loss : 0.264513, train loss_ce: 0.333977, train loss_dice: 0.195049, train dice : 0.804951
[05:29:48.152] epoch : 215, iteration : 3888, train loss : 0.241954, train loss_ce: 0.173869, train loss_dice: 0.310040, train dice : 0.689960
[05:29:48.272] epoch : 215, mean train loss : 0.222846, mean train ce loss: 0.227950, mean train dice : 0.782258
[05:30:06.285] epoch : 215, iteration : 1722, val loss : 0.218661, val loss_ce: 0.208102, val loss_dice: 0.229219, val dice : 0.770781
[05:30:16.165] epoch : 215, iteration : 1728, val loss : 0.290664, val loss_ce: 0.125536, val loss_dice: 0.455791, val dice : 0.544209
[05:30:16.290] epoch : 215, mean val loss : 0.256358, mean val ce loss: 0.247135, mean val dice : 0.734419
[05:31:11.064] epoch : 216, iteration : 3896, train loss : 0.185968, train loss_ce: 0.124036, train loss_dice: 0.247900, train dice : 0.752100
[05:31:48.474] epoch : 216, iteration : 3904, train loss : 0.117150, train loss_ce: 0.137954, train loss_dice: 0.096346, train dice : 0.903654
[05:31:55.956] epoch : 216, mean train loss : 0.228671, mean train ce loss: 0.201998, mean train dice : 0.744657
[05:32:16.005] epoch : 216, iteration : 1734, val loss : 0.329094, val loss_ce: 0.423400, val loss_dice: 0.234789, val dice : 0.765211
[05:32:26.818] epoch : 216, mean val loss : 0.255100, mean val ce loss: 0.234934, mean val dice : 0.724735
[05:33:06.086] epoch : 217, iteration : 3912, train loss : 0.250752, train loss_ce: 0.234484, train loss_dice: 0.267019, train dice : 0.732981
[05:33:43.475] epoch : 217, iteration : 3920, train loss : 0.206705, train loss_ce: 0.236066, train loss_dice: 0.177345, train dice : 0.822655
[05:34:00.189] epoch : 217, mean train loss : 0.243959, mean train ce loss: 0.205836, mean train dice : 0.717918
[05:34:20.708] epoch : 217, iteration : 1740, val loss : 0.100065, val loss_ce: 0.128101, val loss_dice: 0.072028, val dice : 0.927972
[05:34:28.661] epoch : 217, mean val loss : 0.214080, mean val ce loss: 0.201309, mean val dice : 0.773150
[05:35:03.774] epoch : 218, iteration : 3928, train loss : 0.195290, train loss_ce: 0.200488, train loss_dice: 0.190093, train dice : 0.809907
[05:35:41.162] epoch : 218, iteration : 3936, train loss : 0.201113, train loss_ce: 0.194949, train loss_dice: 0.207277, train dice : 0.792723
[05:36:07.248] epoch : 218, mean train loss : 0.222793, mean train ce loss: 0.206970, mean train dice : 0.761384
[05:36:18.578] epoch : 218, iteration : 1746, val loss : 0.168770, val loss_ce: 0.096535, val loss_dice: 0.241006, val dice : 0.758994
[05:36:31.301] epoch : 218, iteration : 1752, val loss : 0.186614, val loss_ce: 0.137824, val loss_dice: 0.235404, val dice : 0.764596
[05:36:31.428] epoch : 218, mean val loss : 0.201292, mean val ce loss: 0.186490, mean val dice : 0.783906
[05:36:51.623] epoch : 219, iteration : 3944, train loss : 0.231425, train loss_ce: 0.168672, train loss_dice: 0.294177, train dice : 0.705823
[05:37:33.053] epoch : 219, iteration : 3952, train loss : 0.175090, train loss_ce: 0.165457, train loss_dice: 0.184723, train dice : 0.815277
[05:38:08.310] epoch : 219, iteration : 3960, train loss : 0.321621, train loss_ce: 0.083666, train loss_dice: 0.559575, train dice : 0.440425
[05:38:08.491] epoch : 219, mean train loss : 0.213990, mean train ce loss: 0.190620, mean train dice : 0.762641
[05:38:28.612] epoch : 219, iteration : 1758, val loss : 0.291998, val loss_ce: 0.387365, val loss_dice: 0.196631, val dice : 0.803369
[05:38:33.620] epoch : 219, mean val loss : 0.232608, mean val ce loss: 0.225622, mean val dice : 0.760406
[05:39:25.080] epoch : 220, iteration : 3968, train loss : 0.197577, train loss_ce: 0.241478, train loss_dice: 0.153676, train dice : 0.846324
[05:40:02.446] epoch : 220, iteration : 3976, train loss : 0.159490, train loss_ce: 0.083454, train loss_dice: 0.235526, train dice : 0.764474
[05:40:09.893] epoch : 220, mean train loss : 0.221600, mean train ce loss: 0.197676, mean train dice : 0.754476
[05:40:25.821] epoch : 220, iteration : 1764, val loss : 0.297173, val loss_ce: 0.317345, val loss_dice: 0.277000, val dice : 0.723000
[05:40:33.120] epoch : 220, mean val loss : 0.222507, mean val ce loss: 0.195924, mean val dice : 0.750910
[05:41:20.107] epoch : 221, iteration : 3984, train loss : 0.207060, train loss_ce: 0.191946, train loss_dice: 0.222173, train dice : 0.777827
[05:41:57.497] epoch : 221, iteration : 3992, train loss : 0.220069, train loss_ce: 0.219678, train loss_dice: 0.220460, train dice : 0.779540
[05:42:14.240] epoch : 221, mean train loss : 0.189556, mean train ce loss: 0.172176, mean train dice : 0.793064
[05:42:26.795] epoch : 221, iteration : 1770, val loss : 0.250305, val loss_ce: 0.149693, val loss_dice: 0.350918, val dice : 0.649082
[05:42:40.549] epoch : 221, iteration : 1776, val loss : 0.345278, val loss_ce: 0.391264, val loss_dice: 0.299292, val dice : 0.700708
[05:42:40.819] epoch : 221, mean val loss : 0.251372, mean val ce loss: 0.165159, mean val dice : 0.662414
[05:43:16.751] epoch : 222, iteration : 4000, train loss : 0.075760, train loss_ce: 0.094220, train loss_dice: 0.057299, train dice : 0.942701
[05:43:54.137] epoch : 222, iteration : 4008, train loss : 0.223714, train loss_ce: 0.244973, train loss_dice: 0.202455, train dice : 0.797545
[05:44:20.495] epoch : 222, mean train loss : 0.185445, mean train ce loss: 0.170627, mean train dice : 0.799738
[05:44:35.099] epoch : 222, iteration : 1782, val loss : 0.430284, val loss_ce: 0.569212, val loss_dice: 0.291356, val dice : 0.708644
[05:44:52.396] epoch : 222, mean val loss : 0.227557, mean val ce loss: 0.226192, mean val dice : 0.771078
[05:45:18.106] epoch : 223, iteration : 4016, train loss : 0.157731, train loss_ce: 0.171140, train loss_dice: 0.144322, train dice : 0.855678
[05:45:55.511] epoch : 223, iteration : 4024, train loss : 0.148179, train loss_ce: 0.166150, train loss_dice: 0.130208, train dice : 0.869792
[05:46:30.766] epoch : 223, iteration : 4032, train loss : 0.158682, train loss_ce: 0.079827, train loss_dice: 0.237538, train dice : 0.762462
[05:46:31.077] epoch : 223, mean train loss : 0.198222, mean train ce loss: 0.177542, mean train dice : 0.781098
[05:46:52.486] epoch : 223, iteration : 1788, val loss : 0.212716, val loss_ce: 0.280412, val loss_dice: 0.145020, val dice : 0.854980
[05:47:02.609] epoch : 223, mean val loss : 0.190489, mean val ce loss: 0.190871, mean val dice : 0.809893
[05:47:02.836] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 224, val dice: 0.8098933100700378
[05:47:52.199] epoch : 224, iteration : 4040, train loss : 0.148936, train loss_ce: 0.063042, train loss_dice: 0.234829, train dice : 0.765171
[05:48:29.574] epoch : 224, iteration : 4048, train loss : 0.134979, train loss_ce: 0.171505, train loss_dice: 0.098454, train dice : 0.901546
[05:48:37.012] epoch : 224, mean train loss : 0.193798, mean train ce loss: 0.155191, mean train dice : 0.767595
[05:48:48.946] epoch : 224, iteration : 1794, val loss : 0.180711, val loss_ce: 0.124875, val loss_dice: 0.236547, val dice : 0.763453
[05:49:03.333] epoch : 224, iteration : 1800, val loss : 0.176860, val loss_ce: 0.140778, val loss_dice: 0.212942, val dice : 0.787058
[05:49:03.786] epoch : 224, mean val loss : 0.209549, mean val ce loss: 0.162423, mean val dice : 0.743326
[05:49:40.571] epoch : 225, iteration : 4056, train loss : 0.230524, train loss_ce: 0.219061, train loss_dice: 0.241987, train dice : 0.758013
[05:50:17.959] epoch : 225, iteration : 4064, train loss : 0.227893, train loss_ce: 0.294492, train loss_dice: 0.161295, train dice : 0.838705
[05:50:34.629] epoch : 225, mean train loss : 0.215079, mean train ce loss: 0.213020, mean train dice : 0.782862
[05:50:53.458] epoch : 225, iteration : 1806, val loss : 0.430427, val loss_ce: 0.506981, val loss_dice: 0.353873, val dice : 0.646127
[05:51:03.188] epoch : 225, mean val loss : 0.285542, mean val ce loss: 0.297885, mean val dice : 0.726801
[05:51:37.533] epoch : 226, iteration : 4072, train loss : 0.058529, train loss_ce: 0.049501, train loss_dice: 0.067557, train dice : 0.932443
[05:52:14.940] epoch : 226, iteration : 4080, train loss : 0.288739, train loss_ce: 0.058368, train loss_dice: 0.519110, train dice : 0.480890
[05:52:41.051] epoch : 226, mean train loss : 0.290976, mean train ce loss: 0.267852, mean train dice : 0.685899
[05:52:56.261] epoch : 226, iteration : 1812, val loss : 0.570641, val loss_ce: 0.773085, val loss_dice: 0.368197, val dice : 0.631803
[05:53:12.147] epoch : 226, mean val loss : 0.370942, mean val ce loss: 0.416550, mean val dice : 0.674666
[05:53:36.506] epoch : 227, iteration : 4088, train loss : 0.248159, train loss_ce: 0.205167, train loss_dice: 0.291151, train dice : 0.708849
[05:54:13.892] epoch : 227, iteration : 4096, train loss : 0.312869, train loss_ce: 0.311603, train loss_dice: 0.314135, train dice : 0.685865
[05:54:49.185] epoch : 227, iteration : 4104, train loss : 0.459373, train loss_ce: 0.321482, train loss_dice: 0.597265, train dice : 0.402735
[05:54:49.410] epoch : 227, mean train loss : 0.312670, mean train ce loss: 0.296296, mean train dice : 0.670956
[05:55:02.121] epoch : 227, iteration : 1818, val loss : 0.328153, val loss_ce: 0.415580, val loss_dice: 0.240726, val dice : 0.759274
[05:55:12.302] epoch : 227, iteration : 1824, val loss : 0.168133, val loss_ce: 0.095692, val loss_dice: 0.240574, val dice : 0.759426
[05:55:12.407] epoch : 227, mean val loss : 0.325906, mean val ce loss: 0.302022, mean val dice : 0.650210
[05:56:06.068] epoch : 228, iteration : 4112, train loss : 0.148819, train loss_ce: 0.166783, train loss_dice: 0.130855, train dice : 0.869145
[05:56:43.457] epoch : 228, iteration : 4120, train loss : 0.211257, train loss_ce: 0.224224, train loss_dice: 0.198291, train dice : 0.801709
[05:56:50.855] epoch : 228, mean train loss : 0.268584, mean train ce loss: 0.252579, mean train dice : 0.715411
[05:57:09.912] epoch : 228, iteration : 1830, val loss : 0.331343, val loss_ce: 0.124395, val loss_dice: 0.538291, val dice : 0.461709
[05:57:20.815] epoch : 228, mean val loss : 0.223273, mean val ce loss: 0.189646, mean val dice : 0.743100
[05:58:03.500] epoch : 229, iteration : 4128, train loss : 0.241706, train loss_ce: 0.252713, train loss_dice: 0.230699, train dice : 0.769301
[05:58:40.897] epoch : 229, iteration : 4136, train loss : 0.344781, train loss_ce: 0.376446, train loss_dice: 0.313116, train dice : 0.686885
[05:58:57.645] epoch : 229, mean train loss : 0.265874, mean train ce loss: 0.252935, mean train dice : 0.721186
[05:59:20.796] epoch : 229, iteration : 1836, val loss : 0.270603, val loss_ce: 0.230505, val loss_dice: 0.310701, val dice : 0.689299
[05:59:32.980] epoch : 229, mean val loss : 0.484775, mean val ce loss: 0.490561, mean val dice : 0.521011
[06:00:11.024] epoch : 230, iteration : 4144, train loss : 0.310850, train loss_ce: 0.337850, train loss_dice: 0.283850, train dice : 0.716150
[06:00:48.416] epoch : 230, iteration : 4152, train loss : 0.345778, train loss_ce: 0.213521, train loss_dice: 0.478035, train dice : 0.521965
[06:01:14.473] epoch : 230, mean train loss : 0.276853, mean train ce loss: 0.276837, mean train dice : 0.723131
[06:01:35.414] epoch : 230, iteration : 1842, val loss : 0.591314, val loss_ce: 0.809005, val loss_dice: 0.373623, val dice : 0.626377
[06:01:44.290] epoch : 230, iteration : 1848, val loss : 0.247259, val loss_ce: 0.174648, val loss_dice: 0.319870, val dice : 0.680130
[06:01:44.468] epoch : 230, mean val loss : 0.288597, mean val ce loss: 0.270419, mean val dice : 0.693225
[06:02:03.094] epoch : 231, iteration : 4160, train loss : 0.423812, train loss_ce: 0.396835, train loss_dice: 0.450790, train dice : 0.549210
[06:02:40.502] epoch : 231, iteration : 4168, train loss : 0.261826, train loss_ce: 0.123201, train loss_dice: 0.400451, train dice : 0.599549
[06:03:15.753] epoch : 231, iteration : 4176, train loss : 0.411724, train loss_ce: 0.357438, train loss_dice: 0.466010, train dice : 0.533990
[06:03:15.883] epoch : 231, mean train loss : 0.267052, mean train ce loss: 0.237810, mean train dice : 0.703705
[06:03:37.515] epoch : 231, iteration : 1854, val loss : 0.281299, val loss_ce: 0.313721, val loss_dice: 0.248877, val dice : 0.751123
[06:03:48.132] epoch : 231, mean val loss : 0.257156, mean val ce loss: 0.249255, mean val dice : 0.734942
[06:04:32.283] epoch : 232, iteration : 4184, train loss : 0.348499, train loss_ce: 0.304827, train loss_dice: 0.392171, train dice : 0.607828
[06:05:09.668] epoch : 232, iteration : 4192, train loss : 0.191661, train loss_ce: 0.260174, train loss_dice: 0.123147, train dice : 0.876853
[06:05:17.039] epoch : 232, mean train loss : 0.230611, mean train ce loss: 0.194096, mean train dice : 0.732873
[06:05:36.807] epoch : 232, iteration : 1860, val loss : 0.316565, val loss_ce: 0.308396, val loss_dice: 0.324735, val dice : 0.675265
[06:05:44.958] epoch : 232, mean val loss : 0.240470, mean val ce loss: 0.243289, mean val dice : 0.762349
[06:06:30.325] epoch : 233, iteration : 4200, train loss : 0.216485, train loss_ce: 0.182242, train loss_dice: 0.250729, train dice : 0.749271
[06:07:07.714] epoch : 233, iteration : 4208, train loss : 0.213893, train loss_ce: 0.187278, train loss_dice: 0.240508, train dice : 0.759492
[06:07:24.550] epoch : 233, mean train loss : 0.262535, mean train ce loss: 0.216455, mean train dice : 0.691386
[06:07:40.571] epoch : 233, iteration : 1866, val loss : 0.166567, val loss_ce: 0.155830, val loss_dice: 0.177303, val dice : 0.822697
[06:07:53.139] epoch : 233, iteration : 1872, val loss : 0.248169, val loss_ce: 0.243657, val loss_dice: 0.252682, val dice : 0.747318
[06:07:53.316] epoch : 233, mean val loss : 0.297089, mean val ce loss: 0.274143, mean val dice : 0.679964
[06:08:24.126] epoch : 234, iteration : 4216, train loss : 0.205772, train loss_ce: 0.265656, train loss_dice: 0.145887, train dice : 0.854113
[06:09:01.498] epoch : 234, iteration : 4224, train loss : 0.133387, train loss_ce: 0.142094, train loss_dice: 0.124679, train dice : 0.875321
[06:09:27.538] epoch : 234, mean train loss : 0.227470, mean train ce loss: 0.193417, mean train dice : 0.738476
[06:09:45.031] epoch : 234, iteration : 1878, val loss : 0.311657, val loss_ce: 0.376477, val loss_dice: 0.246837, val dice : 0.753163
[06:09:58.244] epoch : 234, mean val loss : 0.255482, mean val ce loss: 0.203837, mean val dice : 0.692872
[06:10:23.976] epoch : 235, iteration : 4232, train loss : 0.223247, train loss_ce: 0.137438, train loss_dice: 0.309057, train dice : 0.690943
[06:11:01.361] epoch : 235, iteration : 4240, train loss : 0.345516, train loss_ce: 0.424323, train loss_dice: 0.266710, train dice : 0.733290
[06:11:36.607] epoch : 235, iteration : 4248, train loss : 0.238962, train loss_ce: 0.067127, train loss_dice: 0.410798, train dice : 0.589202
[06:11:36.763] epoch : 235, mean train loss : 0.228345, mean train ce loss: 0.205399, mean train dice : 0.748708
[06:11:56.638] epoch : 235, iteration : 1884, val loss : 0.304336, val loss_ce: 0.311166, val loss_dice: 0.297506, val dice : 0.702494
[06:12:07.273] epoch : 235, mean val loss : 0.235512, mean val ce loss: 0.185654, mean val dice : 0.714630
[06:12:59.087] epoch : 236, iteration : 4256, train loss : 0.242561, train loss_ce: 0.263071, train loss_dice: 0.222051, train dice : 0.777949
[06:13:36.462] epoch : 236, iteration : 4264, train loss : 0.221112, train loss_ce: 0.280066, train loss_dice: 0.162158, train dice : 0.837842
[06:13:44.042] epoch : 236, mean train loss : 0.230664, mean train ce loss: 0.194870, mean train dice : 0.733542
[06:13:57.743] epoch : 236, iteration : 1890, val loss : 0.328189, val loss_ce: 0.365103, val loss_dice: 0.291276, val dice : 0.708724
[06:14:09.103] epoch : 236, iteration : 1896, val loss : 0.194830, val loss_ce: 0.079581, val loss_dice: 0.310079, val dice : 0.689921
[06:14:09.220] epoch : 236, mean val loss : 0.233627, mean val ce loss: 0.188003, mean val dice : 0.720749
[06:15:03.769] epoch : 237, iteration : 4272, train loss : 0.222521, train loss_ce: 0.256427, train loss_dice: 0.188616, train dice : 0.811384
[06:15:41.150] epoch : 237, iteration : 4280, train loss : 0.135095, train loss_ce: 0.151427, train loss_dice: 0.118763, train dice : 0.881237
[06:15:57.938] epoch : 237, mean train loss : 0.178284, mean train ce loss: 0.170512, mean train dice : 0.813944
[06:16:22.534] epoch : 237, iteration : 1902, val loss : 0.408110, val loss_ce: 0.480767, val loss_dice: 0.335453, val dice : 0.664547
[06:16:30.501] epoch : 237, mean val loss : 0.232892, mean val ce loss: 0.190067, mean val dice : 0.724282
[06:17:13.774] epoch : 238, iteration : 4288, train loss : 0.135364, train loss_ce: 0.121408, train loss_dice: 0.149320, train dice : 0.850680
[06:17:51.156] epoch : 238, iteration : 4296, train loss : 0.350222, train loss_ce: 0.363619, train loss_dice: 0.336825, train dice : 0.663175
[06:18:17.172] epoch : 238, mean train loss : 0.199934, mean train ce loss: 0.198919, mean train dice : 0.799051
[06:18:37.976] epoch : 238, iteration : 1908, val loss : 0.124034, val loss_ce: 0.165526, val loss_dice: 0.082543, val dice : 0.917457
[06:18:49.425] epoch : 238, mean val loss : 0.180745, mean val ce loss: 0.144186, mean val dice : 0.782697
[06:19:10.985] epoch : 239, iteration : 4304, train loss : 0.142253, train loss_ce: 0.168313, train loss_dice: 0.116194, train dice : 0.883806
[06:19:48.334] epoch : 239, iteration : 4312, train loss : 0.151816, train loss_ce: 0.204950, train loss_dice: 0.098682, train dice : 0.901318
[06:20:23.601] epoch : 239, iteration : 4320, train loss : 0.317130, train loss_ce: 0.311987, train loss_dice: 0.322273, train dice : 0.677727
[06:20:23.801] epoch : 239, mean train loss : 0.173566, mean train ce loss: 0.174511, mean train dice : 0.827378
[06:20:38.374] epoch : 239, iteration : 1914, val loss : 0.120603, val loss_ce: 0.155307, val loss_dice: 0.085900, val dice : 0.914100
[06:20:48.417] epoch : 239, iteration : 1920, val loss : 0.307581, val loss_ce: 0.328519, val loss_dice: 0.286643, val dice : 0.713357
[06:20:48.516] epoch : 239, mean val loss : 0.242356, mean val ce loss: 0.206327, mean val dice : 0.721614
[06:21:37.861] epoch : 240, iteration : 4328, train loss : 0.269112, train loss_ce: 0.277961, train loss_dice: 0.260264, train dice : 0.739736
[06:22:15.254] epoch : 240, iteration : 4336, train loss : 0.125240, train loss_ce: 0.156892, train loss_dice: 0.093588, train dice : 0.906412
[06:22:22.693] epoch : 240, mean train loss : 0.190315, mean train ce loss: 0.180606, mean train dice : 0.799977
[06:22:41.927] epoch : 240, iteration : 1926, val loss : 0.271508, val loss_ce: 0.329149, val loss_dice: 0.213867, val dice : 0.786133
[06:22:54.827] epoch : 240, mean val loss : 0.219822, mean val ce loss: 0.168693, mean val dice : 0.729050
[06:23:39.718] epoch : 241, iteration : 4344, train loss : 0.134872, train loss_ce: 0.178714, train loss_dice: 0.091031, train dice : 0.908969
[06:24:17.103] epoch : 241, iteration : 4352, train loss : 0.259895, train loss_ce: 0.319472, train loss_dice: 0.200318, train dice : 0.799682
[06:24:33.799] epoch : 241, mean train loss : 0.216527, mean train ce loss: 0.209333, mean train dice : 0.776279
[06:24:54.790] epoch : 241, iteration : 1932, val loss : 0.184332, val loss_ce: 0.143409, val loss_dice: 0.225254, val dice : 0.774746
[06:25:03.624] epoch : 241, mean val loss : 0.178258, mean val ce loss: 0.147865, mean val dice : 0.791349
[06:25:41.285] epoch : 242, iteration : 4360, train loss : 0.162571, train loss_ce: 0.200154, train loss_dice: 0.124988, train dice : 0.875012
[06:26:18.695] epoch : 242, iteration : 4368, train loss : 0.220826, train loss_ce: 0.228463, train loss_dice: 0.213189, train dice : 0.786811
[06:26:44.812] epoch : 242, mean train loss : 0.182959, mean train ce loss: 0.182079, mean train dice : 0.816161
[06:27:00.378] epoch : 242, iteration : 1938, val loss : 0.154628, val loss_ce: 0.082819, val loss_dice: 0.226437, val dice : 0.773562
[06:27:09.467] epoch : 242, iteration : 1944, val loss : 0.175088, val loss_ce: 0.222212, val loss_dice: 0.127963, val dice : 0.872037
[06:27:09.643] epoch : 242, mean val loss : 0.203144, mean val ce loss: 0.172214, mean val dice : 0.765927
[06:27:33.561] epoch : 243, iteration : 4376, train loss : 0.185494, train loss_ce: 0.206717, train loss_dice: 0.164271, train dice : 0.835729
[06:28:10.966] epoch : 243, iteration : 4384, train loss : 0.170790, train loss_ce: 0.180199, train loss_dice: 0.161381, train dice : 0.838619
[06:28:46.309] epoch : 243, iteration : 4392, train loss : 0.224338, train loss_ce: 0.178238, train loss_dice: 0.270437, train dice : 0.729563
[06:28:46.497] epoch : 243, mean train loss : 0.200956, mean train ce loss: 0.193916, mean train dice : 0.792004
[06:29:05.950] epoch : 243, iteration : 1950, val loss : 0.311522, val loss_ce: 0.065823, val loss_dice: 0.557222, val dice : 0.442778
[06:29:10.842] epoch : 243, mean val loss : 0.186839, mean val ce loss: 0.116865, mean val dice : 0.743187
[06:30:03.687] epoch : 244, iteration : 4400, train loss : 0.309386, train loss_ce: 0.045702, train loss_dice: 0.573071, train dice : 0.426929
[06:30:41.084] epoch : 244, iteration : 4408, train loss : 0.176766, train loss_ce: 0.192844, train loss_dice: 0.160687, train dice : 0.839313
[06:30:48.603] epoch : 244, mean train loss : 0.178852, mean train ce loss: 0.152252, mean train dice : 0.794547
[06:31:06.705] epoch : 244, iteration : 1956, val loss : 0.417293, val loss_ce: 0.543745, val loss_dice: 0.290840, val dice : 0.709160
[06:31:18.211] epoch : 244, mean val loss : 0.208753, mean val ce loss: 0.187799, mean val dice : 0.770292
[06:32:04.302] epoch : 245, iteration : 4416, train loss : 0.074717, train loss_ce: 0.102619, train loss_dice: 0.046815, train dice : 0.953185
[06:32:41.680] epoch : 245, iteration : 4424, train loss : 0.207879, train loss_ce: 0.159986, train loss_dice: 0.255773, train dice : 0.744227
[06:32:58.414] epoch : 245, mean train loss : 0.189010, mean train ce loss: 0.160628, mean train dice : 0.782607
[06:33:13.789] epoch : 245, iteration : 1962, val loss : 0.114457, val loss_ce: 0.140232, val loss_dice: 0.088681, val dice : 0.911319
[06:33:24.811] epoch : 245, iteration : 1968, val loss : 0.208047, val loss_ce: 0.138180, val loss_dice: 0.277915, val dice : 0.722085
[06:33:24.936] epoch : 245, mean val loss : 0.186219, mean val ce loss: 0.180287, mean val dice : 0.807848
[06:33:55.855] epoch : 246, iteration : 4432, train loss : 0.186905, train loss_ce: 0.210444, train loss_dice: 0.163366, train dice : 0.836634
[06:34:33.227] epoch : 246, iteration : 4440, train loss : 0.151801, train loss_ce: 0.072215, train loss_dice: 0.231387, train dice : 0.768613
[06:34:59.496] epoch : 246, mean train loss : 0.179499, mean train ce loss: 0.137454, mean train dice : 0.778456
[06:35:17.528] epoch : 246, iteration : 1974, val loss : 0.164533, val loss_ce: 0.116532, val loss_dice: 0.212534, val dice : 0.787466
[06:35:23.131] epoch : 246, mean val loss : 0.153358, mean val ce loss: 0.148836, mean val dice : 0.842120
[06:35:23.387] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 247, val dice: 0.8421195149421692
[06:35:46.319] epoch : 247, iteration : 4448, train loss : 0.199321, train loss_ce: 0.201255, train loss_dice: 0.197386, train dice : 0.802614
[06:36:23.688] epoch : 247, iteration : 4456, train loss : 0.144536, train loss_ce: 0.063547, train loss_dice: 0.225524, train dice : 0.774476
[06:36:58.965] epoch : 247, iteration : 4464, train loss : 0.243608, train loss_ce: 0.076578, train loss_dice: 0.410637, train dice : 0.589363
[06:36:59.082] epoch : 247, mean train loss : 0.193794, mean train ce loss: 0.143003, mean train dice : 0.755415
[06:37:17.832] epoch : 247, iteration : 1980, val loss : 0.198350, val loss_ce: 0.175255, val loss_dice: 0.221445, val dice : 0.778555
[06:37:20.222] epoch : 247, mean val loss : 0.188032, mean val ce loss: 0.142583, mean val dice : 0.766520
[06:38:15.084] epoch : 248, iteration : 4472, train loss : 0.149277, train loss_ce: 0.180459, train loss_dice: 0.118094, train dice : 0.881906
[06:38:52.482] epoch : 248, iteration : 4480, train loss : 0.195964, train loss_ce: 0.259624, train loss_dice: 0.132305, train dice : 0.867695
[06:38:59.810] epoch : 248, mean train loss : 0.187113, mean train ce loss: 0.162437, mean train dice : 0.788211
[06:39:18.289] epoch : 248, iteration : 1986, val loss : 0.371087, val loss_ce: 0.419138, val loss_dice: 0.323036, val dice : 0.676964
[06:39:28.597] epoch : 248, iteration : 1992, val loss : 0.147858, val loss_ce: 0.041866, val loss_dice: 0.253849, val dice : 0.746151
[06:39:28.885] epoch : 248, mean val loss : 0.194185, mean val ce loss: 0.196000, mean val dice : 0.807630
[06:40:07.177] epoch : 249, iteration : 4488, train loss : 0.160748, train loss_ce: 0.151880, train loss_dice: 0.169617, train dice : 0.830383
[06:40:44.563] epoch : 249, iteration : 4496, train loss : 0.301498, train loss_ce: 0.335347, train loss_dice: 0.267649, train dice : 0.732351
[06:41:01.400] epoch : 249, mean train loss : 0.199122, mean train ce loss: 0.168645, mean train dice : 0.770401
[06:41:20.499] epoch : 249, iteration : 1998, val loss : 0.295880, val loss_ce: 0.396041, val loss_dice: 0.195719, val dice : 0.804281
[06:41:30.770] epoch : 249, mean val loss : 0.207261, mean val ce loss: 0.213455, mean val dice : 0.798933
[06:42:10.467] epoch : 250, iteration : 4504, train loss : 0.199560, train loss_ce: 0.211448, train loss_dice: 0.187673, train dice : 0.812327
[06:42:47.864] epoch : 250, iteration : 4512, train loss : 0.250930, train loss_ce: 0.074235, train loss_dice: 0.427626, train dice : 0.572374
[06:43:13.956] epoch : 250, mean train loss : 0.242970, mean train ce loss: 0.192873, mean train dice : 0.706933
[06:43:37.283] epoch : 250, iteration : 2004, val loss : 0.553574, val loss_ce: 0.630958, val loss_dice: 0.476189, val dice : 0.523811
[06:43:44.824] epoch : 250, mean val loss : 0.363609, mean val ce loss: 0.393850, mean val dice : 0.666632
[06:44:03.441] epoch : 251, iteration : 4520, train loss : 0.533342, train loss_ce: 0.649131, train loss_dice: 0.417554, train dice : 0.582446
[06:44:40.805] epoch : 251, iteration : 4528, train loss : 0.224511, train loss_ce: 0.110165, train loss_dice: 0.338857, train dice : 0.661143
[06:45:16.057] epoch : 251, iteration : 4536, train loss : 0.208357, train loss_ce: 0.166103, train loss_dice: 0.250611, train dice : 0.749389
[06:45:16.296] epoch : 251, mean train loss : 0.267488, mean train ce loss: 0.263678, mean train dice : 0.728702
[06:45:34.282] epoch : 251, iteration : 2010, val loss : 0.251195, val loss_ce: 0.311159, val loss_dice: 0.191230, val dice : 0.808770
[06:45:43.347] epoch : 251, iteration : 2016, val loss : 0.150377, val loss_ce: 0.183877, val loss_dice: 0.116878, val dice : 0.883122
[06:45:43.540] epoch : 251, mean val loss : 0.299883, mean val ce loss: 0.340334, mean val dice : 0.740568
[06:46:28.179] epoch : 252, iteration : 4544, train loss : 0.281433, train loss_ce: 0.092006, train loss_dice: 0.470859, train dice : 0.529141
[06:47:05.563] epoch : 252, iteration : 4552, train loss : 0.183941, train loss_ce: 0.226026, train loss_dice: 0.141855, train dice : 0.858145
[06:47:13.149] epoch : 252, mean train loss : 0.238858, mean train ce loss: 0.209208, mean train dice : 0.731492
[06:47:31.969] epoch : 252, iteration : 2022, val loss : 0.284277, val loss_ce: 0.290542, val loss_dice: 0.278012, val dice : 0.721988
[06:47:38.980] epoch : 252, mean val loss : 0.211668, mean val ce loss: 0.189224, mean val dice : 0.765887
[06:48:16.049] epoch : 253, iteration : 4560, train loss : 0.238568, train loss_ce: 0.241110, train loss_dice: 0.236027, train dice : 0.763973
[06:48:53.419] epoch : 253, iteration : 4568, train loss : 0.271238, train loss_ce: 0.312241, train loss_dice: 0.230235, train dice : 0.769765
[06:49:10.104] epoch : 253, mean train loss : 0.221102, mean train ce loss: 0.212004, mean train dice : 0.769800
[06:49:33.350] epoch : 253, iteration : 2028, val loss : 0.228246, val loss_ce: 0.222835, val loss_dice: 0.233658, val dice : 0.766342
[06:49:42.543] epoch : 253, mean val loss : 0.204179, mean val ce loss: 0.212340, mean val dice : 0.803982
[06:50:18.088] epoch : 254, iteration : 4576, train loss : 0.116716, train loss_ce: 0.142082, train loss_dice: 0.091351, train dice : 0.908649
[06:50:55.466] epoch : 254, iteration : 4584, train loss : 0.273052, train loss_ce: 0.086117, train loss_dice: 0.459986, train dice : 0.540014
[06:51:21.473] epoch : 254, mean train loss : 0.214758, mean train ce loss: 0.190185, mean train dice : 0.760670
[06:51:37.604] epoch : 254, iteration : 2034, val loss : 0.121048, val loss_ce: 0.167302, val loss_dice: 0.074794, val dice : 0.925206
[06:51:50.459] epoch : 254, iteration : 2040, val loss : 0.198908, val loss_ce: 0.275511, val loss_dice: 0.122305, val dice : 0.877695
[06:51:50.817] epoch : 254, mean val loss : 0.201719, mean val ce loss: 0.179777, mean val dice : 0.776340
[06:52:13.327] epoch : 255, iteration : 4592, train loss : 0.126607, train loss_ce: 0.142331, train loss_dice: 0.110883, train dice : 0.889117
[06:52:50.708] epoch : 255, iteration : 4600, train loss : 0.259429, train loss_ce: 0.341411, train loss_dice: 0.177446, train dice : 0.822554
[06:53:25.951] epoch : 255, iteration : 4608, train loss : 0.240309, train loss_ce: 0.130670, train loss_dice: 0.349947, train dice : 0.650053
[06:53:26.151] epoch : 255, mean train loss : 0.219545, mean train ce loss: 0.187906, mean train dice : 0.748816
[06:53:43.360] epoch : 255, iteration : 2046, val loss : 0.174503, val loss_ce: 0.167248, val loss_dice: 0.181758, val dice : 0.818242
[06:53:58.843] epoch : 255, mean val loss : 0.222630, mean val ce loss: 0.196188, mean val dice : 0.750929
[06:54:43.250] epoch : 256, iteration : 4616, train loss : 0.258596, train loss_ce: 0.178349, train loss_dice: 0.338844, train dice : 0.661156
[06:55:20.642] epoch : 256, iteration : 4624, train loss : 0.233131, train loss_ce: 0.112025, train loss_dice: 0.354237, train dice : 0.645763
[06:55:28.096] epoch : 256, mean train loss : 0.205796, mean train ce loss: 0.180562, mean train dice : 0.768970
[06:55:55.470] epoch : 256, iteration : 2052, val loss : 0.195723, val loss_ce: 0.213383, val loss_dice: 0.178064, val dice : 0.821936
[06:56:05.605] epoch : 256, mean val loss : 0.286737, mean val ce loss: 0.268488, mean val dice : 0.695014
[06:56:44.031] epoch : 257, iteration : 4632, train loss : 0.258308, train loss_ce: 0.358904, train loss_dice: 0.157713, train dice : 0.842287
[06:57:21.421] epoch : 257, iteration : 4640, train loss : 0.176219, train loss_ce: 0.108799, train loss_dice: 0.243639, train dice : 0.756361
[06:57:38.140] epoch : 257, mean train loss : 0.230125, mean train ce loss: 0.236174, mean train dice : 0.775924
[06:57:52.894] epoch : 257, iteration : 2058, val loss : 0.430793, val loss_ce: 0.517301, val loss_dice: 0.344284, val dice : 0.655716
[06:58:07.420] epoch : 257, iteration : 2064, val loss : 0.443148, val loss_ce: 0.595066, val loss_dice: 0.291230, val dice : 0.708770
[06:58:07.689] epoch : 257, mean val loss : 0.303694, mean val ce loss: 0.283496, mean val dice : 0.676108
[06:58:40.015] epoch : 258, iteration : 4648, train loss : 0.246107, train loss_ce: 0.262460, train loss_dice: 0.229753, train dice : 0.770247
[06:59:17.431] epoch : 258, iteration : 4656, train loss : 0.108271, train loss_ce: 0.131606, train loss_dice: 0.084936, train dice : 0.915065
[06:59:43.596] epoch : 258, mean train loss : 0.236093, mean train ce loss: 0.209810, mean train dice : 0.737624
[07:00:04.474] epoch : 258, iteration : 2070, val loss : 0.286367, val loss_ce: 0.336852, val loss_dice: 0.235882, val dice : 0.764118
[07:00:15.689] epoch : 258, mean val loss : 0.237547, mean val ce loss: 0.219823, mean val dice : 0.744730
[07:00:32.284] epoch : 259, iteration : 4664, train loss : 0.203200, train loss_ce: 0.114436, train loss_dice: 0.291965, train dice : 0.708035
[07:01:10.139] epoch : 259, iteration : 4672, train loss : 0.199254, train loss_ce: 0.114975, train loss_dice: 0.283532, train dice : 0.716468
[07:01:45.390] epoch : 259, iteration : 4680, train loss : 0.285721, train loss_ce: 0.271554, train loss_dice: 0.299887, train dice : 0.700113
[07:01:45.584] epoch : 259, mean train loss : 0.205018, mean train ce loss: 0.187244, mean train dice : 0.777207
[07:02:01.052] epoch : 259, iteration : 2076, val loss : 0.177316, val loss_ce: 0.240699, val loss_dice: 0.113932, val dice : 0.886068
[07:02:14.286] epoch : 259, mean val loss : 0.215912, mean val ce loss: 0.186173, mean val dice : 0.754349
[07:03:14.172] epoch : 260, iteration : 4688, train loss : 0.080872, train loss_ce: 0.090014, train loss_dice: 0.071730, train dice : 0.928270
[07:03:51.547] epoch : 260, iteration : 4696, train loss : 0.110371, train loss_ce: 0.128712, train loss_dice: 0.092031, train dice : 0.907969
[07:03:59.000] epoch : 260, mean train loss : 0.213856, mean train ce loss: 0.192553, mean train dice : 0.764841
[07:04:14.579] epoch : 260, iteration : 2082, val loss : 0.492676, val loss_ce: 0.603504, val loss_dice: 0.381849, val dice : 0.618151
[07:04:31.357] epoch : 260, iteration : 2088, val loss : 0.164994, val loss_ce: 0.096849, val loss_dice: 0.233139, val dice : 0.766861
[07:04:31.535] epoch : 260, mean val loss : 0.213815, mean val ce loss: 0.224223, mean val dice : 0.796593
[07:05:10.598] epoch : 261, iteration : 4704, train loss : 0.144096, train loss_ce: 0.175746, train loss_dice: 0.112447, train dice : 0.887553
[07:05:47.911] epoch : 261, iteration : 4712, train loss : 0.144989, train loss_ce: 0.132536, train loss_dice: 0.157442, train dice : 0.842558
[07:06:04.627] epoch : 261, mean train loss : 0.222655, mean train ce loss: 0.202064, mean train dice : 0.756755
[07:06:22.853] epoch : 261, iteration : 2094, val loss : 0.354582, val loss_ce: 0.415089, val loss_dice: 0.294075, val dice : 0.705925
[07:06:31.731] epoch : 261, mean val loss : 0.250786, mean val ce loss: 0.209026, mean val dice : 0.707453
[07:07:01.149] epoch : 262, iteration : 4720, train loss : 0.404015, train loss_ce: 0.460406, train loss_dice: 0.347624, train dice : 0.652376
[07:07:38.424] epoch : 262, iteration : 4728, train loss : 0.124050, train loss_ce: 0.147834, train loss_dice: 0.100265, train dice : 0.899735
[07:08:04.523] epoch : 262, mean train loss : 0.233041, mean train ce loss: 0.231045, mean train dice : 0.764963
[07:08:21.461] epoch : 262, iteration : 2100, val loss : 0.172680, val loss_ce: 0.114911, val loss_dice: 0.230450, val dice : 0.769550
[07:08:30.231] epoch : 262, mean val loss : 0.229927, mean val ce loss: 0.167628, mean val dice : 0.707774
[07:08:49.007] epoch : 263, iteration : 4736, train loss : 0.118526, train loss_ce: 0.129492, train loss_dice: 0.107559, train dice : 0.892441
[07:09:26.257] epoch : 263, iteration : 4744, train loss : 0.271533, train loss_ce: 0.138607, train loss_dice: 0.404459, train dice : 0.595541
[07:10:01.367] epoch : 263, iteration : 4752, train loss : 0.166775, train loss_ce: 0.093459, train loss_dice: 0.240091, train dice : 0.759909
[07:10:01.498] epoch : 263, mean train loss : 0.194534, mean train ce loss: 0.176605, mean train dice : 0.787537
[07:10:22.230] epoch : 263, iteration : 2106, val loss : 0.143818, val loss_ce: 0.173036, val loss_dice: 0.114600, val dice : 0.885400
[07:10:30.965] epoch : 263, iteration : 2112, val loss : 0.179404, val loss_ce: 0.209635, val loss_dice: 0.149172, val dice : 0.850828
[07:10:31.150] epoch : 263, mean val loss : 0.246754, mean val ce loss: 0.246258, mean val dice : 0.752750
[07:11:31.283] epoch : 264, iteration : 4760, train loss : 0.228798, train loss_ce: 0.186674, train loss_dice: 0.270922, train dice : 0.729078
[07:12:08.511] epoch : 264, iteration : 4768, train loss : 0.250012, train loss_ce: 0.078525, train loss_dice: 0.421499, train dice : 0.578501
[07:12:15.837] epoch : 264, mean train loss : 0.200909, mean train ce loss: 0.173949, mean train dice : 0.772130
[07:12:38.743] epoch : 264, iteration : 2118, val loss : 0.168910, val loss_ce: 0.164547, val loss_dice: 0.173272, val dice : 0.826728
[07:12:46.287] epoch : 264, mean val loss : 0.167222, mean val ce loss: 0.142922, mean val dice : 0.808477
[07:13:37.595] epoch : 265, iteration : 4776, train loss : 0.165022, train loss_ce: 0.221127, train loss_dice: 0.108916, train dice : 0.891084
[07:14:14.806] epoch : 265, iteration : 4784, train loss : 0.157198, train loss_ce: 0.164193, train loss_dice: 0.150204, train dice : 0.849796
[07:14:31.563] epoch : 265, mean train loss : 0.184918, mean train ce loss: 0.155308, mean train dice : 0.785472
[07:14:52.551] epoch : 265, iteration : 2124, val loss : 0.261394, val loss_ce: 0.296605, val loss_dice: 0.226183, val dice : 0.773817
[07:15:05.999] epoch : 265, mean val loss : 0.241972, mean val ce loss: 0.211493, mean val dice : 0.727550
[07:15:47.608] epoch : 266, iteration : 4792, train loss : 0.177493, train loss_ce: 0.097105, train loss_dice: 0.257881, train dice : 0.742119
[07:16:24.844] epoch : 266, iteration : 4800, train loss : 0.145030, train loss_ce: 0.073111, train loss_dice: 0.216948, train dice : 0.783052
[07:16:50.923] epoch : 266, mean train loss : 0.195226, mean train ce loss: 0.164037, mean train dice : 0.773585
[07:17:06.355] epoch : 266, iteration : 2130, val loss : 0.070276, val loss_ce: 0.101194, val loss_dice: 0.039357, val dice : 0.960643
[07:17:18.531] epoch : 266, iteration : 2136, val loss : 0.083523, val loss_ce: 0.107634, val loss_dice: 0.059412, val dice : 0.940588
[07:17:18.838] epoch : 266, mean val loss : 0.139003, mean val ce loss: 0.117121, mean val dice : 0.839115
[07:17:45.089] epoch : 267, iteration : 4808, train loss : 0.300776, train loss_ce: 0.205042, train loss_dice: 0.396511, train dice : 0.603489
[07:18:22.316] epoch : 267, iteration : 4816, train loss : 0.077286, train loss_ce: 0.051565, train loss_dice: 0.103008, train dice : 0.896992
[07:18:57.425] epoch : 267, iteration : 4824, train loss : 0.567069, train loss_ce: 0.215216, train loss_dice: 0.918922, train dice : 0.081078
[07:18:57.556] epoch : 267, mean train loss : 0.180451, mean train ce loss: 0.157772, mean train dice : 0.796870
[07:19:19.829] epoch : 267, iteration : 2142, val loss : 0.194592, val loss_ce: 0.245201, val loss_dice: 0.143982, val dice : 0.856018
[07:19:21.926] epoch : 267, mean val loss : 0.216490, mean val ce loss: 0.171115, mean val dice : 0.738136
[07:20:15.007] epoch : 268, iteration : 4832, train loss : 0.149433, train loss_ce: 0.191668, train loss_dice: 0.107199, train dice : 0.892801
[07:20:52.247] epoch : 268, iteration : 4840, train loss : 0.282188, train loss_ce: 0.307610, train loss_dice: 0.256766, train dice : 0.743234
[07:20:59.781] epoch : 268, mean train loss : 0.157449, mean train ce loss: 0.140046, mean train dice : 0.825148
[07:21:21.886] epoch : 268, iteration : 2148, val loss : 0.218028, val loss_ce: 0.245675, val loss_dice: 0.190380, val dice : 0.809620
[07:21:30.234] epoch : 268, mean val loss : 0.214947, mean val ce loss: 0.208310, mean val dice : 0.778416
[07:22:09.892] epoch : 269, iteration : 4848, train loss : 0.203940, train loss_ce: 0.247129, train loss_dice: 0.160751, train dice : 0.839249
[07:22:47.122] epoch : 269, iteration : 4856, train loss : 0.151859, train loss_ce: 0.169833, train loss_dice: 0.133885, train dice : 0.866115
[07:23:03.770] epoch : 269, mean train loss : 0.188111, mean train ce loss: 0.182331, mean train dice : 0.806110
[07:23:17.659] epoch : 269, iteration : 2154, val loss : 0.122361, val loss_ce: 0.160030, val loss_dice: 0.084693, val dice : 0.915307
[07:23:27.167] epoch : 269, iteration : 2160, val loss : 0.149226, val loss_ce: 0.039006, val loss_dice: 0.259445, val dice : 0.740555
[07:23:27.377] epoch : 269, mean val loss : 0.166991, mean val ce loss: 0.144487, mean val dice : 0.810504
[07:24:05.483] epoch : 270, iteration : 4864, train loss : 0.149128, train loss_ce: 0.193821, train loss_dice: 0.104436, train dice : 0.895564
[07:24:42.718] epoch : 270, iteration : 4872, train loss : 0.188091, train loss_ce: 0.200415, train loss_dice: 0.175767, train dice : 0.824233
[07:25:08.710] epoch : 270, mean train loss : 0.140253, mean train ce loss: 0.128239, mean train dice : 0.847733
[07:25:31.469] epoch : 270, iteration : 2166, val loss : 0.087670, val loss_ce: 0.111398, val loss_dice: 0.063941, val dice : 0.936059
[07:25:32.633] epoch : 270, mean val loss : 0.135570, mean val ce loss: 0.096932, mean val dice : 0.825793
[07:25:58.517] epoch : 271, iteration : 4880, train loss : 0.117457, train loss_ce: 0.151140, train loss_dice: 0.083774, train dice : 0.916226
[07:26:35.746] epoch : 271, iteration : 4888, train loss : 0.185041, train loss_ce: 0.129313, train loss_dice: 0.240769, train dice : 0.759231
[07:27:10.861] epoch : 271, iteration : 4896, train loss : 0.146634, train loss_ce: 0.064912, train loss_dice: 0.228356, train dice : 0.771644
[07:27:11.057] epoch : 271, mean train loss : 0.159404, mean train ce loss: 0.135931, mean train dice : 0.817122
[07:27:24.663] epoch : 271, iteration : 2172, val loss : 0.209000, val loss_ce: 0.226502, val loss_dice: 0.191498, val dice : 0.808502
[07:27:36.495] epoch : 271, mean val loss : 0.189823, mean val ce loss: 0.178771, mean val dice : 0.799125
[07:28:28.283] epoch : 272, iteration : 4904, train loss : 0.290445, train loss_ce: 0.319465, train loss_dice: 0.261425, train dice : 0.738575
[07:29:05.530] epoch : 272, iteration : 4912, train loss : 0.163086, train loss_ce: 0.061288, train loss_dice: 0.264884, train dice : 0.735116
[07:29:12.873] epoch : 272, mean train loss : 0.167575, mean train ce loss: 0.124631, mean train dice : 0.789481
[07:29:37.215] epoch : 272, iteration : 2178, val loss : 0.164152, val loss_ce: 0.143172, val loss_dice: 0.185132, val dice : 0.814868
[07:29:46.174] epoch : 272, iteration : 2184, val loss : 0.129528, val loss_ce: 0.159363, val loss_dice: 0.099693, val dice : 0.900307
[07:29:46.389] epoch : 272, mean val loss : 0.178641, mean val ce loss: 0.131293, mean val dice : 0.774012
[07:30:29.033] epoch : 273, iteration : 4920, train loss : 0.154539, train loss_ce: 0.199451, train loss_dice: 0.109628, train dice : 0.890372
[07:31:06.254] epoch : 273, iteration : 4928, train loss : 0.146282, train loss_ce: 0.089920, train loss_dice: 0.202645, train dice : 0.797355
[07:31:22.940] epoch : 273, mean train loss : 0.171504, mean train ce loss: 0.127089, mean train dice : 0.784081
[07:31:48.396] epoch : 273, iteration : 2190, val loss : 0.156078, val loss_ce: 0.188281, val loss_dice: 0.123874, val dice : 0.876126
[07:31:49.606] epoch : 273, mean val loss : 0.164547, mean val ce loss: 0.108564, mean val dice : 0.779470
[07:32:22.438] epoch : 274, iteration : 4936, train loss : 0.144148, train loss_ce: 0.073924, train loss_dice: 0.214371, train dice : 0.785629
[07:32:59.669] epoch : 274, iteration : 4944, train loss : 0.259370, train loss_ce: 0.058305, train loss_dice: 0.460435, train dice : 0.539565
[07:33:25.587] epoch : 274, mean train loss : 0.191803, mean train ce loss: 0.144158, mean train dice : 0.760551
[07:33:44.054] epoch : 274, iteration : 2196, val loss : 0.054077, val loss_ce: 0.072957, val loss_dice: 0.035197, val dice : 0.964803
[07:33:56.229] epoch : 274, mean val loss : 0.188007, mean val ce loss: 0.171172, mean val dice : 0.795157
[07:34:19.330] epoch : 275, iteration : 4952, train loss : 0.114895, train loss_ce: 0.120103, train loss_dice: 0.109687, train dice : 0.890313
[07:34:56.542] epoch : 275, iteration : 4960, train loss : 0.087623, train loss_ce: 0.107892, train loss_dice: 0.067354, train dice : 0.932646
[07:35:31.696] epoch : 275, iteration : 4968, train loss : 0.252445, train loss_ce: 0.162082, train loss_dice: 0.342808, train dice : 0.657192
[07:35:31.833] epoch : 275, mean train loss : 0.178251, mean train ce loss: 0.176191, mean train dice : 0.819688
[07:35:51.285] epoch : 275, iteration : 2202, val loss : 0.588237, val loss_ce: 0.765667, val loss_dice: 0.410806, val dice : 0.589194
[07:36:05.148] epoch : 275, iteration : 2208, val loss : 0.249722, val loss_ce: 0.296171, val loss_dice: 0.203273, val dice : 0.796727
[07:36:05.328] epoch : 275, mean val loss : 0.310205, mean val ce loss: 0.327219, mean val dice : 0.706809
[07:37:05.811] epoch : 276, iteration : 4976, train loss : 0.264916, train loss_ce: 0.180845, train loss_dice: 0.348988, train dice : 0.651012
[07:37:43.039] epoch : 276, iteration : 4984, train loss : 0.647488, train loss_ce: 0.768540, train loss_dice: 0.526435, train dice : 0.473565
[07:37:50.409] epoch : 276, mean train loss : 0.296643, mean train ce loss: 0.265857, mean train dice : 0.672570
[07:38:14.085] epoch : 276, iteration : 2214, val loss : 0.336120, val loss_ce: 0.351979, val loss_dice: 0.320261, val dice : 0.679739
[07:38:15.281] epoch : 276, mean val loss : 0.393985, mean val ce loss: 0.438960, mean val dice : 0.650991
[07:38:52.699] epoch : 277, iteration : 4992, train loss : 0.228197, train loss_ce: 0.217897, train loss_dice: 0.238496, train dice : 0.761504
[07:39:29.944] epoch : 277, iteration : 5000, train loss : 0.271989, train loss_ce: 0.307173, train loss_dice: 0.236806, train dice : 0.763194
[07:39:46.669] epoch : 277, mean train loss : 0.316355, mean train ce loss: 0.280887, mean train dice : 0.648177
[07:40:04.229] epoch : 277, iteration : 2220, val loss : 0.325278, val loss_ce: 0.392854, val loss_dice: 0.257702, val dice : 0.742298
[07:40:14.418] epoch : 277, mean val loss : 0.260414, mean val ce loss: 0.254793, mean val dice : 0.733966
[07:40:51.568] epoch : 278, iteration : 5008, train loss : 0.317040, train loss_ce: 0.405748, train loss_dice: 0.228333, train dice : 0.771667
[07:41:28.790] epoch : 278, iteration : 5016, train loss : 0.184694, train loss_ce: 0.148276, train loss_dice: 0.221112, train dice : 0.778888
[07:41:54.713] epoch : 278, mean train loss : 0.237596, mean train ce loss: 0.250080, mean train dice : 0.774888
[07:42:12.470] epoch : 278, iteration : 2226, val loss : 0.208136, val loss_ce: 0.205887, val loss_dice: 0.210385, val dice : 0.789615
[07:42:24.715] epoch : 278, iteration : 2232, val loss : 0.547885, val loss_ce: 0.824581, val loss_dice: 0.271188, val dice : 0.728812
[07:42:24.829] epoch : 278, mean val loss : 0.355522, mean val ce loss: 0.433213, mean val dice : 0.722169
[07:42:45.279] epoch : 279, iteration : 5024, train loss : 0.199838, train loss_ce: 0.120899, train loss_dice: 0.278777, train dice : 0.721223
[07:43:22.503] epoch : 279, iteration : 5032, train loss : 0.282293, train loss_ce: 0.103921, train loss_dice: 0.460666, train dice : 0.539334
[07:43:57.702] epoch : 279, iteration : 5040, train loss : 0.286011, train loss_ce: 0.198051, train loss_dice: 0.373972, train dice : 0.626028
[07:43:57.896] epoch : 279, mean train loss : 0.232418, mean train ce loss: 0.228536, mean train dice : 0.763700
[07:44:14.511] epoch : 279, iteration : 2238, val loss : 0.360996, val loss_ce: 0.437911, val loss_dice: 0.284082, val dice : 0.715918
[07:44:27.460] epoch : 279, mean val loss : 0.259017, mean val ce loss: 0.262440, mean val dice : 0.744406
[07:45:21.712] epoch : 280, iteration : 5048, train loss : 0.176932, train loss_ce: 0.193905, train loss_dice: 0.159960, train dice : 0.840040
[07:45:58.964] epoch : 280, iteration : 5056, train loss : 0.206753, train loss_ce: 0.147777, train loss_dice: 0.265729, train dice : 0.734271
[07:46:06.385] epoch : 280, mean train loss : 0.239936, mean train ce loss: 0.244342, mean train dice : 0.764470
[07:46:31.848] epoch : 280, iteration : 2244, val loss : 0.189085, val loss_ce: 0.127155, val loss_dice: 0.251015, val dice : 0.748985
[07:46:44.158] epoch : 280, mean val loss : 0.293510, mean val ce loss: 0.253734, mean val dice : 0.666715
[07:47:27.449] epoch : 281, iteration : 5064, train loss : 0.218148, train loss_ce: 0.263570, train loss_dice: 0.172725, train dice : 0.827275
[07:48:04.704] epoch : 281, iteration : 5072, train loss : 0.122165, train loss_ce: 0.148335, train loss_dice: 0.095995, train dice : 0.904005
[07:48:21.478] epoch : 281, mean train loss : 0.246676, mean train ce loss: 0.242341, mean train dice : 0.748988
[07:48:33.870] epoch : 281, iteration : 2250, val loss : 0.210019, val loss_ce: 0.192008, val loss_dice: 0.228031, val dice : 0.771969
[07:48:45.984] epoch : 281, iteration : 2256, val loss : 0.160764, val loss_ce: 0.087183, val loss_dice: 0.234346, val dice : 0.765654
[07:48:46.158] epoch : 281, mean val loss : 0.182090, mean val ce loss: 0.157497, mean val dice : 0.793317
[07:49:19.886] epoch : 282, iteration : 5080, train loss : 0.189396, train loss_ce: 0.258345, train loss_dice: 0.120447, train dice : 0.879553
[07:49:57.146] epoch : 282, iteration : 5088, train loss : 0.181758, train loss_ce: 0.178935, train loss_dice: 0.184582, train dice : 0.815418
[07:50:23.161] epoch : 282, mean train loss : 0.253294, mean train ce loss: 0.245829, mean train dice : 0.739240
[07:50:44.102] epoch : 282, iteration : 2262, val loss : 0.218461, val loss_ce: 0.287056, val loss_dice: 0.149866, val dice : 0.850134
[07:50:46.873] epoch : 282, mean val loss : 0.203592, mean val ce loss: 0.202503, mean val dice : 0.795320
[07:51:12.863] epoch : 283, iteration : 5096, train loss : 0.221530, train loss_ce: 0.179964, train loss_dice: 0.263097, train dice : 0.736903
[07:51:50.095] epoch : 283, iteration : 5104, train loss : 0.321335, train loss_ce: 0.465533, train loss_dice: 0.177136, train dice : 0.822864
[07:52:25.249] epoch : 283, iteration : 5112, train loss : 0.293936, train loss_ce: 0.186337, train loss_dice: 0.401535, train dice : 0.598465
[07:52:25.442] epoch : 283, mean train loss : 0.217995, mean train ce loss: 0.194174, mean train dice : 0.758184
[07:52:39.041] epoch : 283, iteration : 2268, val loss : 0.157506, val loss_ce: 0.171299, val loss_dice: 0.143713, val dice : 0.856287
[07:52:55.719] epoch : 283, mean val loss : 0.232552, mean val ce loss: 0.160550, mean val dice : 0.695446
[07:53:44.077] epoch : 284, iteration : 5120, train loss : 0.074164, train loss_ce: 0.077379, train loss_dice: 0.070950, train dice : 0.929050
[07:54:21.342] epoch : 284, iteration : 5128, train loss : 0.087928, train loss_ce: 0.114429, train loss_dice: 0.061427, train dice : 0.938573
[07:54:28.836] epoch : 284, mean train loss : 0.224926, mean train ce loss: 0.177519, mean train dice : 0.727666
[07:54:42.328] epoch : 284, iteration : 2274, val loss : 0.191667, val loss_ce: 0.202989, val loss_dice: 0.180345, val dice : 0.819655
[07:54:52.713] epoch : 284, iteration : 2280, val loss : 0.158321, val loss_ce: 0.086556, val loss_dice: 0.230086, val dice : 0.769914
[07:54:52.853] epoch : 284, mean val loss : 0.219418, mean val ce loss: 0.177803, mean val dice : 0.738967
[07:55:40.998] epoch : 285, iteration : 5136, train loss : 0.276937, train loss_ce: 0.320530, train loss_dice: 0.233345, train dice : 0.766655
[07:56:18.263] epoch : 285, iteration : 5144, train loss : 0.220696, train loss_ce: 0.281581, train loss_dice: 0.159812, train dice : 0.840189
[07:56:34.919] epoch : 285, mean train loss : 0.212820, mean train ce loss: 0.183841, mean train dice : 0.758202
[07:56:53.284] epoch : 285, iteration : 2286, val loss : 0.157712, val loss_ce: 0.161862, val loss_dice: 0.153562, val dice : 0.846438
[07:57:03.247] epoch : 285, mean val loss : 0.217909, mean val ce loss: 0.199995, mean val dice : 0.764177
[07:57:37.774] epoch : 286, iteration : 5152, train loss : 0.115380, train loss_ce: 0.162200, train loss_dice: 0.068560, train dice : 0.931440
[07:58:15.017] epoch : 286, iteration : 5160, train loss : 0.227755, train loss_ce: 0.052648, train loss_dice: 0.402862, train dice : 0.597138
[07:58:41.021] epoch : 286, mean train loss : 0.197187, mean train ce loss: 0.153669, mean train dice : 0.759295
[07:59:03.106] epoch : 286, iteration : 2292, val loss : 0.246530, val loss_ce: 0.257853, val loss_dice: 0.235207, val dice : 0.764793
[07:59:14.348] epoch : 286, mean val loss : 0.193936, mean val ce loss: 0.137685, mean val dice : 0.749813
[07:59:30.846] epoch : 287, iteration : 5168, train loss : 0.165210, train loss_ce: 0.105749, train loss_dice: 0.224672, train dice : 0.775328
[08:00:08.101] epoch : 287, iteration : 5176, train loss : 0.149603, train loss_ce: 0.174523, train loss_dice: 0.124683, train dice : 0.875317
[08:00:43.250] epoch : 287, iteration : 5184, train loss : 0.231433, train loss_ce: 0.112242, train loss_dice: 0.350623, train dice : 0.649377
[08:00:43.458] epoch : 287, mean train loss : 0.196798, mean train ce loss: 0.151781, mean train dice : 0.758186
[08:00:56.885] epoch : 287, iteration : 2298, val loss : 0.173163, val loss_ce: 0.220851, val loss_dice: 0.125476, val dice : 0.874524
[08:01:09.203] epoch : 287, iteration : 2304, val loss : 0.176274, val loss_ce: 0.248967, val loss_dice: 0.103582, val dice : 0.896418
[08:01:09.377] epoch : 287, mean val loss : 0.171456, mean val ce loss: 0.169394, mean val dice : 0.826481
[08:01:56.789] epoch : 288, iteration : 5192, train loss : 0.154471, train loss_ce: 0.098195, train loss_dice: 0.210747, train dice : 0.789253
[08:02:34.056] epoch : 288, iteration : 5200, train loss : 0.157485, train loss_ce: 0.079977, train loss_dice: 0.234992, train dice : 0.765008
[08:02:41.373] epoch : 288, mean train loss : 0.199109, mean train ce loss: 0.151059, mean train dice : 0.752842
[08:03:06.903] epoch : 288, iteration : 2310, val loss : 0.233995, val loss_ce: 0.286262, val loss_dice: 0.181728, val dice : 0.818272
[08:03:11.914] epoch : 288, mean val loss : 0.219535, mean val ce loss: 0.167690, mean val dice : 0.728619
[08:03:49.441] epoch : 289, iteration : 5208, train loss : 0.166287, train loss_ce: 0.180344, train loss_dice: 0.152231, train dice : 0.847769
[08:04:26.698] epoch : 289, iteration : 5216, train loss : 0.190476, train loss_ce: 0.142128, train loss_dice: 0.238824, train dice : 0.761176
[08:04:43.388] epoch : 289, mean train loss : 0.187069, mean train ce loss: 0.147041, mean train dice : 0.772904
[08:05:06.930] epoch : 289, iteration : 2316, val loss : 0.224333, val loss_ce: 0.248099, val loss_dice: 0.200568, val dice : 0.799432
[08:05:09.177] epoch : 289, mean val loss : 0.189530, mean val ce loss: 0.177076, mean val dice : 0.798016
[08:05:37.831] epoch : 290, iteration : 5224, train loss : 0.236705, train loss_ce: 0.287266, train loss_dice: 0.186144, train dice : 0.813856
[08:06:15.089] epoch : 290, iteration : 5232, train loss : 0.261213, train loss_ce: 0.225569, train loss_dice: 0.296857, train dice : 0.703143
[08:06:41.101] epoch : 290, mean train loss : 0.183207, mean train ce loss: 0.177175, mean train dice : 0.810762
[08:06:53.857] epoch : 290, iteration : 2322, val loss : 0.186380, val loss_ce: 0.243488, val loss_dice: 0.129271, val dice : 0.870729
[08:07:04.305] epoch : 290, iteration : 2328, val loss : 0.213019, val loss_ce: 0.108602, val loss_dice: 0.317435, val dice : 0.682565
[08:07:04.543] epoch : 290, mean val loss : 0.174576, mean val ce loss: 0.164860, mean val dice : 0.815708
[08:07:28.605] epoch : 291, iteration : 5240, train loss : 0.144281, train loss_ce: 0.166465, train loss_dice: 0.122097, train dice : 0.877903
[08:08:05.840] epoch : 291, iteration : 5248, train loss : 0.122515, train loss_ce: 0.162489, train loss_dice: 0.082541, train dice : 0.917459
[08:08:40.969] epoch : 291, iteration : 5256, train loss : 0.432936, train loss_ce: 0.030882, train loss_dice: 0.834990, train dice : 0.165010
[08:08:41.156] epoch : 291, mean train loss : 0.165432, mean train ce loss: 0.138639, mean train dice : 0.807775
[08:09:00.939] epoch : 291, iteration : 2334, val loss : 0.138959, val loss_ce: 0.181368, val loss_dice: 0.096550, val dice : 0.903450
[08:09:10.593] epoch : 291, mean val loss : 0.148981, mean val ce loss: 0.162097, mean val dice : 0.864134
[08:09:10.884] save best model to ../model/TU_Synapse[16, 336, 448]/TU_pretrain_R50-ViT-B_16_skip3_15k_epo1500_bs1_[16, 336, 448]/best_model.pth at epoch 292, val dice: 0.8641337752342224
[08:09:59.020] epoch : 292, iteration : 5264, train loss : 0.151525, train loss_ce: 0.079835, train loss_dice: 0.223215, train dice : 0.776785
[08:10:36.282] epoch : 292, iteration : 5272, train loss : 0.138596, train loss_ce: 0.083599, train loss_dice: 0.193593, train dice : 0.806407
[08:10:43.591] epoch : 292, mean train loss : 0.141580, mean train ce loss: 0.132504, mean train dice : 0.849343
[08:11:01.506] epoch : 292, iteration : 2340, val loss : 0.134059, val loss_ce: 0.167850, val loss_dice: 0.100268, val dice : 0.899732
[08:11:08.658] epoch : 292, mean val loss : 0.166421, mean val ce loss: 0.152267, mean val dice : 0.819425
[08:11:47.787] epoch : 293, iteration : 5280, train loss : 0.161776, train loss_ce: 0.194557, train loss_dice: 0.128995, train dice : 0.871005
[08:12:25.038] epoch : 293, iteration : 5288, train loss : 0.108067, train loss_ce: 0.110784, train loss_dice: 0.105350, train dice : 0.894650
[08:12:41.798] epoch : 293, mean train loss : 0.157037, mean train ce loss: 0.144650, mean train dice : 0.830576
[08:12:52.284] epoch : 293, iteration : 2346, val loss : 0.150506, val loss_ce: 0.074425, val loss_dice: 0.226587, val dice : 0.773413
[08:13:07.983] epoch : 293, iteration : 2352, val loss : 0.197648, val loss_ce: 0.255662, val loss_dice: 0.139634, val dice : 0.860366
[08:13:08.184] epoch : 293, mean val loss : 0.152226, mean val ce loss: 0.136399, mean val dice : 0.831946
[08:13:45.573] epoch : 294, iteration : 5296, train loss : 0.130430, train loss_ce: 0.124095, train loss_dice: 0.136766, train dice : 0.863234
[08:14:22.846] epoch : 294, iteration : 5304, train loss : 0.188640, train loss_ce: 0.161873, train loss_dice: 0.215407, train dice : 0.784593
[08:14:49.099] epoch : 294, mean train loss : 0.153042, mean train ce loss: 0.141296, mean train dice : 0.835212
[08:15:11.423] epoch : 294, iteration : 2358, val loss : 0.251740, val loss_ce: 0.266232, val loss_dice: 0.237249, val dice : 0.762751
[08:15:16.811] epoch : 294, mean val loss : 0.179711, mean val ce loss: 0.151955, mean val dice : 0.792534
[08:15:38.149] epoch : 295, iteration : 5312, train loss : 0.162002, train loss_ce: 0.065506, train loss_dice: 0.258499, train dice : 0.741501
[08:16:15.388] epoch : 295, iteration : 5320, train loss : 0.109617, train loss_ce: 0.156775, train loss_dice: 0.062460, train dice : 0.937540
[08:16:50.519] epoch : 295, iteration : 5328, train loss : 0.190098, train loss_ce: 0.217449, train loss_dice: 0.162747, train dice : 0.837253
[08:16:50.666] epoch : 295, mean train loss : 0.165323, mean train ce loss: 0.155377, mean train dice : 0.824730
[08:17:07.501] epoch : 295, iteration : 2364, val loss : 0.188575, val loss_ce: 0.170632, val loss_dice: 0.206518, val dice : 0.793482
[08:17:15.727] epoch : 295, mean val loss : 0.189900, mean val ce loss: 0.112164, mean val dice : 0.732364
[08:18:07.601] epoch : 296, iteration : 5336, train loss : 0.114980, train loss_ce: 0.128978, train loss_dice: 0.100981, train dice : 0.899019
[08:18:44.921] epoch : 296, iteration : 5344, train loss : 0.178442, train loss_ce: 0.156326, train loss_dice: 0.200559, train dice : 0.799441
[08:18:52.360] epoch : 296, mean train loss : 0.167607, mean train ce loss: 0.155252, mean train dice : 0.820039
[08:19:10.910] epoch : 296, iteration : 2370, val loss : 0.233834, val loss_ce: 0.223266, val loss_dice: 0.244401, val dice : 0.755599
[08:19:20.136] epoch : 296, iteration : 2376, val loss : 0.175562, val loss_ce: 0.105836, val loss_dice: 0.245288, val dice : 0.754712
[08:19:20.313] epoch : 296, mean val loss : 0.168150, mean val ce loss: 0.120638, mean val dice : 0.784339
[08:20:13.301] epoch : 297, iteration : 5352, train loss : 0.135226, train loss_ce: 0.136345, train loss_dice: 0.134108, train dice : 0.865892
[08:20:50.603] epoch : 297, iteration : 5360, train loss : 0.059427, train loss_ce: 0.081055, train loss_dice: 0.037800, train dice : 0.962200
[08:21:07.336] epoch : 297, mean train loss : 0.140055, mean train ce loss: 0.129184, mean train dice : 0.849074
[08:21:32.424] epoch : 297, iteration : 2382, val loss : 0.235143, val loss_ce: 0.304203, val loss_dice: 0.166082, val dice : 0.833918
[08:21:39.903] epoch : 297, mean val loss : 0.167627, mean val ce loss: 0.156205, mean val dice : 0.820952
[08:22:18.089] epoch : 298, iteration : 5368, train loss : 0.061440, train loss_ce: 0.082493, train loss_dice: 0.040386, train dice : 0.959614
[08:22:55.391] epoch : 298, iteration : 5376, train loss : 0.093373, train loss_ce: 0.116927, train loss_dice: 0.069819, train dice : 0.930180
[08:23:21.579] epoch : 298, mean train loss : 0.166916, mean train ce loss: 0.131728, mean train dice : 0.797896
[08:23:39.619] epoch : 298, iteration : 2388, val loss : 0.152561, val loss_ce: 0.190559, val loss_dice: 0.114563, val dice : 0.885437
[08:24:02.023] epoch : 298, mean val loss : 0.190781, mean val ce loss: 0.159186, mean val dice : 0.777624
[08:24:23.089] epoch : 299, iteration : 5384, train loss : 0.176826, train loss_ce: 0.103302, train loss_dice: 0.250350, train dice : 0.749650
[08:25:00.335] epoch : 299, iteration : 5392, train loss : 0.228093, train loss_ce: 0.079491, train loss_dice: 0.376694, train dice : 0.623306
[08:25:35.493] epoch : 299, iteration : 5400, train loss : 0.395817, train loss_ce: 0.120382, train loss_dice: 0.671251, train dice : 0.328749
[08:25:35.687] epoch : 299, mean train loss : 0.167409, mean train ce loss: 0.113045, mean train dice : 0.778227
[08:25:54.024] epoch : 299, iteration : 2394, val loss : 0.082674, val loss_ce: 0.097140, val loss_dice: 0.068208, val dice : 0.931792
[08:26:01.512] epoch : 299, iteration : 2400, val loss : 0.136932, val loss_ce: 0.162452, val loss_dice: 0.111413, val dice : 0.888587
[08:26:01.753] epoch : 299, mean val loss : 0.140981, mean val ce loss: 0.112463, mean val dice : 0.830501
[08:26:56.594] epoch : 300, iteration : 5408, train loss : 0.299298, train loss_ce: 0.256246, train loss_dice: 0.342350, train dice : 0.657650
[08:27:33.850] epoch : 300, iteration : 5416, train loss : 0.105135, train loss_ce: 0.149021, train loss_dice: 0.061249, train dice : 0.938751
[08:27:41.268] epoch : 300, mean train loss : 0.192562, mean train ce loss: 0.147426, mean train dice : 0.762302
[08:28:02.794] epoch : 300, iteration : 2406, val loss : 0.318912, val loss_ce: 0.070106, val loss_dice: 0.567719, val dice : 0.432281
[08:28:04.261] epoch : 300, mean val loss : 0.230652, mean val ce loss: 0.201793, mean val dice : 0.740490

[AlexYouXin]

For the standard calculating way, Dice scores range in [0, 1]. And the weight for Dice loss in our code is also normalized. So a negative Dice is strictly avoided. Please check the Dice score for each class. Maybe a detailed check for the input/prediction/ground truth is a good way to solve your problem.

Thank you for your prompt response. Can you share your training log to compare to see the training process correctly. Thank you!

This is the log file for the clean lung lobe dataset