-
Notifications
You must be signed in to change notification settings - Fork 1
/
TFTransformer_log.txt
190 lines (190 loc) · 22.3 KB
/
TFTransformer_log.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
2021-09-03 15:23:39,001 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 1 Lr 0.0009999999506011068 Loss 2.824898
2021-09-03 15:24:11,087 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 201 Lr 0.0009980055760310416 Loss 0.288475
2021-09-03 15:24:42,656 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 401 Lr 0.0009920778306883834 Loss 0.252154
2021-09-03 15:25:14,295 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 601 Lr 0.0009822640081969926 Loss 0.270999
2021-09-03 15:25:45,948 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 801 Lr 0.0009686424066622785 Loss 0.214717
2021-09-03 15:26:17,540 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 1001 Lr 0.000951321703981527 Loss 0.229461
2021-09-03 15:26:49,304 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 1201 Lr 0.0009304400907736608 Loss 0.217524
2021-09-03 15:27:20,911 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 1401 Lr 0.0009061641678461934 Loss 0.232213
2021-09-03 15:27:52,691 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 1601 Lr 0.0008786876169957689 Loss 0.302791
2021-09-03 15:28:24,262 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 1801 Lr 0.0008482296557470985 Loss 0.227365
2021-09-03 15:28:55,797 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 2001 Lr 0.0008150332883589441 Loss 0.208030
2021-09-03 15:29:27,473 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 2201 Lr 0.0007793633670512172 Loss 0.243047
2021-09-03 15:29:58,992 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 2401 Lr 0.000741504478921416 Loss 0.167886
2021-09-03 15:30:30,685 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 2601 Lr 0.0007017586754093052 Loss 0.138515
2021-09-03 15:31:02,247 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 2801 Lr 0.0006604430624249766 Loss 0.189793
2021-09-03 15:31:33,649 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 3001 Lr 0.0006178872703670821 Loss 0.214850
2021-09-03 15:32:05,293 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 3201 Lr 0.000574430824216318 Loss 0.204835
2021-09-03 15:32:37,005 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 3401 Lr 0.0005304204346864788 Loss 0.188094
2021-09-03 15:33:08,605 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 3601 Lr 0.0004862072320452168 Loss 0.223791
2021-09-03 15:33:40,285 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 3801 Lr 0.0004421439646740699 Loss 0.211821
2021-09-03 15:34:12,207 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 4001 Lr 0.0003985821847185883 Loss 0.185862
2021-09-03 15:34:43,828 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 4201 Lr 0.00035586944328247006 Loss 0.217519
2021-09-03 15:35:15,053 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 4401 Lr 0.00031434651754339867 Loss 0.215630
2021-09-03 15:35:46,341 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 4601 Lr 0.0002743446919136313 Loss 0.176132
2021-09-03 15:36:17,650 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 4801 Lr 0.00023618311493718536 Loss 0.203161
2021-09-03 15:36:48,980 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 5001 Lr 0.00020016625301122183 Loss 0.181000
2021-09-03 15:37:20,382 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 5201 Lr 0.0001665814612467342 Loss 0.183891
2021-09-03 15:37:51,636 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 5401 Lr 0.00013569669084904869 Loss 0.209007
2021-09-03 15:38:22,848 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 5601 Lr 0.00010775835130947083 Loss 0.153682
2021-09-03 15:38:54,425 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 5801 Lr 8.29893444642522e-05 Loss 0.176696
2021-09-03 15:39:26,138 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 6001 Lr 6.158728610584651e-05 Loss 0.159247
2021-09-03 15:39:57,926 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 6201 Lr 4.3722929335055734e-05 Loss 0.189528
2021-09-03 15:40:29,556 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 6401 Lr 2.953880223312291e-05 Loss 0.170042
2021-09-03 15:41:00,851 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 6601 Lr 1.9148070722888842e-05 Loss 0.179470
2021-09-03 15:41:31,974 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 6801 Lr 1.2633635691514614e-05 Loss 0.228351
2021-09-03 15:42:03,170 - TFTransformer - INFO - [TRAIN] Epoch 0 Iter 7001 Lr 1.0047471578232351e-05 Loss 0.177861
2021-09-03 15:46:55,198 - TFTransformer - INFO - [TEST] AVG P90 Loss on TEST-set: 0.028266 │ T: 287.26 s
2021-09-03 15:46:55,199 - TFTransformer - INFO - [TEST] AVG Loss on TEST-set: 0.211138 │ T: 287.26 s
2021-09-03 15:46:55,683 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 1 Lr 0.001 Loss 0.179772
2021-09-03 15:47:27,146 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 201 Lr 0.000998025358508001 Loss 0.192790
2021-09-03 15:47:58,386 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 401 Lr 0.0009921171884118949 Loss 0.207535
2021-09-03 15:48:29,490 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 601 Lr 0.0009823226271573839 Loss 0.180707
2021-09-03 15:49:00,734 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 801 Lr 0.0009687198191769983 Loss 0.204773
2021-09-03 15:49:31,991 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 1001 Lr 0.0009514172924264802 Loss 0.223575
2021-09-03 15:50:03,216 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 1201 Lr 0.0009305530925108286 Loss 0.192137
2021-09-03 15:50:34,567 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 1401 Lr 0.0009062936813082275 Loss 0.193239
2021-09-03 15:51:04,722 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 1601 Lr 0.0008788326088790084 Loss 0.183640
2021-09-03 15:51:33,565 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 1801 Lr 0.0008483889692556302 Loss 0.214860
2021-09-03 15:52:02,614 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 2001 Lr 0.0008152056524339227 Loss 0.168641
2021-09-03 15:52:31,272 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 2201 Lr 0.0007795474065118175 Loss 0.189952
2021-09-03 15:53:00,820 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 2401 Lr 0.0007416987254365066 Loss 0.230462
2021-09-03 15:53:31,957 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 2601 Lr 0.0007019615792123069 Loss 0.164715
2021-09-03 15:54:03,065 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 2801 Lr 0.0006606530046784414 Loss 0.158549
2021-09-03 15:54:34,165 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 3001 Lr 0.0006181025760783453 Loss 0.190814
2021-09-03 15:55:05,336 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 3201 Lr 0.0005746497756011736 Loss 0.183460
2021-09-03 15:55:36,399 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 3401 Lr 0.0005306412848742632 Loss 0.174573
2021-09-03 15:56:07,518 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 3601 Lr 0.00048642821901595285 Loss 0.159855
2021-09-03 15:56:38,935 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 3801 Lr 0.0004423633253164785 Loss 0.235008
2021-09-03 15:57:10,442 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 4001 Lr 0.00039879816889680595 Loss 0.151719
2021-09-03 15:57:42,002 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 4201 Lr 0.00035608032779924445 Loss 0.174989
2021-09-03 15:58:13,509 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 4401 Lr 0.00031455061988835877 Loss 0.177214
2021-09-03 15:58:45,140 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 4601 Lr 0.00027454038368694315 Loss 0.177605
2021-09-03 15:59:16,552 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 4801 Lr 0.00023636883484149401 Loss 0.182482
2021-09-03 15:59:47,933 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 5001 Lr 0.00020034051930822882 Loss 0.177535
2021-09-03 16:00:19,292 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 5201 Lr 0.00016674288357902263 Loss 0.170216
2021-09-03 16:00:50,613 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 5401 Lr 0.00013584398133284012 Loss 0.166256
2021-09-03 16:01:22,202 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 5601 Lr 0.00010789033480981336 Loss 0.168559
2021-09-03 16:01:53,619 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 5801 Lr 8.310496797065664e-05 Loss 0.190023
2021-09-03 16:02:25,162 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 6001 Lr 6.16856271335726e-05 Loss 0.160877
2021-09-03 16:02:56,631 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 6201 Lr 4.380320328501782e-05 Loss 0.165835
2021-09-03 16:03:28,114 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 6401 Lr 2.9600368651695462e-05 Loss 0.164687
2021-09-03 16:03:59,306 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 6601 Lr 1.9190438411669365e-05 Loss 0.184859
2021-09-03 16:04:30,701 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 6801 Lr 1.2656466626276187e-05 Loss 0.188838
2021-09-03 16:05:02,125 - TFTransformer - INFO - [TRAIN] Epoch 1 Iter 7001 Lr 1.005058360579874e-05 Loss 0.178692
2021-09-03 16:10:57,469 - TFTransformer - INFO - [TEST] AVG P90 Loss on TEST-set: 0.028063 │ T: 350.56 s
2021-09-03 16:10:57,469 - TFTransformer - INFO - [TEST] AVG Loss on TEST-set: 0.209907 │ T: 350.56 s
2021-09-03 16:10:57,942 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 1 Lr 0.001 Loss 0.172156
2021-09-03 16:11:29,647 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 201 Lr 0.000998025358508001 Loss 0.190345
2021-09-03 16:12:01,216 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 401 Lr 0.0009921171884118949 Loss 0.169587
2021-09-03 16:12:32,781 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 601 Lr 0.0009823226271573839 Loss 0.209035
2021-09-03 16:13:04,431 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 801 Lr 0.0009687198191769983 Loss 0.174679
2021-09-03 16:13:35,955 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 1001 Lr 0.0009514172924264802 Loss 0.198299
2021-09-03 16:14:07,595 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 1201 Lr 0.0009305530925108286 Loss 0.155516
2021-09-03 16:14:39,102 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 1401 Lr 0.0009062936813082275 Loss 0.176828
2021-09-03 16:15:10,718 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 1601 Lr 0.0008788326088790084 Loss 0.187589
2021-09-03 16:15:42,138 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 1801 Lr 0.0008483889692556302 Loss 0.169784
2021-09-03 16:16:13,661 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 2001 Lr 0.0008152056524339227 Loss 0.150678
2021-09-03 16:16:45,180 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 2201 Lr 0.0007795474065118175 Loss 0.178095
2021-09-03 16:17:16,667 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 2401 Lr 0.0007416987254365066 Loss 0.214385
2021-09-03 16:17:48,095 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 2601 Lr 0.0007019615792123069 Loss 0.159672
2021-09-03 16:18:19,576 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 2801 Lr 0.0006606530046784414 Loss 0.169776
2021-09-03 16:18:50,992 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 3001 Lr 0.0006181025760783453 Loss 0.245625
2021-09-03 16:19:22,324 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 3201 Lr 0.0005746497756011736 Loss 0.175648
2021-09-03 16:19:53,656 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 3401 Lr 0.0005306412848742632 Loss 0.176674
2021-09-03 16:20:24,958 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 3601 Lr 0.00048642821901595285 Loss 0.158371
2021-09-03 16:20:56,317 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 3801 Lr 0.0004423633253164785 Loss 0.162989
2021-09-03 16:21:27,786 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 4001 Lr 0.00039879816889680595 Loss 0.160757
2021-09-03 16:21:59,141 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 4201 Lr 0.00035608032779924445 Loss 0.160036
2021-09-03 16:22:30,501 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 4401 Lr 0.00031455061988835877 Loss 0.228346
2021-09-03 16:23:01,929 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 4601 Lr 0.00027454038368694315 Loss 0.177525
2021-09-03 16:23:33,484 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 4801 Lr 0.00023636883484149401 Loss 0.163165
2021-09-03 16:24:04,968 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 5001 Lr 0.00020034051930822882 Loss 0.167117
2021-09-03 16:24:36,388 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 5201 Lr 0.00016674288357902263 Loss 0.141205
2021-09-03 16:25:07,809 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 5401 Lr 0.00013584398133284012 Loss 0.175409
2021-09-03 16:25:39,146 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 5601 Lr 0.00010789033480981336 Loss 0.199143
2021-09-03 16:26:10,664 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 5801 Lr 8.310496797065664e-05 Loss 0.166995
2021-09-03 16:26:42,464 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 6001 Lr 6.16856271335726e-05 Loss 0.157313
2021-09-03 16:27:14,637 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 6201 Lr 4.380320328501782e-05 Loss 0.201651
2021-09-03 16:27:45,894 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 6401 Lr 2.9600368651695462e-05 Loss 0.182091
2021-09-03 16:28:17,252 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 6601 Lr 1.9190438411669365e-05 Loss 0.160753
2021-09-03 16:28:48,832 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 6801 Lr 1.2656466626276187e-05 Loss 0.156193
2021-09-03 16:29:19,839 - TFTransformer - INFO - [TRAIN] Epoch 2 Iter 7001 Lr 1.005058360579874e-05 Loss 0.153121
2021-09-03 16:36:12,210 - TFTransformer - INFO - [TEST] AVG P90 Loss on TEST-set: 0.028056 │ T: 407.63 s
2021-09-03 16:36:12,210 - TFTransformer - INFO - [TEST] AVG Loss on TEST-set: 0.210342 │ T: 407.63 s
2021-09-03 16:36:12,583 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 1 Lr 0.001 Loss 0.172025
2021-09-03 16:36:44,174 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 201 Lr 0.000998025358508001 Loss 0.195251
2021-09-03 16:37:15,596 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 401 Lr 0.0009921171884118949 Loss 0.189264
2021-09-03 16:37:47,192 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 601 Lr 0.0009823226271573839 Loss 0.178337
2021-09-03 16:38:18,855 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 801 Lr 0.0009687198191769983 Loss 0.172385
2021-09-03 16:38:50,505 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 1001 Lr 0.0009514172924264802 Loss 0.196923
2021-09-03 16:39:22,217 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 1201 Lr 0.0009305530925108286 Loss 0.166585
2021-09-03 16:39:53,741 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 1401 Lr 0.0009062936813082275 Loss 0.206917
2021-09-03 16:40:25,162 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 1601 Lr 0.0008788326088790084 Loss 0.186853
2021-09-03 16:40:56,818 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 1801 Lr 0.0008483889692556302 Loss 0.143513
2021-09-03 16:41:28,459 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 2001 Lr 0.0008152056524339227 Loss 0.172585
2021-09-03 16:41:59,810 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 2201 Lr 0.0007795474065118175 Loss 0.178991
2021-09-03 16:42:31,230 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 2401 Lr 0.0007416987254365066 Loss 0.174969
2021-09-03 16:43:02,708 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 2601 Lr 0.0007019615792123069 Loss 0.163181
2021-09-03 16:43:33,993 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 2801 Lr 0.0006606530046784414 Loss 0.161615
2021-09-03 16:44:05,493 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 3001 Lr 0.0006181025760783453 Loss 0.185036
2021-09-03 16:44:37,094 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 3201 Lr 0.0005746497756011736 Loss 0.166235
2021-09-03 16:45:08,686 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 3401 Lr 0.0005306412848742632 Loss 0.187205
2021-09-03 16:45:40,344 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 3601 Lr 0.00048642821901595285 Loss 0.216185
2021-09-03 16:46:11,800 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 3801 Lr 0.0004423633253164785 Loss 0.184418
2021-09-03 16:46:43,162 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 4001 Lr 0.00039879816889680595 Loss 0.140129
2021-09-03 16:47:14,631 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 4201 Lr 0.00035608032779924445 Loss 0.167763
2021-09-03 16:47:46,109 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 4401 Lr 0.00031455061988835877 Loss 0.146749
2021-09-03 16:48:17,540 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 4601 Lr 0.00027454038368694315 Loss 0.180058
2021-09-03 16:48:48,957 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 4801 Lr 0.00023636883484149401 Loss 0.159326
2021-09-03 16:49:20,483 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 5001 Lr 0.00020034051930822882 Loss 0.179515
2021-09-03 16:49:52,054 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 5201 Lr 0.00016674288357902263 Loss 0.169625
2021-09-03 16:50:23,478 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 5401 Lr 0.00013584398133284012 Loss 0.158231
2021-09-03 16:50:55,004 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 5601 Lr 0.00010789033480981336 Loss 0.168307
2021-09-03 16:51:26,674 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 5801 Lr 8.310496797065664e-05 Loss 0.146352
2021-09-03 16:51:58,146 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 6001 Lr 6.16856271335726e-05 Loss 0.156581
2021-09-03 16:52:29,711 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 6201 Lr 4.380320328501782e-05 Loss 0.167749
2021-09-03 16:53:01,314 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 6401 Lr 2.9600368651695462e-05 Loss 0.179641
2021-09-03 16:53:32,683 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 6601 Lr 1.9190438411669365e-05 Loss 0.157845
2021-09-03 16:54:04,121 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 6801 Lr 1.2656466626276187e-05 Loss 0.171479
2021-09-03 16:54:35,531 - TFTransformer - INFO - [TRAIN] Epoch 3 Iter 7001 Lr 1.005058360579874e-05 Loss 0.184333
2021-09-03 17:02:32,455 - TFTransformer - INFO - [TEST] AVG P90 Loss on TEST-set: 0.028243 │ T: 472.14 s
2021-09-03 17:02:32,456 - TFTransformer - INFO - [TEST] AVG Loss on TEST-set: 0.211166 │ T: 472.14 s
2021-09-03 17:02:32,850 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 1 Lr 0.001 Loss 0.157066
2021-09-03 17:03:04,468 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 201 Lr 0.000998025358508001 Loss 0.183769
2021-09-03 17:03:36,316 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 401 Lr 0.0009921171884118949 Loss 0.181540
2021-09-03 17:04:07,543 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 601 Lr 0.0009823226271573839 Loss 0.182696
2021-09-03 17:04:38,898 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 801 Lr 0.0009687198191769983 Loss 0.157487
2021-09-03 17:05:10,297 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 1001 Lr 0.0009514172924264802 Loss 0.143891
2021-09-03 17:05:41,495 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 1201 Lr 0.0009305530925108286 Loss 0.172226
2021-09-03 17:06:12,712 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 1401 Lr 0.0009062936813082275 Loss 0.162778
2021-09-03 17:06:43,898 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 1601 Lr 0.0008788326088790084 Loss 0.192393
2021-09-03 17:07:15,125 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 1801 Lr 0.0008483889692556302 Loss 0.184524
2021-09-03 17:07:46,486 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 2001 Lr 0.0008152056524339227 Loss 0.192533
2021-09-03 17:08:18,390 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 2201 Lr 0.0007795474065118175 Loss 0.150952
2021-09-03 17:08:50,221 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 2401 Lr 0.0007416987254365066 Loss 0.156857
2021-09-03 17:09:21,870 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 2601 Lr 0.0007019615792123069 Loss 0.151446
2021-09-03 17:09:53,029 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 2801 Lr 0.0006606530046784414 Loss 0.172409
2021-09-03 17:10:24,023 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 3001 Lr 0.0006181025760783453 Loss 0.182902
2021-09-03 17:10:55,020 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 3201 Lr 0.0005746497756011736 Loss 0.174904
2021-09-03 17:11:26,030 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 3401 Lr 0.0005306412848742632 Loss 0.171373
2021-09-03 17:11:57,149 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 3601 Lr 0.00048642821901595285 Loss 0.145672
2021-09-03 17:12:28,274 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 3801 Lr 0.0004423633253164785 Loss 0.167671
2021-09-03 17:12:59,476 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 4001 Lr 0.00039879816889680595 Loss 0.175784
2021-09-03 17:13:30,872 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 4201 Lr 0.00035608032779924445 Loss 0.153229
2021-09-03 17:14:02,294 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 4401 Lr 0.00031455061988835877 Loss 0.150401
2021-09-03 17:14:33,744 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 4601 Lr 0.00027454038368694315 Loss 0.169285
2021-09-03 17:15:05,225 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 4801 Lr 0.00023636883484149401 Loss 0.182621
2021-09-03 17:15:36,610 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 5001 Lr 0.00020034051930822882 Loss 0.143936
2021-09-03 17:16:08,111 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 5201 Lr 0.00016674288357902263 Loss 0.157285
2021-09-03 17:16:39,456 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 5401 Lr 0.00013584398133284012 Loss 0.159246
2021-09-03 17:17:10,950 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 5601 Lr 0.00010789033480981336 Loss 0.174817
2021-09-03 17:17:42,337 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 5801 Lr 8.310496797065664e-05 Loss 0.163804
2021-09-03 17:18:13,735 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 6001 Lr 6.16856271335726e-05 Loss 0.159630
2021-09-03 17:18:45,236 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 6201 Lr 4.380320328501782e-05 Loss 0.203636
2021-09-03 17:19:16,743 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 6401 Lr 2.9600368651695462e-05 Loss 0.154684
2021-09-03 17:19:48,169 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 6601 Lr 1.9190438411669365e-05 Loss 0.158356
2021-09-03 17:20:19,243 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 6801 Lr 1.2656466626276187e-05 Loss 0.164066
2021-09-03 17:20:48,077 - TFTransformer - INFO - [TRAIN] Epoch 4 Iter 7001 Lr 1.005058360579874e-05 Loss 0.161644
2021-09-03 17:29:43,478 - TFTransformer - INFO - [TEST] AVG P90 Loss on TEST-set: 0.028849 │ T: 530.98 s
2021-09-03 17:29:43,479 - TFTransformer - INFO - [TEST] AVG Loss on TEST-set: 0.213228 │ T: 530.99 s