Stochastic pooling

[zuheli]

Is there anyone having tested the performance of stochastic pooling in the mnist example?In my experiment,it didn't work.Though the training goes on,the accuracy stayed low and the loss didn't drop.
I0914 16:58:11.371650 6664 solver.cpp:232] Iteration 0, Testing net (#0)
I0914 16:58:13.163522 6664 solver.cpp:270] Test score #0: 0.1053
I0914 16:58:13.163574 6664 solver.cpp:270] Test score #1: 11.2434
I0914 16:58:13.212926 6664 solver.cpp:195] Iteration 0, loss = 2.30317
I0914 16:58:13.212963 6664 solver.cpp:365] Iteration 0, lr = 0.01
I0914 16:58:16.516218 6664 solver.cpp:195] Iteration 100, loss = 0.551969
I0914 16:58:16.516258 6664 solver.cpp:365] Iteration 100, lr = 0.00992565
I0914 16:58:19.817391 6664 solver.cpp:195] Iteration 200, loss = 0.358946
I0914 16:58:19.817428 6664 solver.cpp:365] Iteration 200, lr = 0.00985258
I0914 16:58:23.118729 6664 solver.cpp:195] Iteration 300, loss = 0.358588
I0914 16:58:23.118769 6664 solver.cpp:365] Iteration 300, lr = 0.00978075
I0914 16:58:26.420337 6664 solver.cpp:195] Iteration 400, loss = 0.130164
I0914 16:58:26.420377 6664 solver.cpp:365] Iteration 400, lr = 0.00971013
I0914 16:58:29.688901 6664 solver.cpp:232] Iteration 500, Testing net (#0)
I0914 16:58:31.486011 6664 solver.cpp:270] Test score #0: 0.148
I0914 16:58:31.486045 6664 solver.cpp:270] Test score #1: 62.4881
I0914 16:58:31.499441 6664 solver.cpp:195] Iteration 500, loss = 0.148254
I0914 16:58:31.499467 6664 solver.cpp:365] Iteration 500, lr = 0.00964069
I0914 16:58:34.800518 6664 solver.cpp:195] Iteration 600, loss = 0.103436
I0914 16:58:34.800559 6664 solver.cpp:365] Iteration 600, lr = 0.0095724
I0914 16:58:38.102004 6664 solver.cpp:195] Iteration 700, loss = 0.4548
I0914 16:58:38.102042 6664 solver.cpp:365] Iteration 700, lr = 0.00950522
I0914 16:58:41.403699 6664 solver.cpp:195] Iteration 800, loss = 0.479394
I0914 16:58:41.403867 6664 solver.cpp:365] Iteration 800, lr = 0.00943913
I0914 16:58:44.704890 6664 solver.cpp:195] Iteration 900, loss = 0.168024
I0914 16:58:44.704928 6664 solver.cpp:365] Iteration 900, lr = 0.00937411
I0914 16:58:47.972710 6664 solver.cpp:232] Iteration 1000, Testing net (#0)
I0914 16:58:49.769160 6664 solver.cpp:270] Test score #0: 0.1484
I0914 16:58:49.769196 6664 solver.cpp:270] Test score #1: 66.2822
I0914 16:58:49.782620 6664 solver.cpp:195] Iteration 1000, loss = 0.263866
I0914 16:58:49.782645 6664 solver.cpp:365] Iteration 1000, lr = 0.00931012
I0914 16:58:53.084360 6664 solver.cpp:195] Iteration 1100, loss = 0.046007
I0914 16:58:53.084399 6664 solver.cpp:365] Iteration 1100, lr = 0.00924715
I0914 16:58:56.446574 6664 solver.cpp:195] Iteration 1200, loss = 0.105945
I0914 16:58:56.446614 6664 solver.cpp:365] Iteration 1200, lr = 0.00918515
I0914 16:58:59.749254 6664 solver.cpp:195] Iteration 1300, loss = 0.06199
I0914 16:58:59.749294 6664 solver.cpp:365] Iteration 1300, lr = 0.00912412
I0914 16:59:03.050050 6664 solver.cpp:195] Iteration 1400, loss = 0.0346693
I0914 16:59:03.050088 6664 solver.cpp:365] Iteration 1400, lr = 0.00906403
I0914 16:59:06.314366 6664 solver.cpp:232] Iteration 1500, Testing net (#0)
I0914 16:59:08.107100 6664 solver.cpp:270] Test score #0: 0.1459
I0914 16:59:08.107134 6664 solver.cpp:270] Test score #1: nan
I0914 16:59:08.120313 6664 solver.cpp:195] Iteration 1500, loss = 0.109687
I0914 16:59:08.120332 6664 solver.cpp:365] Iteration 1500, lr = 0.00900485
I0914 16:59:11.418623 6664 solver.cpp:195] Iteration 1600, loss = 0.233891
I0914 16:59:11.418774 6664 solver.cpp:365] Iteration 1600, lr = 0.00894657
I0914 16:59:14.719074 6664 solver.cpp:195] Iteration 1700, loss = 0.071758
I0914 16:59:14.719111 6664 solver.cpp:365] Iteration 1700, lr = 0.00888916
I0914 16:59:18.019206 6664 solver.cpp:195] Iteration 1800, loss = 0.0707035
I0914 16:59:18.019244 6664 solver.cpp:365] Iteration 1800, lr = 0.0088326
I0914 16:59:21.317819 6664 solver.cpp:195] Iteration 1900, loss = 0.188843
I0914 16:59:21.317857 6664 solver.cpp:365] Iteration 1900, lr = 0.00877687
I0914 16:59:24.582819 6664 solver.cpp:232] Iteration 2000, Testing net (#0)
I0914 16:59:26.373980 6664 solver.cpp:270] Test score #0: 0.1459
I0914 16:59:26.374014 6664 solver.cpp:270] Test score #1: 68.2534
I0914 16:59:26.387195 6664 solver.cpp:195] Iteration 2000, loss = 0.161515
I0914 16:59:26.387214 6664 solver.cpp:365] Iteration 2000, lr = 0.00872196
I0914 16:59:29.685871 6664 solver.cpp:195] Iteration 2100, loss = 0.0650198
I0914 16:59:29.685910 6664 solver.cpp:365] Iteration 2100, lr = 0.00866784
I0914 16:59:32.992691 6664 solver.cpp:195] Iteration 2200, loss = 0.093393
I0914 16:59:32.992728 6664 solver.cpp:365] Iteration 2200, lr = 0.0086145
I0914 16:59:36.299144 6664 solver.cpp:195] Iteration 2300, loss = 0.26714
I0914 16:59:36.299181 6664 solver.cpp:365] Iteration 2300, lr = 0.00856192
I0914 16:59:39.616719 6664 solver.cpp:195] Iteration 2400, loss = 0.0679288
I0914 16:59:39.616758 6664 solver.cpp:365] Iteration 2400, lr = 0.00851008
I0914 16:59:42.887025 6664 solver.cpp:232] Iteration 2500, Testing net (#0)
I0914 16:59:44.683997 6664 solver.cpp:270] Test score #0: 0.1465
I0914 16:59:44.684036 6664 solver.cpp:270] Test score #1: 68.749
I0914 16:59:44.697453 6664 solver.cpp:195] Iteration 2500, loss = 0.0988776
I0914 16:59:44.697479 6664 solver.cpp:365] Iteration 2500, lr = 0.00845897
I0914 16:59:48.000120 6664 solver.cpp:195] Iteration 2600, loss = 0.239456
I0914 16:59:48.000157 6664 solver.cpp:365] Iteration 2600, lr = 0.00840857
I0914 16:59:51.499128 6664 solver.cpp:195] Iteration 2700, loss = 0.0601836
I0914 16:59:51.499166 6664 solver.cpp:365] Iteration 2700, lr = 0.00835886
I0914 16:59:54.989990 6664 solver.cpp:195] Iteration 2800, loss = 0.0299219
I0914 16:59:54.990030 6664 solver.cpp:365] Iteration 2800, lr = 0.00830984
I0914 16:59:58.334486 6664 solver.cpp:195] Iteration 2900, loss = 0.187015
I0914 16:59:58.334524 6664 solver.cpp:365] Iteration 2900, lr = 0.00826148
I0914 17:00:01.623569 6664 solver.cpp:232] Iteration 3000, Testing net (#0)
I0914 17:00:03.419610 6664 solver.cpp:270] Test score #0: 0.1486
I0914 17:00:03.419646 6664 solver.cpp:270] Test score #1: 69.3736

[shelhamer]

Stochastic pooling isn't a common choice and how to best use it in practice is an exercise for the modeler.

Please ask on caffe-users. Issues are for development discussion. Thanks!