Grid search for hyper-parameter selection (#2)

* Restructured experiments on UCI datasets -

1. Changed folder structure to remove duplicate code.
2. Added cross-validation and grid-search code for getting optimal hyperparameters: dropout_rate and tau.
3. Modified net.py file to allow passing dropout_rate and tau as parameters to the network.`

* Removed .DS_Store files

* Removed .DS_Store files

* Further refinements to code -

1. Created single experiment.py file which can be parameterised with number of hidden layers and epoch multiplier
2. Updated readme file (more updates to come)
3. Deleted previous result directories and modified tau values.

* Modified readme to include updated command line parameters

* Minor bug fixes

* Bug fix

* Even more bug fixes

* Adding experiment results for bostonHousing dataset

* Adding experiment results for concrete dataset

* Made minor changes to experiment.py file and added a shell script to run all experiments.

* Minor temporary changes

* Added code for averaging results from ensembles during both training and cross validation

* Removed ensemble prediction during grid-search to save time.

* Removed dependency on Theano. Now we can use Tensorflow background

* Added code for performing multi-round validation before choosing best validation parameters.

* Removed verbose outputs for grid search

* Added experimental results

* Added the results obtained on all datasets without using ensembling

* Removed unnecessary files and commented the experiment files

* Removed YearPredictionMSD dataset

* Updated readme file

* Cleaned up code and renamed files to better names.

* Removed ensemble code and experiments

* Simplified code in experiment.py

* Removed commented unnecessary code in net.py.

* Update readme.md

* Update readme.md

* Update readme.md

UCI_Datasets/bostonHousing/data/dropout_rates.txt

@@ -0,0 +1,4 @@
+0.005
+0.01
+0.05
+0.1

UCI_Datasets/bostonHousing/data/tau_values.txt

@@ -0,0 +1,3 @@
+0.1
+0.15
+0.2

UCI_Datasets/bostonHousing/results/log_100_xepochs_1_hidden_layers.txt

@@ -0,0 +1,3 @@
+errors 3.608957 +- 1.045561 (stddev) +- 0.233795 (std error), median 3.585592 25p 2.633538 75p 4.067161 
+MC errors 2.903623 +- 0.790853 (stddev) +- 0.176840 (std error), median 2.703667 25p 2.281909 75p 3.631598 
+lls -2.401936 +- 0.186498 (stddev) +- 0.041702 (std error), median -2.379094 25p -2.476636 75p -2.243694 

UCI_Datasets/bostonHousing/results/test_MC_rmse_100_xepochs_1_hidden_layers.txt

@@ -0,0 +1,20 @@
+2.7456764186182667
+2.6407723006975306
+2.34984308942956
+2.6616566967002213
+3.8541530268046373
+1.9960197076622652
+1.859032134705851
+3.595564842208209
+3.7396959501442506
+4.973758481199966
+3.7574957466443437
+2.2472323265570355
+2.2934673512640074
+2.983380964211
+2.1599913796488086
+2.0506266556970996
+2.8071661038842346
+2.8964619205003292
+2.627675944969965
+3.832783478660309

UCI_Datasets/bostonHousing/results/test_ll_100_xepochs_1_hidden_layers.txt

@@ -0,0 +1,20 @@
+-2.443888536053014
+-2.4483468750030273
+-2.2449695537209813
+-2.346866850464276
+-2.7346083521004574
+-2.1788291741009265
+-2.1446279770561327
+-2.4841174240149497
+-2.634877140654101
+-2.850389294755875
+-2.6083354898012274
+-2.1627209577135114
+-2.2995008226440277
+-2.3855348721884617
+-2.239866388395959
+-2.2248633555088224
+-2.3823056896606287
+-2.375882311866735
+-2.3740385828094412
+-2.474141878780316

UCI_Datasets/bostonHousing/results/test_rmse_100_xepochs_1_hidden_layers.txt

@@ -0,0 +1,20 @@
+2.412141919771055
+3.7836351423065477
+2.595727158600059
+2.912478678093925
+4.039550827619564
+2.5789601550917545
+2.3763641816147967
+3.742575160512066
+4.1499919582623805
+5.34692611488699
+4.931053820625412
+2.5442012497236575
+2.888930346407518
+4.188765680575889
+2.646142035982407
+3.6786934800001188
+4.001861241636697
+3.492489630196094
+3.4437995813421782
+6.42485933997255

UCI_Datasets/bostonHousing/results/test_tau_100_xepochs_1_hidden_layers.txt

@@ -0,0 +1,20 @@
+0.1
+0.1
+0.2
+0.2
+0.2
+0.2
+0.2
+0.2
+0.2
+0.2
+0.2
+0.2
+0.15
+0.2
+0.15
+0.2
+0.2
+0.2
+0.15
+0.2

UCI_Datasets/bostonHousing/results/validation_MC_rmse_100_xepochs_1_hidden_layers.txt

@@ -0,0 +1,240 @@
+Dropout_Rate: 0.005 Tau: 0.1 :: 3.146571329088136
+Dropout_Rate: 0.005 Tau: 0.15 :: 3.4312186398337343
+Dropout_Rate: 0.005 Tau: 0.2 :: 3.3817132642163394
+Dropout_Rate: 0.01 Tau: 0.1 :: 3.3521768375084107
+Dropout_Rate: 0.01 Tau: 0.15 :: 3.402403648613082
+Dropout_Rate: 0.01 Tau: 0.2 :: 3.3401508926291523
+Dropout_Rate: 0.05 Tau: 0.1 :: 3.581206352347551
+Dropout_Rate: 0.05 Tau: 0.15 :: 3.5681461688304372
+Dropout_Rate: 0.05 Tau: 0.2 :: 3.3953694694741023
+Dropout_Rate: 0.1 Tau: 0.1 :: 3.7295371690982897
+Dropout_Rate: 0.1 Tau: 0.15 :: 3.7666570576381284
+Dropout_Rate: 0.1 Tau: 0.2 :: 3.782370361578408
+Dropout_Rate: 0.005 Tau: 0.1 :: 4.024664290575444
+Dropout_Rate: 0.005 Tau: 0.15 :: 4.253110167005959
+Dropout_Rate: 0.005 Tau: 0.2 :: 4.14877679244971
+Dropout_Rate: 0.01 Tau: 0.1 :: 4.237389471240054
+Dropout_Rate: 0.01 Tau: 0.15 :: 4.010900772345622
+Dropout_Rate: 0.01 Tau: 0.2 :: 4.45144744375252
+Dropout_Rate: 0.05 Tau: 0.1 :: 3.9997869954068794
+Dropout_Rate: 0.05 Tau: 0.15 :: 4.1382020652469285
+Dropout_Rate: 0.05 Tau: 0.2 :: 4.007432271984684
+Dropout_Rate: 0.1 Tau: 0.1 :: 4.036510032805366
+Dropout_Rate: 0.1 Tau: 0.15 :: 4.036604736645819
+Dropout_Rate: 0.1 Tau: 0.2 :: 4.012423020908353
+Dropout_Rate: 0.005 Tau: 0.1 :: 2.972128854274075
+Dropout_Rate: 0.005 Tau: 0.15 :: 2.8548551637762065
+Dropout_Rate: 0.005 Tau: 0.2 :: 3.1787615654485517
+Dropout_Rate: 0.01 Tau: 0.1 :: 3.0540832701581233
+Dropout_Rate: 0.01 Tau: 0.15 :: 3.242006939147671
+Dropout_Rate: 0.01 Tau: 0.2 :: 2.9672665038460844
+Dropout_Rate: 0.05 Tau: 0.1 :: 2.6266710704597136
+Dropout_Rate: 0.05 Tau: 0.15 :: 2.396560623213283
+Dropout_Rate: 0.05 Tau: 0.2 :: 2.6245269585828486
+Dropout_Rate: 0.1 Tau: 0.1 :: 2.5594568142662766
+Dropout_Rate: 0.1 Tau: 0.15 :: 2.4292778027879995
+Dropout_Rate: 0.1 Tau: 0.2 :: 2.63287560987256
+Dropout_Rate: 0.005 Tau: 0.1 :: 2.7593600085005923
+Dropout_Rate: 0.005 Tau: 0.15 :: 2.81854400220677
+Dropout_Rate: 0.005 Tau: 0.2 :: 2.688969498641645
+Dropout_Rate: 0.01 Tau: 0.1 :: 2.239483803603013
+Dropout_Rate: 0.01 Tau: 0.15 :: 2.425628032522589
+Dropout_Rate: 0.01 Tau: 0.2 :: 2.6160560966432063
+Dropout_Rate: 0.05 Tau: 0.1 :: 2.167021680504325
+Dropout_Rate: 0.05 Tau: 0.15 :: 2.4386070306189382
+Dropout_Rate: 0.05 Tau: 0.2 :: 2.019989327808194
+Dropout_Rate: 0.1 Tau: 0.1 :: 2.34962211349184
+Dropout_Rate: 0.1 Tau: 0.15 :: 2.369230595908994
+Dropout_Rate: 0.1 Tau: 0.2 :: 2.5298645993888242
+Dropout_Rate: 0.005 Tau: 0.1 :: 2.6012079210674695
+Dropout_Rate: 0.005 Tau: 0.15 :: 2.3711949341268097
+Dropout_Rate: 0.005 Tau: 0.2 :: 2.393279373052826
+Dropout_Rate: 0.01 Tau: 0.1 :: 2.435416826909086
+Dropout_Rate: 0.01 Tau: 0.15 :: 2.280939260552221
+Dropout_Rate: 0.01 Tau: 0.2 :: 2.6227240796429805
+Dropout_Rate: 0.05 Tau: 0.1 :: 2.3055204277273904
+Dropout_Rate: 0.05 Tau: 0.15 :: 2.6331862911498827
+Dropout_Rate: 0.05 Tau: 0.2 :: 2.308283610188204
+Dropout_Rate: 0.1 Tau: 0.1 :: 2.4070839337634267
+Dropout_Rate: 0.1 Tau: 0.15 :: 2.457226062306401
+Dropout_Rate: 0.1 Tau: 0.2 :: 2.503068735606781
+Dropout_Rate: 0.005 Tau: 0.1 :: 3.6694922992496206
+Dropout_Rate: 0.005 Tau: 0.15 :: 3.6126738027815657
+Dropout_Rate: 0.005 Tau: 0.2 :: 4.022929103251174
+Dropout_Rate: 0.01 Tau: 0.1 :: 3.9819533233567475
+Dropout_Rate: 0.01 Tau: 0.15 :: 3.7668310333536192
+Dropout_Rate: 0.01 Tau: 0.2 :: 3.77451624763935
+Dropout_Rate: 0.05 Tau: 0.1 :: 3.5271264168869565
+Dropout_Rate: 0.05 Tau: 0.15 :: 3.551580679378907
+Dropout_Rate: 0.05 Tau: 0.2 :: 3.3463717000825075
+Dropout_Rate: 0.1 Tau: 0.1 :: 3.372456868643344
+Dropout_Rate: 0.1 Tau: 0.15 :: 3.403141395519452
+Dropout_Rate: 0.1 Tau: 0.2 :: 3.520041718790112
+Dropout_Rate: 0.005 Tau: 0.1 :: 3.62885019733333
+Dropout_Rate: 0.005 Tau: 0.15 :: 3.607026862967718
+Dropout_Rate: 0.005 Tau: 0.2 :: 3.585850709821047
+Dropout_Rate: 0.01 Tau: 0.1 :: 3.8630377093393746
+Dropout_Rate: 0.01 Tau: 0.15 :: 3.7121308145696443
+Dropout_Rate: 0.01 Tau: 0.2 :: 3.7038756709090292
+Dropout_Rate: 0.05 Tau: 0.1 :: 3.819860014946427
+Dropout_Rate: 0.05 Tau: 0.15 :: 3.507045544032335
+Dropout_Rate: 0.05 Tau: 0.2 :: 3.41294876864793
+Dropout_Rate: 0.1 Tau: 0.1 :: 3.7926593633558423
+Dropout_Rate: 0.1 Tau: 0.15 :: 3.6900182981698073
+Dropout_Rate: 0.1 Tau: 0.2 :: 3.7402029158313552
+Dropout_Rate: 0.005 Tau: 0.1 :: 2.854685630279463
+Dropout_Rate: 0.005 Tau: 0.15 :: 3.0194177506912183
+Dropout_Rate: 0.005 Tau: 0.2 :: 2.7737115430830275
+Dropout_Rate: 0.01 Tau: 0.1 :: 2.8192338580918213
+Dropout_Rate: 0.01 Tau: 0.15 :: 2.797059634423282
+Dropout_Rate: 0.01 Tau: 0.2 :: 2.98165916291536
+Dropout_Rate: 0.05 Tau: 0.1 :: 2.888304538026146
+Dropout_Rate: 0.05 Tau: 0.15 :: 3.0645617894455923
+Dropout_Rate: 0.05 Tau: 0.2 :: 2.679098037230364
+Dropout_Rate: 0.1 Tau: 0.1 :: 2.848428841753909
+Dropout_Rate: 0.1 Tau: 0.15 :: 3.0454640986191936
+Dropout_Rate: 0.1 Tau: 0.2 :: 3.0826913003960374
+Dropout_Rate: 0.005 Tau: 0.1 :: 3.311852297615122
+Dropout_Rate: 0.005 Tau: 0.15 :: 2.807643447498327
+Dropout_Rate: 0.005 Tau: 0.2 :: 2.81453672024952
+Dropout_Rate: 0.01 Tau: 0.1 :: 3.093087458692496
+Dropout_Rate: 0.01 Tau: 0.15 :: 2.979085241537916
+Dropout_Rate: 0.01 Tau: 0.2 :: 2.947252751499548
+Dropout_Rate: 0.05 Tau: 0.1 :: 2.847140914623859
+Dropout_Rate: 0.05 Tau: 0.15 :: 2.9066806684374575
+Dropout_Rate: 0.05 Tau: 0.2 :: 2.8210730495694363
+Dropout_Rate: 0.1 Tau: 0.1 :: 2.824362150731743
+Dropout_Rate: 0.1 Tau: 0.15 :: 2.7881043172054576
+Dropout_Rate: 0.1 Tau: 0.2 :: 2.7878651794545433
+Dropout_Rate: 0.005 Tau: 0.1 :: 2.929409035025775
+Dropout_Rate: 0.005 Tau: 0.15 :: 2.723781875097023
+Dropout_Rate: 0.005 Tau: 0.2 :: 2.7947276402954664
+Dropout_Rate: 0.01 Tau: 0.1 :: 2.5578920642062575
+Dropout_Rate: 0.01 Tau: 0.15 :: 2.684686743102932
+Dropout_Rate: 0.01 Tau: 0.2 :: 2.8427964359407167
+Dropout_Rate: 0.05 Tau: 0.1 :: 2.629239029502718
+Dropout_Rate: 0.05 Tau: 0.15 :: 2.7254453019578366
+Dropout_Rate: 0.05 Tau: 0.2 :: 2.468696968086069
+Dropout_Rate: 0.1 Tau: 0.1 :: 2.693584391055212
+Dropout_Rate: 0.1 Tau: 0.15 :: 2.7526940863401204
+Dropout_Rate: 0.1 Tau: 0.2 :: 2.6361140070165217
+Dropout_Rate: 0.005 Tau: 0.1 :: 2.980044036495408
+Dropout_Rate: 0.005 Tau: 0.15 :: 2.3489646429564464
+Dropout_Rate: 0.005 Tau: 0.2 :: 2.435162722309275
+Dropout_Rate: 0.01 Tau: 0.1 :: 2.3266086610205607
+Dropout_Rate: 0.01 Tau: 0.15 :: 2.332764913468748
+Dropout_Rate: 0.01 Tau: 0.2 :: 2.246633983008252
+Dropout_Rate: 0.05 Tau: 0.1 :: 2.097304567337126
+Dropout_Rate: 0.05 Tau: 0.15 :: 2.330469607381292
+Dropout_Rate: 0.05 Tau: 0.2 :: 2.130524671517193
+Dropout_Rate: 0.1 Tau: 0.1 :: 2.271392414806502
+Dropout_Rate: 0.1 Tau: 0.15 :: 2.115217406428026
+Dropout_Rate: 0.1 Tau: 0.2 :: 2.1833944708443997
+Dropout_Rate: 0.005 Tau: 0.1 :: 2.8606739047914336
+Dropout_Rate: 0.005 Tau: 0.15 :: 2.6238360140105184
+Dropout_Rate: 0.005 Tau: 0.2 :: 2.75671136024382
+Dropout_Rate: 0.01 Tau: 0.1 :: 2.4718197392640504
+Dropout_Rate: 0.01 Tau: 0.15 :: 2.726767631729333
+Dropout_Rate: 0.01 Tau: 0.2 :: 2.4962858936813532
+Dropout_Rate: 0.05 Tau: 0.1 :: 2.3477298597628833
+Dropout_Rate: 0.05 Tau: 0.15 :: 2.2711335569422118
+Dropout_Rate: 0.05 Tau: 0.2 :: 2.441163571560864
+Dropout_Rate: 0.1 Tau: 0.1 :: 2.432457701208484
+Dropout_Rate: 0.1 Tau: 0.15 :: 2.3335762827183686
+Dropout_Rate: 0.1 Tau: 0.2 :: 2.3212478550060616
+Dropout_Rate: 0.005 Tau: 0.1 :: 4.30810932969273
+Dropout_Rate: 0.005 Tau: 0.15 :: 4.346110793277847
+Dropout_Rate: 0.005 Tau: 0.2 :: 3.999241373366635
+Dropout_Rate: 0.01 Tau: 0.1 :: 4.189696027061892
+Dropout_Rate: 0.01 Tau: 0.15 :: 3.9193960856099035
+Dropout_Rate: 0.01 Tau: 0.2 :: 4.248679626606125
+Dropout_Rate: 0.05 Tau: 0.1 :: 4.301378084632108
+Dropout_Rate: 0.05 Tau: 0.15 :: 4.172261727650705
+Dropout_Rate: 0.05 Tau: 0.2 :: 4.3599371677794565
+Dropout_Rate: 0.1 Tau: 0.1 :: 4.599935545611783
+Dropout_Rate: 0.1 Tau: 0.15 :: 4.851069549583766
+Dropout_Rate: 0.1 Tau: 0.2 :: 4.564448334846367
+Dropout_Rate: 0.005 Tau: 0.1 :: 4.4779874492531535
+Dropout_Rate: 0.005 Tau: 0.15 :: 3.949478889345842
+Dropout_Rate: 0.005 Tau: 0.2 :: 4.416552795544607
+Dropout_Rate: 0.01 Tau: 0.1 :: 4.473788962173547
+Dropout_Rate: 0.01 Tau: 0.15 :: 3.8833465809455814
+Dropout_Rate: 0.01 Tau: 0.2 :: 3.713854254195298
+Dropout_Rate: 0.05 Tau: 0.1 :: 3.186750501186634
+Dropout_Rate: 0.05 Tau: 0.15 :: 3.7907262499591963
+Dropout_Rate: 0.05 Tau: 0.2 :: 3.5518033699764486
+Dropout_Rate: 0.1 Tau: 0.1 :: 3.720868596229488
+Dropout_Rate: 0.1 Tau: 0.15 :: 3.722786689891069
+Dropout_Rate: 0.1 Tau: 0.2 :: 3.377250414640822
+Dropout_Rate: 0.005 Tau: 0.1 :: 3.120192151064065
+Dropout_Rate: 0.005 Tau: 0.15 :: 3.3628971490029063
+Dropout_Rate: 0.005 Tau: 0.2 :: 3.2633661599926906
+Dropout_Rate: 0.01 Tau: 0.1 :: 3.2980657545329777
+Dropout_Rate: 0.01 Tau: 0.15 :: 3.2274011489349896
+Dropout_Rate: 0.01 Tau: 0.2 :: 3.1984611446996687
+Dropout_Rate: 0.05 Tau: 0.1 :: 3.4099332709329477
+Dropout_Rate: 0.05 Tau: 0.15 :: 3.211113782240537
+Dropout_Rate: 0.05 Tau: 0.2 :: 3.271567004226289
+Dropout_Rate: 0.1 Tau: 0.1 :: 3.4682236934971202
+Dropout_Rate: 0.1 Tau: 0.15 :: 3.6890786616202935
+Dropout_Rate: 0.1 Tau: 0.2 :: 3.4756248126587685
+Dropout_Rate: 0.005 Tau: 0.1 :: 2.8198706255517734
+Dropout_Rate: 0.005 Tau: 0.15 :: 2.761247980311201
+Dropout_Rate: 0.005 Tau: 0.2 :: 3.1197657000713344
+Dropout_Rate: 0.01 Tau: 0.1 :: 3.0891816835584054
+Dropout_Rate: 0.01 Tau: 0.15 :: 2.7569280001553684
+Dropout_Rate: 0.01 Tau: 0.2 :: 2.937224972251622
+Dropout_Rate: 0.05 Tau: 0.1 :: 2.9026250435892273
+Dropout_Rate: 0.05 Tau: 0.15 :: 2.8614201199541225
+Dropout_Rate: 0.05 Tau: 0.2 :: 3.1206872857590113
+Dropout_Rate: 0.1 Tau: 0.1 :: 3.0597294599202836
+Dropout_Rate: 0.1 Tau: 0.15 :: 3.3018214582561365
+Dropout_Rate: 0.1 Tau: 0.2 :: 3.0186462743394387
+Dropout_Rate: 0.005 Tau: 0.1 :: 3.3440661134643195
+Dropout_Rate: 0.005 Tau: 0.15 :: 3.8275705598871923
+Dropout_Rate: 0.005 Tau: 0.2 :: 3.3769526052784493
+Dropout_Rate: 0.01 Tau: 0.1 :: 3.6511602841438346
+Dropout_Rate: 0.01 Tau: 0.15 :: 3.8634843050860312
+Dropout_Rate: 0.01 Tau: 0.2 :: 3.400498333090723
+Dropout_Rate: 0.05 Tau: 0.1 :: 2.9745553513296024
+Dropout_Rate: 0.05 Tau: 0.15 :: 3.078209750844501
+Dropout_Rate: 0.05 Tau: 0.2 :: 3.2082564217502885
+Dropout_Rate: 0.1 Tau: 0.1 :: 3.1462503392642494
+Dropout_Rate: 0.1 Tau: 0.15 :: 3.008394426669874
+Dropout_Rate: 0.1 Tau: 0.2 :: 3.061077670461937
+Dropout_Rate: 0.005 Tau: 0.1 :: 3.4501877278374273
+Dropout_Rate: 0.005 Tau: 0.15 :: 2.8944979645913547
+Dropout_Rate: 0.005 Tau: 0.2 :: 3.103753419669938
+Dropout_Rate: 0.01 Tau: 0.1 :: 2.9802651098781303
+Dropout_Rate: 0.01 Tau: 0.15 :: 2.81185142088628
+Dropout_Rate: 0.01 Tau: 0.2 :: 2.943762835689973
+Dropout_Rate: 0.05 Tau: 0.1 :: 2.7086280804778156
+Dropout_Rate: 0.05 Tau: 0.15 :: 2.6427773032889195
+Dropout_Rate: 0.05 Tau: 0.2 :: 2.7757984381342187
+Dropout_Rate: 0.1 Tau: 0.1 :: 2.7976884593155735
+Dropout_Rate: 0.1 Tau: 0.15 :: 2.68609004275903
+Dropout_Rate: 0.1 Tau: 0.2 :: 2.731704206337393
+Dropout_Rate: 0.005 Tau: 0.1 :: 2.5818210738085936
+Dropout_Rate: 0.005 Tau: 0.15 :: 2.846190515541407
+Dropout_Rate: 0.005 Tau: 0.2 :: 2.949155902507361
+Dropout_Rate: 0.01 Tau: 0.1 :: 2.720339869916656
+Dropout_Rate: 0.01 Tau: 0.15 :: 2.840850847786197
+Dropout_Rate: 0.01 Tau: 0.2 :: 2.6878962032500184
+Dropout_Rate: 0.05 Tau: 0.1 :: 2.680243708156178
+Dropout_Rate: 0.05 Tau: 0.15 :: 2.71053797144085
+Dropout_Rate: 0.05 Tau: 0.2 :: 2.8201206613207996
+Dropout_Rate: 0.1 Tau: 0.1 :: 2.8813373051898408
+Dropout_Rate: 0.1 Tau: 0.15 :: 2.761538707805824
+Dropout_Rate: 0.1 Tau: 0.2 :: 2.794350319094386
+Dropout_Rate: 0.005 Tau: 0.1 :: 4.477175610865179
+Dropout_Rate: 0.005 Tau: 0.15 :: 3.7992345090886115
+Dropout_Rate: 0.005 Tau: 0.2 :: 3.5975241530290627
+Dropout_Rate: 0.01 Tau: 0.1 :: 3.9797899794228675
+Dropout_Rate: 0.01 Tau: 0.15 :: 3.530455859080459
+Dropout_Rate: 0.01 Tau: 0.2 :: 3.823542002872116
+Dropout_Rate: 0.05 Tau: 0.1 :: 3.4351836581455095
+Dropout_Rate: 0.05 Tau: 0.15 :: 3.974082501750926
+Dropout_Rate: 0.05 Tau: 0.2 :: 3.700460859627545
+Dropout_Rate: 0.1 Tau: 0.1 :: 3.548420306278518
+Dropout_Rate: 0.1 Tau: 0.15 :: 4.16716690057308
+Dropout_Rate: 0.1 Tau: 0.2 :: 3.8643710496141277