Some question about the experimental results

[pangwenfeng]

Thanks for your nice work first! I run the code without any change, but I cannot get the similar result on ShanghaiTech as the paper post, which is lower than the results shown in the paper.
The related parameters are as follows （all parameters are set as default):
Namespace(act='relu', ae_batch_size=512, ae_epochs=10, ae_fn=None, ae_lr=0.0001, ae_lr_decay=0.99, ae_optimizer='adam', ae_sched='tri', ae_test_every=20, ae_weight_decay=1e-05, alpha=0.001, ckpt_dir='data/exp_dir/stc/Dec14_1059/checkpoints/', conv_oper='sagc', data_dir='data/', dcec_batch_size=512, dcec_epochs=25, dcec_fn=None, dcec_lr=0.0008, dcec_lr_decay=0.98, dcec_optimizer='adam', dcec_sched='tri', dcec_weight_decay=1e-05, debug=False, device='cuda:0', dpmm_fn=None, dropout=0.3, exp_dir='data/exp_dir', gamma=0.6, headless=False, k_init_batch=4, k_init_downsample=1, n_clusters=10, norm_scale=0, num_transform=5, num_workers=8, optimizer='adam', patch_db=False, patch_features=False, patch_size=16, pose_path={'train': 'data/pose/training/tracked_person/', 'test': 'data/pose/testing/tracked_person/'}, pretrain_epochs=0, prop_norm_scale=0, res_batch_size=256, save_results=1, seed=14061439651021546021, seg_len=12, seg_stride=8, train_seg_conf_th=0.0, update_interval=2.0, verbose=1, vid_path={'train': 'data/training/videos/', 'test': 'data/testing/frames/'})

I run the code twice and got the AUC of 73.29%, 72.84%, respectively.

I cannot find the experiment setting details such as learning rate, batch size and sampling method in the paper, so If I want to get the results shown in the paper, what should I change to the current code? or did I make any mistakes in the reimplementation? Thank you very much!

[amirmk89]

Hi, Thank you for your question. It may depend on the environment settings, as the two results you got are also very different from one another, and in my experience results from several runs are often closer. You can also try increasing the 'train_seg_conf_th' parameter.

…

On Mon, Dec 14, 2020, 15:06 庞 ***@***.***> wrote: Thanks for your nice work first! I run the code without any change, but I cannot get the similar result on ShanghaiTech as the paper post, which is lower than the results shown in the paper. The related parameters are as follows （all parameters are set as default): Namespace(act='relu', ae_batch_size=512, ae_epochs=10, ae_fn=None, ae_lr=0.0001, ae_lr_decay=0.99, ae_optimizer='adam', ae_sched='tri', ae_test_every=20, ae_weight_decay=1e-05, alpha=0.001, ckpt_dir='data/exp_dir/stc/Dec14_1059/checkpoints/', conv_oper='sagc', data_dir='data/', dcec_batch_size=512, dcec_epochs=25, dcec_fn=None, dcec_lr=0.0008, dcec_lr_decay=0.98, dcec_optimizer='adam', dcec_sched='tri', dcec_weight_decay=1e-05, debug=False, device='cuda:0', dpmm_fn=None, dropout=0.3, exp_dir='data/exp_dir', gamma=0.6, headless=False, k_init_batch=4, k_init_downsample=1, n_clusters=10, norm_scale=0, num_transform=5, num_workers=8, optimizer='adam', patch_db=False, patch_features=False, patch_size=16, pose_path={'train': 'data/pose/training/tracked_person/', 'test': 'data/pose/testing/tracked_person/'}, pretrain_epochs=0, prop_norm_scale=0, res_batch_size=256, save_results=1, seed=14061439651021546021, seg_len=12, seg_stride=8, train_seg_conf_th=0.0, update_interval=2.0, verbose=1, vid_path={'train': 'data/training/videos/', 'test': 'data/testing/frames/'}) I run the code twice and got the AUC of 73.29%, 72.84%, respectively. I cannot find the experiment setting details such as learning rate, batch size and sampling method in the paper, so If I want to get the results shown in the paper, what should I change to the current code? Thank you very much! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#5>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADFQGYUM5G55FTM3F3JNOI3SUYETTANCNFSM4U2W2FRQ> .

[pangwenfeng]

Thanks for your reply! To ensure that I have same environment as yours, I created a new environment with the environment.txt file by conda (By the way, the command to create environment is incorrect, and correct command should be: conda create --name gpec --file environment.txt). However, I got the AUC of 73.52%, which is lower than the result in the paper. I also tried to increase the 'train_seg_conf_th' parameter, and get an AUC of 74.28%. So what else should I do? Should I set a specific value for random seed or any other parameter? Thank you very much.