Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to reproduce the good results (Acc. 56.7%) of ModelNet to ScanNet without SPST. #4

Open
Jhonve opened this issue Nov 5, 2021 · 0 comments

Comments

@Jhonve
Copy link

Jhonve commented Nov 5, 2021

Using the first version of your code (using batch normalization and TransNorm), I got the results of

02-11-21 19:21:08: Test - Target 0, acc: 0.5415, avg acc: 0.4271, cls loss: 4.3767
02-11-21 19:21:08: target test accuracy: 0.5415, target test loss: 4.3767
02-11-21 19:21:08: Test confusion matrix:
02-11-21 19:21:08: 
[[ 14   2   2   0   0   1   2   0   1   4]
 [  0  23   5   0  12   5   3   1   3  33]
 [  0   4  61   0  15  14   8  15   2  27]
 [  0   0  47   4   6  18  39  16   3  16]
 [  0  19  55   1 561  31  32  44   2  56]
 [  0   1   8   3   4   8   0  12   1   4]
 [  0   0  36   0   0   3  15   6   0   1]
 [  0   0   0   0   2   0   0  23   0   0]
 [  2  15   3   0  42  11   3   8  31  19]
 [  0  12  12   3  14  24   1  13   4 218]]

And then, after you updated your code, I got the results of

04-11-21 02:37:22: Test - Target 0, acc: 0.5517, avg acc: 0.4728, cls loss: 4.2702
04-11-21 02:37:22: target test accuracy: 0.5517, target test loss: 4.9488
04-11-21 02:37:22: Test confusion matrix:
04-11-21 02:37:22: 
[[ 12   3   1   0   1   0   0   0   5   4]
 [  0  45   1   1   3   0   5   0   2  28]
 [  0  11  53   0  22  14   6   3   5  32]
 [  0  12  24   3  10  10  53   2   1  34]
 [  2  67  23   1 519  28  64  10  18  69]
 [  0   1   4   3   6  18   2   0   0   7]
 [  0   2  19   0   0   4  27   1   1   7]
 [  0   0   1   0   4   2   0  17   1   0]
 [  5  20   0   0  25   6   2   2  50  24]
 [  3  29   7   2  10   9   2   3   4 232]]

The accuracy of 0.5517 is different from your result of 56.7% (ModelNet to ScanNet without SPST) in Table 1 in your paper and the average accuracy of 10 classes is 0.4728, right? However, the reported accuracy in your paper is 52.2%.
Are there any problems with your current code or any different details from your paper?
Thanks a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant