Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Support ScanNet test results submission to online benchmark #418

Merged
merged 4 commits into from
Apr 12, 2021

Conversation

Wuziyi616
Copy link
Contributor

Convert network output prediction to original label_idx. Save each mask as a txt file.

@Wuziyi616
Copy link
Contributor Author

Wuziyi616 commented Apr 7, 2021

Fix issue#414. The user needs to manually zip the saved prediction results and submit to ScanNet online benchmark.

@codecov
Copy link

codecov bot commented Apr 7, 2021

Codecov Report

Merging #418 (fbb828e) into master (e21e61e) will decrease coverage by 0.12%.
The diff coverage is 90.90%.

❗ Current head fbb828e differs from pull request most recent head e7ce09a. Consider uploading reports for the commit e7ce09a to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master     #418      +/-   ##
==========================================
- Coverage   50.45%   50.32%   -0.13%     
==========================================
  Files         184      183       -1     
  Lines       13320    13164     -156     
  Branches     2127     2094      -33     
==========================================
- Hits         6720     6625      -95     
+ Misses       6158     6104      -54     
+ Partials      442      435       -7     
Flag Coverage Δ
unittests 50.32% <90.90%> (-0.13%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmdet3d/datasets/scannet_dataset.py 94.04% <90.90%> (-1.12%) ⬇️
mmdet3d/core/visualizer/open3d_vis.py 0.00% <0.00%> (-27.95%) ⬇️
mmdet3d/datasets/sunrgbd_dataset.py 72.60% <0.00%> (-3.01%) ⬇️
mmdet3d/datasets/kitti_dataset.py 75.76% <0.00%> (-0.46%) ⬇️
mmdet3d/apis/__init__.py 100.00% <0.00%> (ø)
mmdet3d/datasets/__init__.py 100.00% <0.00%> (ø)
mmdet3d/core/visualizer/__init__.py 100.00% <0.00%> (ø)
mmdet3d/datasets/pipelines/loading.py 83.82% <0.00%> (ø)
mmdet3d/datasets/utils.py
mmdet3d/core/visualizer/show_result.py 75.00% <0.00%> (+9.82%) ⬆️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e21e61e...e7ce09a. Read the comment docs.

mmdet3d/datasets/scannet_dataset.py Outdated Show resolved Hide resolved
mmdet3d/datasets/scannet_dataset.py Outdated Show resolved Hide resolved
@Wuziyi616
Copy link
Contributor Author

Add code for test data generation.

TODO: Need to move the align step into loading pipeline in future PRs!

@Wuziyi616 Wuziyi616 changed the title [Feature] Support test results submission formatting of ScanNet dataset [Feature] Support ScanNet test results submission to online benchmark Apr 7, 2021
@Wuziyi616 Wuziyi616 linked an issue Apr 8, 2021 that may be closed by this pull request
@Wuziyi616
Copy link
Contributor Author

Resolve conflicts.

@ZwwWayne ZwwWayne merged commit a03100e into open-mmlab:master Apr 12, 2021
@Wuziyi616 Wuziyi616 deleted the scannet_test_submission branch April 12, 2021 13:21
tpoisonooo added a commit to tpoisonooo/mmdetection3d that referenced this pull request Sep 5, 2022
* docs(docs/zh_cn): add doc and link checker

* docs(REAME): update

* docs(docs/zh_cn): update

* docs(benchmark): update table

* docs(zh_cn/benchmark): update link

* CI(docs): update link check

* ci(doc): update checker

* docs(zh_cn): update

* style(ci): remove useless para

* style(ci): update

* docs(zh_cn): update

* docs(benchmark.md): fix mobilnet link error

* docs(docs/zh_cn): add doc and link checker

* docs(REAME): update

* docs(docs/zh_cn): update

* docs(benchmark): update table

* docs(zh_cn/benchmark): update link

* CI(docs): update link check

* ci(doc): update checker

* docs(zh_cn): update

* style(ci): remove useless para

* style(ci): update

* docs(zh_cn): update

* docs(benchmark.md): fix mobilnet link error

* docs(zh_cn/do_regression_test.md): rebase

* docs(docs/zh_cn): add doc and link checker

* Update README_zh-CN.md

* Update README_zh-CN.md

* Update index.rst

* Update check-doc-link.yml

* [Fix] Fix ci (open-mmlab#426)

* fix ci

* add nvidia key

* remote torch

* recover pytorch

* ci(codecov): ignore ci

* docs(zh_cn): add get_started.md

* docs(zh_cn): fix review advice

* docs(readthedocs): update

* docs(zh_CN): update

* docs(zh_CN): revert

* fix(docs): review advices

* fix(docs): review advices

* fix(docs): review

Co-authored-by: q.yao <streetyao@live.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Test results submission for ScanNet test set
3 participants