Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix mIoU calculatiton range #471

Merged
merged 3 commits into from
Apr 14, 2021
Merged

Conversation

clownrat6
Copy link
Contributor

@clownrat6 clownrat6 commented Apr 11, 2021

  1. Fix the bug about torch.histc function. When setting arg:max same as arg:bins for torch.histc, some channel of mIoU may be nan.
  2. Modify tests/test_metrics.py. Add some abnormal num_classes test cases.

Pascal Context Dataset

Single-scale, No flip, mIoU test result:

model num classes Lr schd after fix before fix readme
HRNetV2-W48(new) 59 40000 50.37 50.13 None
HRNetV2-w48(new) 59 80000 51.12 50.85 None
HRNetV2-W48(new) 60 40000 45.55 45.55 45.14
HRNetV2-W48(old) 60 40000 45.14 45.14 45.14

before fix:

  • keyboard=nan

Cityscapes Dataset

Single-scale, No flip, mIoU test result:

model Crop Size Lr schd after fix before fix readme
HRNetV2-W48(old) 512x1024 40000 78.52 78.52 78.48
HRNetV2-W48(old) 512x1024 80000 79.98 79.98 79.93

ADE20K Dataset

Single-scale, No flip, mIoU test result:

model Crop Size Lr schd after fix before fix readme
HRNetV2-W48(old) 512x512 80000 41.9 41.76 41.9
HRNetV2-W48(old) 512x512 160000 42.02 41.82 42.02

before fix:

  • fireplace=nan;
  • blind=nan;
  • television receiver=nan;
  • botle=nan;
  • cradle=nan;
  • animal=nan;

@codecov
Copy link

codecov bot commented Apr 11, 2021

Codecov Report

Merging #471 (031883a) into master (e0e985f) will decrease coverage by 0.07%.
The diff coverage is 16.66%.

❗ Current head 031883a differs from pull request most recent head fb73614. Consider uploading reports for the commit fb73614 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master     #471      +/-   ##
==========================================
- Coverage   86.58%   86.50%   -0.08%     
==========================================
  Files          97       97              
  Lines        4964     4966       +2     
  Branches      807      806       -1     
==========================================
- Hits         4298     4296       -2     
- Misses        514      517       +3     
- Partials      152      153       +1     
Flag Coverage Δ
unittests 86.50% <16.66%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmseg/apis/inference.py 63.79% <0.00%> (-2.28%) ⬇️
mmseg/core/evaluation/metrics.py 86.56% <ø> (ø)
mmseg/ops/wrappers.py 80.76% <ø> (-1.99%) ⬇️
mmseg/models/segmentors/encoder_decoder.py 90.84% <33.33%> (-1.16%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e0e985f...fb73614. Read the comment docs.

@xvjiarui xvjiarui changed the title Fix fence(IoU) = 0 when training on PascalContextDataset59; Fix mIoU calculate range Apr 11, 2021
@xvjiarui xvjiarui changed the title Fix mIoU calculate range Fix mIoU calculatiton range Apr 11, 2021
@xvjiarui
Copy link
Collaborator

Could we add some test cases for this?

@xvjiarui
Copy link
Collaborator

For example, we may construct some test cases that fail in the previous version.

@xvjiarui xvjiarui requested a review from xiexinch April 11, 2021 16:38
@clownrat6
Copy link
Contributor Author

We can use trained model file to test on both two version IoU calculation function.
The trained model file BaiduNetDisk(Extract Code: 0l69)

tests/test_metrics.py Outdated Show resolved Hide resolved
tests/test_metrics.py Outdated Show resolved Hide resolved
@xvjiarui
Copy link
Collaborator

How does this PR affect mIoU of cityscapes and ADE20k?

@clownrat6
Copy link
Contributor Author

How does this PR affect mIoU of cityscapes and ADE20k?

mIoU of cityscapes has a little bit wave and doesn't contain nan value. But mIoU of ADE20K has six nan classes.

Co-authored-by: Jerry Jiarui XU <xvjiarui0826@gmail.com>
@xvjiarui xvjiarui merged commit fb24bf5 into open-mmlab:master Apr 14, 2021
bowenroom pushed a commit to bowenroom/mmsegmentation that referenced this pull request Feb 25, 2022
* Fix fence(IoU) = 0 when training on PascalContextDataset59;

* Add a test case in test_metrics() of tests/test_metrics.py to test the bug caused by torch.histc;

* Update tests/test_metrics.py

Co-authored-by: Jerry Jiarui XU <xvjiarui0826@gmail.com>

Co-authored-by: Jerry Jiarui XU <xvjiarui0826@gmail.com>
aravind-h-v pushed a commit to aravind-h-v/mmsegmentation that referenced this pull request Mar 27, 2023
@OpenMMLab-Coodinator
Copy link

Hi @Sennnnn !First of all, we want to express our gratitude for your significant PR in this project. Your contribution is highly appreciated, and we are grateful for your efforts in helping improve this open-source project during your personal time. We believe that many developers will benefit from your PR.

We would also like to invite you to join our Special Interest Group (SIG) private channel on Discord, where you can share your experiences, ideas, and build connections with like-minded peers. To join the SIG channel, simply message moderator— OpenMMLab on Discord or briefly share your open-source contributions in the #introductions channel and we will assist you. Look forward to seeing you there! Join us :https://discord.gg/raweFPmdzG

If you are Chinese or have WeChat,welcome to join our community on WeChat. You can add our assistant :openmmlabwx. Please add "mmsig + Github ID" as a remark when adding friends:)
Thank you again for your contribution❤

wjkim81 pushed a commit to wjkim81/mmsegmentation that referenced this pull request Dec 3, 2023
sibozhang pushed a commit to sibozhang/mmsegmentation that referenced this pull request Mar 22, 2024
* resolve comments

* update changelog

* add with_global option

* f bug

* + contiguous

* update

* add config w. context

* test with_global

* update README

* update changelog
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants