Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Semi-SL Instance Segmentation #2444

Merged
merged 46 commits into from
Sep 6, 2023
Merged

Add Semi-SL Instance Segmentation #2444

merged 46 commits into from
Sep 6, 2023

Conversation

kprokofi
Copy link
Collaborator

@kprokofi kprokofi commented Aug 21, 2023

Summary

This PR includes Semi-SL Instance Segmentation solution based on the Mean Teacher algorithm after conducted PoC .

How to test

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added e2e tests for validation.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

@kprokofi kprokofi added the ALGO Any changes in OTX Algo Tasks implementation label Aug 21, 2023
@github-actions github-actions bot added the API Any changes in OTX API label Aug 21, 2023
@goodsong81 goodsong81 requested a review from eugene123tw August 21, 2023 09:44
@sovrasov sovrasov added this to the 1.5.0 milestone Aug 21, 2023
@codecov
Copy link

codecov bot commented Aug 22, 2023

Codecov Report

Patch coverage: 71.94% and project coverage change: +0.16% 🎉

Comparison is base (a6b809e) 81.18% compared to head (dc245d1) 81.35%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2444      +/-   ##
===========================================
+ Coverage    81.18%   81.35%   +0.16%     
===========================================
  Files          516      516              
  Lines        37610    37686      +76     
===========================================
+ Hits         30534    30658     +124     
+ Misses        7076     7028      -48     
Flag Coverage Δ
py310 81.34% <71.94%> (+0.16%) ⬆️
py38 81.33% <71.94%> (+0.16%) ⬆️
py39 81.33% <71.94%> (+0.16%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
...ion/adapters/mmdet/models/heads/custom_roi_head.py 36.06% <ø> (ø)
src/otx/core/data/adapter/base_dataset_adapter.py 90.05% <ø> (ø)
...ommon/adapters/mmcv/hooks/custom_model_ema_hook.py 75.94% <7.69%> (-13.45%) ⬇️
...ms/common/adapters/mmcv/hooks/mean_teacher_hook.py 35.29% <33.33%> (ø)
.../common/adapters/mmcv/hooks/dual_model_ema_hook.py 76.71% <36.36%> (-1.86%) ⬇️
...dapters/mmdet/models/heads/custom_fcn_mask_head.py 32.43% <50.00%> (+1.00%) ⬆️
...on/adapters/mmdet/models/detectors/mean_teacher.py 79.14% <79.14%> (ø)
...n/adapters/mmdet/models/losses/cross_focal_loss.py 89.55% <81.81%> (-3.79%) ⬇️
...rc/otx/algorithms/detection/adapters/mmdet/task.py 84.71% <100.00%> (ø)
src/otx/api/entities/shapes/polygon.py 97.18% <100.00%> (-2.82%) ⬇️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions github-actions bot added the TEST Any changes in tests label Aug 22, 2023
@kprokofi
Copy link
Collaborator Author

Below the pivot table with PoC and comparison with other candidates inside OTX

Method Tiny Pascal CityScapes 8% old annotations (20+ classes)** Sea garbage segmentation CityScapes 9% new annotations (8 classes WGISD TACO** Mean metric
Baseline (supervised) 22.13 / 24.94 27.5/52.5 25.63 / 44.82 34.81 / 52.69 18.49 / 25.25 6.4 / 16.6 25.26 / 36.92
Enhanced Mean Teacher 24.84 / 27.93 33.23 / 56.51 36.78 / 51.32 40.08 / 57.12 40.01 / 45.26 8.5 / 19.21 35.43 / 45.41 (+10.17 / + 8.49)
MT + Noisy Boundaries Head 23.66 / 27.17   35.47 / 50.87 39.84 / 57.16 39.56 / 44.41   34.63 / 44.9
MT + IoU masks filtering 24.48 / 27.88   35.75 / 49.65 39.84 / 56.15 39.78 / 45.08   34.96 / 44.69
MT + Unbiased Teacher V2 24.32 / 26.59   35.30 / 49.38 39.67 / 56.82 39.36 / 44.88   34.66 / 44.42
MT + PseCo 22.91 / 25.1 30.43 / 53.56 32.89 / 48.38 38.72 / 55.97 38.38 / 42.58   33.23 / 43.01

@kprokofi
Copy link
Collaborator Author

There are mmdet metrics:

Method Tiny Pascal Sea garbage segmentation CityScapes 9% new annotations (8 classes WGISD Mean metric
Baseline (supervised) 58.70 / 59.50 28.30 / 28.70 45.60 / 48.30 25.70 / 24.70 39.57 / 40.30
Enhanced Mean Teacher best 66.80 / 70.20 40.10 / 39.30 50.90 / 53.10 50.30 / 48.70 52.03 / 52.83 (+12.46 / +12.53)

@sungmanc
Copy link
Contributor

sungmanc commented Sep 4, 2023

Could you add a summary for this PR to help to understand the objective and contents of this PR before reviewing?

@kprokofi
Copy link
Collaborator Author

kprokofi commented Sep 4, 2023

This PR includes Semi-SL Instance Segmentation solution based on the Mean Teacher algorithm after conducted PoC .

I added it in the description
Basically, this PR includes Semi-SL Instance Segmentation solution based on the Mean Teacher algorithm.

jaegukhyun
jaegukhyun previously approved these changes Sep 5, 2023
Copy link
Contributor

@sungmanc sungmanc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the effort, I left few comments

Copy link
Contributor

@goodsong81 goodsong81 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Unbiased Teacher" -> general "Mean Teacher" scheme looks good to me.
Hope this would benefit both of det & iseg.
I've left a few minor change requests & qeustions.
Thank you for the great work!

@kprokofi kprokofi merged commit e28026b into develop Sep 6, 2023
@kprokofi kprokofi deleted the kp/semisl_instance_seg branch September 6, 2023 09:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ALGO Any changes in OTX Algo Tasks implementation API Any changes in OTX API TEST Any changes in tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants