Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Add Maskformer to mmdet #7212

Merged
merged 6 commits into from
Feb 22, 2022
Merged

[Feature] Add Maskformer to mmdet #7212

merged 6 commits into from
Feb 22, 2022

Conversation

chhluo
Copy link
Collaborator

@chhluo chhluo commented Feb 21, 2022 •
edited
Loading

Motivation

Add MaskFormer to mmdet. The older one (#6607 ) is too messy, so I make a new one.

Modification

- panoptic single stage segmentor
- maskformer segmentor
- maskformer head
- mask hungrarian assigner
- match cost: BinaryDiceCost, MaskFocalLossCost
- mask pseudo sampler, mask sample result (copy from  https://github.com/ZwwWayne/K-Net/blob/main/knet/det/mask_pseudo_sampler.py)
- one config files: maskformer_r50_mstrain_16x1_75e_coco.py

BC-breaking (Optional)

Use cases (Optional)

Checklist

  1. Pre-commit or other linting tools are used to fix the potential lint issues.
  2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMCls.
  4. The documentation has been modified accordingly, like docstring or example tutorials.

@chhluo chhluo requested review from AronLin and ZwwWayne February 21, 2022 13:10
@codecov
Copy link

codecov bot commented Feb 21, 2022 •
edited
Loading

Codecov Report

Merging #7212 (bd61a83) into dev (0b205c6) will increase coverage by 0.50%.
The diff coverage is 88.80%.

❗ Current head bd61a83 differs from pull request most recent head b028d13. Consider uploading reports for the commit b028d13 to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##              dev    #7212      +/-   ##
==========================================
+ Coverage   62.57%   63.07%   +0.50%     
==========================================
  Files         332      339       +7     
  Lines       26443    26936     +493     
  Branches     4472     4530      +58     
==========================================
+ Hits        16546    16990     +444     
- Misses       9017     9051      +34     
- Partials      880      895      +15
Flag Coverage Δ
unittests 63.05% <88.80%> (+0.49%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmdet/core/bbox/samplers/mask_sampling_result.py 68.75% <68.75%> (ø)
mmdet/models/detectors/maskformer.py 77.77% <77.77%> (ø)
...det/core/bbox/assigners/mask_hungarian_assigner.py 88.63% <88.63%> (ø)
mmdet/models/dense_heads/maskformer_head.py 89.09% <89.09%> (ø)
mmdet/core/bbox/samplers/mask_pseudo_sampler.py 89.47% <89.47%> (ø)
mmdet/models/utils/panoptic_gt_processing.py 91.66% <91.66%> (ø)
mmdet/models/plugins/pixel_decoder.py 93.97% <93.97%> (ø)
mmdet/core/bbox/match_costs/match_cost.py 97.46% <97.05%> (-0.41%) ⬇️
mmdet/core/bbox/assigners/__init__.py 100.00% <100.00%> (ø)
mmdet/core/bbox/match_costs/__init__.py 100.00% <100.00%> (ø)
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0b205c6...b028d13. Read the comment docs.

@chhluo
move model description from config to readme
6a6bd29 
add description for binary_input

add description for dice loss

add a independent panoptic gt processing function

add a independent panoptic gt processing function

remove compatibility of pretrain in maskformer
@chhluo
Merge branch 'upstream/dev' of into maskformer
bd61a83
@ZwwWayne ZwwWayne merged commit cac3563 into open-mmlab:dev Feb 22, 2022
@AronLin AronLin mentioned this pull request Feb 24, 2022
Merged
ZwwWayne added a commit that referenced this pull request Feb 26, 2022
@ZwwWayne @chhluo
@huang-jesse @dvansa @jbwang1997 @shinya7y @siatwangmin @hhaAndroid @del-zhenwu @AronLin @vikashranjan26 @RangeKing @jamiechoi1995 @BIGWangYuDong @haofanwang @RangiLyu @zytx121 @HJoonKwon @yarkable
Bump version to v2.22.0 (#7249)
e045607 
* [Enhancement] Upgrade isort in pre-commit hook (#7130)

* upgrade isort to v5.10.1

* replace known_standard_library with extra_standard_library

* upgrade isort to v5.10.1

replace known_standard_library with extra_standard_library

* imports order changes

* [Fix] cannot to save the best checkpoint when the key_score is None (#7101)

* [Fix] Fix MixUp transform filter boxes failing case. Added test case (#7080)

* [Fix] Update the version limitation of mmcv-full and pytorch in CI. (#7133)

* Update

* Update build.yml

* Update build.yml

* [Feature] Support TIMMBackbone (#7020)

* add TIMMBackbone

based on
open-mmlab/mmpretrain#427
open-mmlab/mmsegmentation#998

* update and clean

* fix unit test

* Revert

* add example configs

* Create 2_new_data_model.md (#6476)

fix some typo

Co-authored-by: PJLAB\huanghaian <1286304229@qq.com>

* [FIX] add Ci of pytorch 1.10 and comments for bbox clamp (#7081) (#7083)

* add comments for bbox clamp

* add CI of pytorch1.10

* add ci of pytorch1.10.1

* mmcv1.9.0->mmcv1.9

* add ci of pytorch1.10

* Add daily issue owners (#7163)

* Add code owners

Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com>

* Update code owners

Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com>

* [Feature] Support visualization for Panoptic Segmentation (#7041)

* First commit of v2

* split the functions

* Support to show panoptic result

* temp

* Support to show gt

* support show gt

* fix lint

* Support to browse datasets

* Fix unit tests

* Fix findContours

* fix comments

* Fix pre-commit

* fix lint

* Add the type of an argument

* [Fix] confusion_matrix.py analysis tool handling NaNs (#7147)

* [Fix] Added missing property in SABLHead (#7091)

* Added missing property in SABLHead

* set pre-commit-hooks to v0.1.0

* set maskdownlint to v0.11.0

* pre-commit-hooks

Co-authored-by: Cedric Luo <luochunhua1996@outlook.com>

* Update config.md (#7215)

* [Fix] Fix wrong img name in onnx2tensorrt.py (#7157)

* [Docs] fix albumentations installed way (#7143)

* Update config.md

fix some typos

Co-authored-by: Jamie <jamiechoi1995@users.noreply.github.com>
Co-authored-by: BigDong <yudongwang@tju.edu.cn>

* [Feature] Support DyHead (#6823)

* add DyHead

* move and update DYReLU

* update

* replace stack with sum to reduce memory

* clean and update

* update to align inference accuracy (incomplete)

* fix pad

* update to align training accuracy and pick #6867

* add README and metafile

* update docs

* resolve comments

* revert picking 6867

* update README.md

* update metafile.yml

* resolve comments and update urls

* Fix broken colab link (#7218)

* [Fix] Fix wrong img name in onnx2tensorrt.py (#7157)

* [Docs] fix albumentations installed way (#7143)

* Fix broken colab link

Co-authored-by: Jamie <jamiechoi1995@users.noreply.github.com>
Co-authored-by: BigDong <yudongwang@tju.edu.cn>

* Remove the inplace addition in `FPN` (#7175)

* [Fix] Fix wrong img name in onnx2tensorrt.py (#7157)

* [Docs] fix albumentations installed way (#7143)

* Remove the inplace addition in `FPN`

* update

Co-authored-by: Jamie <jamiechoi1995@users.noreply.github.com>
Co-authored-by: BigDong <yudongwang@tju.edu.cn>
Co-authored-by: PJLAB\huanghaian <1286304229@qq.com>

* [Feature] Support OpenImages Dataset (#6331)

* [Feature] support openimage group of eval

* [Feature] support openimage group of eval

* support openimage dataset

* support openimage challenge dataset

* fully support OpenImages-V6 and OpenImages Challenge 2019

* Fix some logic error

* update config file

* fix get data_infos error

* fully support OpenImages evaluation

* update OpenImages config files

* [Feature] support OpenImages datasets

* fix bug

* support load image metas from pipeline

* fix bug

* fix get classes logic error

* update code

* support get image metas

* support openimags

* support collect image metas

* support Open Images

* fix openimages logic

* minor fix

* add a new function to compute openimages tpfp

* minor fix

* fix ci error

* minor fix

* fix indication

* minor fix

* fix returns

* fix returns

* fix returns

* fix returns

* fix returns

* minor fix

* update readme

* support loading image level labels and fix some logic

* minor fix

* minor fix

* add class names

* minor fix

* minor fix

* minor fix

* add openimages test unit

* minor fix

* minor fix

* fix test unit

* minor fix

* fix logic error

* minor fix

* fully support openimages

* minor fix

* fix docstring

* fix docstrings in readthedocs

* update get image metas script

* label_description_file -> label_file

* update openimages readme

* fix test unit

* fix test unit

* minor fix

* update readme file

* Update get_image_metas.py

* [Enhance] Speed up SimOTA matching. (#7098)

* [Feature] Add Maskformer to mmdet (#7212)

* first commit

* add README

* move model description from config to readme

add description for binary_input

add description for dice loss

add a independent panoptic gt processing function

add a independent panoptic gt processing function

remove compatibility of pretrain in maskformer

* update comments in maskformer_head

* update docs format

* Add deprecation message for deploy tools (#7242)

* Add CI for windows (#7228)

* [Fix] Fix wrong img name in onnx2tensorrt.py (#7157)

* [Docs] fix albumentations installed way (#7143)

* Add mmrotate (#7216)

* fix description for args in CSPDarknet (#7187)

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* fix test_find_latest_checkpoint

* fix data_infos__default_db_directories

* fix test_custom_classes_override_default

* Update test_custom_dataset.py

* Update test_common.py

* Update assign_result.py

* use os.path.join

* fix bug

* Update test_common.py

* Update assign_result.py

* Update sampling_result.py

* os.path -> osp

* os.path -> osp

Co-authored-by: Jamie <jamiechoi1995@users.noreply.github.com>
Co-authored-by: BigDong <yudongwang@tju.edu.cn>
Co-authored-by: Hyeokjoon Kwon <dsazz801@gmail.com>

* add Chinese version of init_cfg (#7188)

* [Fix] Fix wrong img name in onnx2tensorrt.py (#7157)

* [Docs] fix albumentations installed way (#7143)

* Create init_cfg.md

* Update docs/zh_cn/tutorials/init_cfg.md

Co-authored-by: Haian Huang(深度眸) <1286304229@qq.com>

* update init_cfg.md

* update init_cfg.md

* update init_cfg.md

* update init_cfg.md

Co-authored-by: Jamie <jamiechoi1995@users.noreply.github.com>
Co-authored-by: BigDong <yudongwang@tju.edu.cn>
Co-authored-by: Haian Huang(深度眸) <1286304229@qq.com>

* update MaskFormer readme and docs (#7241)

* update docs for maskformer

* update readme

* update readme format

* update link

* update json link

* update format of ConfigDict

* update format of function returns

* uncomment main in deployment/test.py

* [Feature] ResNet Strikes Back. (#7001)

* [Feature] ResNet Strikes Back.

* add more cfg

* add readme

* update

* update

* update

* update

* update

* update

* Maskformer Visualization (#7247)

* maskformer visualization

* compatible with models that do not contain arg of pretrained

* compatible with models that do not contain arg of pretrained

* Bump versions to v2.22.0 (#7240)

* Bump versions to v2.22.0

* Fix comments and add the latest PRs

* fix the id of contributor

* relax the version of mmcv

* Add ResNet Strikes Back

* Update README_zh-CN.md

* Update README.md

* fix typo

* Update README_zh-CN.md

Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com>

* Maskformer metafile and rsb readme format (#7250)

* [Fix] Fix Open Images testunit to avoid error in Windows CI (#7252)

* [Feature] support openimage group of eval

* [Feature] support openimage group of eval

* support openimage dataset

* support openimage challenge dataset

* fully support OpenImages-V6 and OpenImages Challenge 2019

* Fix some logic error

* update config file

* fix get data_infos error

* fully support OpenImages evaluation

* update OpenImages config files

* [Feature] support OpenImages datasets

* fix bug

* support load image metas from pipeline

* fix bug

* fix get classes logic error

* update code

* support get image metas

* support openimags

* support collect image metas

* support Open Images

* fix openimages logic

* minor fix

* add a new function to compute openimages tpfp

* minor fix

* fix ci error

* minor fix

* fix indication

* minor fix

* fix returns

* fix returns

* fix returns

* fix returns

* fix returns

* minor fix

* update readme

* support loading image level labels and fix some logic

* minor fix

* minor fix

* add class names

* minor fix

* minor fix

* minor fix

* add openimages test unit

* minor fix

* minor fix

* fix test unit

* minor fix

* fix logic error

* minor fix

* fully support openimages

* minor fix

* fix docstring

* fix docstrings in readthedocs

* update get image metas script

* label_description_file -> label_file

* update openimages readme

* fix test unit

* fix test unit

* minor fix

* update readme file

* Update get_image_metas.py

* fix oid testunit to avoid some error in windows

* minor fix to avoid some error in windows

* minor fix

* add comments in oid test unit

* minor fix

Co-authored-by: Cedric Luo <luochunhua1996@outlook.com>
Co-authored-by: LuooChen <33743370+LuooChen@users.noreply.github.com>
Co-authored-by: Daniel van Sabben Alsina <13849741+dvansa@users.noreply.github.com>
Co-authored-by: jbwang1997 <jbwang1997@gmail.com>
Co-authored-by: Yosuke Shinya <42844407+shinya7y@users.noreply.github.com>
Co-authored-by: siatwangmin <csustwangmin@163.com>
Co-authored-by: PJLAB\huanghaian <1286304229@qq.com>
Co-authored-by: del-zhenwu <dele.zhenwu@gmail.com>
Co-authored-by: Guangchen Lin <347630870@qq.com>
Co-authored-by: VIKASH RANJAN <ranjanvikash26@gmail.com>
Co-authored-by: Range King <RangeKingHZ@gmail.com>
Co-authored-by: Jamie <jamiechoi1995@users.noreply.github.com>
Co-authored-by: BigDong <yudongwang@tju.edu.cn>
Co-authored-by: Haofan Wang <frankmiracle@outlook.com>
Co-authored-by: Zhijian Liu <zhijianliu.cs@gmail.com>
Co-authored-by: BigDong <yudongwang1226@gmail.com>
Co-authored-by: RangiLyu <lyuchqi@gmail.com>
Co-authored-by: Yue Zhou <592267829@qq.com>
Co-authored-by: Hyeokjoon Kwon <dsazz801@gmail.com>
Co-authored-by: Kevin Ye <1752391457@qq.com>
ZwwWayne pushed a commit that referenced this pull request Jul 18, 2022
@chhluo
[Feature] Add Maskformer to mmdet (#7212)
e705d36 
* first commit

* add README

* move model description from config to readme

add description for binary_input

add description for dice loss

add a independent panoptic gt processing function

add a independent panoptic gt processing function

remove compatibility of pretrain in maskformer

* update comments in maskformer_head

* update docs format
ZwwWayne pushed a commit to ZwwWayne/mmdetection that referenced this pull request Jul 19, 2022
@chhluo @ZwwWayne
[Feature] Add Maskformer to mmdet (open-mmlab#7212)
28befc2 
* first commit

* add README

* move model description from config to readme

add description for binary_input

add description for dice loss

add a independent panoptic gt processing function

add a independent panoptic gt processing function

remove compatibility of pretrain in maskformer

* update comments in maskformer_head

* update docs format
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ZwwWayne ZwwWayne ZwwWayne left review comments

@AronLin AronLin Awaiting requested review from AronLin

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@chhluo @ZwwWayne