Adjust vision transformer backbone architectures #524
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
April 28, 2021 19:31
* Add DropPath, trunc_normal_ for VisionTransformer implementation; * Add class token buring intermediate period and remove it during final period;
Codecov Report
@@ Coverage Diff @@
## master #524 +/- ##
==========================================
+ Coverage 86.60% 86.68% +0.07%
==========================================
Files 99 101 +2
Lines 5160 5234 +74
Branches 834 848 +14
==========================================
+ Hits 4469 4537 +68
- Misses 533 535 +2
- Partials 158 162 +4
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
* Remove class token and reshape entire token feature from NLC to NCHW;
added 2 commits
April 29, 2021 14:56
* Add related unit test;
xvjiarui
reviewed
Apr 30, 2021
mmseg/models/backbones/vit.py
Outdated
| final feature map. Default: False. | ||
| interpolate_mode (str): Select the interpolate mode for position | ||
| embeding vector resize. Default: bilinear. | ||
| input_cls_token (bool): If concatenating class token into image tokens |
There was a problem hiding this comment.
Suggested change
| input_cls_token (bool): If concatenating class token into image tokens | |
| with_cls_token (bool): If concatenating class token into image tokens |
xvjiarui
reviewed
Apr 30, 2021
mmseg/models/backbones/vit.py
Outdated
| self.pos_drop = nn.Dropout(p=drop_rate) | ||
|
|
||
| self.blocks = nn.Sequential(*[ | ||
| self.num_stages = depth | ||
| self.out_indices = tuple(range(self.num_stages)) |
There was a problem hiding this comment.
We may make this an argument, default to (12, )
xvjiarui
reviewed
Apr 30, 2021
| self.drop_prob = drop_prob | ||
| self.keep_prob = 1 - drop_prob | ||
|
|
||
| def forward(self, x): |
There was a problem hiding this comment.
tests/test_models/test_utils/test_drop.py
xvjiarui
reviewed
Apr 30, 2021
mmseg/models/backbones/vit.py
Outdated
| x = x[:, 1:] | ||
|
|
||
| outs = [] | ||
| block_len = len(self.blocks) |
There was a problem hiding this comment.
This variable is not needed.
xvjiarui
reviewed
Apr 30, 2021
mmseg/models/backbones/vit.py
Outdated
| block_len = len(self.blocks) | ||
| for i, blk in enumerate(self.blocks): | ||
| x = blk(x) | ||
| if i == block_len - 1: |
There was a problem hiding this comment.
Suggested change
| if i == block_len - 1: | |
| if i == len(self.blocks) - 1: |
* Add unit test for DropPath;
xvjiarui
reviewed
Apr 30, 2021
xvjiarui
reviewed
Apr 30, 2021
xvjiarui
approved these changes
Apr 30, 2021
bowenroom
pushed a commit
to bowenroom/mmsegmentation
that referenced
this pull request
Feb 25, 2022
* Adjust vision transformer backbone architectures; * Add DropPath, trunc_normal_ for VisionTransformer implementation; * Add class token buring intermediate period and remove it during final period; * Fix some parameters loss bug; * * Store intermediate token features and impose no processes on them; * Remove class token and reshape entire token feature from NLC to NCHW; * Fix some doc error * Add a arg for VisionTransformer backbone to control if input class token into transformer; * Add stochastic depth decay rule for DropPath; * * Fix output bug when input_cls_token=False; * Add related unit test; * * Add arg: out_indices to control model output; * Add unit test for DropPath; * Apply suggestions from code review Co-authored-by: Jerry Jiarui XU <xvjiarui0826@gmail.com>
wjkim81
pushed a commit
to wjkim81/mmsegmentation
that referenced
this pull request
Dec 3, 2023
* Add 3D pose pipeline * Add transforms on joint coordinates in pipelines * Add camera projection in pipelines * Add camera interface in mmpose.core * Add 3D pose pipeline * Revise code * fix variable name * Add 3D pose pipeline * update unittests for better codecov rate * Add 3D pose pipeline * update unittests for better codecov rate * Add 3D pose pipeline * update unittests for better codecov rate * Add 3D pose pipeline * Add PoseSequenceToTensor * minor fix according to review comments * Revise according to review comments * rebase to master * extend fliplr_regression to handle 2D/3D * add remove_root option to JointRelativization * Fix docstring * update unittest * update unittest * update camera parameters to be in meter * minor fix to docstrings * minor fix * fix importing
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add DropPath, trunc_normal_ for VisionTransformer implementation;
Add arg for VisionTransformer backbone to control if input class token into transformer;
Class token will be removed during output stage;