[Feature] Add Satrn #405

2793145003 · 2021-08-02T07:34:30Z

Hi, 我在我自己的数据集（10w条数据）上使用了 SATRN（https://arxiv.org/pdf/1910.04396.pdf）的 small 版本，最开始的几天 acc 完全没有变化，但之后上升得很快，目前 word_acc=0.9245 并且还在缓慢上升中（执行时间 266:17:00）。
不过我没有足够的资源和时间使用公开数据集，也没有足够显存运行完整的模型，所以没有相应的指标，也不能保证 satrn_academic（论文中的结构）也能得到好的效果……

codecov · 2021-08-02T08:04:08Z

Codecov Report

Merging #405 (421c57b) into main (7571763) will increase coverage by 0.16%.
The diff coverage is 100.00%.

❗ Current head 421c57b differs from pull request most recent head ed821c0. Consider uploading reports for the commit ed821c0 to get more accurate results

@@            Coverage Diff             @@
##             main     #405      +/-   ##
==========================================
+ Coverage   85.34%   85.50%   +0.16%     
==========================================
  Files         139      142       +3     
  Lines        9380     9502     +122     
  Branches     1343     1353      +10     
==========================================
+ Hits         8005     8125     +120     
- Misses       1065     1066       +1     
- Partials      310      311       +1

Flag	Coverage Δ
unittests	`85.50% <100.00%> (+0.16%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmocr/utils/ocr.py	`75.69% <ø> (ø)`
mmocr/models/textrecog/backbones/shallow_cnn.py	`100.00% <100.00%> (ø)`
mmocr/models/textrecog/encoders/satrn_encoder.py	`100.00% <100.00%> (ø)`
mmocr/models/textrecog/layers/transformer_layer.py	`99.44% <100.00%> (+0.32%)`	⬆️
mmocr/models/textrecog/recognizer/satrn.py	`100.00% <100.00%> (ø)`
mmocr/datasets/pipelines/transforms.py	`81.18% <0.00%> (-0.34%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7571763...ed821c0. Read the comment docs.

gaotongxiao

Thanks for your contribution! Your work is excellent and really constructive to MMOCR!

I had compared your implementation with the official release and found some differences that should be addressed. Some are not significant but might slightly affect the performance. (We want to be conservative on this model due to its long training cycle) I had also commented on some parts that I'm not quite sure and we can discuss.

Meanwhile, we are also training the full Satrn model so we'll see if the model can match the claimed performance.

mmocr/models/textrecog/backbones/shallow_cnn.py

configs/textrecog/satrn/satrn_academic.py

configs/textrecog/satrn/satrn_small.py

mmocr/models/textrecog/layers/transformer_layer.py

configs/textrecog/satrn/satrn_academic.py

configs/textrecog/satrn/satrn_small.py

tests/test_models/test_ocr_encoder.py

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

gaotongxiao

Changed the config to get a closer implementation to the official implementation.

configs/textrecog/satrn/satrn_small.py

configs/textrecog/satrn/satrn_academic.py

mmocr/models/textrecog/layers/transformer_layer.py

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

gaotongxiao · 2021-08-09T06:04:41Z

For now, I've trained your original Satrn implementation on Synthtext + Mjsynth for 5 epochs, but only got ~ 10%-20% accuracy on test sets. The modified version is still in its first training epoch but gets an average loss of ~0.02 which is 10 times smaller than the original one. I think we are on the right track.

gaotongxiao · 2021-08-11T01:53:00Z

Well, here's the best result after the 4th epoch:

IIIT5k	SVT	IC13	IC15	SVTP	CT80
0.9607	0.9351	0.9567	0.8406	0.8853	0.9028

It's even better than what the original paper have claimed. I feel the model is good enough for now, and I will continue to wrap it up and publish this model soon. Good job!

2793145003 · 2021-08-11T02:30:40Z

好耶！感谢大佬的review和指点！

gaotongxiao

Final code style pass, just a bit of refactoring. I'll push a new readme and ocr.py to this PR soon.
BTW, Github now supports applying multiple suggestions in a single commit :)

mmocr/models/textrecog/backbones/shallow_cnn.py

mmocr/models/textrecog/layers/transformer_layer.py

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

gaotongxiao · 2021-08-18T11:38:08Z

@innerlee @cuhk-hbsun Any general comments? If not, it will be merged tomorrow

* Add SATRN * Create satrn_small_academic.py * Update README.md * change config name * Update mmocr/models/textrecog/backbones/shallow_cnn.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update configs/textrecog/satrn/satrn_academic.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update configs/textrecog/satrn/satrn_small.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update shallow_cnn.py * Update mmocr/models/textrecog/layers/transformer_layer.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update mmocr/models/textrecog/layers/transformer_layer.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update test_ocr_encoder.py * change keep_aspect_ratio=False * Update transformer_layer.py * Update configs/textrecog/satrn/satrn_small.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update configs/textrecog/satrn/satrn_academic.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update mmocr/models/textrecog/layers/transformer_layer.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update mmocr/models/textrecog/layers/transformer_layer.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update mmocr/models/textrecog/layers/transformer_layer.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update mmocr/models/textrecog/layers/transformer_layer.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update mmocr/models/textrecog/layers/transformer_layer.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update mmocr/models/textrecog/layers/transformer_layer.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update mmocr/models/textrecog/layers/transformer_layer.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update mmocr/models/textrecog/layers/transformer_layer.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update mmocr/models/textrecog/layers/transformer_layer.py Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update transformer_layer.py * Apply suggestions from code review Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * Update transformer_layer.py * update satrn readme * add satrn to ocr.py * add satrn_sm and fix configs * add a test for config * add copyright info * use mmocr registry Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

2793145003 added 4 commits August 2, 2021 15:03

Add SATRN

480d87e

Create satrn_small_academic.py

870639c

Update README.md

55f70b5

change config name

0566fba

gaotongxiao added the New Model label Aug 2, 2021

gaotongxiao reviewed Aug 5, 2021

View reviewed changes

2793145003 and others added 10 commits August 6, 2021 10:33

Update mmocr/models/textrecog/backbones/shallow_cnn.py

9a4962b

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update configs/textrecog/satrn/satrn_academic.py

25149c1

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update configs/textrecog/satrn/satrn_small.py

22684df

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update shallow_cnn.py

e203bbe

Update mmocr/models/textrecog/layers/transformer_layer.py

97d0ed4

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update mmocr/models/textrecog/layers/transformer_layer.py

1fe5038

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update test_ocr_encoder.py

f57f734

Merge branch 'satrn' of https://github.com/2793145003/mmocr into satrn

9e4d0fd

change keep_aspect_ratio=False

0c72c91

Update transformer_layer.py

0a9b629

gaotongxiao reviewed Aug 9, 2021

View reviewed changes

2793145003 and others added 12 commits August 9, 2021 12:44

Update configs/textrecog/satrn/satrn_small.py

0abe8b5

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update configs/textrecog/satrn/satrn_academic.py

7e365dc

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update mmocr/models/textrecog/layers/transformer_layer.py

7bc05e0

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update mmocr/models/textrecog/layers/transformer_layer.py

1c2fb34

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update mmocr/models/textrecog/layers/transformer_layer.py

2f8b5bf

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update mmocr/models/textrecog/layers/transformer_layer.py

e6189ff

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update mmocr/models/textrecog/layers/transformer_layer.py

ff90147

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update mmocr/models/textrecog/layers/transformer_layer.py

4df80bb

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update mmocr/models/textrecog/layers/transformer_layer.py

deab520

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update mmocr/models/textrecog/layers/transformer_layer.py

68bf232

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update mmocr/models/textrecog/layers/transformer_layer.py

0c3c674

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update transformer_layer.py

141c998

gaotongxiao changed the title ~~Add Satrn~~ [Feature] Add Satrn Aug 9, 2021

gaotongxiao reviewed Aug 11, 2021

View reviewed changes

2793145003 and others added 5 commits August 12, 2021 10:45

Apply suggestions from code review

d3ca21d

Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

Update transformer_layer.py

61c710e

update satrn readme

afc6249

Merge branch 'main' into satrn

bbd1374

add satrn to ocr.py

1555680

gaotongxiao requested a review from Harold-lkk August 12, 2021 05:10

gaotongxiao added 3 commits August 16, 2021 11:58

add satrn_sm and fix configs

ceb4e26

add a test for config

9d8b93d

Merge branch 'main' into satrn_pr

3f25e24

gaotongxiao approved these changes Aug 16, 2021

View reviewed changes

gaotongxiao added 3 commits August 19, 2021 19:10

add copyright info

5969965

Merge branch 'main' into satrn_pr

b34e8d2

use mmocr registry

ed821c0

open-mmlab deleted a comment from 2793145003 Aug 19, 2021

gaotongxiao merged commit 8f377f2 into open-mmlab:main Aug 19, 2021

2793145003 deleted the satrn branch August 20, 2021 03:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Add Satrn #405

[Feature] Add Satrn #405

2793145003 commented Aug 2, 2021

codecov bot commented Aug 2, 2021 •

edited

Loading

gaotongxiao left a comment •

edited

Loading

gaotongxiao left a comment

gaotongxiao commented Aug 9, 2021

gaotongxiao commented Aug 11, 2021

2793145003 commented Aug 11, 2021

gaotongxiao left a comment

gaotongxiao commented Aug 18, 2021

[Feature] Add Satrn #405

[Feature] Add Satrn #405

Conversation

2793145003 commented Aug 2, 2021

codecov bot commented Aug 2, 2021 • edited Loading

Codecov Report

gaotongxiao left a comment • edited Loading

Choose a reason for hiding this comment

gaotongxiao left a comment

Choose a reason for hiding this comment

gaotongxiao commented Aug 9, 2021

gaotongxiao commented Aug 11, 2021

2793145003 commented Aug 11, 2021

gaotongxiao left a comment

Choose a reason for hiding this comment

gaotongxiao commented Aug 18, 2021

codecov bot commented Aug 2, 2021 •

edited

Loading

gaotongxiao left a comment •

edited

Loading