Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QA Spec #225

Merged
merged 57 commits into from
Mar 1, 2023
Merged
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
e2d4038
wip
kba Sep 6, 2022
0cc4a0b
Merge remote-tracking branch 'origin/master' into qa-spec
kba Sep 14, 2022
5aa6bd5
rewrite eval schema and saple according to OCR-D/zenhub#123
kba Sep 21, 2022
6cd0caf
add metrics to ocrd_eval.md
kba Sep 21, 2022
b529531
ocrd_eval: \begin{array}{ll} instead of .. {2}
kba Sep 22, 2022
18333b8
style(ocrd_eval.md): linting, formatting and correcting images
mweidling Sep 26, 2022
fe9d6ff
stlye: add new line
mweidling Sep 26, 2022
d7854a1
Apply suggestions from code review
kba Sep 27, 2022
ee67881
Apply suggestions from code review
kba Sep 27, 2022
5b35358
retcon JSON changes to YAML
mweidling Sep 27, 2022
1aa048c
comment EvaluationMetrics back in
kba Sep 27, 2022
5840476
generate minimal JSON from YAML src
kba Sep 27, 2022
c9d313f
comment out undiscussed CER metrics
mweidling Sep 27, 2022
a814c89
feat: move workflow_steps to ocr_workflow object
mweidling Nov 24, 2022
a881e08
remove schema from this branch, cf. #236
kba Dec 19, 2022
c7ae88d
integrate Uwe's feedback
mweidling Jan 23, 2023
7ce6c1a
Update ocrd_eval.md
mweidling Jan 23, 2023
ef8aeea
Update ocrd_eval.md
mweidling Jan 23, 2023
e2d2ec9
Update ocrd_eval.md
mweidling Feb 3, 2023
ad97594
Update ocrd_eval.md
mweidling Feb 3, 2023
34a78cb
Update ocrd_eval.md
mweidling Feb 3, 2023
c95ce0b
Update ocrd_eval.md
mweidling Feb 3, 2023
13b2bcd
Update ocrd_eval.md
mweidling Feb 3, 2023
b823afc
Update ocrd_eval.md
mweidling Feb 3, 2023
8ab1391
Apply suggestions from code review
mweidling Feb 3, 2023
5e1da31
Apply suggestions from code review
mweidling Feb 3, 2023
0183ea9
Update ocrd_eval.md
mweidling Feb 3, 2023
5519120
update character definition wrt. white spaces
mweidling Feb 7, 2023
dd2d63b
refine paragraph about characters
mweidling Feb 9, 2023
0253720
move character section before edit distance section
mweidling Feb 9, 2023
19deddd
add placeholder for letter accuracy
mweidling Feb 9, 2023
149b271
fix link
mweidling Feb 9, 2023
7d0bbf6
Update ocrd_eval.md
mweidling Feb 9, 2023
851aeb7
Update ocrd_eval.md
mweidling Feb 9, 2023
c81079a
Update ocrd_eval.md
mweidling Feb 10, 2023
fab6202
Update ocrd_eval.md
mweidling Feb 10, 2023
f678050
implement feedback
mweidling Feb 10, 2023
a680c70
be more precise about CER/WER granularity
mweidling Feb 10, 2023
5e94aa0
change GPU metrics
mweidling Feb 10, 2023
e8dc864
change citation hint
mweidling Feb 10, 2023
ee330c9
adjust WER definition
mweidling Feb 10, 2023
9ea4b62
Apply suggestions from code review
mweidling Feb 10, 2023
c910f0e
add bow metric
mweidling Feb 14, 2023
8c22169
format document
mweidling Feb 14, 2023
149a2eb
gpu mem instead of util
mweidling Feb 14, 2023
2999ef4
Update ocrd_eval.md
mweidling Feb 14, 2023
87f9438
GPU Peak Memory definition
mweidling Feb 14, 2023
5e80c94
Update ocrd_eval.md
mweidling Feb 14, 2023
5cd5efb
Update ocrd_eval.md
mweidling Feb 15, 2023
492b6ee
Update ocrd_eval.md
mweidling Feb 15, 2023
d8d4cef
Update ocrd_eval.md
mweidling Feb 15, 2023
48e69f8
add letter accuracy
mweidling Feb 15, 2023
3cc5bee
rephrase layout eval intro
mweidling Feb 15, 2023
f817521
add reading order evaluation
mweidling Feb 15, 2023
04c5c27
implement Uwe's feedback reg. Letter Accuracy
mweidling Feb 15, 2023
d078b1b
Apply suggestions from code review
mweidling Feb 16, 2023
43b364a
eval: Improvements to TeX formulas
kba Feb 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
json: \
bagit-profile.json \
ocrd_tool.schema.json \
openapi.json

deps:
pip install yaml click
json: $(shell find -name '*.json')

%.json: %.yml
python3 scripts/yaml-to-json.py $< $@
python3 scripts/yaml-to-json.py --indent 0 $< $@

validate: json
jsonschema --output pretty --validator Draft201909Validator --instance ocrd_eval.sample.json ocrd_eval.schema.json

deps:
pip install yaml click jsonschema
2 changes: 1 addition & 1 deletion bagit-profile.json
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"BagIt-Profile-Info":{"BagIt-Profile-Identifier":"https://ocr-d.de/en/spec/bagit-profile.json","BagIt-Profile-Version":"1.2.0","Source-Organization":"OCR-D","External-Description":"BagIt profile for OCR data","Contact-Name":"Konstantin Baierer","Contact-Email":"konstantin.baierer@sbb.spk-berlin.de","Version":0.1},"Bag-Info":{"Bagging-Date":{"required":false},"Source-Organization":{"required":false},"Ocrd-Mets":{"required":false,"default":"mets.xml"},"Ocrd-Identifier":{"required":true},"Ocrd-Checksum":{"required":false,"default":"cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e"}},"Manifests-Required":["sha512"],"Tag-Manifests-Required":[],"Tag-Files-Required":[],"Tag-Files-Allowed":["README.md","Makefile","build.sh","sources.csv","metadata/*.xml","metadata/*.txt"],"Allow-Fetch.txt":false,"Serialization":"required","Accept-Serialization":"application/zip","Accept-BagIt-Version":["1.0"]}
{"BagIt-Profile-Info": {"BagIt-Profile-Identifier": "https://ocr-d.de/en/spec/bagit-profile.json", "BagIt-Profile-Version": "1.2.0", "Source-Organization": "OCR-D", "External-Description": "BagIt profile for OCR data", "Contact-Name": "Konstantin Baierer", "Contact-Email": "konstantin.baierer@sbb.spk-berlin.de", "Version": 0.1}, "Bag-Info": {"Bagging-Date": {"required": false}, "Source-Organization": {"required": false}, "Ocrd-Mets": {"required": false, "default": "mets.xml"}, "Ocrd-Identifier": {"required": true}, "Ocrd-Checksum": {"required": false, "default": "cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e"}}, "Manifests-Required": ["sha512"], "Tag-Manifests-Required": [], "Tag-Files-Required": [], "Tag-Files-Allowed": ["README.md", "Makefile", "build.sh", "sources.csv", "metadata/*.xml", "metadata/*.txt"], "Allow-Fetch.txt": false, "Serialization": "required", "Accept-Serialization": "application/zip", "Accept-BagIt-Version": ["1.0"]}
Loading