Fix conversion from ALTO to PAGE and vice versa #106

stweil · 2019-12-31T09:40:59Z

Fix order of arguments passed
Remove shell debugging (-x)
Handle input from STDIN
Add -convert-to ALTO argument needed for conversion from PAGE to ALTO

stweil · 2019-12-31T09:52:03Z

The prima-page-converter does not format the output file. Should we add post processing which formats it nicely? Otherwise the web interface only shows a single (very lengthy) line.

zuphilip · 2019-12-31T12:15:48Z

Yes, pretty print would be good 👍 Can we use Saxon for that? Would it maybe even make sense to have a CLI option for that in ocr-transform?

I tried some examples in the Web GUI and see an error for alto__page transformation with https://rawgit.com/kba/ocr-fileformat-samples/master/samples/alto/2.0/wetzel_reisebegleiter_1901_0021.alto as well as with http://chroniclingamerica.loc.gov/lccn/sn86069133/1910-10-31/ed-1/seq-1/ocr.xml . But the third example works fine. Are these upstream problems?

zuphilip · 2019-12-31T13:47:37Z

There seems to be a side effect in the abbyy2hocr transformation. In the Web GUI this transformation output nothing with the input https://digi.bib.uni-mannheim.de/~stweil/ocr-praxis/Testseiten/abbyy/417576986_0031.xml . However, this works in the docker run on the master branch. (Sorry for the the previous message which was unrelated to this issue.)

stweil · 2019-12-31T15:54:16Z

There seems to be a side effect in the abbyy2hocr transformation.

That's strange because ABBYY to hOCR does not use a transformer script (and this PR does not change anything else).

zuphilip · 2019-12-31T16:12:06Z

Okay, I did another test with this branch here and can confirm that abbyy2hocr works fine in my docker container. There seems to be a problem with the instance on digi.

stweil · 2020-01-01T11:39:26Z

Yes, pretty print would be good. Can we use Saxon for that? Would it maybe even make sense to have a CLI option for that in ocr-transform?

There are a lot of options how to implement pretty printing. I added a commit which uses Saxon, so the usual command line argument can be used to enable it (currently only implemented for output to STDOUT). The web interface now uses pretty printing for all PAGE related conversions by default.

zuphilip · 2020-01-01T13:35:48Z

Is this ready to merge? It looks good to me!

Do you agree that the problems with the alto files I described above are upstream problems?

Signed-off-by: Stefan Weil <sw@weilnetz.de>

The output is formatted when a Saxon serialization parameter is given on the command line. The web interface automatically uses `!indent=yes`. Signed-off-by: Stefan Weil <sw@weilnetz.de>

It used an undefined macro SAXON_JAR. Signed-off-by: Stefan Weil <sw@weilnetz.de>

stweil · 2020-01-02T01:01:31Z

There seems to be a side effect in the abbyy2hocr transformation. In the Web GUI this transformation output nothing with the input https://digi.bib.uni-mannheim.de/~stweil/ocr-praxis/Testseiten/abbyy/417576986_0031.xml .

Commit OCR-D/format-converters@5b9568f was missing in our installation (fixed now). I noticed that just running make all or make install does not update any of the existing cloned code from external git repositories. I think we should start using git submodules to get explicit dependencies.

stweil · 2020-01-02T01:13:42Z

I tried some examples in the Web GUI and see an error for alto__page transformation with https://rawgit.com/kba/ocr-fileformat-samples/master/samples/alto/2.0/wetzel_reisebegleiter_1901_0021.alto as well as with http://chroniclingamerica.loc.gov/lccn/sn86069133/1910-10-31/ed-1/seq-1/ocr.xml.

The first one throws a Java nullpointer exception, the second one looks like a vendor specific variant of ALTO. So both problems are not caused by the ocr-fileformat code.

zuphilip · 2020-01-02T06:55:22Z

The first one throws a Java nullpointer exception, the second one looks like a vendor specific variant of ALTO. So both problems are not caused by the ocr-fileformat code.

Okay, we should report them upstream such that they will hopefully been fixed there in the future.

Moreover, I created an issue about a better update mechanism.

So, let me ask again: Is this ready to merge? It looks good to me! 👍

zuphilip · 2020-01-02T10:09:16Z

Thank you @stweil for all the work on this! 🙇‍♂️

kba · 2020-01-02T10:42:54Z

bin/ocr-transform.sh

@@ -50,6 +49,8 @@ main () {
        fi
    fi

+    declare -a script_args


Why did you move that line?

I simply wanted to have it close to the first use of script_args.

kba · 2020-01-02T10:43:57Z

bin/ocr-transform.sh

@@ -86,8 +87,7 @@ main () {
        [[ "$outfile" != '-' ]] &&  script_args=("${script_args[@]}" "-o:$outfile")
        exec_saxon "${script_args[@]}"
    else
-        script_args=("${script_args[@]}" "$infile")
-        script_args=("${script_args[@]}" "$outfile")
+        script_args=("$infile" "$outfile" "${script_args[@]}")


This is reverse sorted then before.

It is now sorted as described in script/transform/README.md.

It was wrong before, weird we never noticed this...

stweil mentioned this pull request Dec 31, 2019

[WIP] Fix page__alto and alto__page #105

Closed

zuphilip mentioned this pull request Dec 31, 2019

GCV2hocr not working: no file #109

Closed

stweil force-pushed the fix-page-to-alto branch from 5cd607c to 509aac2 Compare December 31, 2019 15:47

zuphilip mentioned this pull request Dec 31, 2019

Add hocr__page transformation #113

Closed

stweil mentioned this pull request Jan 1, 2020

Convert Google Cloud Vision OCR output to hocr. #22

Closed

stweil added 9 commits January 1, 2020 23:14

Fix order of arguments for transformer script

3405ffb

Signed-off-by: Stefan Weil <sw@weilnetz.de>

Fix conversion from PAGE to ALTO

31e97e6

Signed-off-by: Stefan Weil <sw@weilnetz.de>

Fix conversion from ALTO to PAGE

0643c13

Signed-off-by: Stefan Weil <sw@weilnetz.de>

Fix conversion from GCV to hOCR

6505a88

Signed-off-by: Stefan Weil <sw@weilnetz.de>

Add transformation from hOCR to PAGE

06d037e

Signed-off-by: Stefan Weil <sw@weilnetz.de>

Add transformation from ABBYY FineReader XML to PAGE

18dd27b

Signed-off-by: Stefan Weil <sw@weilnetz.de>

Add transformation from Google Cloud Vision JSON to PAGE

7332869

Signed-off-by: Stefan Weil <sw@weilnetz.de>

Format output from PageConverter

8228c4c

The output is formatted when a Saxon serialization parameter is given on the command line. The web interface automatically uses `!indent=yes`. Signed-off-by: Stefan Weil <sw@weilnetz.de>

Fix debug message

d1c4477

It used an undefined macro SAXON_JAR. Signed-off-by: Stefan Weil <sw@weilnetz.de>

stweil force-pushed the fix-page-to-alto branch from c34ae24 to d1c4477 Compare January 1, 2020 22:14

zuphilip mentioned this pull request Jan 2, 2020

Add update mechanism #119

Open

stweil merged commit 20886fe into master Jan 2, 2020

stweil deleted the fix-page-to-alto branch January 2, 2020 08:36

kba reviewed Jan 2, 2020

View reviewed changes

zuphilip mentioned this pull request Jan 6, 2020

Support conversion from and to PAGE XML #79

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix conversion from ALTO to PAGE and vice versa #106

Fix conversion from ALTO to PAGE and vice versa #106

stweil commented Dec 31, 2019

stweil commented Dec 31, 2019

zuphilip commented Dec 31, 2019

zuphilip commented Dec 31, 2019

stweil commented Dec 31, 2019

zuphilip commented Dec 31, 2019

stweil commented Jan 1, 2020 •

edited

Loading

zuphilip commented Jan 1, 2020

stweil commented Jan 2, 2020

stweil commented Jan 2, 2020

zuphilip commented Jan 2, 2020

zuphilip commented Jan 2, 2020

kba Jan 2, 2020

stweil Jan 2, 2020

kba Jan 2, 2020

stweil Jan 2, 2020

kba Jan 2, 2020

Fix conversion from ALTO to PAGE and vice versa #106

Fix conversion from ALTO to PAGE and vice versa #106

Conversation

stweil commented Dec 31, 2019

stweil commented Dec 31, 2019

zuphilip commented Dec 31, 2019

zuphilip commented Dec 31, 2019

stweil commented Dec 31, 2019

zuphilip commented Dec 31, 2019

stweil commented Jan 1, 2020 • edited Loading

zuphilip commented Jan 1, 2020

stweil commented Jan 2, 2020

stweil commented Jan 2, 2020

zuphilip commented Jan 2, 2020

zuphilip commented Jan 2, 2020

kba Jan 2, 2020

Choose a reason for hiding this comment

stweil Jan 2, 2020

Choose a reason for hiding this comment

kba Jan 2, 2020

Choose a reason for hiding this comment

stweil Jan 2, 2020

Choose a reason for hiding this comment

kba Jan 2, 2020

Choose a reason for hiding this comment

stweil commented Jan 1, 2020 •

edited

Loading