Support training without lstmf files

[stweil]

The list of the training or validation files may now contain .png files instead of .lstmf files, so it is no longer necessary to spend time and disc space for creating .box and .lstmf files for a training.

[zdenop]

Why not use to use std::filesystem::path as in

tesseract/src/training/unicharset_extractor.cpp

Line 74 in b217cfc

if (filePath.extension() == ".box") {

[egorpugin]

Path refactoring should be done as more general procedure. It is ok to keep std::string in very local code.

[stweil]

So would you prefer using std::filesystem::path::extension() here or not?

[egorpugin]

It's ok as is. fs::paths can be refactored for 6.0 wit api changes/breakage.

[zdenop]

@egorpugin: document_name_ is not part of tesseract API - it is the private object used in imagedata.cpp only, So I do not think it will break something or change API.

@stweil: I am fine with the current PR code, just I think we should step by step use c++17 features.

[yaofuzhou]

The list of the training or validation files may now contain .png files instead of .lstmf files, so it is no longer necessary to spend time and disc space for creating .box and .lstmf files for a training.

@stweil

Hi - In my understanding of Tesseract before your merge, list.train and list.eval list the .lstmf files used in training and evaluation, where each .lstmf file is the product of an image and its .box file. Without the .lstmf files, how should I tell the Tesseract training program which files to use in the training process? Thanks!

[stweil]

You still use list.train and list.eval, but they can now include .png files instead of .lstmf files.

[yaofuzhou]

You still use list.train and list.eval, but they can now include .png files instead of .lstmf files.

Sorry for the silly question, but may I assume that the corresponding .box files will be automatically fetched?

I care about this because I am getting the error of

Compute CTC targets failed for xyz.lstmf

from time to time. I was wondering if this is due to insufficient LSTM cells to accommodate an entire training image, and I wonder if training without the .lstmf file would make any difference. Thanks!

[stweil]

Neither .box nor .lstmf files are used if you provide only .png files in the lists.

[amitdo]

I assume the textline within the png image should not have any padding, is that right?

[stweil]

Our line images are from real prints and typically have random padding. As tesstrain uses PSM=13 by default, the .lstmf files would also contain line images with the same padding.