large file management

[Salvodif]

is there a way to work on large files page by page?
like telling the app now do page 1 to 150 and then in a second process be able to tell it do page 151 to 300?
so that the memory usage problem can be solved?

[jbarlow83]

You can split the file with an pdf splitter program and recombine, but the easiest thing to do is increase the allocation of your tmpfs partition or set an environment variable TMPDIR=/home/user/tmp to allow use of a large storage device for temporary files.

Improving the efficiency of temporary storage is not something that can easily be done right now because concurrent execution and plugins means all intermediate temporary files need to be available all until processing is done. I may change this one day, but it would be a breaking change that would affect plugins.

OCRmyPDF needs a lot of temp storage for large files. That is how it is. People have used ocrmypdf in production for 10,000+ page PDFs so 300 is no problem, if you provide ample /tmp storage. This requirement is spelled out in the documentation.

[Salvodif]

You can split the file with an pdf splitter program and recombine, but the easiest thing to do is increase the allocation of your tmpfs partition or set an environment variable TMPDIR=/home/user/tmp to allow use of a large storage device for temporary files.

I've 32Gb of Ram and
tmpfs 16300248 8 16300240 1% /tmp

but I got issue with 1300 pages. But I'll try with splitting the book