Dangerzone fails mid-way through conversion, claiming I have run out of space

[pizzadude]

I get this error message (seen in the picture above) when trying to convert a PDF with the latest dangerzone ( 0.4.2 ) on Fedora 38, it claims there is no space left on the device, but I have plenty of space available still.

This might be related: #517

[apyrgio]

Hm, from this log, we see two things:

1. You are converting a document with at least 148 pages.
2. The code running within the Podman container cannot write files under a mounted directory (/tmp/dangerzone).

If it's a typical PDF document, point (1) tells us that it fails after writing ~200MiB of data (1 A4 page x 72 DPI x 3 color channels ~= 1.5MiB).

Point (2) may be more interesting. The mounted directory that runs out of space is a temporary directory. This means that it may be backed by an in-memory filesystem, and thus limited by your available RAM.

So, to debug further can you:

1. Try the conversion with a single-page document, just to ensure that it works.

2. Check if tmpfs is mounted on /tmp:

   mount | grep /tmp
   

3. Convert the failing document once more, and keep an eye on your disk and memory stats during the conversion, for any of those that seems to run out. For instance, running this in a separate terminal may help:

   watch 'free -m; df -h; df -i'
   

[pizzadude]

Try the conversion with a single-page document, just to ensure that it works.

Yes that works. (6 page pdf)

mount | grep /tmp                                                                                                                        
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,seclabel,nr_inodes=1048576,inode64)

Also, I have only 8GB of RAM.

/tmp size in df -h:

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           3.8G  2.6G  1.3G  67% /tmp

It would be nice if we could specify the temporary directory used.

[apyrgio]

Thanks for the command output @pizzadude . I'd like to zoom into the last part:

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           3.8G  2.6G  1.3G  67% /tmp

Was this output taken while Dangerzone was running? Does it reach to 100% at some point?

If not, and it's the /tmp usage at rest, then I see two weird things:

1. You are using 2.6GiB of temporary objects. It might be worth checking it out.
2. Dangerzone manages to fill 1.3GiB when converting 148 pages. This is 9MiB per page, or ~6x more than what I expected, which is a problem of its own. Can you give us the output of pdfinfo <file>, assuming that the document you sanitize is a PDF? It could be that we are miscalculating the document's DPI.

It would be nice if we could specify the temporary directory used.

I can suggest some options for you:

1. You can increase the size of /tmp through /etc/fstab. See https://www.looklinux.com/how-to-resize-tmpfs-on-linux/
2. You can specify a different temp dir using the TEMP environment variable (e.g., TEMP=/home/tmp dangerzone)

[apyrgio]

Hi @pizzadude. Just note that I have opened a related issue to this: #526

It seems that my assumption was wrong:

2. Dangerzone manages to fill 1.3GiB when converting 148 pages. This is 9MiB per page, or ~6x more than what I expected, which is a problem of its own.

By default, the A4 pages seem to be converted to pixels with 150 DPI, whereas I assumed we used 72 DPI. The former results to a page size of 6.22 MiB, which is much closer to the one you reported. If you can help us a bit with this part:

Was this output taken while Dangerzone was running? Does it reach to 100% at some point?

then we can be 100% sure that this is the issue that affects you. It's ok if you have switched to something else, we plan to solve this regardless. Thanks again for letting us know 🙂.

[pizzadude]

Hi, sorry for the late reply.

Was this output taken while Dangerzone was running? Does it reach to 100% at some point?

It was after running dangerzone and trying to convert a few times.

This is my current tmpfs, I haven't run dangerzone at all in the 5 hours of uptime I have:

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           3.8G  992K  3.8G   1% /tmp

I hope that helps. Let me know if you need any more info.

[apyrgio]

Thanks a lot for the output. So, Dangerzone starts with 3.8GB of RAM, and at some point eats it all 😱 ? I start to think that something else is at play here as well...

A few more questions:

1. For the document you have trouble converting, does Dangerzone crash consistently at around the 148th page mark? If yes, we're talking about 26.2 MiB per page here, which is veeery weird.
2. If the document is a PDF, can you share with us the output of pdfinfo <file>?
3. Is there a way you can share with us the document, or a page of that document? In general, anything that sticks out about this document will help us to dig deeper.

Feel free to reply with more details here, or reach out to my GitHub username [at] gmail.com.

[pizzadude]

Hi, I don't want to share the documents, but I just tried to convert them again and it worked this time. I'm not sure why it worked this time, but that's good news I guess.

Here's pdfinfo of both documents though:

Document 1:
Creator:         Adobe Acrobat 7.0
Producer:        Adobe Acrobat 7.0 Image Conversion Plug-in
Custom Metadata: no
Metadata Stream: yes
Tagged:          no
UserProperties:  no
Suspects:        no
Form:            none
JavaScript:      no
Pages:           217
Encrypted:       yes (print:yes copy:yes change:no addNotes:no algorithm:RC4)
Page size:       660 x 1000 pts
Page rot:        0
File size:       21904170 bytes
Optimized:       yes
PDF version:     1.6

Document 2:
Creator:         LaTeX with hyperref package
Producer:        xdvipdfmx (0.7.8)
CreationDate:    Tue Feb  2 17:09:17 2016 EST
Custom Metadata: no
Metadata Stream: no
Tagged:          no
UserProperties:  no
Suspects:        no
Form:            none
JavaScript:      no
Pages:           883
Encrypted:       no
Page size:       419.53 x 558.43 pts
Page rot:        0
File size:       7416886 bytes
Optimized:       no
PDF version:     1.5