Reading and writing common image formats, ppm, tiff, jpeg, png

[certik]

Here is an example implementation of loading and saving ppm: https://github.com/certik/fortran-utils/blob/b43bd24cd421509a5bc6d3b9c3eeae8ce856ed88/src/ppm.f90. The advantage of the ppm format is that it is simple to write such readers and writers. Then one can use external tools (such as pnmtopng) to convert to more common formats. So perhaps tiff, jpeg and png are not initially needed and ppm might be enough to allow to work with images in Fortran.

Prior art:

• Matlab: imread can read BMP, JPEG, PNG, CUR, PPM, GIF, PBM, RAS, HDF4, PCX, TIFF, ICO, PGM and XWD files. See also imwrite.
• SciPy: imread uses Python Imaging Library (PIL) to read the image; PIL supports: BMP, DIB, EPS, GIF, ICNS, ICO, IM, JPEG, MSP, PCX, PNG, PPM, SGI, SPIDER, TGA, TIFF, WebP, XBM. See also imsave.
• Julia: Images.jl package has a nice comparison page with SciPy and Matlab; can read at least PNG, GIF, TIFF, JPEG.

[certik]

If we only do PPM, that can be easily done in pure Fortran. For almost all the other formats we would have to depend on a 3rd party library.

It feels like supporting all the formats might be a better fit for a separate library from stdlib, because at least initially I think we should stay in pure Fortran and only have minimal external dependencies to make the distribution of stdlib easier.

But since PPM can be supported easily in pure Fortran, it would allow codes to export and import images by using some external tool to convert to and from PPM first, which seems like it might be valuable enough to include in stdlib.

[milancurcic]

PPM is good enough for start! I agree on the advantage of pure Fortran implementation. We can adopt your implementations, they're good. I also have a P3 writer (same as P6, but ascii).

[certik]

Perfect, let's do it. Yes, we can support both ascii and binary PPM and a few related formats.

[milancurcic]

What module would this belong to? stdlib_experimental_io or its own stdlib_experimental_image? On one hand, I think this fits thematically in io. However, we anticipate that io will grow further and would be split in smaller modules eventually, so maybe a dedicated module is more fitting.

I'm happy to prepare a PR with loadppm and saveppm implementations for P3 and P6 formats -- basically take your code and add P3 implementations. Other formats may be implemented later if desired.

How should we handle choosing a format? For loadppm I think we can have just one specific procedure, because the format is specified in the file header and the user doesn't have to specify which format it is. However, for saveppm, user should somehow specify which format we're storing into. We can just use different procedures directly:

• saveppm_p3
• saveppm_p6

We could also have a thin wrapper and specify it with the format argument:

subroutine saveppm(filename, img, format)
  ...
  if (format == 'P3') then
    call saveppm_p3(filename, img)
  else if (format == 'P6') then
    call saveppm_p3(filename, img)
  else
    call error_stop()
  end if
end subroutine saveppm

[certik]

Let's use stdlib_experimental_io for now. I would use stdlib_experimental_image if we plan to support more image routines. In SciPy it is in scipy.misc. Also, SciPy and Matlab calls this imread. However, their imread can read all kinds of image formats. Since we will only do PPM at first, I think it's fine to call it load / saveppm. We can later add imread as we support more formats.

Regarding the API, I agree with your choice. I would further make the format argument optional, and if it is not present, probably choose P6 (binary) as the default.

P.S. Isn't format a "reserved" word? If so, you can use fmt instead.

[milancurcic]

Isn't format a "reserved" word? If so, you can use fmt instead.

I don't think Fortran has reserved words. fmt is also a keyword in read() and write() statements. I do prefer fmt here because it's shorter and clear what it means.

I like the idea of P6 as default.

[ivan-pi]

For some applications I had in the past a graymap PPM (P2 or P5) would have sufficed. Specifically, I could use a ppm as a porosity map in a Lattice Boltzmann simulation. Since these only require a rank 2 array (i and j image coordinates) vs a rank 3 array (i, j, and rgb) it would be easy to support these too.

[jacobwilliams]

FYI: pure Fortran gif (even animated) library here: https://github.com/jacobwilliams/FGIF

[certik]

@jacobwilliams this is great! It might not be that difficult to provide pure Fortran readers / writers to other image formats also.

[gronki]

IMHO this is easily implemented as part of the ecosystem with iso_c_binding to currently existing libraries. There is no need complicating stuff by making competing solutions in stdlib (even for ppm). And one would need unsigned types to make it work in any decent way.

[zbeekman]

There is no need complicating stuff by making competing solutions in stdlib (even for ppm). And one would need unsigned types to make it work in any decent way.

Often there is a compelling case to write something in Fortran so that the community can contribute and fix it, but for certain cases, especially file formats, if there is a widely used existing library that stdlib can provide a Fortran interface for then that reduces complexity and reduces maintenance burden, while ensuring a higher quality of implementation.

In this particular case we should be pragmatic and include implementations only where:

1. There is already a popular and well tested Fortran version we can adopt
2. It is easy to write and maintain (e.g., PPM)
3. There is no suitable extant implementation for which a Fortran interface can be created

[milancurcic]

@gronki Can you list C library candidates? I know of stb and libpng. What do you have in mind?

We should absolutely weigh-out the complexity of pure Fortran vs. C-interop implementations. In case of ppm, the pure Fortran implementation is so simple that I can't imagine a similar Fortran-C interface would be any simpler. I may be wrong though.

Involving any external C library would add the complexity of a dependency. While I'm not against external dependencies in stdlib (Python has them), I think we should consider them as a last resort.

[gronki]

But is reading and writing ppm's worth the burden of the maintenance? Don't get me wrong but it's not a serious image format. PPM is short for "i don't feel like googling libpng manual". I see no value of having a ppm implementation but maybe my opinion is isolated.

I have never heard about stb before. I personally used libgdal, libpng and cfitsio in my field (from Fortran). All of them mature, stable and insanely easy to use by iso_c_binding (or even to wrap in a derived type). (edit: Actually, cfitsio has a f77 interface included.) I think it would be nice to have a stable and up-to-date bindings as a part of the ecosystem (whether they make it into stdlib or not) but I see absolutely no need to put any effort in implementing a worse version of what is already done.

One a side note, I feel that not having certain bindings for popular format handling libraries is the actual problem we are trying to solve here. Fortran does have PNG or JPG handling. But it takes too much effort (one has to write their own binding rather than have a ready-to-use interface such as C or Python).

[zbeekman]

We should absolutely weigh-out the complexity of pure Fortran vs. C-interop implementations. In case of ppm, the pure Fortran implementation is so simple that I can't imagine a similar Fortran-C interface would be any simpler. I may be wrong though.

@milancurcic Exactly how I feel. Well said.

Involving any external C library would add the complexity of a dependency. While I'm not against external dependencies in stdlib (Python has them), I think we should consider them as a last resort.

This is very true. But in the case where an external library already does it---and does it well---and adding a roll-your-own implementation in the standard library adds unacceptable complexity, then I think a choice should be made about either:

1. Creating an interface for the extant external library
2. Not including it in the standard library

Elon Musk recently said something like this when talking about SpaceX Starship:

The best design is an un-design. It has no moving parts, no points of failure, no maintenance and development costs, no documentation to write or protocols to create and follow.

Obviously we want convenience and utility in a standard library but it needs to be provided in a sustainable way.

(And, yes, I know Musk is a bit mad, but he has some moments of wisdom/genius.)

[milancurcic]

But in the case where an external library already does it---and does it well---and adding a roll-your-own implementation in the standard library adds unacceptable complexity, then I think a choice should be made about either:

1. Creating an interface for the extant external library
2. Not including it in the standard library

I agree! It seems to me that any format that would need compression like zlib would fall into one of these two categories. (if we need to include zlib as a dependency, this lowers the bar to entry to an external image library as a dependency as well).

@gronki Let's focus on PNG and libpng -- do you have example interfaces that we can look at? Would you like to contribute them to stdlib?

[certik]

The idea of PPM is that they made the format simple enough so that anybody can easily read and write it from any language. Then you can use external tools to convert it to more common formats.

The cost of any dependency should not be underestimated. For example, I would suggest to make such dependencies optional, at least initially. Not every (future) user of stdlib wants to use images, and having them require to install 10 libraries for the 10 image formats that we will support (but they don't need), that I think is not worth it.

[jvdp1]

I would suggest to avoid non-optional dependencies as much as possible, except if there are installed on several OSes per default (or are very common and expected to be already installed).
Otherwise, if a user wants to use stdlib that depends on (exotic) dependencies, on a machine on which he is not an admin (e.g. on a HPC), that user may have issues to install the depencies and stdlib itself. stdlib should help the user with Fortran, not to learn him how to install various libraries (automatically or not;locally or not).
Having said that, I also think we should avoid to "translate" existing and well-developed C-librairies into Fortran.
Optional dependencies would then be a good solution IMHO.

[jacobwilliams]

I also agree that we shouldn't re-implement rock solid libs written in other languages (I was also thinking of something like libcurl). The stdlib CMake project needs to be set up so that if you don't have some external lib installed, it builds it without that component.

But, also keep in mind that this is a volunteer project. If somebody wants to spend their time writing the world's most amazing Fortran PPM interface, I have absolutely no objection to them doing that. :)

[certik]

Looks like we are all in agreement on the following:

1. Let's figure out the natural API, no matter how it is actually implemented

2. Let's start using external libraries optionally to implement as many formats as we can. This should always be available as an option.

3. Let's also have pure Fortran backends, as people voluntarily write them. (E.g., the PPM one is already written.) Setup our CMake / Make so that people can choose which one to use (whether our "reference" pure Fortran implementation if available, or a 3rd party library if available).

Let me know if you all agree with the above, and if you think this will satisfy everybody.

[urbanjost]

M_pixel

was designed as a simple self-contained vector library for PPM files for use in a graphics class. The Fortran-based GIF utilities mentioned previously worked great in conjunction with it for making simple GIF and animated GIF examples. PPM is very good (of course) for programs that are basically manipulating pixels; this could use some polish and needs a higher-level axis routine added, and prettier fonts but if anyone ones something in self-contained Fortran to start with feel free to take a look.

In a batch environment the venerable NetPBM and ImageMagik filters worked seamlessly to convert the PPM files to just about any image file format around. With command-level packages like that the PPM format was basically just as good for single-image graphics files as any other, and easiest to convert to other formats. It is a bit verbose so it was almost always converted to something else on the spot. We actually let the programs write multiple images sequentially to a single file and then split them with a post-processor to make multi-plot formats like Adobe PDF. The main irritation with PPM files is they are only designed to represent one frame.

It was designed to be compatible (Originally) with the VOGLE/M_graph library, which supports a lot of vector output formats but very little pixel-level graphics; so anyone that used VOGLE in the past
should find it very familiar.

[vmagnin]

2\. Let's start using external libraries optionally to implement as many formats as we can.

The C GdkPixbuf library can save files in "jpeg", "png", "tiff", "ico" or "bmp":
https://developer.gnome.org/gdk-pixbuf/stable/

See my Discourse post for a Fortran example:
https://fortran-lang.discourse.group/t/making-computer-graphics-in-fortran-without-gui-just-creating-a-png/

[ivan-pi]

I think Portable pixmaps would be valuable. I used a portable graymap (P2) in a recent Discourse thread.

Now I doubt a Fortran-lang Netpbm module would ever be standardized, but it's just nice to have something light-weight you can drop in to a project, without having to install any libraries.

Here is a very basic example, with the image given as an array of characters:

   !> Save as plain Portable GrayMap (use .pgm extension)
   subroutine save_pgm(filename,icode,ngray)
      character(len=*), intent(in) :: filename
      character(1), intent(in) :: icode(:,:)
      integer, intent(in) :: ngray
         ! number of gray values

      integer :: i, j, unit

      open(newunit=unit,file=filename,status='unknown')

      ! a description of the format is available at
      !   https://en.wikipedia.org/wiki/Netpbm

      ! header part
      write(unit,'(A)') 'P2'
      write(unit,'(I0,1X,I0)') size(icode,1), size(icode,2)
      write(unit,'(I0)') ngray

      ! pixel data, gray values
      do j = 1, size(icode,2)
         write(unit,'(*(A1,:,1X))') (icode(i,j), i = 1, size(icode,1))
      end do

      close(unit)

   end subroutine

I would note that the "official" library for reading and writing Netpbm files is libnetpbm.

[milancurcic]

There's also stb_image.

[ivan-pi]

That's a nice find for C programmers. There is also PNM in C++ (similar single source file concept). But what we really want is something Fortran-friendly. Implementation issues aside, as has been said times and times before it's mainly about having a community-approved interface.

Having a quick and simple way to export images can be very handy for developing and demonstrating PDE solvers. Here's an example in C++ where the authors included a BMP writer, directly in their single source file heat-equation solver:
https://gitlab.com/unigehpfs/paralg/-/blob/master/heatEquation/heatEquation.cpp#L287
Obviously, custom code like this doesn't belong in stdlib. I just wanted mention it as a use case which would have been valuable for me in the past.

When I first learned Fortran, I wanted to read a gray image in BMP format as the initial conditions for a fluid simulation. All I could find where uncomplete snippets on comp.lang.fortran (https://groups.google.com/g/comp.lang.fortran/c/bvrvFLCT-uc/m/fU9jQTR8u8oJ). Implementing something from this like scratch was just not something I was capable of at the time. In Python or Matlab on the other hand it might have been as simple as imread("file.bmp").

[lewisfish]

I've started making bindings to stb and an image container for my own use. The project is still in a rough stage currently. Can currently save images as PNG, BMP, TGA and can read a bunch of formats.

[certik]

Yes, the ppm, pbm style is super nice, because it is text only, easy to write, easy to read. And you can use other software to convert to/from png or other formats. We definitely need this as part of stdlib.