Rename memmap?

[timholy]

memmap is used for incremental writing. However, as pointed out in #79,it's not actually using Mmap.mmap and perhaps should be renamed. What about empty_tiff?

Alternatively, you can use (warning: untested)

img = open(filename, "w") do io
    Mmap.mmap(io, Gray{N0f16}, sz3)
end
for i = 1:n
    img[:,:,i] = create_slice_data(i, sz2)   # this will write to the file
end

This should work at least on Linux despite apparently closing the file before values are assigned; Linux will defer the close until img gets garbage-collected. I am not sure about other platforms.

Of course, that doesn't create a TIFF. You'd need to do some work to write the header and set up empty chunks for the slice planes, essentially as MmappedTIFF does in #79. Basically, you mmap the entire file-buffer and then assign slices to correctly-placed views within the full buffer. Of course, if it's possible to pack all slice planes densely, then just a single view will suffice for an entire 3d array.

This may be more of a pain than any benefit you'd derive from it, so just renaming memmap seems expeditious.

[tlnagy]

In light of #79, it would make sense to rename the memmap function. I don't love the empty_tiff name since it's not immediately apparent to me that it is on-disk instead of in memory. I would prefer

• TiffImages.incremental
• TiffImages.ondisk
• Something else?

Longer term I think it would be nice to add push! support to MmappedTIFF and unified all disk ops under that type. I'm fine with throwing a deprecation warning for memmap in v0.6.0 and then removing it in the next minor release.

[timholy]

I'd be fine with either of those names.

push! support to MmappedTIFF would be possible but it seems likely to have constraints: I think that once the file is mmapped its size is fixed. I'm not entirely sure, though.

So push! could work, but we'd have to know the final size at the beginning and keep track of the "existing size" to know where to insert new data.

[tlnagy]

Actually, the better solution is likely something like

img = empty(LazyTIFF, T, "filepath.tif")

where LazyTIFF is DiskTaggedImage renamed. I think that would be a more Julian approach.

What do you think?

[timholy]

That works for me!

[tlnagy]

Throwing the memmap deprecation and introducing the new empty syntax should probably happen before I tag v0.6.0 for release and then memmap can be removed in v0.7.0

[timholy]

I think you'll have to do the deprecation simultaneously with the 0.6.0 release. LazyBufferedTIFF will only exist in 0.6 and higher, and I don't think you really want to direct people to use empty(DiskTaggedImage, ...) for a single point release?

EDIT: or maybe I misunderstood you. I thought you were planning a 0.5.6 release in which memmap is deprecated, but perhaps not.

[tlnagy]

No, you understood me properly. I'm about to submit the PR for the new empty function plus deps, just have to actually push it 😅

I'll tag the v0.6.0 release then.