mmap_array's 'offset' parameter cannot be used to skip a header

[descmdr]

I am using Julia 0.2.1.

The documentation for the mmap_array function states:

you can specify an offset (in bytes) if, for example, you want to skip over a header in the file.

However, the value is passed directly to the FileOffset parameters in the CreateFileMappingA call. According to MSDN,

The combination of the high and low offsets must specify an offset within the file mapping. They must also match the memory allocation granularity of the system. That is, the offset must be a multiple of the allocation granularity. To obtain the memory allocation granularity of the system, use the GetSystemInfo function, which fills in the members of a SYSTEM_INFO structure.

I haven't dug any deeper, but I have seen that the allocation granularity is 64K. So unless your header is precisely 64K, the offset value cannot be used for this purpose. I think this should be fixed.

There are two solutions I have off the top of my head. One is to query allocation granularity and mod it against the offset value. The other is to create another offset argument. Does anyone have a preference? I have written my own version of mmap_array that does the latter.

function mmap_array{T,N,TInt<:Integer}(::Type{T}, dims::NTuple{N,TInt}, s::IO, map_offset::FileOffset, array_offset::FileOffset)
    shandle = Base._get_osfhandle(RawFD(fd(s)))
    if int(shandle.handle) == -1
        error("Could not get handle")
    end
    ro = isreadonly(s)
    flprotect = ro ? 0x02 : 0x04
    len = prod(dims)*sizeof(T) + array_offset
    if len > typemax(Int)
        error("File is too large to memory-map on this platform")
    end
    szarray = convert(Csize_t, len)
    szfile = szarray + convert(Csize_t, map_offset)
    mmaphandle = ccall(:CreateFileMappingA, stdcall, Ptr{Void}, (Ptr{Void}, Ptr{Void}, Cint, Cint, Cint, Ptr{Void}), shandle.handle, C_NULL, flprotect, szfile>>32, szfile&0xffffffff, C_NULL)
    if mmaphandle == C_NULL
        error("Could not create file mapping")
    end
    access = ro ? 4 : 2
    viewhandle = ccall(:MapViewOfFile, stdcall, Ptr{Void}, (Ptr{Void}, Cint, Cint, Cint, Csize_t), mmaphandle, access, map_offset>>32, map_offset&0xffffffff, szarray)
    if viewhandle == C_NULL
        error("Could not create mapping view")
    end
    A = pointer_to_array(pointer(T, viewhandle + array_offset), dims)
    finalizer(A, x->Base.munmap(viewhandle, mmaphandle))
    return A
end

[tkelman]

closed by #7242 (apparently github doesn't auto-close b if you do "fixes #a and #b")