Support for Fortran-layout bigarrays

[yallop]

This PR adds support for Fortran-layout bigarrays.

Background: existing bigarray support in ctypes

Bigarrays are already closely integrated into ctypes. For example, it is possible to

• store bigarrays in larger ctypes values (e.g. as struct fields or array elements)
• pass pointers to bigarrays to C functions
• return pointers to bigarrays from C functions
• allocate bigarray values using ctypes
• create bigarray values from arbitrary pointers
• convert between bigarrays and ctypes arrays

All these features are available via a small number of operations in the interface:

• bigarray, genarray, array1, array2, array3 construct bigarray type descriptions
• bigarray_start and bigarray_of_ptr convert between bigarrays and pointers
• bigarray_of_array and array_of_bigarray convert between bigarrays and ctypes arrays

These operations construct values that can be used with the generic ctypes functions for operating on C types and values.

Background: bigarray type descriptions

OCaml bigarrays are divided into several classes with distinct types: Array1.t, Array2.t, Array3.t and Genarray.t. In ctypes these different types of array are all described by a single type, bigarray_class, which is indexed by information about the type it describes. For example, here is the type of the description of Array2.t:

val array2 :
  < element: 'a;
    ba_repr: 'b;
    bigarray: ('a, 'b, Bigarray.c_layout) Bigarray.Array2.t;
    carray: 'a carray carray;
    dims: int * int > bigarray_class

The various fields of the object -- element, ba_repr, dims, etc. -- describe the characteristics of the Array2.t type. For example, dims: int * int indicates that the dimensions are represented as a pair of integers (since Array2.t has rank 2), and carray:a carray carrayindicates that anArray2.t` value can be converted to a ctypes array of arrays.

These fields are used to constrain the types of the generic bigarray operations in ctypes. For example, here is the type of array_of_bigarray:

val array_of_bigarray : < element: _;
                          ba_repr: _;
                          bigarray: 'b;
                          carray: 'c;
                          dims: _ > bigarray_class -> 'b -> 'c

Since array_of_bigarray is parameterised by a bigarray_class value it can operate on any kind of bigarray. The return type of the function, 'c is constrained to match the value of the carray field in the bigarray description, and so it varies with the type of the first argument. For example, array_of_bigarray can convert a rank-1 bigarray to a ctypes array:

# array_of_bigarray array1;;
- : ('_a, '_b, Bigarray.c_layout) Bigarray.Array1.t -> '_a carray =
<fun>

or convert a rank-2 bigarray to a ctypes array of arrays:

# array_of_bigarray array2
- : ('_a, '_b, Bigarray.c_layout) Bigarray.Array2.t -> '_a carray carray
= <fun>

Interface changes

OCaml provides support for both C-layout (0-indexed, row-major) and Fortran-layout (1-indexed, column-major) bigarrays. However, at present all of the ctypes bigarray operations only support C layout.

This PR generalizes the interface to add support for Fortran layout. The changes fall into two classes. First, the types of most of the bigarray operations are generalized to operate on either C-layout or Fortran-layout bigarrays. This involves the addition of an extra field, layout, to the bigarray_class index. For example, here is the new type of array2:

val array2 :
  < element: 'a;
    ba_repr: 'b;
    layout: 'l;
    bigarray: ('a, 'b, 'l) Bigarray.Array2.t;
    carray: 'a carray carray;
    dims: int * int > bigarray_class

The operations genarray, array1, array2, array3 and bigarray_start are generalized in this way. Second, two functions, bigarray and bigarray_of_ptr, which could not be generalized without breaking backwards compatibility now have new Fortran-layout counterparts. These functions could not be generalized because they construct bigarray type representations or bigarray values, which requires supplying them with information about layout. It would have been possible to add an additional layout parameter to the functions, but to avoid breaking existing code this PR instead adds new functions fortran_bigarray and fortran_bigarray_of_ptr.

Finally, the operations bigarray_of_array and array_of_bigarray have not been generalized, since they convert between bigarrays and ctypes arrays, which are defined as having C layout. If necessary, it is possible to convert between Fortran-layout bigarrays and ctypes arrays in two steps, by using one of the Bigarray change_layout functions, or by a combination of bigarray_start and fortran_bigarray_of_ptr.

@nilsbecker, @berke: comments on the interface would be very welcome here, if you have time to take a look.

This closes #509.

[nilsbecker]

when reading, i was wondering if bigarray and bigarray_of_ptr could be generalized by introducing an optional layout argument which defaults to C-layout. is this too hackish/ugly/slow?

[yallop]

I'd have liked to have done that, but I don't think it's possible, unfortunately, since c_layout and fortran_layout have different types. This thread has some related discussion.

[nilsbecker]

ah ok. wasn't aware that they are actually different types. i think i kind of get why it can't work. the function would have to be polymorphic but this is not possible as an optional argument. well. other option would be to add not the fortran functions but general functions with non-optional layout argument. then potentially deprecate the c-only function. it's more symmetric eventually but i suspect that the vast majority of uses will be c layout (?) so maybe forcing another argument would be intrusive...

[yallop]

Yes; I went back and forth a bit on this, but that was also my conclusion in the end.