undefined behaviour due to LAPACK prototypes missing "hidden" variables required by Fortran

[conradsnicta]

The prototypes for Fortran LAPACK functions appear to be incomplete in Dlib. This can lead to undefined behaviour with recent releases of gcc due to more aggressive compiler optimisations.

According to gfortran passing conventions (followed by other fortran compilers as well), every char* argument also has an associated "hidden" argument which specifies the number of characters. The type of the "hidden" argument may vary across compilers and/or compiler versions. The position of each "hidden" argument is also compiler dependent, though seems to be typically tacked onto the end of the function definition.
https://gcc.gnu.org/onlinedocs/gfortran/Argument-passing-conventions.html

As an example in Dlib, the current prototype for dsyev in
https://github.com/davisking/dlib/blob/master/dlib/matrix/lapack/syev.h
is defined as:
void DLIB_FORTRAN_ID(dsyev) (const char *jobz, const char *uplo, const integer *n, double *a, const integer *lda, double *w, double *work, const integer *lwork, integer *info);

But it probably should be:
void DLIB_FORTRAN_ID(dsyev) (const char *jobz, const char *uplo, const integer *n, double *a, const integer *lda, double *w, double *work, const integer *lwork, integer *info, blas_len jobz_len, blas_len uplo_len);

where blas_len is typically a 32 bit unsigned int on 32 bit platforms, and 64 bit unsigned int on 64 bit platforms. (This does not apply to gcc versions <= 7, where it is always int).

Avoiding the use of the blas_len arguments appeared to work, but recently gcc has started to optimise more heavily, leading to stack overwrites when these arguments are not used.

The same bug affects Armadillo, R, and a lot of other software in C and C++ that uses LAPACK:

• https://gitlab.com/conradsnicta/armadillo-code/issues/123
• https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90329
• https://gcc.gnu.org/ml/gcc-patches/2019-05/msg00915.html
• C -> Fortran ABI issues in CBLAS & LAPACKE Reference-LAPACK/lapack#339

CC: @dodomorandi

[davisking]

Can you submit a PR for this?

[conradsnicta]

I got my hands full fixing the same issue in Armadillo :)

[davisking]

Ha, ok, thanks for the heads up :) Are there known cases where this results in incorrect runtime behavior?

[conradsnicta]

• New BLAS for some platforms have issues with upper-triangular inverses RcppCore/RcppArmadillo#254
• BLAS/LAPACK routine 'DTRTI2' gave error code -1 in test-omega-chol.R:14 nlmixrdevelopment/RxODE#84
• https://bugzilla.redhat.com/show_bug.cgi?id=1709538

The above first looked like a compiler problem.

A while back I've also had a few reports of strange crashes involving code compiled by non-gcc compilers. I couldn't reproduce them on my setup, but I now strongly suspect that the "hidden" args were the culprit.