io_uring_prep_link_timeout return -EINVAL with valid timespec and flag

[buracchi]

A snippet to reproduce this behavior was not trivial, I could try to provide it if required.

I have a single thread worker running jobs that can submit SQEs into the worker io_uring.

One such job execute something equivalent to

    sqe = io_uring_get_sqe(ring);
    io_uring_prep_recvmsg(sqe, fd, &msghdr, 0);
    io_uring_sqe_set_data(sqe, &listener_job);
    io_uring_sqe_set_flags(sqe, IOSQE_IO_LINK);
    sqe = io_uring_get_sqe(ring);
    io_uring_prep_link_timeout(sqe, &timeout_timespec, 0);
    io_uring_sqe_set_data(sqe, &timeout_job);
    io_uring_submit(ring);

If I try to overload with a high enough number of parallel requests the worker (having a load balancing mechanism to guarantee the CQ to not overflow) after a sufficient ammount of execution time a cqe resulting from the timeout with a ret value of -EINVAL is produced.

The timeout_timespec is a valid memory location from the submission to the completion of the timeout entry and it is set to a reasonable timeout value.

As a side note, I once encountered the undocumented ret value -ESUCCESS from the linked timeout while testing the -EINVAL issue but I couldn't reproduce it again.

[axboe]

-EINVAL or -EINTR? Subject was one thing, body another. And by -ESUCCESS, do you mean 0?

[buracchi]

I didn't notice, my bad.
By -ESUCCESS I mean 0, yes.

[axboe]

Just a guess, is your timespec struct going out of scope before you submit the IO? That could cause garbage in the tv_sec/tv_nsec, and that could cause an -EINVAL to be returned (eg if either of them ended up being less than 0).

[buracchi]

At first I thought this was the problem too, so I replaced my previous compound literal expression

io_uring_prep_link_timeout(sqe, &(struct __kernel_timespec) {.tv_sec = 1}, 0);

with a named object whose lifetime persists for the entire duration of the program

io_uring_prep_link_timeout(sqe, &timeout_timespec, 0);

[axboe]

There aren't really any post-prep side -EINVAL possible for a linked timeout, hence why I was suspecting it's something that the prep side doesn't like (like, for example, an invalid timeout).

timeout_flags will always get set, and that's 0 in your case, so won't cause anything.
That then really leaves the timeout itself being invalid (less than 0 for tv_sec/tv_nsec), or not having something to link against. The linked timeout must be submitted within the same submit call as the request being linked against. Is it possible that you ever submit in between those two preparations?

Outside of those few conditions, nothing else would cause -EINVAL.