Asynchronous file I/O

[SDLBugzilla]

This bug report was migrated from our old Bugzilla tracker.

Reported in version: HG 2.1
Reported for operating system, platform: All, All

Comments on the original bug report:

On 2014-02-24 01:55:02 +0000, Nathaniel Fries wrote:

Currently, file RWops are blocking and synchronous (just like what's provided by the C FILE* and C++ fstream). This is all fine in good for non-interactive programs and initialization-time loading. But most applications of SDL are interactive, and many are games and media players, which need to read from the harddisk while remaining interactive.

Because file RWops are blocking (take control away from the main thread), reading too much data from the hard disk on the main thread makes programs non-interactive. This leads to messy solutions like creating another thread just to load data ( "Main thread renders loading animation while background thread uploads whole level along with textures. In fact I did notice that this takes slightly longer than doing everything in main but user experience is much better with main thread still operational, showing anims and gameplay tips.
": http://forums.libsdl.org/viewtopic.php?t=9897 ).

So, I request the addition of asynchronous I/O features to SDL.

POSIX non-blocking I/O is simple: pass the flag O_NONBLOCK to open(), and all calls to read/write will return without waiting for the disk (the return value will be the number of bytes written).
Asynchronous file I/O was added to POSIX in 2001. Most versions of the interface do not work on sockets. See: http://pubs.opengroup.org/onlinepubs/009604599/basedefs/aio.h.html
Non-blocking I/O can be multiplexed using select() in order to reduce the number of read()/write() syscalls: http://pubs.opengroup.org/onlinepubs/7908799/xsh/select.html
With non-blocking I/O, SDL will have to maintain the total number of bytes written in order to determine when an I/O operation has completed.

Windows does not support synchronous (in order) non-blocking I/O. Instead it only supports asynchronous (out-of-order, or in Windows terms, overlapped) I/O. This is implemented by passing FILE_FLAG_OVERLAPPED to CreateFile and maintaining an OVERLAPPED structure for each I/O operation. See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365683%28v=vs.85%29.aspx
There are several ways to get indications of I/O completions using asynchronous I/O. One is to associate a kernel event with the OVERLAPPED structure passed to ReadFile/WriteFile. Another is to call GetOverlappedResult.
But because an application that has real use of asynchronous I/O is probably performing operations on multiple files, it would be better to reduce the number of syscalls. This means using Read/WriteFileEx and entering an alertable wait state in the main loop (a new API function, SDL_PumpIO, for other threads; and probably inside SDL_PumpEvents) to execute I/O completion handlers. See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa363772%28v=vs.85%29.aspx

It is entirely possible to implement this using the existing SDL_RWops interface. But an asynchronous SDL_RWop would break old code using SDL_RWops by its nature.

Instead, I recommend an additional API just for asynchronous file ops. While this could certainly be done as a third party library, I feel that it would be a better fit for the main library, using the SDL event queue to report completed/failed async ops with a new event.

On 2017-08-14 20:42:19 +0000, Sam Lantinga wrote:

This is a good idea for SDL 2.1. Do you have a proposed first pass implementation?

[icculus]

I might take a shot at this for 2.0.20 (2.0.18 is getting quite heavy already).

[icculus]

Actually, let's bump this to SDL3, if we're serious about starting on that soon.

[nfries88]

Most of what I wrote in that proposal won't work for regular files because unfortunately as far as the OS is concerned, those are always ready to read and write. I learned that trying to do this myself. So non-blocking I/O isn't viable.

Given developments in cloud storage and asset streaming, having a generalized extensible asynchronous interface may be even more valuable than just asynchronous disk I/O.

Because of how file caching works, async file I/O is most valuable for concurrent random reads and large reads and writes, and we can just ignore offsets required by a random read/write interface when working on a socket or whatever.

We can also minimize blocking times for reads by slicing large reads into smaller reads no larger than the maximum readahead window. Writes generally don't block because they're cached by the OS, but that might not be true on all targets and the OS can run out of cache if we're doing a lot of I/O, so breaking up writes might be prudent too.

Fortunately there's a lot of systems with support for real asynchronous I/O now: Windows has had IOCP since before XP, macOS and iOS have had dispatch_io in the C interface to Grand Central Dispatch for like a decade, Linux has had io_uring for a couple of years, and FreeBSD has has a kernel implementation of POSIX aio for quite awhile (it was only an optional module for a long time, but that changed in FreeBSD 11 iirc), despite the problems with that interface. Unfortunately when it comes to Android assets, I think the only real choice is between letting the AssetManager mmap them or doing normal blocking reads at the current file offset, but it's been a few years since I even touched any code running on Android.

So I propose something of this sort: SDL provides an async-exclusive user-overloadable interface similar to RWops, and another that manages them, something like:

struct SDL_Proactor;

typedef struct SDL_IOObject
{
void* impldata;
void* userdata;
struct SDL_Proactor* proactor;
// function pointers generally set up by the proactor
void (*asyncreadsome)(void* self, void* buffer, Sint64 offset, Sint64 size, SDL_IOOperation* state);
void (*asyncwritesome)(void* impl, void* buffer, Sint64 offset, Sint64 size, SDL_IOOperation* state);
void (*asyncsync)(void* impl, SDL_IOOperation* state); // flushwritebuffers/fsync for local files, might be more important for remote files
void (*asyncclose)(void* impl, SDL_IOOperation* state); // 
} SDL_IOObject

typedef struct SDL_Proactor
{
    // intermediary function pointers for SDL_Async* and ioobject->async*
};

void SDL_AsyncRead(SDL_IOObject* ioobject, void* buffer, Sint64 offset, Sint64 size, SDL_IOOperation* state, SDL_IOCallback)
 // etc

SDL_IOOperation should probably be made to contain enough information that we don't need to pass the buffer, offset, and size arguments around like that; if you think this kind of interface is something worth me sinking some time into mocking up better let me know.

[nfries88]

I recommend adjusting this to "asynchronous storage I/O". Steamworks has asynchronous I/O for its storage interface and emscripten's asynchronous APIs are a good fit for the storage interface as well.

The most beneficial use case of asynchronous disk I/O to simpler games, which is enabling concurrent random reads in a large on-disk file, is most typically used for loading assets packed into an archive format. The storage interface is quite suitable for this broader pattern IMO. For this pattern a relatively naiive threadpool using blocking native position-independent file I/O functions (eg ReadFile/pread) may be suboptimal but is generally adequate, it is reasonable to leave implementation of such an interface to third parties.

The other main beneficial use case of asynchronous disk I/O would be for streaming large numbers of assets, or perhaps a more modest very large assets, within a handful of frames. I don't think a naive implementation would be adequate here, but I also have my doubts that a simple general purpose interface would be terribly satisfactory either. This kind of need is probably best served by a tightly integrated asset loading pipeline that can exploit backend-specific peculiarities throughout, which seems out of scope for SDL.

[icculus]

This is where I am. I'll probably tweak this as I implement this.

/**
 * The asynchronous I/O operation structure.
 *
 * This operates as an opaque handle. There are several APIs to create various
 * types of I/O, or an app can supply an SDL_IOAsyncInterface to
 * SDL_OpenIOAsync() to provide their own async implementation behind this
 * struct's abstract interface.
 *
 * \since This struct is available since SDL 3.0.0.
 */
typedef struct SDL_IOAsync SDL_IOAsync;

typedef enum SDL_IOAsyncStatus {
    SDL_IOASYNC_PENDING,
    SDL_IOASYNC_COMPLETE,
    SDL_IOASYNC_FAILURE,
    SDL_IOASYNC_CANCELLED
} SDL_IOAsyncStatus;


/* This entire struct is read-only! It will be updated by SDL in calls to SDL_GetIOAsyncResults. */
typedef struct SDL_IOAsyncTask
{
    SDL_IOAsync *async;
    SDL_IOAsyncStatus status;
    Uint64 offset;
    void *buffer;
    Uint64 requested_size;
    Uint64 result_size;
    void *internal;
} SDL_IOAsyncTask;

/**
 * The function pointers that drive an SDL_IOAsync.
 *
 * Applications can provide this struct to SDL_OpenIOAsync() to create their own
 * implementation of SDL_IOAsync. This is not necessarily required, as SDL
 * already offers several common types of I/O streams, via functions like
 * SDL_IOAsyncFromFile().
 *
 * \since This struct is available since SDL 3.0.0.
 */
typedef struct SDL_IOAsyncInterface
{
    /**
     * Read up to `size` bytes from the data stream to the area pointed
     * at by `ptr`.
     *
     * \return 0 if async task has started, or -1 on error.
     */
    int (SDLCALL *read)(void *userdata, void *ptr, Uint64 offset, Uint64 size, SDL_IOAsyncTask *task);

    /**
     * Write exactly `size` bytes from the area pointed at by `ptr`
     * to data stream.
     *
     * On an incomplete write, you should set `*status` to a value from the
     * SDL_IOStatus enum. You do not have to explicitly set this on
     * a complete, successful write.
     *
     * \return 0 if async task has started, or -1 on error.
     */
    int (SDLCALL *write)(void *userdata, const void *ptr, Uint64 offset, Uint64 size, SDL_IOAsyncTask *task);

    /**
     * Request an async i/o task be cancelled.
     *
     * Note that a task might complete normally between when this request
     * is made and when the cancellation can be processed. The caller should
     * still call task_results() to see the final result; the buffer used
     * for this task must remain available until the task is no longer marked
     * as PENDING (whether it completes, fails, or cancels).
     */
    int (SDLCALL *cancel_task)(void *userdata, SDL_IOAsyncTask *task);

    /**
     * This method is called to query if a task has completed. The
     * implementation must update the fields of `task` and _must not block_.
     */
    int (SDLCALL *task_results)(void *userdata, SDL_IOAsyncTask *task);

    /**
     * Close and free any allocated resources.
     *
     * This will cancel any pending tasks, including writes!
     *
     * This will _block_ until all pending tasks have cancelled or completed.
     *
     * \return 0 if successful or -1 on write error when flushing data.
     */
    int (SDLCALL *close)(void *userdata);
} SDL_IOAsyncInterface;


/**
 * Create a custom SDL_IOAync.
 *
 * Applications do not need to use this function unless they are providing
 * their own SDL_IOAsync implementation. If you just need an SDL_IOAsync to
 * read/write a common data source, you should use the built-in
 * implementations in SDL, like SDL_IOAsyncFromFile(), etc.
 *
 * You must free the returned pointer with SDL_CloseIOAsync().
 *
 * This function makes a copy of `iface` and the caller does not need to keep
 * this data around after this call.
 *
 * \param iface the function pointers that implement this SDL_IOAsync.
 * \param userdata the app-controlled pointer that is passed to iface's
 *                 functions when called.
 * \returns a pointer to the allocated memory on success or NULL on failure;
 *          call SDL_GetError() for more information.
 *
 * \since This function is available since SDL 3.0.0.
 *
 * \sa SDL_CloseIOAsync
 * \sa SDL_IOAsyncFromFile
 */
extern SDL_DECLSPEC SDL_IOAsync * SDLCALL SDL_OpenIOAsync(const SDL_IOAsyncInterface *iface, void *userdata);


/**
 * Use this function to create a new SDL_IOAsync structure for reading from
 * and/or writing to a named file.
 *
 * The `mode` string understands the following values:
 *
 * - "r": Open a file for reading only. It must exist.
 * - "w": Open a file for writing only. It will create missing files or truncate existing ones.
 * - "a": Open a file for writing only. Appending doesn't mean anything here; this just doesn't truncate existing files.
 * - "a+": Open a file for reading and writing. Appending doesn't mean anything here; this just doesn't truncate existing files.
 *
 * There is no "b" mode, as there is only "binary" style i/o.
 *
 * This function supports Unicode filenames, but they must be encoded in UTF-8
 * format, regardless of the underlying operating system.
 *
 * \param file a UTF-8 string representing the filename to open.
 * \param mode an ASCII string representing the mode to be used for opening
 *             the file.
 * \returns a pointer to the SDL_IOAsync structure that is created or NULL on
 *          failure; call SDL_GetError() for more information.
 *
 * \since This function is available since SDL 3.0.0.
 *
 * \sa SDL_CloseIOAsync
 * \sa SDL_ReadIOAsync
 * \sa SDL_WriteIOAsync
 */
extern SDL_DECLSPEC SDL_IOAsync * SDLCALL SDL_IOAsyncFromFile(const char *file, const char *mode);


/**
 * Start an async read.
 *
 * This function reads up `size` bytes from `offset` position in the data
 * source to the area pointed at by `ptr`. This function may read less bytes
 * than requested.
 *
 * This function returns as quickly as possible; it does not wait for the
 * read to complete. On a successful return, `task` will be initialized
 * and can be used to manage the asynchronous work. Even failure is
 * asynchronous: a failing return value from this function means the task
 * couldn't start at all.
 *
 * `ptr` and `task` must remain available until the task is done, and may be
 * written to by the system at any time until the task is finished. Do not
 * allocate either on the stack!
 *
 * \param context a pointer to an SDL_IOAsync structure.
 * \param ptr a pointer to a buffer to read data into.
 * \param offset the position to start reading in the number of bytes to read
 *               from the data source.
 * \param size the number of bytes to read from the data source.
 * \returns 0 on success or a negative error code on failure; call
 *          SDL_GetError() for more information.
 *
 * \since This function is available since SDL 3.0.0.
 *
 * \sa SDL_WriteIOAsync
 * \sa SDL_GetIOStatus
 */
extern SDL_DECLSPEC int SDLCALL SDL_ReadIOAsync(SDL_IOAsync *context, void *ptr, Uint64 offset, Uint64 size, SDL_IOAsyncTask *task)

/**
 * Start an async write.
 *
 * This function writes up to `size` bytes to `offset` position in the data
 * source from the area pointed at by `ptr`.
 *
 * This function returns as quickly as possible; it does not wait for the
 * write to complete. On a successful return, `task` will be initialized
 * and can be used to manage the asynchronous work. Even failure is
 * asynchronous: a failing return value from this function means the task
 * couldn't start at all.
 *
 * `ptr` and `task` must remain available until the task is done, and may be
 * written to by the system at any time until the task is finished. Do not
 * allocate either on the stack!
 *
 * \param context a pointer to an SDL_IOAsync structure.
 * \param ptr a pointer to a buffer to write data from.
 * \param offset the position to start writing in the number of bytes to write
 *               to the data source.
 * \param size the number of bytes to read from the data source.
 * \returns 0 on success or a negative error code on failure; call
 *          SDL_GetError() for more information.
 *
 * \since This function is available since SDL 3.0.0.
 *
 * \sa SDL_ReadIOAsync
 */
extern SDL_DECLSPEC int SDLCALL SDL_WriteIOAsync(SDL_IOAsync *context, void *ptr, Uint64 offset, Uint64 size, SDL_IOAsyncTask *task)

/**
 * Check the status of an async task.
 *
 * This function will update `task` with appropriate status updates.
 *
 * \param task the task to query.
 * \returns 0 on success or a negative error code on failure; call
 *          SDL_GetError() for more information.
 *
 * \since This function is available since SDL 3.0.0.
 *
 * \sa SDL_ReadIOAsync
 */
extern SDL_DECLSPEC int SDLCALL SDL_GetIOAsyncResults(SDL_IOAsyncTask *task)

/**
 * Cancel an in-progress async i/o task.
 *
 * This will attempt to stop a task in progress. Cancellation is _also_
 * async; you still have to check for final results; further, there are no
 * guarantees that the task won't legitimately complete before the cancellation
 * request goes through.
 *
 * Cancelling a task might also means _some_ of the data was transferred.
 *
 * \param task the task to cancel.
 * \returns 0 on success or a negative error code on failure; call
 *          SDL_GetError() for more information.
 *
 * \since This function is available since SDL 3.0.0.
 *
 * \sa SDL_ReadIOAsync
 */
extern SDL_DECLSPEC int SDLCALL SDL_CancelIOAsync(SDL_IOAsyncTask *task);

/**
 * Close and free any allocated resources for an async I/O object.
 *
 * This will cancel any pending tasks, including writes!
 *
 * This will _block_ until all pending tasks have cancelled or completed.
 *
 * \param context a pointer to an SDL_IOAsync structure.
 * \returns 0 if successful or -1 on write error when flushing data.
 */
extern SDL_DECLSPEC int SDLCALL SDL_CloseIOAsync(SDL_IOAsync *context);

I'm also just realizing the Storage interface doesn't ever generate SDL_IOStreams (reads and writes are synchronous and intended to operate on an entire file at a time), but it seems like we need a way to read massive files in chunks and in the background (not just for disk i/o speed, but network i/o speed on cloud services). This can be a single extra API call that provides a storage-specific SDL_IOAsync pointer and that's it.

Using this API might look like this, sans error checking:

SDL_IOAsync *a = SDL_IOAsyncFromFile("level3.map", "r");

const Uint64 chunksize = 1024 * 1024 * 4;  // 4 megabytes.
Uint8 mapbuffer[MAP_BUFFER_SIZE];  // !!! FIXME: the API needs a way to ask for this number.
Uint64 offset = 0;
SDL_IOAsyncTask task;

// this reads this in chunks, but you could just read the whole thing in one block
// and wait for it to finish it. Or read multiple things from different parts of the file
// in separate tasks that run in parallel.
SDL_ReadIOAsync(a, mapbuffer, offset, chunksize, &task);

while (offset < MAP_BUFFER_SIZE) {
    SDL_GetIOAsyncResults(&task);
    if (task.status != SDL_IOASYNC_PENDING) { // one should check for errors here, too!
        offset += task.result_size;  // start the next read.
        SDL_ReadIOAsync(a, mapbuffer + offset, offset, chunksize, &task);
    }
    draw_the_loading_screen();  // this never slows down because the filesystem burped.
}

SDL_CloseIOAsync(a);

[nfries88]

A more efficient/scalable way of getting a completion notification would polish that off nicely. Polling each pending I/O task individually every frame gets pretty wasteful pretty quick. Aside for that I think you have a good interface.

[slouken]

My first thought is this isn’t simple. Do we need this in the SDL API?

[slouken]

My second thought is, maybe the callback based API fits in better with the rest of SDL, and this could be internal implementation?

[icculus]

My first thought is this isn’t simple.

There is no such thing as simple asynchronous i/o. :)

I'm making adjustments, then we'll make a call. Stay tuned.

[icculus]

Okay, the interface for creating your own still stands (apps usually don't need it, SDL talks to it internally).

The public API will have a callback...

typedef void (SDLCALL *SDL_IOAsyncCallback)(void *userdata, SDL_IOAsync *context, SDL_IOAsyncStatus status, void *ptr, Uint64 offset, Uint64 bytes_requested, Uint64 bytes_transferred);

...and read/write functions will take a callback...

extern SDL_DECLSPEC int SDLCALL SDL_WriteIOAsync(SDL_IOAsync *context, void *ptr, Uint64 offset, Uint64 size, SDL_IOAsyncCallback callback, void *userdata);

...when the task completes, it fires the callback, and the pointer needs to be valid until the callback fires.

We'll remove the ability to cancel tasks from the public API, so we don't have to provide a handle. Internally, cancelling will still exist so we can use it when closing a file that still has tasks in-flight.

So ignoring the implement-my-own-custom-IOAsync interface, the API looks like:

• Open a file for async i/o
• Read from an async i/o, trigger a callback when done.
• Write to an async i/o, trigger a callback when done.
• Close an async i/o.

extern SDL_DECLSPEC SDL_IOAsync * SDLCALL SDL_IOAsyncFromFile(const char *file, const char *mode);


typedef void (SDLCALL *SDL_IOAsyncCallback)(void *userdata, SDL_IOAsync *context, SDL_IOAsyncStatus status, void *ptr, Uint64 offset, Uint64 bytes_requested, Uint64 bytes_transferred);

extern SDL_DECLSPEC int SDLCALL SDL_ReadIOAsync(SDL_IOAsync *context, void *ptr, Uint64 offset, Uint64 size, SDL_IOAsyncCallback callback, void *userdata);
extern SDL_DECLSPEC int SDLCALL SDL_WriteIOAsync(SDL_IOAsync *context, void *ptr, Uint64 offset, Uint64 size, SDL_IOAsyncCallback callback, void *userdata);
extern SDL_DECLSPEC int SDLCALL SDL_CloseIOAsync(SDL_IOAsync *context);

...and using it looks like this (just read the whole thing in one big block instead of chunks)...

static void SDLCALL read_done(void *userdata, SDL_IOAsync *context, SDL_IOAsyncStatus status, void *ptr, Uint64 offset, Uint64 bytes_requested, Uint64 bytes_transferred)
{
    // you should check for errors, etc here.
    SDL_AtomicSet((SDL_AtomicInt *) userdata), 1);  // signal we are finished.
}

// ...

SDL_IOAsync *a = SDL_IOAsyncFromFile("level3.map", "r");
SDL_AtomicInt done;
const Uint64 chunksize = 1024 * 1024 * 4;  // 4 megabytes.
Uint8 mapbuffer[MAP_BUFFER_SIZE];  // !!! FIXME: the API needs a way to ask for this number.
Uint64 offset = 0;

SDL_AtomicSet(&done, 0);
SDL_ReadIOAsync(a, mapbuffer, offset, MAP_BUFFER_SIZE, read_done, &done);

while (!SDL_AtomicGet(&done)) {
    draw_the_loading_screen();  // this never slows down because the filesystem burped.
}

SDL_CloseIOAsync(a);

[slouken]

This looks pretty good. Can we also have SDL_LoadFileAsync()?
I assume that we'll have storage versions of these as well?

[icculus]

And we definitely want the storage interface to only load a whole file at once, right? The sync version only does the whole file, so I assume the async version should do the same, just without the blocking.

[slouken]

And we definitely want the storage interface to only load a whole file at once, right? The sync version only does the whole file, so I assume the async version should do the same, just without the blocking.

We should look at some cloud APIs and see what's common here.

[icculus]

And we definitely want the storage interface to only load a whole file at once, right? The sync version only does the whole file, so I assume the async version should do the same, just without the blocking.

We should look at some cloud APIs and see what's common here.

(Paging @flibitijibibo back into this conversation.)

[nfries88]

And we definitely want the storage interface to only load a whole file at once, right? The sync version only does the whole file, so I assume the async version should do the same, just without the blocking.

We should look at some cloud APIs and see what's common here.

Only supporting whole files for the storage interface seems like the right call.

IIRC Steam only supports writing a whole file, emscripten only supports reading and writing whole files, and Android doesn't support position-independent reads and writes to assets except by decompressing the entire file into a memory object and giving the caller a file descriptor.

For hypothetical backends that implement the storage interface over a compressed archive, it's generally impractical to support position-independent reads and writes.

Not sure about other potential cloud backends, but given the overhead of making a request in the first place it probably isn't worth doing for typical game assets or cloud saves even if they do support it. It may make sense for some other contexts though, so exposing it as an optional interface might be useful.

[slouken]

If the storage interface only does whole file, should the SDL API also be whole file? I assume this will be implemented on top of the storage interface?

[icculus]

For non-async stuff outside of the Storage interface, we have something that reads in pieces, and also SDL_LoadFile. Both are useful.

There are scenarios where you'd not want to suck a whole file into RAM (.zip and other packfiles, some massive dataset for, say, the game map of Spider-Man 2, etc).

[nfries88]

There are scenarios where you'd not want to suck a whole file into RAM (.zip and other packfiles...

right, and it would also be easier to implement async whole-file reads in a packfile storage backend in terms of position-independent async file reads than working around a seeking interface. (this is actually my own primary reason for wanting async file I/O in the first place). and performance will be better if the async reads actually temporally overlap (which is not strictly a given even with a conforming pread implementation, but at least we can try)