Merge pull request #745 from kjbracey-arm/FileHandle

Add FileHandle/poll documentation

Author: Amanda Butler

docs/api/platform/FileHandle.md

@@ -0,0 +1,214 @@
+## FileHandle
+
+For general information on [files](file.html) and [filing systems](filesystem.html) in persistant storage, see their documentation. This chapter covers the abstract API, with an emphasis on devices.
+
+`FileHandle` is an abstract class representing a device that supports file-like operations, such as `read` and `write`. This may be an actual `File` on a storage device provided by a `FileSystem` or a device such as `UARTSerial`.
+
+The `FileHandle` abstraction represents an already-opened file or device, so it has no `open` method of its own - the opening may take the form of a call to another class that returns a `FileHandle`, such as `FileSystem::open`, or it may be implicit in the construction of an object such as `UARTSerial`.
+
+The `FileHandle` abstraction permits stream-handling code to be device-independent, rather than tied to a specific device like a serial port. Examples of such code in Mbed OS are:
+
+- The console input and output streams (`stdin` and `stdout`).
+- The `ATCmdParser` helper.
+- The PPP connection to lwIP.
+
+Exactly which operations a `FileHandle` supports depends on the underlying device, and in turn restricts what applications it is suitable for. For example, a database application might require random-access and `seek`, but this may not be available on a limited file system, and certainly not on a stream device. Only a `File` on a full `FileSystem`, such as `FATFileSystem`, would generally implement the entire API. Specialized devices may have particular limitations or behavior, which limit their general utility. Devices that do not implement a particular call indicate it by an error return - often `ENOSYS`, but sometimes more specific errors, such as `ESPIPE` apply; please see the POSIX specifications for details.
+
+### Relationship of FileHandle to other APIs
+
+You can use a `FileHandle` directly, or you can use standard POSIX or C/C++ APIs to manipulate it. Stdio calls taking `FILE *stream` call the POSIX APIs taking `int fd`, which call methods on `FileHandle` objects.
+
+<span class="images">![](https://s3-us-west-2.amazonaws.com/mbed-os-docs-images/filehandle_callstack.png)</span>
+
+The `FileHandle` may be implicitly created by a higher layer, as in a call to `fopen`. In this case, the name lookup produces a `FileHandle` and POSIX file descriptor internally.
+
+The three APIs provide different levels of capability:
+
+API                            | C/C++                   | POSIX                  | Mbed
+-------------------------------|-------------------------|------------------------|-----------------------------
+Headers                        | stdio.h, iostream       | mbed_retarget.h        | FileHandle.h, mbed_poll.h
+Main type                      | FILE *                  | int                    | FileHandle object
+Blocking I/O                   | Yes (always)            | Yes (default)          | Yes (default)
+Nonblocking I/O                | No                      | Yes                    | Yes
+Poll                           | No                      | Yes (struct pollfd)    | Yes (struct pollfh)
+Sigio                          | No                      | No                     | Yes
+Device-specific extensions     | No                      | No                     | Possible using derived types
+Newline conversion             | Yes (enabled with JSON) | No                     | No
+Error indications              | EOF, ferror, set errno  | Return -1, set errno   | Return negative error code
+Portability                    | High                    | Medium                 | Low
+
+You can mix the APIs if you're careful, for example setting up a callback initially with `FileHandle::sigio` but performing all subsequent operations using POSIX.
+
+<span class="notes">**Note:** `errno` is not thread-local on all toolchains. This may cause problems with error handling if multiple threads are using POSIX or C file APIs simultaneously.</span>
+ 
+### Mapping between APIs
+
+Calls are provided to attach already-opened lower levels to the higher levels:
+
+- `int mbed_bind_to_fd(FileHandle *)` bind a FileHandle to a POSIX file descriptor.
+- `FILE *fdopen(int fd, const char *mode)` bind a POSIX file descriptor to a stdio FILE.
+- `FILE *fdopen(FileHandle *fh, const char *mode)` bind a FileHandle to a stdio FILE.
+
+The standard POSIX function `int fileno(FILE *stream)` may be available to map from `FILE` to file descriptor, depending on the toolchain and C library in use - it is not usable in fully portable Mbed OS code.
+
+As it is not possible to map from higher levels to lower levels, if code needs to access the lower levels, use a lower-level open call, so the lower-level handle is known. Then, bind that to the higher level..
+
+The POSIX file descriptors for the console are available as `STDIN_FILENO`, `STDOUT_FILENO` and `STDERR_FILENO`, permitting operations such as `fsync(STDERR_FILENO)`, which would for example drain `UARTSerial`s output buffer.
+
+### Redirecting the console
+
+If a target has serial support, by default a serial port is used for the console. The pins and settings for the port selection come from target header files and JSON settings. This uses either an internal `DirectSerial` if unbuffered (for backwards compatibility) or `UARTSerial` if `platform.stdio-buffered-serial` is `true`.
+
+The target can override this by providing `mbed::mbed_target_override_console` to specify an alternative `FileHandle`. For example, a target using SWO might have:
+
+```
+    namespace mbed
+    {
+        FileHandle *mbed_target_override_console(int)
+        {
+            static SerialWireOutput swo;
+            return &swo;
+        }
+    }
+```
+
+Then any program using `printf` on that target sends its output over the SWO, rather than serial.
+
+Because targets can redirect the console in this way, portable applications should not use constructs like `Serial(USBTX, USBRX)`, assuming that this will access the console. Instead they should use `stdin`/`stdout`/`stderr` or `STDIN_FILENO`/`STDOUT_FILENO`/`STDERR_FILENO`.
+
+```
+    // Don't do:
+    Serial serial(USBTX, USBRX);
+    serial.printf("Hello!\r\n");
+    
+    // Do do:
+    printf("Hello!\n");  // assume platform.stdio-convert-newlines is true
+```
+
+Beyond the target-specific override, an application can override the target's default behavior itself by providing `mbed::mbed_override_console`. 
+
+Alternatively, an application could use the standard C `freopen` function to redirect `stdout` to a named file or device while running. However there is no `fdreopen` analogue to redirect to an unnamed device by file descriptor or `FileHandle` pointer. 
+
+### Polling and nonblocking
+
+By default `FileHandle`s conventionally block until a `read` or `write` operation completes. This is the only behavior supported by normal `File`s, and is expected by the C library's `stdio` functions.
+
+Device-type `FileHandle`s, such as `UARTSerial`, are expected to also support nonblocking operation, which permits the `read` and `write` calls to return immediately when unable to transfer data. Please see the API reference pages of these functions for more information.
+
+For a timed wait for data, or to monitor multiple `FileHandle`s, see [`poll`](poll.html)
+
+### Event-driven I/O
+
+If using nonblocking I/O, you probably want to know when to next attempt a `read` or `write` if they indicate no data is available. `FileHandle::sigio` lets you attach a `Callback`, which is called whenever the `FileHandle` becomes readable or writable.
+
+Important notes on sigio:
+
+- The sigio may be issued from interrupt context. You cannot portably issue `read` or `write` calls directly from this callback, so you should queue an [`Event`](event.html) or wake a thread to perform the `read` or `write`.
+- The sigio callback is only guaranteed when a `FileHandle` _becomes_ readable or writable. If you do not fully drain the input or fully fill the output, no sigio may be generated. This is also important on start-up - don't wait for sigio before attempting to read or write for the first time, but only use it as a "try again" signal after seeing an `EAGAIN` error.
+- Spurious sigios are permitted - you can't assume data will be available after a sigio.
+- Given all the above, use of sigio normally implies use of nonblocking mode or possibly `poll`. 
+
+Ordinary files do not generate sigio callbacks because they are always readable and writable.
+
+### Stream-derived FileHandles
+
+`Stream` is a legacy class that provides an abstract interface for streams similar to the `FileHandle` class. The difference is that the `Stream` API is built around the `getc` and `putc` set of functions, whereas `FileHandle` is built around `read` and `write`. This makes implementations simpler but limits what is possible with the API. Because of this, implementing the `FileHandle` API directly is suggested API for new device drivers.
+
+Note that `FileHandle` implementations derived from `Stream`, such as `Serial`, have various limitations:
+
+- `Stream` does not support nonblocking I/O, poll or sigio.
+- `Stream` does not have correct `read` semantics for a device - it always waits for the entire input buffer to fill.
+- `Stream` returns 0 from `isatty`, which can slightly confuse the C library (for example defeating newline conversion and causing buffering).
+
+As such, you can only use `Stream`-based devices for blocking I/O, such as through the C library, so we don't recommend use of `Stream` to implement a `FileHandle` for more general use.
+ 
+### FileHandle class reference
+
+[![View code](https://www.mbed.com/embed/?type=library)](http://os-doc-builder.test.mbed.com/docs/development/mbed-os-api-doxy/classmbed_1_1_file_handle.html)
+
+### FileHandle using C library example
+
+```
+// Continuously monitor a serial device, and every time it outputs a
+// character, send it to the console and toggle LED2. Can use the C library
+// to access the device as only using blocking I/O.
+//
+// Note that the console is accessed using putchar - this will be accessing
+// a FileHandle-based device under the surface, but the particular device can be
+// target-dependent. This makes the program portable to different devices
+// with different console types, with the only target-dependence being
+// knowledge of which pins the serial device we're monitoring is attached to,
+// which can be configured using JSON.
+
+static DigitalOut led2(LED2);
+
+// UARTSerial derives from FileHandle
+static UARTSerial device(MBED_CONF_APP_DEVICE_TX, MBED_CONF_APP_DEVICE_RX);
+
+int main()
+{
+    // Perform device-specific setup
+    device.set_baud(19200);
+    
+    // Once set up, access through the C library
+    FILE *devin = fdopen(&device, "r");
+    
+    while (1) {
+        putchar(fgetc(devin));
+        led2 = !led2;
+    }
+}
+```
+
+### FileHandle sigio example
+
+```
+// Main thread flashes LED1, while we monitor a serial-attached device
+// in the background. Every time that device outputs a character, we echo
+// it to the console and toggle LED2.
+#include "mbed.h"
+
+static DigitalOut led1(LED1);
+static DigitalOut led2(LED2);
+
+static UARTSerial device(MBED_CONF_APP_DEVICE_TX, MBED_CONF_APP_DEVICE_RX);
+
+static void callback_ex()
+{
+    // always read until data is exhausted - we may not get another
+    // sigio otherwise
+    while (1) {
+        char c;
+        if (device.read(&c, 1) != 1) {
+            break;
+        }
+        putchar(c);
+        led2 = !led2;
+    }
+}
+
+int main()
+{
+    // UARTSerial-specific method - all others are from FileHandle base class
+    device.set_baud(19200);
+    
+    // Ensure that device.read() returns -EAGAIN when out of data
+    device.set_blocking(false);
+    
+    // sigio callback is deferred to event queue, as we cannot in general
+    // perform read() calls directly from the sigio() callback.
+    device.sigio(mbed_event_queue()->event(callback_ex));
+
+    while (1) {
+        led1 = !led1;
+        wait(0.5);
+    }
+}
+
+```
+
+### Related content
+
+- [File](file.html).
+- [FileSystem](filesystem.html).
+- [Poll](poll.html).

docs/api/platform/Poll.md

@@ -0,0 +1,83 @@
+## Poll
+
+The `poll` functions perform timed waits on one or more [file handles](filehandle.html).
+
+Mbed OS provides `poll()` in two forms:
+
+- `int poll(struct pollfd fds[], nfds_t nfds, int timeout)` the standard POSIX form taking an array of integer file descriptors.
+- `int mbed::poll(pollfh fhs[], unsigned nfhs, int timeout)`, the variant taking an array of `FileHandle *`.
+
+See the POSIX specification of `poll` for full details of the API. Mbed OS provides the `POLLIN`, `POLLOUT`, `POLLERR` and `POLLHUP` event flags, but implementation depends on the device.
+
+Note that the `poll` is not affected by the blocking or nonblocking setting of the individual file handles. The `timeout` argument always determines the wait behavior: 0 for immediate (nonblocking) return, -1 for wait forever or positive for a limited wait.
+
+As per the POSIX specification, `poll` always indicates that ordinary files (as opposed to devices) are readable and writable.
+
+### Poll reference
+
+[![View code](https://www.mbed.com/embed/?type=library)](https://os-doc-builder.test.mbed.com/docs/development/mbed-os-api-doxy/group__platform__poll.html)
+
+### Poll example
+
+```
+// Transfer bidirectional data between two ports, acting as a virtual link.
+// poll() is used to monitor both ports for input.
+#include "mbed.h"
+
+// Pins for each port are specified using mbed_app.json. Assume no flow control.
+// (If there were flow control, being blocked by it could break the implementation,
+// as you would stop reading input in the opposite direction).
+UARTSerial device1(MBED_CONF_APP_UART1_TX, MBED_CONF_APP_UART1_RX);
+UARTSerial device2(MBED_CONF_APP_UART2_TX, MBED_CONF_APP_UART2_RX);
+
+// Precondition: "in" is readable
+static void copy_some(FileHandle *out, FileHandle *in)
+{
+    // To ensure performance, try to read multiple bytes at once,
+    // but don't expect to read many in practice.
+    char buffer[32];
+
+    // Despite the FileHandle being in its default blocking mode,
+    // read() must return immediately with between 1-32 bytes, as
+    // you've already checked that `in` is ready with poll()
+    ssize_t read = in->read(buffer, sizeof buffer);
+    if (read <= 0) {
+        error("Input error");
+    }
+
+    // Write everything out. Assuming output port is same speed as input,
+    // and no flow control, this may block briefly but not significantly.
+    ssize_t written = out->write(buffer, read);
+    if (written < read) {
+        error("Output error");
+    }
+}
+
+int main()
+{
+    char buffer[32];
+    pollfh fds[2];
+
+    fds[0].fh = &device1;
+    fds[0].events = POLLIN;
+    fds[1].fh = &device2;
+    fds[1].events = POLLIN;
+
+    while (1) {
+        // Block indefinitely until either of the 2 ports is readable (or has an error)
+        poll(fds, 2, -1);
+
+        // Transfer some data in either or both directions
+        if (fds[0].revents) {
+            copy_some(fds[1].fh, fds[0].fh);
+        }
+        if (fds[1].revents) {
+            copy_some(fds[0].fh, fds[1].fh);
+        }
+    }
+}
+```
+
+### Related content
+
+- [FileHandle](filehandle.html).