Unify IO errors as Eio.Io

This makes it easy to catch and log all IO errors if desired. The
exception payload gives the type and can be used for matching specific
errors. It also allows attaching extra information to exceptions.

When multiple exceptions occur, we now keep and report all the backtraces.
If multiple IO exceptions occur, the result is also an IO exception (so
you can still catch all IO errors).

Various functions now add extra context to IO exceptions:
- `accept_fork` adds the remote address.
- Various functions add their arguments (`Net.connect`, `Path.load`, etc).

Also:

- `Eio_linux.openfile` is gone (was just a hack for testing in the early
  days).

- `connect` functions no longer need to wrap all errors in
  `Connection_failure`. The fact you got an IO error from `connect` is
  enough.

- Fixed non-tail-recursive continue in Eio_luv (`Socket_of_fd`).

README.md

@@ -24,12 +24,12 @@ Eio replaces existing concurrency libraries such as Lwt
 * [Cancellation](#cancellation)
 * [Racing](#racing)
 * [Switches](#switches)
-* [Design Note: Results vs Exceptions](#design-note-results-vs-exceptions)
 * [Performance](#performance)
 * [Networking](#networking)
 * [Design Note: Capabilities](#design-note-capabilities)
 * [Buffered Reading and Parsing](#buffered-reading-and-parsing)
 * [Buffered Writing](#buffered-writing)
+* [Error Handling](#error-handling)
 * [Filesystem Access](#filesystem-access)
 * [Time](#time)
 * [Multicore Support](#multicore-support)
@@ -364,18 +364,6 @@ Every switch also creates a new cancellation context.
 You can use `Switch.fail` to mark the switch as failed and cancel all fibers within it.
 The exception (or exceptions) passed to `fail` will be raised by `run` when the fibers have exited.
 
-## Design Note: Results vs Exceptions
-
-The OCaml standard library uses exceptions to report errors in most cases.
-Many libraries instead use the `result` type, which has the advantage of tracking the possible errors in the type system.
-However, using `result` is slower, as it requires more allocations, and explicit code to propagate errors.
-
-As part of the effects work, OCaml is expected to gain a [typed effects][] extension to the type system,
-allowing it to track both effects and exceptions statically.
-In anticipation of this, the Eio library prefers to use exceptions in most cases,
-reserving the use of `result` for cases where the caller is likely to want to handle the problem immediately
-rather than propagate it.
-
 ## Performance
 
 As mentioned above, Eio allows you to supply your own implementations of its abstract interfaces.
@@ -707,6 +695,97 @@ let send_response socket =
 
 Now the first two writes were combined and sent together.
 
+## Error Handling
+
+Errors interacting with the outside world are indicated by the `Eio.Io (err, context)` exception.
+This is roughly equivalent to the `Unix.Unix_error` exception from the OCaml standard library.
+
+The `err` field describes the error using nested error codes,
+allowing you to match on either specific errors or whole classes of errors at once.
+For example:
+
+```ocaml
+let test r =
+  try Eio.Buf_read.line r
+  with
+  | Eio.Io (Eio.Net.E Connection_reset Eio_luv.Luv_error _, _) -> "Luv connection reset"
+  | Eio.Io (Eio.Net.E Connection_reset _, _) -> "Connection reset"
+  | Eio.Io (Eio.Net.E _, _) -> "Some network error"
+  | Eio.Io _ -> "Some I/O error"
+```
+
+For portable code, you will want to avoid matching backend-specific errors, so you would avoid the first case.
+The `Eio.Io` type is extensible, so libraries can also add additional top-level error types if needed.
+
+`Io` errors also allow adding extra context information to the error.
+For example, this HTTP GET function adds the URL to any IO error:
+
+```ocaml
+# let get ~net ~host ~path =
+    try
+      Eio.Net.with_tcp_connect net ~host ~service:"http" @@ fun _flow ->
+      "..."
+    with Eio.Io _ as ex ->
+      let bt = Printexc.get_raw_backtrace () in
+      Eio.Exn.reraise_with_context ex bt "fetching http://%s/%s" host path;;
+val get : net:#Eio.Net.t -> host:string -> path:string -> string = <fun>
+```
+
+If we test it using a mock network that returns a timeout,
+we get a useful error message telling us the IP address and port of the failed attempt,
+extended with the hostname we used to get that,
+and then extended again by our `get` function with the full URL:
+
+```ocaml
+# Eio_mock.Backend.run @@ fun () ->
+  let net = Eio_mock.Net.make "mocknet" in
+  Eio_mock.Net.on_getaddrinfo net [`Return [`Tcp (Eio.Net.Ipaddr.V4.loopback, 80)]];
+  Eio_mock.Net.on_connect net [`Raise (Eio.Net.err (Connection_failure Timeout))];
+  get ~net ~host:"example.com" ~path:"index.html";;
++mocknet: getaddrinfo ~service:http example.com
++mocknet: connect to tcp:127.0.0.1:80
+Exception:
+Eio.Io Net Connection_failure Timeout,
+  connecting to tcp:127.0.0.1:80,
+  connecting to "example.com":http,
+  fetching http://example.com/index.html
+```
+
+To get more detailed information, you can enable backtraces by setting `OCAMLRUNPARAM=b`
+or by calling `Printexc.record_backtrace true`, as usual.
+
+When writing MDX tests that depend on getting the exact error output,
+it can be annoying to have the full backend-specific error displayed:
+
+<!-- $MDX non-deterministic=command -->
+```ocaml
+# Eio_main.run @@ fun env ->
+  let net = Eio.Stdenv.net env in
+  Switch.run @@ fun sw ->
+  Eio.Net.connect ~sw net (`Tcp (Eio.Net.Ipaddr.V4.loopback, 1234));;
+Exception:
+Eio.Io Net Connection_failure Refused Eio_luv.Luv_error(ECONNREFUSED) (* connection refused *),
+  connecting to tcp:127.0.0.1:1234
+```
+
+If we ran this using e.g. the Linux io_uring backend, the `Luv_error` part would change.
+To avoid this problem, you can use `Eio.Exn.Backend.show` to hide the backend-specific part of errors:
+
+```ocaml
+# Eio.Exn.Backend.show := false;;
+- : unit = ()
+
+# Eio_main.run @@ fun env ->
+  let net = Eio.Stdenv.net env in
+  Switch.run @@ fun sw ->
+  Eio.Net.connect ~sw net (`Tcp (Eio.Net.Ipaddr.V4.loopback, 1234));;
+Exception:
+Eio.Io Net Connection_failure Refused _,
+  connecting to tcp:127.0.0.1:1234
+```
+
+We'll leave it like that for the rest of this file,
+so the examples can be tested automatically by MDX.
 
 ## Filesystem Access
 
@@ -756,13 +835,13 @@ Access to `cwd` only grants access to that sub-tree:
 ```ocaml
 let try_save path data =
   match Eio.Path.save ~create:(`Exclusive 0o600) path data with
-  | () -> traceln "save %a -> ok" Eio.Path.pp path
-  | exception ex -> traceln "save %a -> %a" Eio.Path.pp path Fmt.exn ex
+  | () -> traceln "save %a : ok" Eio.Path.pp path
+  | exception ex -> traceln "%a" Eio.Exn.pp ex
 
 let try_mkdir path =
   match Eio.Path.mkdir path ~perm:0o700 with
-  | () -> traceln "mkdir %a -> ok" Eio.Path.pp path
-  | exception ex -> traceln "mkdir %a -> %a" Eio.Path.pp path Fmt.exn ex
+  | () -> traceln "mkdir %a : ok" Eio.Path.pp path
+  | exception ex -> traceln "%a" Eio.Exn.pp ex
 ```
 
 ```ocaml
@@ -771,9 +850,9 @@ let try_mkdir path =
   try_mkdir (cwd / "dir1");
   try_mkdir (cwd / "../dir2");
   try_mkdir (cwd / "/tmp/dir3");;
-+mkdir <cwd:dir1> -> ok
-+mkdir <cwd:../dir2> -> Eio__Fs.Permission_denied("../dir2", _)
-+mkdir <cwd:/tmp/dir3> -> Eio__Fs.Permission_denied("/tmp/dir3", _)
++mkdir <cwd:dir1> : ok
++Eio.Io Fs Permission_denied _, creating directory <cwd:../dir2>
++Eio.Io Fs Permission_denied _, creating directory <cwd:/tmp/dir3>
 - : unit = ()
 ```
 
@@ -788,9 +867,9 @@ The checks also apply to following symlinks:
   try_save (cwd / "dir1/file1") "A";
   try_save (cwd / "link-to-dir1/file2") "B";
   try_save (cwd / "link-to-tmp/file3") "C";;
-+save <cwd:dir1/file1> -> ok
-+save <cwd:link-to-dir1/file2> -> ok
-+save <cwd:link-to-tmp/file3> -> Eio__Fs.Permission_denied("link-to-tmp/file3", _)
++save <cwd:dir1/file1> : ok
++save <cwd:link-to-dir1/file2> : ok
++Eio.Io Fs Permission_denied _, opening <cwd:link-to-tmp/file3>
 - : unit = ()
 ```
 
@@ -802,8 +881,8 @@ You can use `open_dir` (or `with_open_dir`) to create a restricted capability to
   Eio.Path.with_open_dir (cwd / "dir1") @@ fun dir1 ->
   try_save (dir1 / "file4") "D";
   try_save (dir1 / "../file5") "E";;
-+save <dir1:file4> -> ok
-+save <dir1:../file5> -> Eio__Fs.Permission_denied("../file5", _)
++save <dir1:file4> : ok
++Eio.Io Fs Permission_denied _, opening <dir1:../file5>
 - : unit = ()
 ```
 
@@ -1446,7 +1525,6 @@ Some background about the effects system can be found in:
 [Lwt_eio]: https://github.com/ocaml-multicore/lwt_eio
 [mirage-trace-viewer]: https://github.com/talex5/mirage-trace-viewer
 [structured concurrency]: https://en.wikipedia.org/wiki/Structured_concurrency
-[typed effects]: https://www.janestreet.com/tech-talks/effective-programming/
 [capability-based security]: https://en.wikipedia.org/wiki/Object-capability_model
 [Emily]: https://www.hpl.hp.com/techreports/2006/HPL-2006-116.pdf
 [gemini-eio]: https://gitlab.com/talex5/gemini-eio

doc/rationale.md

@@ -151,3 +151,22 @@ or add extra convenience functions without forcing every implementor to add them
 
 Note that the use of objects in Eio is not motivated by the use of the "Object Capabilities" security model.
 Despite the name, that is not specific to objects at all.
+
+## Results vs Exceptions
+
+The OCaml standard library uses exceptions to report errors in most cases.
+Many libraries instead use the `result` type, which has the advantage of tracking the possible errors in the type system.
+However, using `result` is slower, as it requires more allocations, and explicit code to propagate errors.
+
+As part of the effects work, OCaml is expected to gain a [typed effects][] extension to the type system,
+allowing it to track both effects and exceptions statically.
+In anticipation of this, the Eio library prefers to use exceptions in most cases,
+reserving the use of `result` for cases where the caller is likely to want to handle the problem immediately
+rather than propagate it.
+
+In additional, while result types work well
+for functions with a small number of known errors which can be handled at the call-site,
+they work poorly for IO errors where there are typically a large and unknown set of possible errors
+(depending on the backend).
+
+[typed effects]: https://www.janestreet.com/tech-talks/effective-programming/

lib_eio/core/eio__core.mli

@@ -358,19 +358,112 @@ end
 module Exn : sig
   type with_bt = exn * Printexc.raw_backtrace
 
-  exception Multiple of exn list
-  (** Raised if multiple fibers fail, to report all the exceptions. *)
+  type err = ..
+  (** Describes the particular error that occurred.
+
+      They are typically nested (e.g. [Fs (Permission_denied (Unix_error ...))])
+      so that you can match e.g. all IO errors, all file-system errors, all
+      permission denied errors, etc.
+
+      If you extend this, use {!register_pp} to add a printer for the new error. *)
+
+  type context
+  (** Extra information attached to an IO error.
+      This provides contextual information about what caused the error. *)
+
+  exception Io of err * context
+  (** A general purpose IO exception.
+
+      This is used for most errors interacting with the outside world,
+      and is similar to {!Unix.Unix_error}, but more general.
+      An unknown [Io] error should typically be reported to the user, but does
+      not generally indicate a bug in the program. *)
+
+  type err += Multiple_io of (err * context * Printexc.raw_backtrace) list
+  (** Error code used when multiple IO errors occur.
+
+      This is useful if you want to catch and report all IO errors. *)
+
+  val create : err -> exn
+  (** [create err] is an {!Io} exception with an empty context. *)
+
+  val add_context : exn -> ('a, Format.formatter, unit, exn) format4 -> 'a
+  (** [add_context ex msg] returns a new exception with [msg] added to [ex]'s context,
+      if [ex] is an {!Io} exception.
+
+      If [ex] is not an [Io] exception, this function just returns the original exception. *)
+
+  val reraise_with_context : exn -> Printexc.raw_backtrace -> ('a, Format.formatter, unit, 'b) format4 -> 'a
+  (** [reraise_with_context ex bt msg] raises [ex] extended with additional information [msg].
+
+      [ex] should be an {!Io} exception (if not, is re-raised unmodified).
+
+      Example:
+      {[
+         try connect addr
+         with Eio.Io _ as ex ->
+           let bt = Printexc.get_raw_backtrace () in
+           reraise_with_context ex bt "connecting to %S" addr
+      ]}
+
+      You must get the backtrace before calling any other function
+      in the exception handler to prevent corruption of the backtrace. *)
+
+  val register_pp : (Format.formatter -> err -> bool) -> unit
+  (** [register_pp pp] adds [pp] as a pretty-printer of errors.
+
+      [pp f err] should format [err] using [f], if possible.
+      It should return [true] on success, or [false] if it didn't
+      recognise [err]. *)
+
+  val pp : exn Fmt.t
+  (** [pp] is a formatter for exceptions.
+
+      This is similar to {!Fmt.exn}, but can do a better job on {!Io} exceptions
+      because it can format them directly without having to convert to a string first. *)
+
+  (** Extensible backend-specific exceptions. *)
+  module Backend : sig
+    type t = ..
+
+    val show : bool ref
+    (** Controls the behaviour of {!pp}. *)
+
+    val register_pp : (Format.formatter -> t -> bool) -> unit
+    (** [register_pp pp] adds [pp] as a pretty-printer of backend errors.
+
+        [pp f err] should format [err] using [f], if possible.
+        It should return [true] on success, or [false] if it didn't
+        recognise [err]. *)
+
+    val pp : t Fmt.t
+    (** [pp] behaves like {!pp} except that if display of backend errors has been turned off
+        (with {!show}) then it just prints a place-holder.
+
+        This is useful for formatting the backend-specific part of exceptions,
+        which should be hidden in expect-style testing that needs to work on multiple backends. *)
+  end
+
+  type err += X of Backend.t
+  (** A top-level code for backend errors that don't yet have a cross-platform classification in Eio.
+
+      You should avoid matching on these (in portable code). Instead, request a proper Eio code for them. *)
+
+  exception Multiple of with_bt list
+  (** Raised if multiple fibers fail, to report all the exceptions.
+
+      This usually indicates a bug in the program.
+
+      Note: If multiple {b IO} errors occur, then you will get [Io (Multiple_io _, _)] instead of this. *)
 
   val combine : with_bt -> with_bt -> with_bt
   (** [combine x y] returns a single exception and backtrace to use to represent two errors.
 
-      Only one of the backtraces will be kept.
       The resulting exception is typically just [Multiple [y; x]],
       but various heuristics are used to simplify the result:
       - Combining with a {!Cancel.Cancelled} exception does nothing, as these don't need to be reported.
         The result is only [Cancelled] if there is no other exception available.
-      - If [x] is a [Multiple] exception then [y] is added to it, to avoid nested [Multiple] exceptions.
-      - Duplicate exceptions are removed (using physical equality of the exception). *)
+      - If both errors are [Io] errors, then the result is [Io (Multiple_io _)]. *)
 end
 
 (** @canonical Eio.Cancel *)