Skip to content

Commit

Permalink
Complete first attempt at review of error handling frameworks for the…
Browse files Browse the repository at this point in the history
… docs.
  • Loading branch information
ned14 committed Jan 4, 2022
1 parent 5ac38c0 commit af7e487
Show file tree
Hide file tree
Showing 6 changed files with 144 additions and 20 deletions.
6 changes: 3 additions & 3 deletions doc/src/content/alternatives/_index.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
+++
title = "Review of Outcome Alternatives"
title = "Review of Error Handling Frameworks"
weight = 8
+++

Outcome [started life in 2014]({{% relref "/history" %}}), entered Boost as Boost.Outcome in 2018, and therefore was amongst the very first of the major alternative error handling frameworks to C++ exception throws in C++. Since then, and sometimes in reaction to Outcome's choice of design, alternative frameworks have appeared. This page tries to give a fairly even handed summary of those alternatives, and how they compare to Outcome in this author's opinion.
Outcome [started life in 2014]({{% relref "/history" %}}), entered Boost as Boost.Outcome in 2018, and therefore was amongst the very first of the major alternative error handling frameworks to standard exception throws in C++. Since then, and sometimes in reaction to Outcome's choice of design, alternative frameworks have appeared. This page tries to give a fairly even handed summary of those alternatives, and how they compare to Outcome in this author's opinion.

These are listed in order of approximate availability to the C++ ecosystem.
These are listed in order of approximate availability to the C++ ecosystem i.e. in order of appearance.

{{% children description="true" depth="2" %}}
14 changes: 9 additions & 5 deletions doc/src/content/alternatives/error_code.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
+++
title = "C++ error codes"
description = "How `std::error_code` compares to Outcome"
title = "std error codes"
description = "Advantages and disadvantages of `std::error_code`"
weight = 20
+++

`std::error_code` came originally from `boost::error_code` which was designed around 2008 as part of implementing Filesystem and Networking. They are a simple trivially copyable type offering improved type safety and functionality over C enumerations. [You can read more about how `std::error_code` works here]({{% relref "/motivation/std_error_code" %}}). They were standardised in the C++ 11 standard.

Pros:
#### Pros:

- Predictable runtime overhead on the happy path.

- Predictable runtime overhead on the sad path.

- Unbiased syntax equal for both success and failure requiring explicit code written to handle both.

- Very little bloat added to binaries.
Expand All @@ -18,9 +20,11 @@ Pros:

- Works well in all configurations of C++, including C++ exceptions and RTTI globally disabled.

Cons:
- Works well on all niche architectures, such as HPC, GPUs, DSPs and microcontrollers.

- Predictable runtime overhead on the sad path.
- Ships with every standard library since C++ 11.

#### Cons:

- Failure to write handling code for failure means failures get silently dropped. This is disturbingly easy to do.

Expand Down
14 changes: 9 additions & 5 deletions doc/src/content/alternatives/exceptions.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,24 @@
+++
title = "C++ exception throws"
description = "How C++ exception throws compare to Outcome"
title = "std exception throws"
description = "Advantages and disadvantages of C++ exception throws"
weight = 10
+++

(Note that we assume a table-based EH implementation here, a SJLJ EH implementaton would be a lot more even with Outcome. Table-based EH implementations are almost universal on x64, ARM and AArch64 targets).
(Note that we assume a table-based EH implementation here, a SJLJ EH implementaton would have even happy and sad path runtime overhead. Table-based EH implementations are almost universal on x64, ARM and AArch64 targets).

C++ exception throws came in the original C++ 98 standard -- at that time, not all the major compilers implemented them yet, and several of those who did did not have efficient implementations, plus in the original days some compiler vendors still patented things like EH implementation techniques to try and maintain a competitive advantage over rival compilers. Unlike other C++ features, enabling C++ exceptions on a code base not written with them in mind is not safe, so this led to the C++ ecosystem becoming bifurcated into exceptions-enabled and exceptions-disabled camps.

Pros:
#### Pros:

- Zero runtime overhead on the happy path.

- Success-orientated syntax declutters code of failure control flow paths.

- As a built-in language feature, probably has the least impact on optimisation of any failure handling mechanism here.

Cons:
- Ships with every standard toolchain (though may not work in some, and cannot be safely enabled for many codebases).

#### Cons:

- Unpredictable runtime overhead on the sad path.

Expand All @@ -28,4 +30,6 @@ Cons:

- Not available in several major parts of the C++ ecosystem (embedded, games, audio, to a lesser extent financial).

- Not available in many niche architectures such as HPC, GPUs, DSPs and microcontrollers.

- Most codebases do not invest in adequate correctness testing of the silent proliferation of failure control flow paths which result in C++ exception throwing code.
22 changes: 15 additions & 7 deletions doc/src/content/alternatives/expected.md
Original file line number Diff line number Diff line change
@@ -1,38 +1,46 @@
+++
title = "C++ expected"
description = "How `std::expected<T, E>` compares to Outcome"
title = "std expected"
description = "Advantages and disadvantages of `std::expected<T, E>`"
weight = 30
+++

`std::expected<T, E>` came originally from an experimental monadic and generic programming library outside of Boost written by Boost and WG21 developers around 2013. Before Outcome v1, I deployed the then Expected into a large codebase and I was dismayed with the results, especially on build times. [You can read here how those experiences led me to develop Outcome v1]({{% relref "/history" %}}).

`std::expected<T, E>` is a constrained variant type with a strong preference for the successful type `T` which it models like a `std::optional<T>`. If, however, there is no `T` value then it supplies an 'unexpected' `E` value instead. `std::expected<T, E>` was standardised in the C++ 20 standard.
`std::expected<T, E>` is a constrained variant type with a strong preference for the successful type `T` which it models like a `std::optional<T>`. If, however, there is no `T` value then it supplies an 'unexpected' `E` value instead. `std::expected<T, E>` was standardised in the C++ 23 standard.

Outcome's Result type [can be configured to act just like Expected if you want that]({{% relref "/faq#how-far-away-from-the-proposed-std-expected-t-e-is-outcome-s-checked-t-e" %}}), however ultimately [Outcome's Result doesn't solve the same problem as Expected]({{% relref "/faq#why-doesn-t-outcome-duplicate-std-expected-t-e-s-design" %}}), plus Outcome models `std::variant<T, E>` rather than `std::optional<T>` which we think much superior for many use cases:
Outcome's Result type [can be configured to act just like Expected if you want that]({{% relref "/faq#how-far-away-from-the-proposed-std-expected-t-e-is-outcome-s-checked-t-e" %}}), however ultimately [Outcome's Result doesn't solve the same problem as Expected]({{% relref "/faq#why-doesn-t-outcome-duplicate-std-expected-t-e-s-design" %}}), plus Outcome models `std::variant<T, E>` rather than `std::optional<T>` which we think much superior for many use cases, which to summarise:

- If you are parsing input which may rarely contain unexpected values, Expected is the right choice here.

- If you want an alternative to C++ exception handling i.e. a generalised whole-program error handling framework, Expected is an inferior choice to alternatives.

Outcome recognises Expected-like types and will construct from them, which aids interoperability.

Pros:

#### Pros:

- Predictable runtime overhead on the happy path.

- Predictable runtime overhead on the sad path.

- Very little bloat added to binaries.

- Variant storage means storage overhead is minimal, except when either `T` or `E` has a throwing move constructor which typically causes storage blowup.

- Works well in all configurations of C++, including C++ exceptions and RTTI globally disabled.

Cons:
- Works well on all niche architectures, such as HPC, GPUs, DSPs and microcontrollers.

- Predictable runtime overhead on the sad path.
- Ships with every standard library since C++ 23.

#### Cons:

- Success-orientated syntax makes doing anything with the `E` type is awkward and clunky.

- Results in branchy code, which is slow -- though predictably so -- for embedded controller CPUs.

- Failure to examine an Expected generates a compiler diagnostic, but failure to handle both failure and success does not. This can mean failures or successes get accidentally dropped.

- Lack of a `try` operator makes use tedious and verbose.

- Variant storage does have an outsize impact on build times in the same way widespread use of `std::variant` has. This is because implementing exception guarantees during copies and moves of non-trivially-copyable types in union storage involves a lot of work for the compiler on every use of copy and move.
54 changes: 54 additions & 0 deletions doc/src/content/alternatives/leaf.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
+++
title = "LEAF"
description = "Advantages and disadvantages of Lightweight Error Augmentation Framework"
weight = 50
+++

As much as Outcome originated in a negative reaction to the then originally proposed `std::expected<T, E>`, [LEAF](https://boostorg.github.io/leaf/) originated in a negative reaction to Outcome. Some of the perceived issues with Outcome were ([LEAF's own rendition of this can be viewed here](https://boostorg.github.io/leaf/#rationale)):

- Outcome based code is visually cluttered, as both happy and sad paths appear in code.

- Outcome based code generates branchy code at runtime, which impacts low end CPUs and first time code execution.

- Outcome's Result type encodes the type of the error in the function signature, which could be considered as more brittle and problematic for large scale code refactoring[^1].

- Outcome is intended to be the ultimate error handling framework in a program (i.e. all third party custom error handling flows into Outcome via customisation point adapters), whereas LEAF is intended to be used ad hoc as needed.

LEAF therefore looks a lot more like standard C++ exception handling, but without the non-deterministic sad path at the cost of a slight impact on happy path runtime performance. LEAF's current design was completed in 2020.

If you need an error handling framework which has predictable sad path overhead unlike C++ exceptions, but you otherwise want similar syntax and use experience to C++ exceptions, LEAF is a very solid choice.


#### Pros:

- Very low runtime overhead on the happy path.

- Very low runtime overhead on the sad path.

- Does not cause branchy code to the same extent as Outcome, and the sad path is deterministic unlike with C++ exceptions.

- Very little bloat added to binaries.

- Sad path control flow is implied in code, same as with C++ exceptions, so code only shows the happy path. This reduces visual code clutter, and eliminates any need for a `try` operator.

- Unlike with any of the preceding options, failures nor successes cannot get unintentionally dropped. This is the same strength of guarantee as with C++ exceptions.

- Works well in most configurations of C++, including C++ exceptions and RTTI globally disabled. Does not dynamically allocate memory.

#### Cons:

- Sad path control flow is implied which suits code which almost never fails, but is less suited for code where sad path control flow is not uncommon and/or where auditing during code review of sad path logic is particularly important.

- Requires out of band storage for state e.g. thread local storage, or a global synchronised ring buffer.

- If thread local storage is chosen as the out of band storage, transporting LEAF state across threads requires manual intervention.

- If a global ring buffer is chosen as the out of band storage, thread synchronisation with global state is required and the ring buffer can wrap which drops state.

- Thread local storage can be problematic or even a showstopper in many niche architectures such as HPC, GPUs, DSPs and microcontrollers. Global synchronised state can introduce an unacceptable performance impact on those architectures.

- Current compilers at the time of writing do not react well to use of thread local storage, it would seem that elision of code generation is inhibited if thread local state gets touched due to pessimistic assumptions about escape analysis. Given that this impacts all of C and C++ due to the same problem with `errno`, it is hoped that future compilers will improve this. Until then, any code which touches thread local storage or magic statics[^2] will not optimise as well as code which does neither.

[^1]: In Outcome, it is strongly recommended that one chooses a single universal error type for all public APIs such as `std::error_code` or `error` from Experimental.Outcome, so if the programmer is disciplined then the function signature does not expose internal error types. Such single universal error types type erase the original error object, but still allow the original error object to be inspected. This avoids 'exception specifications' which are well known to not scale well.

[^2]: `std::error_code` construction touches a magic static, and therefore Outcome when combined with `std::error_code` also sees a codegen pessimisation. Experimental Outcome's `error` fixes this historical oversight.
54 changes: 54 additions & 0 deletions doc/src/content/alternatives/outcome.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
+++
title = "Outcome (proposed std result)"
description = "Advantages and disadvantages of Outcome and its proposed `std::result<T>`"
weight = 40
+++

Outcome (the library described by this documentation) originated in a negative reaction to then originally proposed `std::expected<T, E>`, though what got eventually standardised as `std::expected<T, E>` looks much more like Outcome's `result<T, E>` than the original Expected. [You can read here how those experiences led me to develop Outcome v1]({{% relref "/history" %}}). Outcome comes in both standalone and Boost editions, and its current design was completed in 2018.

Outcome's core is two workhorse types and a macro:

- {{% api "basic_result<T, E, NoValuePolicy>" %}}

- {{% api "basic_outcome<T, EC, EP, NoValuePolicy>" %}}

- {{% api "OUTCOME_TRY(var, expr)" %}}

These three core items are then mixed into a veritable cornucopia of [convenience typedefs]({{% relref "/reference/aliases" %}}) and variations to support a wide range of use cases, including [in C++ coroutines]({{% relref "/tutorial/essential/coroutines" %}}), plus there is extensive plumbing and customisation points for deciding how incompatible types ought to interact, or being notified of lifecycle events (e.g. capture a stack backtrace if a `result<T, E>` is constructed with an error).

Outcome perfectly propagates constexpr, triviality and `noexcept`-ness of each individual operation of the types you configure it with. It never touches dynamic memory allocation, and it has been carefully written so the compiler will optimise it out of codegen entirely wherever that is possible. It is capable of 'true moves' for types which declare themselves 'move bitcopying compatible' i.e. destructor calls on moved-from values are elided. 'True moves' can have a game changing performance gain on types with virtual destructors.

Outcome takes a lot of care to have the least possible impact on build times, and it guarantees that a binary built with it will have stable ABI so it is safe to use in _really_ large C++ codebases (standalone edition only). For interoperation with other languages, it guarantees that C code can work with Outcome data types, and it provides a C macro API header file to help with that.

Outcome recognises Expected-like types and will construct from them, which aids interoperability. [A simplified Result type is proposed for standardisation as `std::result<T>`](https://wg21.link/P1028) where the `E` type is hard coded to a proposed `std::error`. This proposed standardisation has been deployed on billions of devices at the time of writing, and [you can use it today via Experimental.Outcome]({{% relref "/experimental" %}}), the reference implementation.


#### Pros:

- Predictable runtime overhead on the happy path.

- Predictable runtime overhead on the sad path.

- Very little bloat added to binaries.

- Neither success nor failure is prioritised during use -- types will implicitly construct from both `T` and `E` if it is unambiguous, so no clunky added markup needed to return an `E`.

- Sad path control flow is required to be explicitly specified in every situation. This significantly reduces the number of sad control flow paths in a code base, making it much easier to test all of them. It also means that sad paths get audited and checked during code reviews.

- Macro `TRY` operator feels a bit unnatural to use, but is a god send to saving visual code clutter when all you want to say is 'handle this failure by asking my caller to handle it'. It also works with non-Outcome types, and has its own suite of customisation points for third party extension.

- Works well in all configurations of C++, including C++ exceptions and RTTI globally disabled.

- Works well on all niche architectures, such as HPC, GPUs, DSPs and microcontrollers, and does not dynamically allocate memory.

#### Cons:

- Sad path control flow is required to be explicitly specified in every situation. For code where failure is extremely unlikely, or is not important because it always results in aborting the current operation, the added visual code clutter is unhelpful.

- Results in branchy code, which is slow -- though predictably so -- for embedded controller CPUs.

- Failure to examine an Outcome type generates a compiler diagnostic, but failure to handle both failure and success does not. This can mean failures or successes get accidentally dropped.

- To prevent variant storage having an outsize impact on build times in the same way widespread use of `std::variant` has, Outcome only implements union storage when both `T` and `E` are trivially copyable or move bitcopying. Otherwise struct storage is used, which means Outcome's types are larger than Expected's. This is because implementing exception guarantees during copies and moves of non-trivially-copyable types in union storage involves a lot of work for the compiler on every use of copy and move, so by using struct storage Outcome reduces build time impact from copies and moves significantly.

Note that one of the major uses of Outcome types is as the return type from a function, in which case copy elision would happen in C++ 14 and is guaranteed from C++ 17 onwards. This means that the larger footprint of struct storage typically has much less impact in optimised code than might be the case if you store these types inside other types.

0 comments on commit af7e487

Please sign in to comment.