Exceptions and Exception Handling

[vansnyder]

Block-structured exception handling has been provided in other languages for decades. According to "Programming Languages: Principles and Practice," 2nd edition, by Kenneth C. Louden (a notable textbook on programming languages), ``Exception handling was pioneered by the language PL/I in the 1960s and significantly advanced in CLU in the 1970s. However, it was only in the 1980s and early 1990s that design questions were largely resolved.'' Ada 83, for which development began in 1976, is probably the first language that provided block-structured exception handling in the modern form. It is generally agreed that exception handling is commonplace in all modern languages.

It has been observed that the CHANGE TEAM construct described in the 6 November 2014 draft of TS 18508 (ISO/IEC JTC1/SC22/WG5 paper N2033) is, in effect, an exception block, but with an intrinsic (and invisible) exception handler that only manages necessary synchronization and
deallocations.

It's time for Fortran to have a complete block-structured exception handling mechanism. Block-structured exception handling has been proposed for Fortran, but has never been implemented.

Objections have been raised to block-structured exception handling, usually citing performance degradation. While some implementations of block-structured exception handling impose expense even if an exception does not occur, block-structured exception handling does not inevitably
impose a significant execution-time penalty if an exception does not occur. For example, the Janus and Verdix Ada compilers' block-structured exception handling mechanisms impose very low cost if an exception does not occur, and the exception handling mechanism provided by the GNU Ada Translator (GNAT) imposes zero cost if an exception does not occur.

The cost of exception handling should not be confused with the cost of exception detection. An exception cannot be handled unless it is detected. Most processors have methods to specify whether certain exceptions, such as subscripts out of bounds, are detected. The cost of an exception handler is the additional cost to provide for handling an exception, if one is detected. Providing a mechanism to handle an exception does not require or imply that the processor is instructed to detect it. If the processor does detect it, the additional overhead to handle it is very
small, or nonexistent, until the exception is detected. If the processor does not detect it, the additional overhead to handle an exception that cannot occur is very small, or nonexistent.
Exceptions_3.pdf

[FortranFan]

It's exceptionally important for Fortran to include support for exception handling as a modern language must. Coders can then decide whether to make use of it and where and how in their codes just as it is with IEEE exception handling introduced in Fortran starting with the 2003 standard.

One of my biggest disappointments is with WG5 and how WG5 has reduced the work-list for Fortran 202X to drop items such as exception handling and generics. Interested readers can see this link at comp,lang.fortran. As seen in this link, many a Fortranner - with no real voice or representation due to their physical/virtual locale which is well outside the 2 to 5 country-based and rather institutionally controlled org structure - have been requesting support for exception handling but to no avail.

The way things function with an ISO-driven process and how arcane, archaic, and inefficient the process is when it comes to language advancement, it could very well be decades from now before the ducks get in a row for the standard process to finally work attentively toward exception handling in Fortran. And even then there will always be the argument Fortran has survived that long without explicit exception handling support for coders, so why bother!?

In the meantime, there is another issue that too gets overlooked with this topic: in the industry where I work and the computing domains I am familiar with, code base after code base has moved away Fortran, a lack of support for exception handling and the demands that such a scenario places on the most important resource - developer time and productivity - is among the reasons for the flight away from Fortran.

For a simple example, there is a particular formula based on the pioneering scientific advancements by Gibbs and van der Waals which literally gets computed billions of times by engineers and scientists where I work. This formula, y = f(x1, x2, .. , xN), can be written in a few lines of code and computed trivially. Except when the values of the N independent variables x1 thru' xN are outside the physically acceptable range for practicable design and manufacturing. The occurrence of this is truly an exceptional circumstance but which needs to be handled very carefully, for otherwise there are considerable consequences including that of safety.

Owing to the lack of exception handling support in language, the Fortran codes were rendered exceptionally complex and verbose with developers having paid incalculable amount of attention to the handling of exceptional situations where the input values are outside the physically acceptable range. This was so much so that a new engineer/scientist tasked to work on the code (and mostly with background in MATLAB, Python, C++, Java, .NET) would entirely fail to notice the important formula in the subprogram, it was a mind-blowing departure from the concept of Formula Translation and black-board abstraction.

As I mentioned earlier, the computations involving this formula are now performed in any number of compute environments where Fortran plays little to no role. The code written in other languages is rather compact, readable and maintainable, albeit full of curly braces!! For the purse-holders and budget managers though, the new code is overall better performant in spite of the use of structured exception handling. So they can't be happier with their push away from Fortran.

This is an aspect where implementations such as LFortran, LLVM, etc. can be of great help with prototyping.

[certik]

I think it is clear that many (if not the most) people in the wide Fortran community want some form of better exception handling.

I also agree that the Committee in general is not handling these requests efficiently. As you know I am running for the WG5 Convenor position and in my platform I propose that we work on these proposals continuously and when they are ready, we will consider them for inclusion. However, no matter who gets selected in the end, let us work on these things, and when they are ready, I will bring them to the Committee to consider.

So let's get to work.

There are several approaches to exception handling. The above proposal is one. Here are other issues with other proposals: #6, #66. There are more.

Ultimately, this must be prototyped in a compiler to show that there is no performance overhead, and also to iron out corner cases, and to see how difficult it is to implement it. We will be able to do such prototypes very soon (if things go well, hopefully in a year or so). Until then, let's at least narrow down the options of what ideas should be prototyped.

Also, let's collect the most common objections and have solid answers to them. Here are some off top of my head:

• What are the performance implications?
• Why are many C++ applications turning off exceptions for performance (in gaming industry for example)?
• The C++ committee does not seem to be super happy with their traditional exceptions, and are proposing a different mechanism: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r0.pdf; Wouldn't it be a mistake to do what C++ found is not such a great idea?

And there is a lot more, see the thread at the J3 mailinglist starting with this email:

https://mailman.j3-fortran.org/pipermail/j3/2019-February/011256.html

We have to collect all the objections there and have a good answer, before we bring it back to the committee. Otherwise the same objections will be given and we will not move the discussion forward.

This is the real work that we have to do. I currently don't have time to drive this particular effort, but if somebody is interested in leading this, I will help as much as I can.

[klausler]

Here is what I think Fortran can do.

1. Add a statement (say FAIL) that signifies failure with an error code. As you will see, it can appear only within a dynamic context that can handle the error. It causes transfer of control to the most deeply nested handler for its error code. Memory allocations are automatically cleaned up between the FAIL and its handler.
2. Add a new specifier (say FAILING) to ALLOCATE statements, I/O statements, coarray references, &c. to give them permission to fail on error, END, and EOR conditions, thus allowing these conditions to be handled by the same handlers as if a FAIL statement had been executed.
3. Add a construct (say TRY <block> [ CATCH (error) <block> ]... [ CATCH DEFAULT <block> ] END TRY) that establishes an error handling context. If a FAIL statement is executed, the appropriate handler block is executed. A TRY construct with no CATCH DEFAULT, or with fallible code in its CATCH DEFAULT block, may appear only in a dynamic context that can handle the uncaught errors.
4. Add a attribute (I'll spell it FALLIBLE here) to procedures. It signifies that the procedure may fail without catching all errors. A reference to a FALLIBLE procedure may appear only where a FAIL statement could appear -- i.e., a hermetic TRY construct or in another FALLIBLE procedure. The use of FALLIBLE implies an explicit interface.

The FAIL statement, and references to FALLIBLE procedures, may appear only within a hermetic TRY construct if they occur in a DO CONCURRENT loop or main program.

Why I favor this approach:

• It has no effect on existing code.
• Static checking can guarantee that all errors will be caught.
• Runtime overhead is cheap; no more expensive than alternate returns.
• Implementation would be straightforward; no magic in the linker or runtime library.

TBD:

• interaction with finalization
• whether ALLOCATE statements, I/O statements, coarray references, &c. should automatically be fallible if they appear in a TRY construct.
• how distinct user error codes would be defined

[certik]

@klausler thanks for this proposal. What I like about it is that it is essentially equivalent to manual alternate returns, and thus does not require any magic at runtime with stack frames or runtime library.

At the February 2019 meeting, we fleshed out two similar / alternative proposals:

https://mailman.j3-fortran.org/pipermail/j3/2019-February/011257.html

See the links at the bottom of the email:

• example code in current Fortran: https://gist.github.com/certik/bd8235d2e1d049b22fcd1016f4914430
• my original "try" proposal: https://gist.github.com/certik/d950b7468228ff6f7d8bbc680a0943a7
• new "status" proposal: https://gist.github.com/certik/2293270ac39f2589ae702ed751f405f8

it is also equivalent to manual alternate returns, no magic. I'll try to do this example using your proposal also, so that we can see how it would look like in a code.

[vansnyder]

I uploaded a complete exception-handling proposal a few days ago. I don't remember the number. 1. Exceptions are identified by enumerators of an extensible enumeration type. 2. Instead of a new construct, allow ENABLE ( <exception-list> ) suffix on BLOCK, CHANGE TEAM, DO CONCURENT, CRITICAL, SUBROUTINE and FUNCTION statements. 3. Allow any number of HANDLE ( <exception-list> ) ... <block> sub constructs before the END of a construct that has an ENABLE suffix. The <exception-list> in the HANDLE should be required to be a subset of the list in the ENABLE statement. The <exception-lists> in HANDLE blocks shall be disjoint. 4. The HANDLE block accesses its containing construct or program unit by host association, so the containing scope is NOT finalized before the exception handler starts. 5. When the exception handler finishes, its scope, and the containing scope, are finalized. 6. Users can define exceptions by extending the EXCEPTION type. Any exception, intrinsic or user-defined, can be explicitly raised by a RAISE statement. 7. If a scoping unit does not handle an exception, it is raised in the enclosing scoping unit. If there is no enclosing scoping unit, it is raised in the invoking scoping unit, if any. If it is raised in the main program, the rutime support handles it, probably by printing a message and ERROR STOPping with a non-zero status. 8. Interaction with interoperability TBD. 9. More details in the uploaded paper.

…

On Wed, 2020-05-13 at 13:40 -0700, Peter Klausler wrote: Here is what I think Fortran can do. Add a statement (say FAIL) that signifies failure with an error code. As you will see, it can appear only within a dynamic context that can handle the error. It causes transfer of control to the most deeply nested handler for its error code. Memory allocations are automatically cleaned up between the FAIL and its handler. Add a new specifier (say FAILING) to ALLOCATE statements, I/O statements, coarray references, &c. to give them permission to fail on error, END, and EOR conditions, thus allowing these conditions to be handled by the same handlers as if a FAIL statement had been executed. Add a construct (say TRY <block> [ CATCH (error) <block> ]... [ CATCH DEFAULT <block> ] END TRY) that establishes an error handling context. If a FAIL statement is executed, the appropriate handler block is executed. A TRY construct with no CATCH DEFAULT, or with fallible code in its CATCH DEFAULT block, may appear only in a dynamic context that can handle the uncaught errors. Add a attribute (I'll spell it FALLIBLE here) to procedures. It signifies that the procedure may fail without catching all errors. A reference to a FALLIBLE procedure may appear only where a FAIL statement could appear -- i.e., a hermetic TRY construct or in another FALLIBLE procedure. The use of FALLIBLE implies an explicit interface. The FAIL statement, and references to FALLIBLE procedures, may appear only within a hermetic TRY construct if they occur in a DO CONCURRENT loop or main program. Why I favor this approach: It has no effect on existing code. Static checking can guarantee that all errors will be caught. Runtime overhead is cheap; no more expensive than alternate returns. Implementation would be straightforward; no magic in the linker or runtime library. TBD: interaction with finalization whether ALLOCATE statements, I/O statements, coarray references, &c. should automatically be fallible if they appear in a TRY construct. how distinct user error codes would be defined — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": " #172 (comment) ", "url": " #172 (comment) ", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

[klausler]

Looks a lot to me like the current C++ exception feature that many significant codebases specifically go out of their way to disable and that C++ is working hard to replace.

[certik]

@klausler I implemented an example using your proposal, please let me know if I got it right:

• Current valid Fortran: https://gist.github.com/certik/bd8235d2e1d049b22fcd1016f4914430
• Peter's proposal: https://gist.github.com/certik/8dc9f960f7c58a2cc17f7f5b911eb4fa

[certik]

@vansnyder I have implemented an example using your proposal, please let me know if I got it right:

• Current valid Fortran: https://gist.github.com/certik/bd8235d2e1d049b22fcd1016f4914430
• Van's proposal: https://gist.github.com/certik/012ed004257e14fc2a7557c7943eeaf1

[vansnyder]

On Wed, 2020-05-13 at 17:17 -0700, Peter Klausler wrote: Looks a lot to me like the current C++ exception feature that many significant codebases specifically go out of their way to disable and that C++ is working hard to replace.

It's based on the Ada exception mechanism, and John Reid's proposal. One important difference from John's proposal is that John's represented exceptions by integers.

…

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": " #172 (comment) ", "url": " #172 (comment) ", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

[klausler]

@klausler I implemented an example using your proposal, please let me know if I got it right:

• Current valid Fortran: https://gist.github.com/certik/bd8235d2e1d049b22fcd1016f4914430
• Peter's proposal: https://gist.github.com/certik/8dc9f960f7c58a2cc17f7f5b911eb4fa

The fallible subroutine can't be called at line 15 from the main program outside an error handling context.

[certik]

The fallible subroutine can't be called at line 15 from the main program outside an error handling context.

I updated the gist (still the same link), you can see the changes in the "Revisions" tab.

I was wondering about this --- I can see both ways here. Your proposal disallows calling the fallible subroutine outside of an error handling context, which is very explicit and will cause all errors to be explicitly handled, which I think is a nice feature.

Two questions:

1. How would this be implemented in the compiler? Would there be an implicit additional intent(out) argument __failed for every subroutine that is fallible? And then an implicit if statement right after calling any fallible subroutine that will check this __failed argument and if it is .true., it would immediately return and propagate __failed to the parent subroutine or main program?

2. Assume you are in the main program or in some non-fallible subroutine. And you need to call a subroutine that is fallible, but you don't want to be handling errors, you just want the program to fail at runtime with a stacktrace, just like currently allocate will fail if you don't set the stat argument. In your proposal currently you have to do:

try
    call mysum(5, s)
catch default
    error stop
end try

Which is quite verbose given that you don't want to handle the error. Would it make sense to create a shortcut for the above code as follows:

try mysum(5, s)

Which would be exactly equivalent to the above code. So in practice, if you have an existing code that has call mysum(5, s) and then later if the mysum subroutine becomes fallible, but you don't care about handling the error, all you have to do is to change call into try at each call site. Which is a simple change and the compiler would tell you which places need changing, and you can decide at each place if you want the default handling (error stop + stacktrace at runtime) or if you want to handle the error yourself. And also even later on, you will always see which places in your code can fail using the default error handling (error stop) and change them to handle the possible failure yourself. So that seems like a robust design.

[klausler]

The implementation would be very much like that of alternate returns, except that it would also work for functions (not just subroutines). The value returned has to be an error code, not just a simple flag.

If a call fails, the caller either handles the failure, or fails itself. In the latter case, the caller's return is almost immediate -- it still deallocates its local state.

[FortranFan]

The implementation would be very much like that of alternate returns, except that it would also work for functions (not just subroutines). The value returned has to be an error code, not just a simple flag.

If a call fails, the caller either handles the failure, or fails itself. In the latter case, the caller's return is almost immediate -- it still deallocates its local state.

I like the sound of this. Plus @klausler point about "The implementation would be very much like that of alternate returns," is really good to read. For I've always had a simpleton's view the alternate return facility (given it's still part of the standard even if labeled obsolescent) in a given implementation for Fortran might be able to assist with illustration and prototyping.

Re: @klausler other point, "The value returned has to be an error code," it will be really beneficial and user friendly from a coder perspective if the return value is of an enumeration type as proposed by @vansnyder above.

[FortranFan]

@certik wrote on 12 May 2020:

I think it is clear that many (if not the most) people in the wide Fortran community want some form of better exception handling.

I also agree that the Committee in general is not handling these requests efficiently. As you know I am running for the WG5 Convenor position and in my platform I propose that we work on these proposals continuously and when they are ready, we will consider them for inclusion. However, no matter who gets selected in the end, let us work on these things, and when they are ready, I will bring them to the Committee to consider.

So let's get to work.
..

• The C++ committee does not seem to be super happy with their traditional exceptions, and are proposing a different mechanism: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r0.pdf; Wouldn't it be a mistake to do what C++ found is not such a great idea?
  ..

Thank you @certik , your feedback and suggestions are very inclusive of the broader needs of the community and as such, they are very helpful to achieve progress in the Fortran language with this important feature.

As I mentioned above, my experience in industry is along the same lines as the IEEE Spectrum ranking in terms of the popular languages employed for computing and where each and every one of the top languages offers support for exception handling. In addition to C++, the languages used and all of whom support exception handling include C# and Java and Python and R and also some new languages such as Swift which has gained a lot of influence due to the influence of iOS ecosystem: https://docs.swift.org/swift-book/LanguageGuide/ErrorHandling.html. But not to be overlooked are other recent languages such as Julia which do compete with C++ and Fortran in the high-performance computing space also e.g., https://docs.julialang.org/en/v1/manual/control-flow/#Exception-Handling-1. And as mentioned by @vansnyder, Ada with its focus on embedded devices and controls and task-based concurrent programming with highly demanding environments such as aircraft systems has also provided robust support for exception handling.

Thus it appears many paths are available beyond that taken by early C++and I hope Fortran can also quickly learn from other approaches such as Julia, Swift, C#, etc. and offer up some performant facility.

[certik]

@FortranFan great links. The Swift link is I think very close if not exactly what we want. Very similar to Peter's proposal, although with different syntax.