-
-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add erroring and widening arithmetic operators #14320
Comments
Only looking at this proposal:
This feels consistent, but not very ergonomic to write or read. Also I am curious: Do you know use cases, where non-global handling (not at statement level) of overflow and widening would be strictly necessary?
Statement level means Counter-proposal (wip thought process): // return error on failed operation
var x: u128 = try @errorop(a + b - d*c + g*f - z)
var x: u128 = @errorop(a + b - d*c + g*f - z) catch unreachable;
// widen
var x = @widen(a + b - d*c + g*f - z); // error: widen requires type on result location
var x: u128 = @widen(a + b - d*c + g*f - z);
// mixing @errorop with other errors must be forbidden. arithmetic errors of local context
must always be handled local Shortcomings with this counter-proposal:
|
I feel more like widening and error operation for arithmetic-only (no other errors possible in that statement) belong to the assignment, because the alternative is less readable, less copy-pastable and arithmetic is too common of an operation. The typical use case for "add overflow checks to everything" is to me to have effortless + relative fast sanitation of a limited number of inputs not known to be safe to use, because the developer did not check input bounds of the arithmetic. Or do I misunderstand the use case for widening and error handling in arithmetic-only statements? |
I see your point, but I went with
Well, no widening operator was used in that expression, so that would just give you the status quo integer type unioned with an error. If you meant to use the widening operators in all those places, I would agree the result of that widening would not be obvious, and I think that would be a very contrived and bad piece of code to write. The point of these operators is to make it easier to express (good and safe) intent, not to throw a bunch of syntax together and guess what it means. I'd expect that widening operators will typically be used in expressions with only one or two widening operators in them. I'd expect things like these to be way more common: x +_ @boolToInt(b)
@boolToInt(b1) +_ @boolToInt(b2)
a *_ b
I am not sure which way to interpret this question, so I will try to answer the different things you might mean. For one, Zig already handles expressions independently of the assignment statement they occur in. E.g. If you are asking about the need for being able to express different arithmetic semantics within a single expression, I gave a theoretical example of a situation where you know that adding up some numbers will give an answer larger than another number. var x = a +! b - c;
// the programmer knows that a+b>c If you are asking about whether it would ever make sense to
Well you can distinguish where it happened, if you write it in such a way that you can by breaking up into multiple statements or
The problem with adding built-ins like
The last part of this statement is the problem, in my view. I want programmers to actually express in their program where overflow can occur and where it can't. E.g. imagine I calculate how much space to allocate, allocate that much space, and then write the important data into that buffer. When calculating how much space to allocate in the first pass, it might overflow. When calculating the index to write into that buffer, we know it can't overflow because we are repeating the calculations from the first pass: var required_space: error{Overflow}!u64 = 0;
for (data) |b| { // imagine some real logic
required_space +!= 1;
// we might overflow if we don't know how big `data` is
// imagine we're streaming `data` or the calculation for space usage is superlinear
}
var buffer = try allocator.alloc(try required_space);
var index: u64 = 0;
for (data) |b| {
buffer[index] = b;
index += 1;
// we know this can't overflow because we are repeating
// the calculations from the previous, already checked loop.
} |
Unfortunately, we need to take into consideration all edge cases or we can create unnecessary + terrible footguns. I think allowing error handling together with trapping in tedious and long expressions like Likewise stands my argument for widening: Each nesting of a multiplication multiplies the necessary bit size by 2, which obliterates performance very quickly if not forcing to be catched via explicit type bounds. Mixing trapping and widening code is here also a problem. I do agree with you on the reasoning for
Agreed. I would phrase the proposal according to exactly this use case instead of "Add erroring and widening arithmetic operators".
|
After thinking about this for a while, I'm starting to like this. I like the idea of making all the different operator types equally hard/easy to write. I think An argument against |
The one thing I don't understand about this proposal is the auto-over-widening rounding up to a power of two.
I understand that most architectures won't provide specific arithmetic instructions for, say, With the proposed over-widened result types, if I need/want slimmer ranges, I'd have to
With slimmer result types, you'd only have to fiddle with types in response to the generated assembly - |
I'm not fully understanding what specific situations you may be thinking of, but I don't think this is a bad feature just because someone could abuse it. I think in order for something to be a footgun you have to be trying to get it right but some other subtlety thwarts your efforts. E.g. With I also am not sure I agree with the statement "without the typo it just works". For one, when you have an expression that produces an error vs one that doesn't, that will be handled very differently by the rest of the code and so that will require multiple lines to change (or at least some If you are just critiquing the possibility of someone properly specifying which operations can overflow for most of the operations but specifying one operation incorrectly, then to that I would say that that could happen in status quo anyway. In fact, the easiest thing to do is to just not specify which operations can overflow at all. So from that perspective, any stray
The mere presence of these operators in the language will encourage programmers to ask themselves whether each arithmetic operation might overflow and what should happen in each case. It will change the way programmers think: it will make them think more carefully about arithmetic operations in general. So I really am not afraid that someone might do
Yes, safe arithmetic is slower on most current ISAs, but that is an implementation detail. Don't get me wrong, I care deeply about speed, and power consumption too. On better ISA's like the Mill, which hopefully takes over the world some day, safe arithmetic is equally as fast as wrapping arithmetic. On ISA's like x86_64, it requires a To that, I would say that correctness is more important than speed. One of Zig's fundamental beliefs is that allocation can fail, even though allocation succeeds 99.9999% of the time (made up statistic but probably pretty close). Zig would be faster if it didn't check whether allocation failed and didn't pass around allocator pointers. But would it be better? To me, one of the most attractive features of Zig is the ability to ensure all edge-cases are properly handled. Zig should encourage handling arithmetic operations properly too. |
You are probably right. I was not sure what behavior would be expected but now that I think about it, it is probably more Zig-like to produce integer types that are not rounded up to a power of two. And like you said, the compiler should be able to figure out that a |
@rohlem Do you believe it would make sense to have |
If we wanted to be able to narrow through division with comptime_int, you wouldn't need a whole new operator, you'd just need to make it so that that's what happens when you divide with a comptime_int. |
I like and support @InKryption's suggestion above, I don't see a reason why we shouldn't always narrow the result type when dividing by comptime_int. @Validark I do think One special case that an actual widening division would be useful would be if we had distinct float types for finite-only and full IEEE numbers. |
In Zig, the division operator does not work with signed integers unless they are known to be positive at compile-time, because you have to choose whether you want it to round towards 0 or -inf ( |
Update: I removed over-widening (rounding up to a power of 2) from the proposal. |
Problem
Currently in Zig, the way to express a program which properly handles all potential errors, in this case integer overflows, is not the easiest, laziest thing to do. To use addition as the canonical example, most people reach for the
+
operator for nearly every case where they want to do addition (with+%
and+|
being used when called for by particular needs). This means the programmer is not expressing which additions can overflow and which cannot, because both are being expressed the same way.Gibbon1's comment on Hackernews, although indicative of a lack of understanding that programs can be properly expressed in Zig, does indicate that this is a real problem.
See comment
abainbridge:
From the Zig language reference: Zig has many instances of undefined behavior. If undefined behavior is detected at compile-time, Zig emits a compile error and refuses to continue. Most undefined behavior that cannot be detected at compile-time can be detected at runtime. In these cases, Zig has safety checks. [...] When a safety check fails, Zig crashes with a stack trace
Gibbon1:
I hope the people behind Zig understand that in critical applications that's totally unacceptable.
Here is a YouTube video touring the facilities provided by various languages for integer overflow, and unfortunately for Zig, it is mainly in the same boat as C/C++, with
std.math.add
(potentially erroring addition) andstd.math.mulWide
(casting to a bigger integer size) being relegated to a single comment in the comments section (at the time of writing). I believe I have also seen other sentiments and comments over time of a similar nature. Many people think the way Zig deals with overflow is by using ReleaseSafe and throwing up its hands in defeat when it happens or using ReleaseFast and just letting wraparound behavior do ungodly things to your program invariants should an overflow occur (like allowing buffer overruns).In short,
std.math.add
,@addWithOverflow
, and casting integers to a bigger size are not nearly as ubiquitous as they probably should be, because they have a lot more friction than just using+
. Zig strives to apply friction to make it harder to do things you likely should not be doing and make it easy to do things the right way, but in the status quo, I believe it accidentally encourages people to write suboptimal software that does not adequately express their intent when it comes to overflowable integer operations.Solution
To remedy this, I propose adding two new classes of operators, which for now can be thought of as syntactic sugar for the standard library functions like
std.math.add
(erroring behavior) andstd.math.mulWide
(widening behavior).+!
error.Overflow
error upon overflow.+_
This should complement the existing operators:
+
a + b
is equivalent to(a +! b) catch unreachable
+%
+|
There should be similar operators for the other operators than can produce an
error.Overflow
. I.e.-!
,*!
,-_
,*_
. The erroring operators also conveniently eliminate the need for@addWithOverflow
,@subWithOverflow
,@mulWithOverflow
, simplifying the language a little.Example usage
I also think it would be a welcome feature to allow (these?) errors to propagate across operations, kind of like NaN.
Here is an implementation of the new operators, with the semantics I think are most desirable.
Note that the widening operator actually adapts according to what the range of potential variables are. That means when dealing with comptime ints we can be more precise:
Assignment operators
+!=
makes sense as an operator so long as it is operating on an error union. It would be necessary to check the value using an if statement after a+!=
operation, but it would work.+_=
does not make sense as an operator, as it would require changing the destination variable type, but in Zig variables cannot change their type dynamically.Discussions
There could also be
<<!
,>>!
which could replace@shlWithOverflow
, as well as@shrExact
and@shlExact
when paired withcatch unreachable
. However, having these might imply thata << b
is equivalent to(a <<! b) catch unreachable
, which in status quo it isn't. It's worth consideration, but probably not a good fit for Zig at the moment.Note that the
/
operator only works on unsigned values in Zig, so overflowing viastd.math.minInt(Type) / -1
is not possible. However, there is still safety checked/undefined behavior for divisions by zero, so/!
could check evaluate toerror.DivisionByZero
when applicable, although it is less compelling of use-case to support.I think the
!
and_
operators should not work for floats. E.g.f32 +_ f32
should not work, and I don't even know what the!
operators would do for floats anyway.Parsing
+!
et al. should mostly integrate into Zig just fine, becausea+!b
has never worked because the arithmetic operators do not work on booleans.+_
could conflict with existing Zig programs in the case ofa+_b
. Thus, the+_
operator must require a space afterwards in order to be considered a valid widening addition. Ambiguous+
operators without a space following it where the next operand is a variable that starts with an underscore should be banned. E.g.a+_b
should be banned because it could meana +_ b
ora + _b
, anda+_1
should be banned because it could meana +_ 1
ora + _1
.Prior art
Others have had this idea: #7821 (comment)
Update: Removed rounding up to power of 2.
Side note: I also believe the error should be changed to
error.IntegerOverflow
, so as to not be confused with stack overflow or any other overflow you can think of.The text was updated successfully, but these errors were encountered: