From eac94fa097934e9329c65a3471949b228b098d1d Mon Sep 17 00:00:00 2001 From: Steve Klabnik Date: Fri, 20 Mar 2015 17:35:52 -0400 Subject: [PATCH] Re-write closures chapter --- src/doc/trpl/closures.md | 538 +++++++++++++++++++++++++++++---------- 1 file changed, 401 insertions(+), 137 deletions(-) diff --git a/src/doc/trpl/closures.md b/src/doc/trpl/closures.md index bf4c2d903570b..01b8163ffd372 100644 --- a/src/doc/trpl/closures.md +++ b/src/doc/trpl/closures.md @@ -1,214 +1,478 @@ % Closures -So far, we've made lots of functions in Rust, but we've given them all names. -Rust also allows us to create anonymous functions. Rust's anonymous -functions are called *closures*. By themselves, closures aren't all that -interesting, but when you combine them with functions that take closures as -arguments, really powerful things are possible. +Rust not only has named functions, but anonymous functions as well. Anonymous +functions that have an associated environment are called 'closures', because they +close over an environment. Rust has a really great implementation of them, as +we'll see. -Let's make a closure: +# Syntax -```{rust} -let add_one = |x| { 1 + x }; +Closures look like this: -println!("The sum of 5 plus 1 is {}.", add_one(5)); +```rust +let plus_one = |x: i32| x + 1; + +assert_eq!(2, plus_one(1)); +``` + +We create a binding, `plus_one`, and assign it to a closure. The closure's +arguments go between the pipes (`|`), and the body is an expression, in this +case, `x + 1`. Remember that `{ }` is an expression, so we can have multi-line +closures too: + +```rust +let plus_two = |x| { + let mut result: i32 = x; + + result += 1; + result += 1; + + result +}; + +assert_eq!(4, plus_two(2)); +``` + +You'll notice a few things about closures that are a bit different than regular +functions defined with `fn`. The first of which is that we did not need to +annotate the types of arguments the closure takes or the values it returns. We +can: + +```rust +let plus_one = |x: i32| -> i32 { x + 1 }; + +assert_eq!(2, plus_one(1)); +``` + +But we don't have to. Why is this? Basically, it was chosen for ergonomic reasons. +While specifying the full type for named functions is helpful with things like +documentation and type inference, the types of closures are rarely documented +since they’re anonymous, and they don’t cause the kinds of error-at-a-distance +that inferring named function types can. + +The second is that the syntax is similar, but a bit different. I've added spaces +here to make them look a little closer: + +```rust +fn plus_one_v1 ( x: i32 ) -> i32 { x + 1 } +let plus_one_v2 = |x: i32 | -> i32 { x + 1 }; +let plus_one_v3 = |x: i32 | x + 1 ; ``` -We create a closure using the `|...| { ... }` syntax, and then we create a -binding so we can use it later. Note that we call the function using the -binding name and two parentheses, just like we would for a named function. +Small differences, but they're similar in ways. -Let's compare syntax. The two are pretty close: +# Closures and their environment -```{rust} -let add_one = |x: i32| -> i32 { 1 + x }; -fn add_one (x: i32) -> i32 { 1 + x } +Closures are called such because they 'close over their environment.' It +looks like this: + +```rust +let num = 5; +let plus_num = |x: i32| x + num; + +assert_eq!(10, plus_num(5)); ``` -As you may have noticed, closures infer their argument and return types, so you -don't need to declare one. This is different from named functions, which -default to returning unit (`()`). +This closure, `plus_num`, refers to a `let` binding in its scope: `num`. More +specifically, it borrows the binding. If we do something that would conflict +with that binding, we get an error. Like this one: + +```rust,ignore +let mut num = 5; +let plus_num = |x: i32| x + num; -There's one big difference between a closure and named functions, and it's in -the name: a closure "closes over its environment." What does that mean? It means -this: +let y = &mut num; +``` -```{rust} +Which errors with: + +```text +error: cannot borrow `num` as mutable because it is also borrowed as immutable + let y = &mut num; + ^~~ +note: previous borrow of `num` occurs here due to use in closure; the immutable + borrow prevents subsequent moves or mutable borrows of `num` until the borrow + ends + let plus_num = |x| x + num; + ^~~~~~~~~~~ +note: previous borrow ends here fn main() { - let x: i32 = 5; + let mut num = 5; + let plus_num = |x| x + num; + + let y = &mut num; +} +^ +``` + +A verbose yet helpful error message! As it says, we can't take a mutable borrow +on `num` because the closure is already borrowing it. If we let the closure go +out of scope, we can: + +```rust +let mut num = 5; +{ + let plus_num = |x: i32| x + num; + +} // plus_num goes out of scope, borrow of num ends - let printer = || { println!("x is: {}", x); }; +let y = &mut num; +``` + +If your closure requires it, however, Rust will take ownership and move +the environment instead: + +```rust,ignore +let nums = vec![1, 2, 3]; + +let takes_nums = || nums; + +println!("{:?}", nums); +``` + +This gives us: + +```text +note: `nums` moved into closure environment here because it has type + `[closure(()) -> collections::vec::Vec]`, which is non-copyable +let takes_nums = || nums; + ^~~~~~~ +``` + +`Vec` has ownership over its contents, and therefore, when we refer to it +in our closure, we have to take ownership of `nums`. It's the same as if we'd +passed `nums` to a function that took ownership of it. + +## `move` closures + +We can force our closure to take ownership of its environment with the `move` +keyword: - printer(); // prints "x is: 5" +```rust +let num = 5; + +let owns_num = move |x: i32| x + num; +``` + +Now, even though the keyword is `move`, the variables follow normal move semantics. +In this case, `5` implements `Copy`, and so `owns_num` takes ownership of a copy +of `num`. So what's the difference? + +```rust +let mut num = 5; + +{ + let mut add_num = |x: i32| num += x; + + add_num(5); } + +assert_eq!(10, num); ``` -The `||` syntax means this is an anonymous closure that takes no arguments. -Without it, we'd just have a block of code in `{}`s. +So in this case, our closure took a mutable reference to `num`, and then when +we called `add_num`, it mutated the underlying value, as we'd expect. We also +needed to declare `add_num` as `mut` too, because we’re mutating its +environment. -In other words, a closure has access to variables in the scope where it's -defined. The closure borrows any variables it uses, so this will error: +We also had to declare `add_num` as mut, since we will be modifying its +environment. -```{rust,ignore} -fn main() { - let mut x: i32 = 5; +If we change to a `move` closure, it's different: + +```rust +let mut num = 5; - let printer = || { println!("x is: {}", x); }; +{ + let mut add_num = move |x: i32| num += x; - x = 6; // error: cannot assign to `x` because it is borrowed + add_num(5); } + +assert_eq!(5, num); ``` -## Moving closures +We only get `5`. Rather than taking a mutable borrow out on our `num`, we took +ownership of a copy. + +Another way to think about `move` closures: they give a closure its own stack +frame. Without `move`, a closure may be tied to the stack frame that created +it, while a `move` closure is self-contained. This means that you cannot +generally return a non-`move` closure from a function, for example. + +But before we talk about taking and returning closures, we should talk some more +about the way that closures are implemented. As a systems language, Rust gives +you tons of control over what your code does, and closures are no different. -Rust has a second type of closure, called a *moving closure*. Moving -closures are indicated using the `move` keyword (e.g., `move || x * -x`). The difference between a moving closure and an ordinary closure -is that a moving closure always takes ownership of all variables that -it uses. Ordinary closures, in contrast, just create a reference into -the enclosing stack frame. Moving closures are most useful with Rust's -concurrency features, and so we'll just leave it at this for -now. We'll talk about them more in the "Concurrency" chapter of the book. +# Closure implementation -## Accepting closures as arguments +Rust's implementation of closures is a bit different than other languages. They +are effectively syntax sugar for traits. You'll want to make sure to have read +the [traits chapter][traits] before this one, as well as the chapter on [static +and dynamic dispatch][dispatch], which talks about trait objects. -Closures are most useful as an argument to another function. Here's an example: +[traits]: traits.html +[dispatch]: static-and-dynamic-dispatch.html -```{rust} -fn twice i32>(x: i32, f: F) -> i32 { - f(x) + f(x) +Got all that? Good. + +The key to understanding how closures work under the hood is something a bit +strange: Using `()` to call a function, like `foo()`, is an overloadable +operator. From this, everything else clicks into place. In Rust, we use the +trait system to overload operators. Calling functions is no different. We have +three separate traits to overload with: + +```rust +# mod foo { +pub trait Fn : FnMut { + extern "rust-call" fn call(&self, args: Args) -> Self::Output; } -fn main() { - let square = |x: i32| { x * x }; +pub trait FnMut : FnOnce { + extern "rust-call" fn call_mut(&mut self, args: Args) -> Self::Output; +} + +pub trait FnOnce { + type Output; - twice(5, square); // evaluates to 50 + extern "rust-call" fn call_once(self, args: Args) -> Self::Output; } +# } ``` -Let's break the example down, starting with `main`: +You'll notice a few differences between these traits, but a big one is `self`: +`Fn` takes `&self`, `FnMut` takes `&mut self`, and `FnOnce` takes `self`. This +covers all three kinds of `self` via the usual method call syntax. But we've +split them up into three traits, rather than having a single one. This gives us +a large amount of control over what kind of closures we can take. -```{rust} -let square = |x: i32| { x * x }; -``` +The `|| {}` syntax for closures is sugar for these three traits. Rust will +generate a struct for the environment, `impl` the appropriate trait, and then +use it. + +# Taking closures as arguments + +Now that we know that closures are traits, we already know how to accept and +return closures: just like any other trait! + +This also means that we can choose static vs dynamic dispatch as well. First, +let's write a function which takes something callable, calls it, and returns +the result: + +```rust +fn call_with_one(some_closure: F) -> i32 + where F : Fn(i32) -> i32 { + + some_closure(1) +} -We've seen this before. We make a closure that takes an integer, and returns -its square. +let answer = call_with_one(|x| x + 2); -```{rust} -# fn twice i32>(x: i32, f: F) -> i32 { f(x) + f(x) } -# let square = |x: i32| { x * x }; -twice(5, square); // evaluates to 50 +assert_eq!(3, answer); ``` -This line is more interesting. Here, we call our function, `twice`, and we pass -it two arguments: an integer, `5`, and our closure, `square`. This is just like -passing any other two variable bindings to a function, but if you've never -worked with closures before, it can seem a little complex. Just think: "I'm -passing two variables: one is an i32, and one is a function." +We pass our closure, `|x| x + 2`, to `call_with_one`. It just does what it +suggests: it calls the closure, giving it `1` as an argument. -Next, let's look at how `twice` is defined: +Let's examine the signature of `call_with_one` in more depth: -```{rust,ignore} -fn twice i32>(x: i32, f: F) -> i32 { +```rust +fn call_with_one(some_closure: F) -> i32 +# where F : Fn(i32) -> i32 { +# some_closure(1) } ``` -`twice` takes two arguments, `x` and `f`. That's why we called it with two -arguments. `x` is an `i32`, we've done that a ton of times. `f` is a function, -though, and that function takes an `i32` and returns an `i32`. This is -what the requirement `Fn(i32) -> i32` for the type parameter `F` says. -Now `F` represents *any* function that takes an `i32` and returns an `i32`. +We take one parameter, and it has the type `F`. We also return a `i32`. This part +isn't interesting. The next part is: -This is the most complicated function signature we've seen yet! Give it a read -a few times until you can see how it works. It takes a teeny bit of practice, and -then it's easy. The good news is that this kind of passing a closure around -can be very efficient. With all the type information available at compile-time -the compiler can do wonders. +```rust +# fn call_with_one(some_closure: F) -> i32 + where F : Fn(i32) -> i32 { +# some_closure(1) } +``` + +Because `Fn` is a trait, we can bound our generic with it. In this case, our closure +takes a `i32` as an argument and returns an `i32`, and so the generic bound we use +is `Fn(i32) -> i32`. -Finally, `twice` returns an `i32` as well. +There's one other key point here: because we're bounding a generic with a +trait, this will get monomorphized, and therefore, we'll be doing static +dispatch into the closure. That's pretty neat. In many langauges, closures are +inherently heap allocated, and will always involve dynamic dispatch. In Rust, +we can stack allocate our closure environment, and statically dispatch the +call. This happens quite often with iterators and their adapters, which often +take closures as arguments. -Okay, let's look at the body of `twice`: +Of course, if we want dynamic dispatch, we can get that too. A trait object +handles this case, as usual: -```{rust} -fn twice i32>(x: i32, f: F) -> i32 { - f(x) + f(x) +```rust +fn call_with_one(some_closure: &Fn(i32) -> i32) -> i32 { + some_closure(1) } + +let answer = call_with_one(&|x| x + 2); + +assert_eq!(3, answer); ``` -Since our closure is named `f`, we can call it just like we called our closures -before, and we pass in our `x` argument to each one, hence the name `twice`. +Now we take a trait object, a `&Fn`. And we have to make a reference +to our closure when we pass it to `call_with_one`, so we use `&||`. -If you do the math, `(5 * 5) + (5 * 5) == 50`, so that's the output we get. +# Returning closures -Play around with this concept until you're comfortable with it. Rust's standard -library uses lots of closures where appropriate, so you'll be using -this technique a lot. +It’s very common for functional-style code to return closures in various +situations. If you try to return a closure, you may run into an error. At +first, it may seem strange, but we'll figure it out. Here's how you'd probably +try to return a closure from a function: -If we didn't want to give `square` a name, we could just define it inline. -This example is the same as the previous one: +```rust,ignore +fn factory() -> (Fn(i32) -> Vec) { + let vec = vec![1, 2, 3]; -```{rust} -fn twice i32>(x: i32, f: F) -> i32 { - f(x) + f(x) + |n| vec.push(n) } -fn main() { - twice(5, |x: i32| { x * x }); // evaluates to 50 -} +let f = factory(); + +let answer = f(4); +assert_eq!(vec![1, 2, 3, 4], answer); ``` -A named function's name can be used wherever you'd use a closure. Another -way of writing the previous example: +This gives us these long, related errors: + +```text +error: the trait `core::marker::Sized` is not implemented for the type +`core::ops::Fn(i32) -> collections::vec::Vec` [E0277] +f = factory(); +^ +note: `core::ops::Fn(i32) -> collections::vec::Vec` does not have a +constant size known at compile-time +f = factory(); +^ +error: the trait `core::marker::Sized` is not implemented for the type +`core::ops::Fn(i32) -> collections::vec::Vec` [E0277] +factory() -> (Fn(i32) -> Vec) { + ^~~~~~~~~~~~~~~~~~~~~ +note: `core::ops::Fn(i32) -> collections::vec::Vec` does not have a constant size known at compile-time +fa ctory() -> (Fn(i32) -> Vec) { + ^~~~~~~~~~~~~~~~~~~~~ -```{rust} -fn twice i32>(x: i32, f: F) -> i32 { - f(x) + f(x) -} +``` -fn square(x: i32) -> i32 { x * x } +In order to return something from a function, Rust needs to know what +size the return type is. But since `Fn` is a trait, it could be various +things of various sizes: many different types can implement `Fn`. An easy +way to give something a size is to take a reference to it, as references +have a known size. So we'd write this: -fn main() { - twice(5, square); // evaluates to 50 +```rust,ignore +fn factory() -> &(Fn(i32) -> Vec) { + let vec = vec![1, 2, 3]; + + |n| vec.push(n) } + +let f = factory(); + +let answer = f(4); +assert_eq!(vec![1, 2, 3, 4], answer); +``` + +But we get another error: + +```text +error: missing lifetime specifier [E0106] +fn factory() -> &(Fn(i32) -> i32) { + ^~~~~~~~~~~~~~~~~ ``` -Doing this is not particularly common, but it's useful every once in a while. +Right. Because we have a reference, we need to give it a lifetime. But +our `factory()` function takes no arguments, so elision doesn't kick in +here. What lifetime can we choose? `'static`: -Before we move on, let us look at a function that accepts two closures. +```rust,ignore +fn factory() -> &'static (Fn(i32) -> i32) { + let num = 5; -```{rust} -fn compose(x: i32, f: F, g: G) -> i32 - where F: Fn(i32) -> i32, G: Fn(i32) -> i32 { - g(f(x)) + |x| x + num } -fn main() { - compose(5, - |n: i32| { n + 42 }, - |n: i32| { n * 2 }); // evaluates to 94 +let f = factory(); + +let answer = f(1); +assert_eq!(6, answer); +``` + +But we get another error: + +```text +error: mismatched types: + expected `&'static core::ops::Fn(i32) -> i32`, + found `[closure :7:9: 7:20]` +(expected &-ptr, + found closure) [E0308] + |x| x + num + ^~~~~~~~~~~ + +``` + +This error is letting us know that we don't have a `&'static Fn(i32) -> i32`, +we have a `[closure :7:9: 7:20]`. Wait, what? + +Because each closure generates its own environment `struct` and implementation +of `Fn` and friends, these types are anonymous. They exist just solely for +this closure. So Rust shows them as `closure `, rather than some +autogenerated name. + +But why doesn't our closure implement `&'static Fn`? Well, as we discussed before, +closures borrow their environment. And in this case, our environment is based +on a stack-allocated `5`, the `num` variable binding. So the borrow has a lifetime +of the stack frame. So if we returned this closure, the function call would be +over, the stack frame would go away, and our closure is capturing an environment +of garbage memory! + +So what to do? This _almost_ works: + +```rust,ignore +fn factory() -> Box i32> { + let num = 5; + + Box::new(|x| x + num) } +# fn main() { +let f = factory(); + +let answer = f(1); +assert_eq!(6, answer); +# } ``` -You might ask yourself: why do we need to introduce two type -parameters `F` and `G` here? Evidently, both `f` and `g` have the -same signature: `Fn(i32) -> i32`. +We use a trait object, by `Box`ing up the `Fn`. There's just one last problem: -That is because in Rust each closure has its own unique type. -So, not only do closures with different signatures have different types, -but different closures with the *same* signature have *different* -types, as well! +```text +error: `num` does not live long enough +Box::new(|x| x + num) + ^~~~~~~~~~~ +``` + +We still have a reference to the parent stack frame. With one last fix, we can +make this work: -You can think of it this way: the behavior of a closure is part of its -type. Therefore, using a single type parameter for both closures -will accept the first of them, rejecting the second. The distinct -type of the second closure does not allow it to be represented by the -same type parameter as that of the first. We acknowledge this, and -use two different type parameters `F` and `G`. +```rust +fn factory() -> Box i32> { + let num = 5; -This also introduces the `where` clause, which lets us describe type -parameters in a more flexible manner. + Box::new(move |x| x + num) +} +# fn main() { +let f = factory(); + +let answer = f(1); +assert_eq!(6, answer); +# } +``` -That's all you need to get the hang of closures! Closures are a little bit -strange at first, but once you're used to them, you'll miss them -in other languages. Passing functions to other functions is -incredibly powerful, as you will see in the following chapter about iterators. +By making the inner closure a `move Fn`, we create a new stack frame for our +closure. By `Box`ing it up, we've given it a known size, and allowing it to +escape our stack frame.