Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Compat blog post #1226

Closed
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 113 additions & 0 deletions _posts/2018-01-01-compatibility-layer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
---
layout: post
title: "Compatibility Layer"
subtitle: "0.1 ❤ 0.3"
author: "Josef Brandl"
author_github: "MajorBreakfast"
date: 2018-01-01
categories: blog
---

# Futures 0.1 Compatibility Layer

Rust's futures ecosystem is currenlty split in two: On the one hand we have the vibrant ecosystem built around `futures@0.1` with its many libraries working on stable Rust and on the other hand there's the unstable `futures@0.3` ecosystem with support for the ergonomic and powerful `async`/`await` language feature. To bridge the gap between these two worlds we have introduced a compatibility layer (first released as part of futures 0.3.0-alpha.3). This blog post aims to give an overview over how to use it.

## `Cargo.toml`

The compatibility layer can be enabled by setting the `compat` or `tokio-compat` feature in your `Cargo.toml`:

```toml
futures-preview = { version = "0.3.0-alpha.3", features = ["tokio-compat"] }
```

To use `futures@0.1` and `futures@0.3` together in a single project, we can make use of the new cargo feature for renaming dependencies. Why? Because, even though the `futures@0.3` crate is called `futures-preview` on crates.io, it's lib name is also `futures`. By renaming `futures` version 0.1 to `futures01`, we can avoid a name collision:

```toml
# A the top:
cargo-features = ["rename-dependency"]

[dependencies]
futures01 = { package = "futures", version = "0.1", optional = true }
```

**Note: Renaming the crate is only required if you specify it as a dependency. If your project depends on Tokio and thus only indirectly on `futures@0.1`, then no renaming is required.**

## Async functions on 0.1 executors

The compatibility layer makes it possible to run 0.3 futures on executors built for 0.1. This makes it for instance possible to run futures created via `async`/`await` on Tokio's executor. Here's how this looks like:

```rust
#![feature(async_await, await_macro, futures_api)]
use futures::future::{FutureExt, TryFutureExt};
use futures::compat::TokioDefaultSpawner;

let future03 = async {
println!("Running on the pool");
};

let future01 = future03
.unit_error()
.boxed()
.compat(TokioDefaultSpawner);

tokio::run(future01);
```

Turning a 0.3 future into a 0.1 future requires three steps:
- First, the future needs to be a `TryFuture`, i.e. a future with `Output = Result<T, E>`. If your future isn't a `TryFuture` yet, you can quickly make it one using the `unit_error` combinator which wraps the output in a `Result<T, ()>`.
- Next, the future needs to be `Unpin`. If your future isn't `Unpin` yet, you can use the `boxed` combinator which wraps the future in a `PinBox`.
- The final step is to call the `compat` combinator which converts it into a future that can run both on 0.1 and 0.3 executors. This method requires a `spawner` parameter because 0.1 futures don't get passed a context that contains a spawner. If you use Tokio's default executor, you can do it like in the example above. Otherwise, take a look at the code example for `Executor01CompatExt::compat` if you want to specify a custom spawner.

## 0.1 futures in async functions

The conversion from a 0.1 future to a 0.3 future also works via a `compat` combinator method:

```rust
use futures::compat::Futures01CompatExt;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be use futures::compat::Future01CompatExt;


let future03 = future01.compat();
```

It converts a 0.1 `Future<Item = T, Error = E>` into a 0.3 `Future<Output = Result<T, E>>`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a note that even with this it's not possible to run a tokio based future on a 0.3 executor because there is no reactor available? (I have an idea for another combinator that would allow this, but haven't gotten round to trying it out yet).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't aware of this limitation. I'm not sure how to phrase this. Could you maybe formulate how to explain this best?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try and throw together a short failing example, I think that might be the best way to show why it doesn't work.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, damn, it's not even just that there's no reactor available, the current compat01to03 implementation won't work on any futures(0.3) executor because there's no thread-local Task to store the Notify in:

#![feature(async_await, await_macro, futures_api)]

use std::net::ToSocketAddrs;
use futures::compat::Future01CompatExt;

const MSG: &[u8] = &[0x44, 0x01, 0x6d, 0xb3, 0xb3, 0xb1, 0x28, 0xba, 0xb4, 0x74, 0x65, 0x73, 0x74];

async fn do_request() {
    let addr = ("coap.me", 5683).to_socket_addrs().unwrap().next().unwrap();
    let bind_addr = "0.0.0.0:0".parse().unwrap();
    let socket = tokio::net::UdpSocket::bind(&bind_addr).unwrap();
    let (socket, _) = await!(socket.send_dgram(MSG, &addr).compat()).unwrap();
    let buffer = vec![0; 256];
    let (_, buffer, length, _) = await!(socket.recv_dgram(buffer).compat()).unwrap();
    let result = String::from_utf8_lossy(&buffer[..length]);
    assert!(result.contains("welcome to the ETSI plugtest"))
}

fn main() {
    futures::executor::block_on(do_request())
}

gives an error:

thread 'main' panicked at 'no Task is currently running', libcore/option.rs:1000:5
stack backtrace:
[...]
   9: <futures::task_impl::NotifyHandle as core::clone::Clone>::clone
             at /Users/travis/build/rust-lang/rust/src/libcore/option.rs:312
  10: futures::task_impl::with
             at /Users/Nemo157/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.23/src/task_impl/mod.rs:43
  11: futures::task_impl::with_notify
              at /Users/Nemo157/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.23/src/task_impl/mod.rs:471
  12: futures_util::compat::compat01to03::<impl core::future::future::Future for futures_util::compat::compat::Compat<Fut, ()>>::poll
             at /Users/Nemo157/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-preview-0.3.0-alpha.3/src/compat/compat01to03.rs:21
[...]
  26: futures_executor::local_pool::block_on
              at /Users/Nemo157/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-preview-0.3.0-alpha.3/src/local_pool.rs:224
[...]


## Streams

Converting between 0.1 and 0.3 streams is possible via the `TryStreamExt::compat` and `Stream01CompatExt::compat` methods. Both combinators work analogously to their future equivalents.

## Conclusion

The compatiblity layer offers conversions in both directions and thus enables gradual migrations and experiments with futures 0.3. With that it manages to bridge the gap between the futures 0.1 and futures 0.3 ecosystems.

Finally a self contained example that shows how to fetch a website from a server:

```rust
#![feature(pin, async_await, await_macro)]
use futures::compat::{Future01CompatExt, Stream01CompatExt, TokioDefaultSpawner};
use futures::stream::{StreamExt};
use futures::future::{TryFutureExt, FutureExt};
use hyper::Client;
use pin_utils::pin_mut;
use std::io::{self, Write};

fn main() {
let future03 = async {
let url = "http://httpbin.org/ip".parse().unwrap();

let client = Client::new();
let res = await!(client.get(url).compat()).unwrap();
println!("{}", res.status());

let body = res.into_body().compat();
pin_mut!(body);
while let Some(Ok(chunk)) = await!(body.next()) {
io::stdout()
.write_all(&chunk)
.expect("example expects stdout is open");
}
};

tokio::run(future03.unit_error().boxed().compat(TokioDefaultSpawner))
}
```

Special thanks goes to [@tinaun](https://www.github.com/tinaun) and [@Nemo157](https://www.github.com/Nemo157) for developing the compatibility layer.