Conversion to tokio 0.2 #68

dbcfd · 2019-09-09T23:40:55Z

Convert to tokio 0.2

dbcfd · 2019-09-09T23:41:30Z

currently hung up on tokio-dns, looks like the future it returns isn't correct, Box<Future> rather than impl Future or Pin<Box<Future>>

najamelan · 2019-09-10T13:07:52Z

Have you filed an issue at tokio-dns to resolve the blocker?

dbcfd · 2019-09-10T14:16:55Z

Master has the required updates, so using that to keep moving forward.

…

On Tue, Sep 10, 2019, 7:07 AM Naja Melan ***@***.***> wrote: Have you filed an issue at tokio-dns to resolve the blocker? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#68?email_source=notifications&email_token=AALIKFGWLR6I3CVQD2WTRKLQI6L2TA5CNFSM4IVBHKW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6LAOPA#issuecomment-529925948>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AALIKFEKHTFMHI4M7MJYOK3QI6L2TANCNFSM4IVBHKWQ> .

najamelan

Just asking some questions, I haven't fully reviewed everything.

I do feel this lacks integration testing, verifying every possible scenario, but then again, tokio-tungstenite didn't have those before either.

src/compat.rs

src/lib.rs

src/handshake.rs

src/stream.rs

dbcfd · 2019-09-10T22:37:07Z

Just asking some questions, I haven't fully reviewed everything.

I do feel this lacks integration testing, verifying every possible scenario, but then again, tokio-tungstenite didn't have those before either.

Will look at add at least a couple more once I get something that seems like it might work.

dbcfd · 2019-09-12T19:14:36Z

@najamelan @application-developer-DA

Everything compiling, existing tests passing. Examples upgraded as well. Will be seeing what I can come up with for tests.

CryZe · 2019-09-15T18:38:03Z

This doesn't build without the tls feature. In particular here: https://github.com/dbcfd/tungstenite-rs/blob/expose-machine/src/client.rs#L49

error[E0432]: unresolved import `error`                                                                    
  --> C:\Users\Christopher Serr\.cargo\git\checkouts\tungstenite-rs-0e70558ccb2effd7\02684b4\src\client.rs:49:9
   |
49 |     use error::{Error, Result};
   |         ^^^^^ help: a similar path exists: `crate::error`
   |
   = note: `use` statements changed in Rust 2018; read more at <https://doc.rust-lang.org/edition-guide/rust-2018/module-system/path-clarity.html>

error[E0432]: unresolved import `stream`
  --> C:\Users\Christopher Serr\.cargo\git\checkouts\tungstenite-rs-0e70558ccb2effd7\02684b4\src\client.rs:50:9
   |
50 |     use stream::Mode;
   |         ^^^^^^ help: a similar path exists: `crate::stream`
   |
   = note: `use` statements changed in Rust 2018; read more at <https://doc.rust-lang.org/edition-guide/rust-2018/module-system/path-clarity.html>

dbcfd · 2019-09-16T12:41:33Z

@CryZe This was actually an issue in tungstenite and is fixed in snapview/tungstenite-rs#73. cargo update and this branch should build now.

CryZe · 2019-09-16T13:08:57Z

Nice, another issue I ran into was that the pin-project 0.4.0-alpha.10 pulled in here is causing a cargo update failure when the new hyper is also in the dependency graph. But I don't think it's reasonable to ask for you to downgrade to their version. That's a general problem with all these alpha versions because they are all semver incompatible (to the point where you can't even have a dependency on multiple, even if they don't interact with each other). Although maybe one can specify 0.4.0-alpha.*? Idk, I haven't tried, but maybe that would be better.

dbcfd · 2019-09-16T15:19:09Z

Matching hyper for now, since one of the reasons for this work was to use it in warp.

CryZe · 2019-09-16T22:04:50Z

If I'm sending two messages before trying to receive any I get:

thread 'main' panicked at 'assertion failed: !self.context.is_null()', C:\Users\Christopher Serr\.cargo\git\checkouts\tokio-tungstenite-42c59291ea04b35c\bea935e\src\compat.rs:44:13

dbcfd · 2019-09-19T14:33:20Z

@CryZe Added a test which reproduces this behavior, attempting to determine cause and fix it.

dbcfd · 2019-09-19T15:14:15Z

The multiple write failure is due to tungstenite write_message calling flush. This means that Sink cannot be used for send, since it does not include a context.

CryZe · 2019-09-20T08:01:29Z

That seems to have fixed the bug for me :)

CryZe · 2019-09-23T17:30:20Z

Is there a reason this doesn't use Sink for sending messages?

dbcfd · 2019-09-23T18:35:48Z

@CryZe Sink cannot be used because tungstenite write_message is also calling flush. Sink write does not provide a context, so no context is present when write_message then calls flush.

One possibility is to separate the the write and flush into separate functions in tungstenite (that write_message uses), so then we could implement Sink in tokio-tungstenite.

dbrgn · 2019-10-01T09:09:17Z

Async/await is now available on Beta, so we don't even need nightly anymore! 🎉

https://blog.rust-lang.org/2019/09/30/Async-await-hits-beta.html

CryZe · 2019-10-02T06:58:58Z

Yeah looks like tokio and futures have released versions that are compatible with the beta, so I'd appreciate if this PR would be updated to those versions as well.

I did the changes locally here: CryZe@d4e5872

stkfd · 2019-10-08T00:05:41Z

Just wanted to say I'm pretty excited about this PR, I've been trying the most recent version out for a side project the last few days. It's just very smooth integration, I'd say currently this is shaping up to become the go-to for anyone building a websocket based application using async/await.

The only issue I've noticed so far is due to not implementing Sink in this version, as mentioned above - it means you can't split the stream into Sender and Receiver because StreamExt::split requires Self: Sink, which can be a killer for some applications

anlumo · 2019-10-08T00:47:30Z

Can someone point me to a way to read and write to the same Websocket with this implementation without Sink? Both poll_next and send need a &mut self, so I can't do both without splitting.

agalakhov · 2019-10-08T00:54:14Z

It is doable via .into_future() of the Stream trait for reading. The whole is likely to be called from unfold() or loop_fn(). This technique is useful for implementing state machines.

anlumo · 2019-10-08T00:58:35Z

I just discovered that it does work when I remove my call to map on the stream. However, I need that to be able to select on two different streams with different item types. The idea is that I have an futures::channel::mpsc as the incoming message channel from other parts of the program, so these other parts can communicate with the websocket.

I also can't use forward, since that requires a Sink as well.

Seems like this is one of the applications where the Sink issue is a killer, as @skaufhold mentioned.

dbcfd · 2019-10-08T01:15:29Z

@skaufhold I will look about adding a split method to this, so that Sink is not required.

@anlumo It will look something like this

while let Some(try_msg) = websocket_stream.next().await {
    let msg = try_msg.expect("Failed to receive message");
    sender.send(msg);
    if let Some(resp) = receiver.next().await {
        websocket_stream.send(resp).await.expect("Failed to receive message");
    }
}

You can also do something like

while let Some(try_msg) = websocket_stream.next().await {
  let msg = try_msg.expect("Failed to receive message");
  let response = transform(msg);
  websocket_stream.send(response).await.expect("Failed to send response");
}

The examples (especially autobahn) show this off better https://github.com/snapview/tokio-tungstenite/pull/68/files#diff-9da84c2001004ed41d861e7b1fc0850fR37

mcseemk · 2019-10-08T01:16:59Z

I just discovered that it does work when I remove my call to map on the stream. However, I need that to be able to select on two different streams with different item types. The idea is that I have an futures::channel::mpsc as the incoming message channel from other parts of the program, so these other parts can communicate with the websocket.

I also can't use forward, since that requires a Sink as well.

Seems like this is one of the applications where the Sink issue is a killer, as @skaufhold mentioned.

I use the following workaround:

let (mut ws_stream, _) = match connect_async(...).await {
    Ok(x) => x,
    Err(e) => {}
};

let mut heartbeat = Interval::new(Duration::from_secs(3));

'inner: loop {
    let either1 = select(receive_channel.next(), select(heartbeat.next(), ws_stream.next())).await;
    match either1 {
        Either::Left((msg, _)) => {
            // received message via channel, process it
        }
        Either::Right((either2, _)) => {
            match either2 {
                Either::Left(_) => {
                    // heartbeat, do whatever
                }
                Either::Right((msg, _)) => {
                    // websocket message received, process it
                }
            }
        }
    }
}

nemasu · 2019-12-03T05:50:20Z

Hello @dbcfd , I'm fiddling with the server and client examples.
The description says it broadcasts to all connected clients, but it seems to only echo messages back to the client that sent it.

nemasu · 2019-12-05T09:54:27Z

Hello @dbcfd , I made a PR to update the server example. Seems like you need to hit return in the client terminal(s) before any new messages show up, I'm pretty sure that's the clients fault though.

CryZe · 2019-12-10T12:15:38Z

What's the status of this?

bstrie · 2019-12-10T19:33:10Z

I believe that the acceptance of this PR was blocked on the next stable hyper release? That's just happened ( https://seanmonstar.com/post/189594157852/hyper-v013 ), so are there any remaining issues here?

This reverts commit dd9cad6.

dbcfd · 2019-12-10T19:57:18Z

@bstrie Nope :D

dbcfd · 2019-12-10T19:57:57Z

@application-developer-DA ready for review/merge

Cargo.toml

bstrie · 2019-12-12T19:18:54Z

Cargo.toml

@@ -21,7 +21,7 @@ stream = []
 log = "0.4"
 futures = "0.3"
 pin-project = "0.4"
-tokio = "0.2"
+tokio = { version = "0.2", default-features = false, features = ["io-util"] }


Nit, but tokio's Cargo.toml has:

# Include nothing by default default = []

so default-features = false is a no-op.

Well that's assuming they won't add some. But I guess that's somewhat unlikely.

vorot93 · 2019-12-19T02:56:35Z

@application-developer-DA Any chance this could be merged ASAP?

kellytk · 2019-12-23T17:10:12Z

What, if anything, could be done to progress this?

bstrie · 2019-12-23T17:23:53Z

I would suspect that the maintainers are reasonably predisposed during the holidays, although I suppose it wouldn't hurt to ping @agalakhov as well.

agalakhov · 2019-12-23T18:43:40Z

I'll check this during the holidays.

vorot93 · 2019-12-28T06:45:03Z

@application-developer-DA while examples are definitely important, I strongly suggest merging this as is if it works. There is too much downstream blocked from upgrading to Tokio 0.2 because this PR has not been merged yet.

agalakhov · 2019-12-30T21:19:05Z

Checked this partially. This pull request requires more cleanup to be merged. I'm working on it. Sorry, it can't be merged in its current state.

vorot93 · 2020-01-03T18:13:29Z

@agalakhov If it's in the working state it could be possible to merge as is and keep iterating in subsequent commits. You can slap an -alpha tag if you feel uncomfortable about the current quality of the port. At this point some working release is much better than no release at all!

agalakhov · 2020-01-03T19:47:48Z

I have to check for some potential security and stability issues first. Especially handshake code has to be carefully checked. Now it is much better than initially (no more unsafe), but I'm still suspicious about some places. I just don't want to see any CVE filed.

agalakhov · 2020-01-07T20:22:33Z

Ok, I checked this. While handshake submodule could be improved, I did not found any security related problems yet. It seems to be safe to merge. Thank you.

najamelan reviewed Sep 10, 2019

View reviewed changes

src/compat.rs Outdated Show resolved Hide resolved

src/lib.rs Show resolved Hide resolved

src/lib.rs Outdated Show resolved Hide resolved

dbcfd commented Sep 10, 2019

View reviewed changes

src/handshake.rs Outdated Show resolved Hide resolved

dbcfd commented Sep 10, 2019

View reviewed changes

src/stream.rs Show resolved Hide resolved

dbcfd mentioned this pull request Sep 12, 2019

Tokio-Tungstenite Async Changes snapview/tungstenite-rs#73

Merged

dbcfd force-pushed the tokio2 branch from d7efd25 to d72a2d5 Compare September 12, 2019 19:13

dbcfd mentioned this pull request Sep 12, 2019

Support tokio 0.2 and futures 0.3 #64

Closed

dbcfd mentioned this pull request Sep 16, 2019

update to std::future::Future seanmonstar/warp#265

Merged

Downgrade to tokio alpha

dd9cad6

dbcfd force-pushed the tokio2 branch from b9b1c7d to dd9cad6 Compare November 30, 2019 18:48

sdroege mentioned this pull request Dec 2, 2019

Client example no longer compiles #70

Closed

CryZe mentioned this pull request Dec 9, 2019

Support the event stream again LiveSplit/splits-io-api#5

Open

Revert "Downgrade to tokio alpha"

d06001d

This reverts commit dd9cad6.

dbcfd mentioned this pull request Dec 10, 2019

broadcast_server example and server description update. dbcfd/tokio-tungstenite#9

Closed

bstrie reviewed Dec 10, 2019

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

Drop alpha notation

d25475c

samrg472 reviewed Dec 10, 2019

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

Change doc link, reduce tokio features

6dc2018

bstrie reviewed Dec 12, 2019

View reviewed changes

Clean up examples

5a50cf8

This was referenced Jan 5, 2020

Towards a v1.0 interledger/interledger-rs#557

Open

Migration to 0.3 & Async/Await interledger/interledger-rs#582

Closed

agalakhov merged commit 2474ba4 into snapview:master Jan 7, 2020

Conversion to tokio 0.2 #68

Conversion to tokio 0.2 #68

Conversation

dbcfd commented Sep 9, 2019

dbcfd commented Sep 9, 2019

najamelan commented Sep 10, 2019

dbcfd commented Sep 10, 2019 via email

najamelan left a comment

Choose a reason for hiding this comment

dbcfd commented Sep 10, 2019

dbcfd commented Sep 12, 2019

CryZe commented Sep 15, 2019 • edited Loading

dbcfd commented Sep 16, 2019

CryZe commented Sep 16, 2019 • edited Loading

dbcfd commented Sep 16, 2019

CryZe commented Sep 16, 2019

dbcfd commented Sep 19, 2019

dbcfd commented Sep 19, 2019

CryZe commented Sep 20, 2019

CryZe commented Sep 23, 2019 • edited Loading

dbcfd commented Sep 23, 2019

dbrgn commented Oct 1, 2019

CryZe commented Oct 2, 2019 • edited Loading

stkfd commented Oct 8, 2019

anlumo commented Oct 8, 2019

agalakhov commented Oct 8, 2019 • edited Loading

anlumo commented Oct 8, 2019 • edited Loading

dbcfd commented Oct 8, 2019 • edited Loading

mcseemk commented Oct 8, 2019

nemasu commented Dec 3, 2019

nemasu commented Dec 5, 2019

CryZe commented Dec 10, 2019

bstrie commented Dec 10, 2019

dbcfd commented Dec 10, 2019

dbcfd commented Dec 10, 2019

bstrie Dec 12, 2019

Choose a reason for hiding this comment

CryZe Dec 12, 2019

Choose a reason for hiding this comment

vorot93 commented Dec 19, 2019

kellytk commented Dec 23, 2019

bstrie commented Dec 23, 2019

agalakhov commented Dec 23, 2019

vorot93 commented Dec 28, 2019

agalakhov commented Dec 30, 2019 • edited Loading

vorot93 commented Jan 3, 2020 • edited Loading

agalakhov commented Jan 3, 2020

agalakhov commented Jan 7, 2020

CryZe commented Sep 15, 2019 •

edited

Loading

CryZe commented Sep 16, 2019 •

edited

Loading

CryZe commented Sep 23, 2019 •

edited

Loading

CryZe commented Oct 2, 2019 •

edited

Loading

agalakhov commented Oct 8, 2019 •

edited

Loading

anlumo commented Oct 8, 2019 •

edited

Loading

dbcfd commented Oct 8, 2019 •

edited

Loading

agalakhov commented Dec 30, 2019 •

edited

Loading

vorot93 commented Jan 3, 2020 •

edited

Loading