-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Play nice with Web Assembly #2
Comments
WASM is great. Yeah, maybe we write the "wormhole core" approach (in #1) in a way that can be compiled to wasm, and then the IO wrapper gets written in JS (since wasm is a purely computational environment, and has no IO except exchanging messages with its host). |
Mind you, the amount of JS you'd need to write for the platform-specific IO layer might be considerable.. |
Does this require |
I started experimenting with this over the weekend. Two stumbling blocks I've hit so far:
I don't know what other problems we'll hit. I was able to bypass the first one by just replacing all our symmetric crypto with no-ops, but I don't know how to get past the second one. Maybe there's some Cargo.toml/Cargo.lock wizardry that lets you set features for a transitive-dependency? Or maybe I should check out local copies of the whole dependency chain, and modify all of their Cargo.toml's to use |
Once we get the core library compiling to WASM, the next step is to write JS glue code to satisfy its IO needs. The core is written as a function that accepts "events" from the outside world, and returns a list of actions. The application-facing API consists of API events like So the JS glue needs to react to these actions with things like We don't have any Any thoughts on what the JS API ought to be? My instinct is to make it look like the python Deferred API, where you create a |
Ok the second problem will be solved by an upcoming release of |
@burdges pointed out that cargo features are additive, so if the glue code declares its own dependency upon |
This helps with #2 (wasm support) because 'rustc-serialize' has some C code, whereas 'hex' is pure-Rust. We were already using `hex`, I just didn't realize that we could use it for everything.
The sodiumoxide issue is resolved now, can anyone knowing and interested in WASM give this a second try? |
@piegamesde I would be happy to take a run at it. I have been thinking especially since I love magic-wormhole. I would be interesting to have a static site up that was a WASM version for my non-cli friends and family. Would you be interested in that being a direct part of the project or should I just test as a library and possibly build it separately? |
@corbinu Great! Yeah, we're all dreaming of this :)
Tough question. I suggest you to try to include Wormhole as a library first, and if this doesn't work, we'll can talk about integrating it into the repository. |
I wanted to poke around this a little since I'd really love to use magic wormhole on the go in a browser (since a browser is just ubiquitous these days). I don't have any problem with writing the JavaScript wrapper/interop, but I don't really have much Rust experience. So I tried to include magic-wormhole as a library, as you suggested @piegamesde. magic-wormhole = { git = "https://github.com/magic-wormhole/magic-wormhole.rs", tag = "0.2.0"} But unfortunately I can't get it to build as wasm. I could get around missing C-libraries by adjusting the build target I have created a minimal example repository based on the Could you please point me in the right direction on how to include/build this for wasm as a library? |
@andipabst thanks for looking into this. First of all, please use some recent commit instead of a release. The last release has been a while ago (sorry about that). As far as I can tell |
@piegamesde Thanks for your support! Now the only thing left is the direct TCP Connection in transit.rs. How should this be approached? Maybe this could be implemented using Websockets and I also thought about using WebRTC for the communication. But I think in the end the transit relay might have to be adapted for this (e.g. to not only handle tcp connections, but also websocket connections). |
Indeed. We are in the process of extending the relay protocol to support WebSockets connections in addition to plain TCP. Some links that may help you with this:
For a start, I suggest that you simply hardcode the default relay server and ignore the rest, as we already know that it supports WebSockets. Once it works, we can talk about implementing the necessary protocol extensions (I'd be willing to help with that). |
@andipabst I see that you have some commits on your fork. Can you please take the changes that prepare the repository for WASM support (dependencies, feature flags, code gates) and PR them? |
Hi @piegamesde, I tried again for some time, but unfortunately I was not able to get it working completely. I have pushed my work in progress state in my fork and in a separate repository for the webapp. Should you decide to have a look at it and get it working, I would be happy to assist with the web part! But my Rust skills are simply not enough to get that part running at the moment. |
I started copying some of your code over, please check out what I've done in the Most notably, I try to go without a I'll go ahead and try to integrate your |
I had a look and it looks really good! Basing it on the target platform is a good idea, since I really had some trouble with the feature flag. I think it should be possible to include a |
Thanks! Actually I'm not that happy about all the
Yes, that's the plan |
I think the changes should now be ready for actually trying them out in a browser. Since I've never done any web development beyond most basic HTML+CSS, I fear I won't be able to help out much there. As far as I understand, one of the next steps would be to provide a JavaScript interop API, because most websites are JS based. Let me know if I can help anything there |
Thanks for your changes! I implemented a basic wasm integration. I had to change some parts, since Then I run into an issue, as you predicted: The handshake via the websocket relay server fails since it opens a new websocket connection to the relay server which then gets a "welcome" response instead of the expected handshake messages. |
Actually, the handshake should work fine, even though I haven't really tested it yet. What you describes sound like you are trying to connect to the rendezvous server as relay. I'm not sure if the public relay server even has a WebSocket endpoint. You can use my test server though, it should be reachable as |
Thanks for the hint, that was indeed the issue 🤦 I could use with your test server for developing on my machine, but for making a hosted version available I needed a secure websocket endpoint (wss://), so I had to dust off my old Hetzner server a bit and run my own relay server (behind a nginx proxy for letsencrypt ssl termination). You can see the result here: https://magic-wormhole-wasm.netlify.app/ So going forward, I will polish the code a bit more and then plan to separate the wasm package and my reference implementation of the web-app using it. |
Oh I forgot to add the TLS reverse proxy to the endpoint, hence no WSS. To be honest though, I'm not sure that using TLS provides any additional security given that the transit protocol is built to work with plain TCP connections too. But I'll see if I can add it anyways, shouldn't hurt either. |
No worries. I totally agree with you, but when serving the website over https (which should definitely be done), browsers also require using secure websocket connections. (See e.g. https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_client_applications#security_considerations) |
Thanks for explaining. |
Thanks! As mentioned above I polished and documented the code a bit and copied it over to include it in this repo, see #163. The next step would be to publish this artifact to the JavaScript package registry npm. See the wasm-pack documentation for more details on the technical process. @piegamesde Can you do please do this, as you are a maintainer and also publishing the rust parts to crates.io? Meanwhile, I would go on writing the JavaScript/TypeScript reference implementation I mentioned above. |
This is pretty cool, many thanks! However, I must admit that I have little knowledge about JavaScript and the npm ecosystem, so I wouldn't feel comfortable with pushing things there at the moment. Would you mind becoming the maintainer for the JavaScript bindings of wormhole-rs? |
EditI found the error and it was very simple. I am stupid. Look at #189 for the fix. Original postHi, I'm a beginner in Rust and even more in Wasm but I tried to make a web application using yew and magic-wormhole.rs and I got this error while trying to upload a file. Logs from the console (
I think I understood that curve25519_dalek wants to generate a random number but it can't because rand_core doesn't support wasm32-unknown-unknown: I also tried andipabst's code without much success. It doesn't produce the same Traceback |
So now that this is settled, I have a new problem. It is likely that this is a misunderstanding of your module on my part and maybe this is not the best place to ask if that is the case. But, here is the problem: when handshaking with the peer, I get in the console My code is available here if you want to try: https://github.com/Sky-NiniKo/magic-wormhole-yew |
Your code looks okay on a first glance. Can you get some better error messages or maybe even a stack trace? |
Full stack trace
Video demoFinal.webm |
Ah well, that's easy: the default relay server deployment does not have WebSocket transport enabled yet, however that's the only thing web clients support. So you need to host your own relay, and configure the client correctly. |
The Winden servers that LeastAuthority.com runs support WebSocket (indeed, https://winden.app is a Web client using WASM etc). The addresses are in the FAQ in the linked app, and are currently Note that a "tcp-only" client can talk to a "websocket-only" client via that relay too. |
It works! Thank you everyone. |
This is a big pony (🐴 ), but it'd be amazing if magic-wormhole.rs worked well under Web Assembly.
My reason for wanting this is that I've long wanted to package magic-wormhole up as a browser extension, but my desire to write JS is nil. I've often joked about wanting to do a magic-wormhole Rust implementation and use WASM, so the fact that you're working on this is amzing!
The text was updated successfully, but these errors were encountered: