Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support wasm (in some form) #268

Open
Thomasdezeeuw opened this issue Sep 20, 2021 · 24 comments · May be fixed by #532
Open

Support wasm (in some form) #268

Thomasdezeeuw opened this issue Sep 20, 2021 · 24 comments · May be fixed by #532

Comments

@Thomasdezeeuw
Copy link
Collaborator

We can support wasm via wasi. Last I looked (for tokio-rs/mio#1395, back in November 2020) it doesn't support creating sockets, but it does support some things like read/write.

Alternatively we could start with an implementation that just panics, just so socket2 can build on wasm.

@ChenKS12138
Copy link

ChenKS12138 commented Feb 14, 2022

We can support wasm via wasi. Last I looked (for tokio-rs/mio#1395, back in November 2020) it doesn't support creating sockets, but it does support some things like read/write.

Alternatively we could start with an implementation that just panics, just so socket2 can build on wasm.

I'm interested in using socket2 in wasm too. I found that symbols like AF_INET, sa_family_t, sockaddr_in are missing in wasi, so it doesn't have the concept about the address of socket, and we could barely define SockAddr for wasm. wasi supports socket_accept, socket_recv, socket_send, socket_shutdown by far. accept in socket2 needs to return a SockAddr as the address of peer socket. So I think it seems hard to start with an implementation that just panics.

@sascha1337
Copy link

the day, wasi will have them features, will be gold for rust and thus the planet ... but wen X_X

@notgull
Copy link
Contributor

notgull commented Apr 23, 2022

Isn’t wasi an unstable API at the moment? It might not be the best idea to build on top of it.

@Thomasdezeeuw
Copy link
Collaborator Author

The API is in preview I believe, but considering it's implemented by browser backwards compatibility is everything. So I think it's find to add experimental support for wasm/wasi, this is what we did for Mio.

@thaodt
Copy link

thaodt commented Apr 29, 2022

The API is in preview I believe, but considering it's implemented by browser backwards compatibility is everything. So I think it's find to add experimental support for wasm/wasi, this is what we did for Mio.

hi @Thomasdezeeuw do we have any ways to work around now? My lib depends on tokio and tokio depends on socket2. As a result, it built failed under target wasm32-unknown-unknown (wasm-pack build --target web)

@sascha1337
Copy link

sascha1337 commented Apr 29, 2022

Just do some draft PoC please, we need em sockets and tokio

@Thomasdezeeuw
Copy link
Collaborator Author

hi @Thomasdezeeuw do we have any ways to work around now? My lib depends on tokio and tokio depends on socket2. As a result, it built failed under target wasm32-unknown-unknown (wasm-pack build --target web)

You might be able to disable Tokio's network feature, not sure though.

Just do some draft PoC please, we need em sockets and tokio

Prs are welcome.

@Darksonn
Copy link
Collaborator

Tokio only depends on socket2 if you enable the net feature, and even if you make socket2 compile under wasm somehow, I'm pretty sure that there would also be other problems with getting Tokio's net feature to work.

@miketwenty1
Copy link

miketwenty1 commented Jun 25, 2022

can verify trying to use either crate:
awc = "3.0.0"
actix-rt = "2.7.0"
Will result in a bunch of socket2 errors when trying to compile to wasm.
such as:

/..../.cargo/registry/src/github.com-1ecc6299db9ec823/socket2-0.4.4/src/socket.rs:1781:15
     |
1781 | from!(Socket, net::UdpSocket);
     |               ^^^^^^^^^^^^^^ expected struct `UdpSocket`, found `()`

Is it assumed until this issue is resolved that we won't be able to use socket2 with wasm or any libs/crates that rely on socket2?

@Lunarequest
Copy link

this should be easier to solve now that wasix exists, I'm not to sure how sockets work so I haven't taken more then a cursory build which resulting in more errors(yes you are reading that right, it seems a lot of the wasix sockets stuff expects more/less args)

@Thomasdezeeuw
Copy link
Collaborator Author

I don't know about wasix, it's doesn't seem like a standard anyone but wasmer is following. I don't really fancy having different code for different wasm runtimes.

@Lunarequest
Copy link

Wasix seems more like a pollyfill. It produces a wasm binary that should work any where

@Thomasdezeeuw
Copy link
Collaborator Author

I highly doubt that. I can't imagine how you would create a socket if the underlying wasm runtime doesn't provide a function/"system call" to create sockets.

@Lunarequest
Copy link

They are able to do this via a npm package and have also forked this repo to allow it to be built to wasm. Having it upstreamed would be very cool

@Thomasdezeeuw
Copy link
Collaborator Author

Cool, a pr with those changes would be welcome. Ideally we can setup some CI as well, running cargo check is good enough, but of course cargo test would be preferred.

@theduke
Copy link

theduke commented Jun 7, 2023

We'd love to have wasix support upstreamed.

It might require some work, but testing with cargo wasix test is doable.
(https://github.com/wasix-org/cargo-wasix)

@LetsMelon
Copy link

Copied from #78:


I believe WASI can be supported since their lib-c exposes sockets. I'd be interested in adding it.

I would be interested in giving it a try. However, after looking a bit through the codebase, I feel that it might be somewhat complicated and not that easy at the moment.

Sounds good @jkelleyrtp. Although last I check (I think version 9/10) WASI didn't allow for the creation of sockets, however that could have changed. In either case the various socket options could still be useful.

I think that's still true, but there is an open and active proposal to add the functionality to create sockets (https://github.com/WebAssembly/wasi-sockets/blob/f2ffc745086ec7ab168f1244f2b3ef0342bf9ddc/wit/tcp-create-socket.wit#L26 and https://github.com/WebAssembly/wasi-sockets/blob/f2ffc745086ec7ab168f1244f2b3ef0342bf9ddc/wit/udp-create-socket.wit#L26) which is currently in 'Phase 3 - Implementation Phase (CG + WG)'. The proposal has been already introduced in WASI Preview 2 which the newly announced target wasm32-wasip2 follows. In the introduction blog post under Conclusion it's encouraged to use cargo-component until the newly created target achieves tier-2. In cargo-component I found a reply from @sunfishcode to an issue that 'work on supporting that (Preview 2, sockets) in libc is underway'.

At the time of writing, WASIX has support via sock_open() to create sockets.


I'm just a random user who stumbled upon this problem: socket2 cannot be built with the target wasm32-wasi. I'm not a wasm/wasi wizard, so please correct me if I have made an error or overlooked something.

@Thomasdezeeuw
Copy link
Collaborator Author

I'm ok with supporting WASI preview 2, but rustc isn't quite ready yet I think, see https://blog.rust-lang.org/2024/04/09/updates-to-rusts-wasi-targets.html (which you also linked).

I think WASIX is a no-go as that's vendor specific. Maybe as an alternative target, but I'm not how that would work at the moment.

Overall I'm also ok with reducing the socket2 to something that is supported on wasi preview 1 (and only wasi) so that it can at least build.

@LetsMelon
Copy link

LetsMelon commented Apr 16, 2024

I'm ok with supporting WASI preview 2, but rustc isn't quite ready yet I think, see https://blog.rust-lang.org/2024/04/09/updates-to-rusts-wasi-targets.html (which you also linked).

Yeah, I think it's a bit too early for WASI preview 2. However, I believe that when the target is released, it can be implemented into socket2.

I think WASIX is a no-go as that's vendor specific. Maybe as an alternative target, but I'm not how that would work at the moment.

I'm also not quite sure how a WASIX implementation should look, and to my knowledge, WASIX is only meant to be a polyfill until the official standard is fully developed.

Overall I'm also ok with reducing the socket2 to something that is supported on wasi preview 1 (and only wasi) so that it can at least build.

I think it would also help when implementing wasm32-wasip2, if the imported functions work the same as in WASI preview 2.


I would be really interested in giving this a try, but because of university, I don't have much free time. Do you know if there is some kind of tracking issue for wasm32-wasip2 and the implementation of WASI preview 2 in rustc?

Edit:

Or some documentation on how to introduce a new target into socket2?

@Thomasdezeeuw
Copy link
Collaborator Author

I think it would also help when implementing wasm32-wasip2, if the imported functions work the same as in WASI preview 2.

I would be really interested in giving this a try, but because of university, I don't have much free time. Do you know if there is some kind of tracking issue for wasm32-wasip2 and the implementation of WASI preview 2 in rustc?

After a quick search I can't find any tracking issues, but let's focus on preview 1 initially.

Or some documentation on how to introduce a new target into socket2?

There aren't a lot of of docs, but I can give a quick overview.

The first thing would be to start running cargo check --target wasm32-wasi (or wasm32-wasip1 in the future I suppose). For each failure add the appropriate cfg attributes, e.g.

#[cfg(all(feature = "all", any(target_os = "freebsd", target_os = "linux")))]
There will be a lot of these. But like I said I'm totally fine with just adding not(wasm) to most of the API to begin with.

Once cargo check works, try adding --features all, that enables even more API.

Once it builds/checks, ideally we find a way to run the tests. Since wasm can't create their own sockets (yet), this will be painful as most if not all of the tests do this. Feel free to skip this entirely.

Final step would be to add some kind of CI, again tests would be ideal, but running cargo check would be sufficient. That is (or should be) as easy as adding the wasm target to

target: ["aarch64-apple-ios", "aarch64-linux-android", "x86_64-apple-darwin", "x86_64-unknown-fuchsia", "x86_64-pc-windows-msvc", "x86_64-pc-solaris", "x86_64-unknown-freebsd", "x86_64-unknown-illumos", "x86_64-unknown-linux-gnu", "x86_64-unknown-linux-musl", "x86_64-unknown-netbsd", "x86_64-unknown-redox", "armv7-linux-androideabi", "i686-linux-android"]
,
target: ["aarch64-apple-ios", "aarch64-linux-android", "x86_64-apple-darwin", "x86_64-unknown-fuchsia", "x86_64-pc-windows-msvc", "x86_64-pc-solaris", "x86_64-unknown-freebsd", "x86_64-unknown-illumos", "x86_64-unknown-linux-gnu", "x86_64-unknown-linux-musl", "x86_64-unknown-netbsd", "x86_64-unknown-redox", "armv7-linux-androideabi", "i686-linux-android"]
and
TARGETS ?= "aarch64-apple-ios" "aarch64-linux-android" "x86_64-apple-darwin" "x86_64-unknown-fuchsia" "x86_64-pc-windows-msvc" "x86_64-pc-solaris" "x86_64-unknown-freebsd" "x86_64-unknown-illumos" "x86_64-unknown-linux-gnu" "x86_64-unknown-linux-musl" "x86_64-unknown-netbsd" "x86_64-unknown-redox" "armv7-linux-androideabi" "i686-linux-android"

@Thomasdezeeuw
Copy link
Collaborator Author

@LetsMelon if you have any questions feel free to open a draft pr and ping me. Or you can leave them here.

@LetsMelon
Copy link

@Thomasdezeeuw thanks for your quick reply and your help!

I will try to look into the codebase over the weekend. My current plan is to fix all cfg's and copy all functions from src/sys/unix.rs into something like src/sys/wasm.rs, replacing their bodies with todo!. Maybe that will be enough for socket2 to compile (but not work) with wasm.

@mirsella
Copy link

any update ?

@Thomasdezeeuw
Copy link
Collaborator Author

No update, nobody is working on this.

@nickrum nickrum linked a pull request Oct 4, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.