Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable cfg_fs for wasi #6822

Merged
merged 4 commits into from
Sep 11, 2024
Merged

Enable cfg_fs for wasi #6822

merged 4 commits into from
Sep 11, 2024

Conversation

namse
Copy link
Contributor

@namse namse commented Sep 4, 2024

Motivation

We are currently using wasip1-threads and have implemented our own wasi fs. However, there is a restriction that prevents using the file system feature when cfg_fs is set to wasi. I believe this restriction should be lifted, as it contradicts the intention stated in the Tokio code, where enabling the tokio_unstable flag allows the use of the file system feature (relevant code link).

Solution

I have removed the wasi condition from cfg_fs to enable the file system feature. After forking Tokio and applying this change, I tested it and found that it works perfectly. This change will allow support for the file system feature in the wasip1-threads environment.

@namse
Copy link
Contributor Author

namse commented Sep 7, 2024

Is there a maintainer who can review this PR? Our team primarily uses WASI, but this issue is a major obstacle in using Tokio with WASI. We would greatly appreciate any help.

@Darksonn Darksonn added A-tokio Area: The main tokio crate M-fs Module: tokio/fs T-wasm Topic: Web Assembly labels Sep 10, 2024
@Darksonn
Copy link
Contributor

I believe that this was added because std::fs relies on spawn_blocking but wasm is single-threaded. I understand that the threading situation has improved recently. Are you able to confirm that this actually works?

@namse
Copy link
Contributor Author

namse commented Sep 10, 2024

Yes, this works well. I have already forked Tokio and am using Tokio FS in a web browser with the fixes from this PR. However, I decided to create this PR because using the fork continuously could cause compatibility issues with other crates.

Not all WASM, or more specifically, WASI, is single-threaded. Among Rust targets, there are wasm32-wasip1, wasm32-wasip1-thread, and wasm32-wasip2. wasm32-wasip1 does not support threads, but wasm32-wasip1-thread supports pthread-based threading. Honestly, I’m not too familiar with wasip2.

WASI support in Tokio is already experimental. Also, since WASI itself is quite experimental, I believe most WASI users would typically check whether the WASI runtime they are using supports threads or not. Therefore, I don't think there would be a significant issue in opening up the configuration to allow using Tokio FS in WASI environments that do support threads.

@Darksonn
Copy link
Contributor

Ok, I don't mind adding this. There's a compilation failure, though.

/// Returns a mutable reference to the underlying `std::fs::OpenOptions`
pub(super) fn as_inner_mut(&mut self) -> &mut StdOpenOptions {
&mut self.0
}

We can fix it like this:

/// Returns a mutable reference to the underlying `std::fs::OpenOptions`
#[cfg(any(windows, unix))]
pub(super) fn as_inner_mut(&mut self) -> &mut StdOpenOptions {
    &mut self.0
}

@namse
Copy link
Contributor Author

namse commented Sep 10, 2024

If the compile error is due to the function not being used in WASI, would it be okay to include a fix for that in this PR as well?

It might be a good idea for me to modify the PR to ensure there are no compile errors overall and then request a review from you.

@Darksonn
Copy link
Contributor

Go ahead.

@namse
Copy link
Contributor Author

namse commented Sep 11, 2024

I thought I would need more changes, but it's not. There's nothing more except for the modifications you mentioned. Please check.

Copy link
Contributor

@Darksonn Darksonn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you.

@Darksonn Darksonn enabled auto-merge (squash) September 11, 2024 06:52
@namse
Copy link
Contributor Author

namse commented Sep 11, 2024

@Darksonn
Copy link
Contributor

You don't need to do anything. I will rerun.

@Darksonn Darksonn merged commit d621359 into tokio-rs:master Sep 11, 2024
81 checks passed
@namse
Copy link
Contributor Author

namse commented Sep 11, 2024

@Darksonn If it's not too much trouble, may I ask when this change is expected to be released?

@namse namse deleted the patch-1 branch September 11, 2024 13:00
@Darksonn
Copy link
Contributor

We don't have very many changes since the last release, so it may be another month. But if you are willing to do the work of preparing the release, we can make one now.

@namse
Copy link
Contributor Author

namse commented Sep 11, 2024

After reviewing this document (https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md#releasing), I believe I can handle it. I have some other tasks to finish first, but once I'm done, I'll create a PR for the release.

@Darksonn
Copy link
Contributor

The part about path dependencies isn't needed anymore, so you can skip that step.

kodiakhq bot pushed a commit to pdylanross/fatigue that referenced this pull request Oct 23, 2024
Bumps tokio from 1.40.0 to 1.41.0.

Release notes
Sourced from tokio's releases.

Tokio v1.41.0
1.41.0 (Oct 22th, 2024)
Added

metrics: stabilize global_queue_depth (#6854, #6918)
net: add conversions for unix SocketAddr (#6868)
sync: add watch::Sender::sender_count (#6836)
sync: add mpsc::Receiver::blocking_recv_many (#6867)
task: stabilize Id apis (#6793, #6891)

Added (unstable)

metrics: add H2 Histogram option to improve histogram granularity (#6897)
metrics: rename some histogram apis (#6924)
runtime: add LocalRuntime (#6808)

Changed

runtime: box futures larger than 16k on release mode (#6826)
sync: add #[must_use] to Notified (#6828)
sync: make watch cooperative (#6846)
sync: make broadcast::Receiver cooperative (#6870)
task: add task size to tracing instrumentation (#6881)
wasm: enable cfg_fs for wasi target (#6822)

Fixed

net: fix regression of abstract socket path in unix socket (#6838)

Documented

io: recommend OwnedFd with AsyncFd (#6821)
io: document cancel safety of AsyncFd methods (#6890)
macros: render more comprehensible documentation for join and try_join (#6814, #6841)
net: fix swapped examples for TcpSocket::set_nodelay and TcpSocket::nodelay (#6840)
sync: document runtime compatibility (#6833)

#6793: tokio-rs/tokio#6793
#6808: tokio-rs/tokio#6808
#6810: tokio-rs/tokio#6810
#6814: tokio-rs/tokio#6814
#6821: tokio-rs/tokio#6821
#6822: tokio-rs/tokio#6822
#6826: tokio-rs/tokio#6826
#6828: tokio-rs/tokio#6828
#6833: tokio-rs/tokio#6833
#6836: tokio-rs/tokio#6836
#6838: tokio-rs/tokio#6838
#6840: tokio-rs/tokio#6840


... (truncated)


Commits

01e04da chore: prepare Tokio v1.41.0 (#6917)
92ccade runtime: fix stability feature flags for docs (#6909)
fbfeb9a metrics: rename *_poll_count_* to *_poll_time_* (#6924)
da745ff metrics: add H2 Histogram option to improve histogram granularity (#6897)
ce1c74f metrics: fix deadlock in injection_queue_depth_multi_thread test (#6916)
28c9a14 metrics: rename injection_queue_depth to global_queue_depth (#6918)
32e0b43 ci: freeze FreeBSD and wasm-unknown-unknown on rustc 1.81 (#6911)
1656d8e sync: add mpsc::Receiver::blocking_recv_many (#6867)
c9e998e ci: print the correct sort order of the dictionary on failure (#6905)
512e9de rt: add LocalRuntime (#6808)
Additional commits viewable in compare view




Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tokio Area: The main tokio crate M-fs Module: tokio/fs T-wasm Topic: Web Assembly
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants