Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support wasm32-unknown-unknown architecture #458

Merged
merged 4 commits into from
Feb 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,3 +91,30 @@ jobs:
with:
command: test
args: -p rspotify -p rspotify-http -p rspotify-model -p rspotify-macros --no-default-features --features=${{ matrix.features }}

test-wasm:
name: Test WASM client
runs-on: ubuntu-latest
env:
RUST_BACKTRACE: 1

steps:
- name: Checkout sources
uses: actions/checkout@v2

- name: Install stable toolchain
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
components: clippy

- name: Install node
uses: actions/setup-node@v4

- name: Install wasm-pack
run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh

- name: Run wasm-pack test
run: wasm-pack test --node
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
## 0.13.0 (2024.02.28)

**New features**
- ([#458](https://github.com/ramsayleung/rspotify/pull/458)) Support for the `wasm32-unknown-unknown` build target

## 0.12.1 (Unreleased)
**Bugfixes**
- ([#440](https://github.com/ramsayleung/rspotify/issues/440)) Add Smartwatch device type, fix for json parse error: unknown variant Smartwatch.
Expand Down
17 changes: 15 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ base64 = "0.21.2"
chrono = { version = "0.4.19", features = ["serde"] }
dotenvy = { version = "0.15.0", optional = true }
futures = { version = "0.3.17", optional = true }
getrandom = "0.2.3"

log = "0.4.14"
maybe-async = "0.2.6"
serde = { version = "1.0.130", default-features = false }
Expand All @@ -46,10 +46,23 @@ thiserror = "1.0.29"
url = "2.2.2"
webbrowser = { version = "0.8.0", optional = true }

[target.'cfg(target_arch = "wasm32")'.dependencies]
getrandom = { version = "0.2.3", features = ["js"] }

[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
getrandom = "0.2.3"

[dev-dependencies]
env_logger = { version = "0.11.0", default-features = false }
tokio = { version = "1.11.0", features = ["rt-multi-thread", "macros"] }
futures-util = "0.3.17"
wasm-bindgen-test = "0.3.34"

[target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies]
tokio = { version = "1.11.0", features = ["rt-multi-thread", "macros"] }

[target.'cfg(target_arch = "wasm32")'.dev-dependencies]
tokio = { version = "1.11.0", features = ["rt", "macros"] }
Copy link
Contributor Author

@gelendir gelendir Feb 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strictly speaking tokio isn't needed for running wasm tests, but to removing it would have made the #[cfg... attributes on tests more complex

dotenvy_macro = { version = "0.15.7" }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used to read .env when running wasm tests

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it meanenv:var in the standard library won't work in the wasm32 architecture?

Shouldn't we add a #[cfg...] flag for it, because it's wasm32 specific, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, std::env::var won't work on wasm32-unknown-unknown.Wasm is designed to run in a web browser which is isolated from the host OS.

From my understanding of cargo the #[cfg...] on line 63 creates a block section, so it should already be imported only for wasm. It doesn't seem to get built when running cargo build --tests:

% cargo build --tests
   Compiling wasm-bindgen-backend v0.2.91
   Compiling tokio v1.35.1
   Compiling futures-util v0.3.30
   Compiling serde v1.0.195
   Compiling thiserror v1.0.56
   Compiling strum v0.26.1
   Compiling enum_dispatch v0.3.12
   Compiling async-stream v0.3.5
   Compiling wasm-bindgen-macro-support v0.2.91
   Compiling wasm-bindgen-macro v0.2.91
   Compiling wasm-bindgen v0.2.91
   Compiling futures-executor v0.3.30
   Compiling serde_json v1.0.111
   Compiling serde_urlencoded v0.7.1
   Compiling chrono v0.4.31
   Compiling futures v0.3.30
   Compiling js-sys v0.3.68
   Compiling console_error_panic_hook v0.1.7
   Compiling tokio-util v0.7.10
   Compiling tokio-native-tls v0.3.1
   Compiling tokio-socks v0.5.1
   Compiling h2 v0.3.23
   Compiling rspotify-model v0.12.0 (/Users/gregory/rust/rspotify/rspotify-model)
   Compiling wasm-bindgen-futures v0.4.41
   Compiling wasm-bindgen-test v0.3.41
   Compiling hyper v0.14.28
   Compiling hyper-tls v0.5.0
   Compiling reqwest v0.11.23
   Compiling rspotify-http v0.12.0 (/Users/gregory/rust/rspotify/rspotify-http)
   Compiling rspotify v0.12.0 (/Users/gregory/rust/rspotify)
    Finished dev [unoptimized + debuginfo] target(s) in 6.82s
    ```

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For comparison, here's the output with wasm:

% cargo build --tests --target wasm32-unknown-unknown
   Compiling proc-macro2 v1.0.76
   Compiling unicode-ident v1.0.12
   Compiling wasm-bindgen-shared v0.2.91
   Compiling bumpalo v3.14.0
   Compiling log v0.4.20
   Compiling autocfg v1.1.0
   Compiling wasm-bindgen v0.2.91
   Compiling cfg-if v1.0.0
   Compiling serde v1.0.195
   Compiling version_check v0.9.4
   Compiling pin-project-lite v0.2.13
   Compiling typenum v1.17.0
   Compiling futures-core v0.3.30
   Compiling itoa v1.0.10
   Compiling tinyvec_macros v0.1.1
   Compiling futures-sink v0.3.30
   Compiling tinyvec v1.6.0
   Compiling syn v1.0.109
   Compiling futures-channel v0.3.30
   Compiling percent-encoding v2.3.1
   Compiling futures-task v0.3.30
   Compiling rustversion v1.0.14
   Compiling pin-utils v0.1.0
   Compiling memchr v2.7.1
   Compiling serde_json v1.0.111
   Compiling generic-array v0.14.7
   Compiling ryu v1.0.16
   Compiling slab v0.4.9
   Compiling futures-io v0.3.30
   Compiling num-traits v0.2.17
   Compiling form_urlencoded v1.2.1
   Compiling unicode-bidi v0.3.14
   Compiling thiserror v1.0.56
   Compiling fnv v1.0.7
   Compiling bytes v1.5.0
   Compiling quote v1.0.35
   Compiling unicode-normalization v0.1.22
   Compiling async-trait v0.1.77
   Compiling heck v0.4.1
   Compiling syn v2.0.48
   Compiling http v0.2.11
   Compiling tower-service v0.3.2
   Compiling idna v0.5.0
   Compiling base64 v0.21.7
   Compiling env_filter v0.1.0
   Compiling dotenvy v0.15.7
   Compiling rspotify-macros v0.12.0 (/Users/gregory/rust/rspotify/rspotify-macros)
   Compiling scoped-tls v1.0.1
   Compiling env_logger v0.11.1
   Compiling url v2.5.0
   Compiling block-buffer v0.10.4
   Compiling crypto-common v0.1.6
   Compiling digest v0.10.7
   Compiling sha2 v0.10.8
   Compiling wasm-bindgen-backend v0.2.91
   Compiling maybe-async v0.2.7
   Compiling dotenvy_macro v0.15.7
   Compiling wasm-bindgen-macro-support v0.2.91
   Compiling serde_derive v1.0.195
   Compiling futures-macro v0.3.30
   Compiling thiserror-impl v1.0.56
   Compiling strum_macros v0.26.1
   Compiling async-stream-impl v0.3.5
   Compiling enum_dispatch v0.3.12
   Compiling wasm-bindgen-test-macro v0.3.41
   Compiling tokio-macros v2.2.0
   Compiling async-stream v0.3.5
   Compiling wasm-bindgen-macro v0.2.91
   Compiling futures-util v0.3.30
   Compiling tokio v1.35.1
   Compiling strum v0.26.1
   Compiling js-sys v0.3.68
   Compiling console_error_panic_hook v0.1.7
   Compiling futures-executor v0.3.30
   Compiling futures v0.3.30
   Compiling wasm-bindgen-futures v0.4.41
   Compiling web-sys v0.3.67
   Compiling getrandom v0.2.12
   Compiling serde_urlencoded v0.7.1
   Compiling chrono v0.4.31
   Compiling wasm-bindgen-test v0.3.41
   Compiling rspotify-model v0.12.0 (/Users/gregory/rust/rspotify/rspotify-model)
   Compiling reqwest v0.11.23
   Compiling rspotify-http v0.12.0 (/Users/gregory/rust/rspotify/rspotify-http)
   Compiling rspotify v0.12.0 (/Users/gregory/rust/rspotify)
    Finished dev [unoptimized + debuginfo] target(s) in 9.77s


[features]
default = ["client-reqwest", "reqwest-default-tls"]
Expand Down
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,16 @@ $ cargo build --all-features

Because in order to switch between clients, the different clients have to implement the same base trait in [src/http/mod.rs](https://github.com/ramsayleung/rspotify/blob/master/src/http/mod.rs), so if you build with all features, you'll get `duplicate definitions` error. As every coin has two sides, you can only have one side at a time, not all sides of it.

## WASM support

RSpotify supports building for the `wasm32-unknown-unknown` target. It should be as easy as:

```sh
$ cargo build --target wasm32-unknown-unknown
```

Refer to the [documentation](https://docs.rs/rspotify/latest/rspotify/#webassembly) for more details

## License

[MIT](./LICENSE)
3 changes: 2 additions & 1 deletion rspotify-http/src/common.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ pub type Form<'a> = HashMap<&'a str, &'a str>;
/// different ways (`Value::Null`, an empty `Value::Object`...), so this removes
/// redundancy and edge cases (a `Some(Value::Null), for example, doesn't make
/// much sense).
#[maybe_async]
#[cfg_attr(target_arch = "wasm32", maybe_async(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), maybe_async)]
pub trait BaseHttpClient: Send + Default + Clone + fmt::Debug {
type Error;

Expand Down
17 changes: 16 additions & 1 deletion rspotify-http/src/reqwest.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
use super::{BaseHttpClient, Form, Headers, Query};

use std::convert::TryInto;

#[cfg(not(target_arch = "wasm32"))]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added to remove a warning on unused imports

use std::time::Duration;

use maybe_async::async_impl;
Expand Down Expand Up @@ -56,6 +58,7 @@ pub struct ReqwestClient {
client: reqwest::Client,
}

#[cfg(not(target_arch = "wasm32"))]
impl Default for ReqwestClient {
fn default() -> Self {
let client = reqwest::ClientBuilder::new()
Expand All @@ -67,6 +70,17 @@ impl Default for ReqwestClient {
}
}

#[cfg(target_arch = "wasm32")]
impl Default for ReqwestClient {
fn default() -> Self {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reqwest doesn't implement the timeout function when built for wasm, hence the need for a separate implementation.

let client = reqwest::ClientBuilder::new()
.build()
// building with these options cannot fail
.unwrap();
Self { client }
}
}

impl ReqwestClient {
async fn request<D>(
&self,
Expand Down Expand Up @@ -109,7 +123,8 @@ impl ReqwestClient {
}
}

#[async_impl]
#[cfg_attr(target_arch = "wasm32", async_impl(?Send))]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering is there any way to encapsulate these cfg_attr flags, because they are duplicated, and we might need to do the same thing for other architecture in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering the same thing. I found in the rust docs that the #![cfg_attr] form applies to an entire file, but that doesn't help much here. Another idea would be to abstract this within a custom macro.

I've never written custom macros yet but could experiment if you think it's a good idea.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Custom macros is not an ideal idea for me, it will also increase complexity and mental effort to maintain the codebase.

#[cfg_attr(not(target_arch = "wasm32"), async_impl)]
impl BaseHttpClient for ReqwestClient {
type Error = ReqwestError;

Expand Down
6 changes: 4 additions & 2 deletions src/auth_code.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,8 @@ pub struct AuthCodeSpotify {
}

/// This client has access to the base methods.
#[maybe_async]
#[cfg_attr(target_arch = "wasm32", maybe_async(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), maybe_async)]
impl BaseClient for AuthCodeSpotify {
fn get_http(&self) -> &HttpClient {
&self.http
Expand Down Expand Up @@ -126,7 +127,8 @@ impl BaseClient for AuthCodeSpotify {

/// This client includes user authorization, so it has access to the user
/// private endpoints in [`OAuthClient`].
#[maybe_async]
#[cfg_attr(target_arch = "wasm32", maybe_async(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), maybe_async)]
impl OAuthClient for AuthCodeSpotify {
fn get_oauth(&self) -> &OAuth {
&self.oauth
Expand Down
6 changes: 4 additions & 2 deletions src/auth_code_pkce.rs
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,8 @@ pub struct AuthCodePkceSpotify {
}

/// This client has access to the base methods.
#[maybe_async]
#[cfg_attr(target_arch = "wasm32", maybe_async(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), maybe_async)]
impl BaseClient for AuthCodePkceSpotify {
fn get_http(&self) -> &HttpClient {
&self.http
Expand Down Expand Up @@ -88,7 +89,8 @@ impl BaseClient for AuthCodePkceSpotify {

/// This client includes user authorization, so it has access to the user
/// private endpoints in [`OAuthClient`].
#[maybe_async]
#[cfg_attr(target_arch = "wasm32", maybe_async(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), maybe_async)]
impl OAuthClient for AuthCodePkceSpotify {
fn get_oauth(&self) -> &OAuth {
&self.oauth
Expand Down
3 changes: 2 additions & 1 deletion src/client_creds.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ pub struct ClientCredsSpotify {
}

/// This client has access to the base methods.
#[maybe_async]
#[cfg_attr(target_arch = "wasm32", maybe_async(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), maybe_async)]
impl BaseClient for ClientCredsSpotify {
fn get_http(&self) -> &HttpClient {
&self.http
Expand Down
3 changes: 2 additions & 1 deletion src/clients/base.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ use serde_json::Value;
/// This trait implements the basic endpoints from the Spotify API that may be
/// accessed without user authorization, including parts of the authentication
/// flow that are shared, and the endpoints.
#[maybe_async]
#[cfg_attr(target_arch = "wasm32", maybe_async(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), maybe_async)]
pub trait BaseClient
where
Self: Send + Sync + Default + Clone + fmt::Debug,
Expand Down
3 changes: 2 additions & 1 deletion src/clients/oauth.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ use url::Url;
/// [`user_playlist`](crate::clients::BaseClient::user_playlist). This trait
/// only separates endpoints that *always* need authorization from the base
/// ones.
#[maybe_async]
#[cfg_attr(target_arch = "wasm32", maybe_async(?Send))]
#[cfg_attr(not(target_arch = "wasm32"), maybe_async)]
pub trait OAuthClient: BaseClient {
fn get_oauth(&self) -> &OAuth;

Expand Down
12 changes: 10 additions & 2 deletions src/clients/pagination/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,18 @@

#[cfg(feature = "__sync")]
mod iter;
#[cfg(feature = "__async")]

#[cfg(all(feature = "__async", not(target_arch = "wasm32")))]
mod stream;

#[cfg(all(feature = "__async", target_arch = "wasm32"))]
mod wasm_stream;

#[cfg(feature = "__sync")]
pub use iter::{paginate, paginate_with_ctx, Paginator};
#[cfg(feature = "__async")]

#[cfg(all(feature = "__async", not(target_arch = "wasm32")))]
pub use stream::{paginate, paginate_with_ctx, Paginator};

#[cfg(all(feature = "__async", target_arch = "wasm32"))]
pub use wasm_stream::{paginate, paginate_with_ctx, Paginator};
62 changes: 62 additions & 0 deletions src/clients/pagination/wasm_stream.rs
Copy link
Contributor Author

@gelendir gelendir Feb 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This stumped me the most. The async runtime in wasm doesn't support Send so it had to be removed from all trait bounds in stream.rs.

At first I tried to extract the trait bounds into "aliases" with the goal of annotating them with #[cfg...] but didn't find a way of making it work. There's trait_alias that might help, but it's available only on nightly.

Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
//! Asynchronous implementation of automatic pagination requests.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need a separate stream module for wasm32 architecture, is there anything the existing stream module is unable to satisfy?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like my previous comment dissapeared, I explained it here: #458 (comment)

Basically I didn't find a way of encapsulating the Send trait bounds on the function signature behind a #[cfg...]


use crate::{model::Page, ClientResult};

use std::pin::Pin;

use futures::{future::Future, stream::Stream};

/// Alias for `futures::stream::Stream<Item = T>`, since async mode is enabled.
pub type Paginator<'a, T> = Pin<Box<dyn Stream<Item = T> + 'a>>;

pub type RequestFuture<'a, T> = Pin<Box<dyn 'a + Future<Output = ClientResult<Page<T>>>>>;

/// This is used to handle paginated requests automatically.
pub fn paginate_with_ctx<'a, Ctx: 'a, T, Request>(
ctx: Ctx,
req: Request,
page_size: u32,
) -> Paginator<'a, ClientResult<T>>
where
T: 'a + Unpin,
Request: 'a + for<'ctx> Fn(&'ctx Ctx, u32, u32) -> RequestFuture<'ctx, T>,
{
use async_stream::stream;
let mut offset = 0;
Box::pin(stream! {
loop {
let request = req(&ctx, page_size, offset);
let page = request.await?;
offset += page.items.len() as u32;
for item in page.items {
yield Ok(item);
}
if page.next.is_none() {
break;
}
}
})
}

pub fn paginate<'a, T, Fut, Request>(req: Request, page_size: u32) -> Paginator<'a, ClientResult<T>>
where
T: 'a + Unpin,
Fut: Future<Output = ClientResult<Page<T>>>,
Request: 'a + Fn(u32, u32) -> Fut,
{
use async_stream::stream;
let mut offset = 0;
Box::pin(stream! {
loop {
let request = req(page_size, offset);
let page = request.await?;
offset += page.items.len() as u32;
for item in page.items {
yield Ok(item);
}
if page.next.is_none() {
break;
}
}
})
}
20 changes: 19 additions & 1 deletion src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,21 @@
//! the [`.env` file](https://github.com/ramsayleung/rspotify/blob/master/.env)
//! for more details.
//!
//! ### WebAssembly
//!
//! RSpotify supports the `wasm32-unknown-unknown` target in combination
//! with the `client-reqwest` feature. HTTP requests must be processed async.
//! Other HTTP client configurations are not supported.
//!
//! [Spotify recommends][spotify-auth-flows] using [`AuthCodePkceSpotify`] for
//! authorization flows on the web.
//!
//! Importing the Client ID via `RSPOTIFY_CLIENT_ID` is not possible since WASM
//! web runtimes are isolated from the host environment. The client ID must be
//! passed explicitly to [`Credentials::new_pkce`]. Alternatively, it can be
//! embedded at compile time with the [`std::env!`] or
//! [`dotenv!`](https://crates.io/crates/dotenvy) macros.
//!
//! ### Examples
//!
//! There are some [available examples on the GitHub
Expand Down Expand Up @@ -442,11 +457,13 @@ impl OAuth {
}

#[cfg(test)]
mod test {
pub mod test {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pub is needed so that wasm-pack can find the tests.

use crate::{alphabets, generate_random_string, Credentials};
use std::collections::HashSet;
use wasm_bindgen_test::*;

#[test]
#[wasm_bindgen_test]
fn test_generate_random_string() {
let mut containers = HashSet::new();
for _ in 1..101 {
Expand All @@ -456,6 +473,7 @@ mod test {
}

#[test]
#[wasm_bindgen_test]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A naive question, what does wasm_bindgen_test mean? Do we need this macro?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regualar #[test] isn't compatible with wasm. Excerpt from the wasm-bindgen-test README:

The normal #[test] cannot be used and will not work. Eventually it's intended that the #[wasm_bindgen_test] attribute could gain arguments like "run in a browser" or something like a minimum Node version.

I believe the macro exports the tests into a separate WASM module to make it easier to run in a browser.

fn test_basic_auth() {
let creds = Credentials::new_pkce("ramsay");
let headers = creds.auth_headers();
Expand Down
Loading
Loading