New system for remote / custom / external types

CHANGELOG.md

@@ -29,6 +29,16 @@
  No consumers of any languages are impacted, only the maintainers of these language bindings.
  ([#2066](https://github.com/mozilla/uniffi-rs/issues/2066)), ([#2094](https://github.com/mozilla/uniffi-rs/pull/2094))
 
+### ⚠️ Breaking Changes ⚠️
+
+- Handling of remote/custom/external types has changed significantly:
+ - UDL users need to add the `[Remote]` attribute for remote types
+ - The `UniffiCustomTypeConverter` trait is no longer used. Instead, use the `custom_type!` macro.
+ - The `use_udl_*` macros are no longer needed and have been removed.
+ - To share remote type implementations between crates, use the `use_remote_type` macro.
+ - The UDL syntax for external types in the UDL has been changed.
+ - See https://mozilla.github.io/uniffi-rs/udl/remote_custom_ext_types.html for details
+
 - The async runtime can be specified for constructors/methods, this will override the runtime specified at the impl block level.
 
 - Removed the dependency on the `oneshot' crate (https://github.com/mozilla/uniffi-rs/issues/1736)

Cargo.toml

@@ -18,6 +18,7 @@ members = [
  "examples/custom-types",
  "examples/futures",
  "examples/geometry",
+ "examples/remote-types",
  "examples/rondpoint",
  "examples/sprites",
  "examples/todolist",
@@ -52,7 +53,6 @@ members = [
  "fixtures/regressions/swift-callbacks-omit-labels",
  "fixtures/regressions/swift-dictionary-nesting",
  "fixtures/regressions/unary-result-alias",
- "fixtures/regressions/nested-module-import",
  "fixtures/regressions/wrong-lower-check",
  "fixtures/trait-methods",
  "fixtures/uitests",

docs/manual/src/internals/lifting_and_lowering.md

@@ -108,9 +108,10 @@ To work around this we do several things:
  - We generate a unit struct named `UniFfiTag` in the root of each UniFFIed crate.
  - Each crate uses the `FfiConverter<crate::UniFfiTag>` trait to lower/lift/serialize values for its scaffolding functions.
 
-This allows us to work around the orphan rules when defining `FfiConverter` implementations.
- - UniFFI consumer crates can implement lifting/lowering/serializing types for their own scaffolding functions, for example `impl FfiConverter<crate::UniFfiTag> for serde_json::Value`. This is allowed since `UniFfiTag` is a local type.
+This allows us to work around the orphan rules when defining ffi trait implementations.
  - The `uniffi` crate can implement lifting/lowering/serializing types for all scaffolding functions using a generic impl, for example `impl<UT> FfiConverter<UT> for u8`. "UT" is short for "UniFFI Tag"
- - We don't currently use this, but crates can also implement lifting/lowering/serializing their local types for all scaffolding functions using a similar generic impl (`impl<UT> FfiConverter<UT> for MyLocalType`).
+ - UniFFI consumer crates usually implement lifting/lowering/serializing types the same way.
+ - However, for remote types, they must only implement ffi traits for their local tag, for example `impl FfiConverter<crate::UniFfiTag> for serde_json::Value`. This is valid since `UniFfiTag` is a local type.
+ - If other crates also want to use the same remote type implementation, the need to implement the ffi traits for their local tag as well. This is what the `use_remote_type!` macro does.
 
 For more details on the specifics of the "orphan rule" and why these are legal implementations, see the [Rust Chalk Book](https://rust-lang.github.io/chalk/book/clauses/coherence.html#the-orphan-rules-in-rustc)

docs/manual/src/kotlin/configuration.md

@@ -10,7 +10,7 @@ The generated Kotlin modules can be configured using a `uniffi.toml` configurati
 | `cdylib_name` | `uniffi_{namespace}`[^1] | The name of the compiled Rust library containing the FFI implementation (not needed when using `generate --library`). |
 | `generate_immutable_records` | `false` | Whether to generate records with immutable fields (`val` instead of `var`). |
 | `custom_types` | | A map which controls how custom types are exposed to Kotlin. See the [custom types section of the manual](../udl/custom_types.md#custom-types-in-the-bindings-code)|
-| `external_packages` | | A map of packages to be used for the specified external crates. The key is the Rust crate name, the value is the Kotlin package which will be used referring to types in that crate. See the [external types section of the manual](../udl/ext_types_external.md#kotlin)
+| `external_packages` | | A map of packages to be used for the specified external crates. The key is the Rust crate name, the value is the Kotlin package which will be used referring to types in that crate. See the [external types section of the manual](../udl/remote_ext_types.md#kotlin)
 | `android` | `false` | Used to toggle on Android specific optimizations
 | `android_cleaner` | `android` | Use the [`android.system.SystemCleaner`](https://developer.android.com/reference/android/system/SystemCleaner) instead of [`java.lang.ref.Cleaner`](https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ref/Cleaner.html). Fallback in both instances is the one shipped with JNA.
 

docs/manual/src/proc_macro/index.md

@@ -354,44 +354,8 @@ impl Foo {
 
 ## The `uniffi::custom_type` and `uniffi::custom_newtype` macros
 
-There are 2 macros available which allow procmacros to support "custom types" as described in the
-[UDL documentation for Custom Types](../udl/custom_types.md)
-
-The `uniffi::custom_type!` macro requires you to specify the name of the custom type, and the name of the
-builtin which implements this type. Use of this macro requires you to manually implement the
-`UniffiCustomTypeConverter` trait for for your type, as shown below.
-```rust
-pub struct Uuid {
- val: String,
-}
-
-// Use `Uuid` as a custom type, with `String` as the Builtin
-uniffi::custom_type!(Uuid, String);
-
-impl UniffiCustomTypeConverter for Uuid {
- type Builtin = String;
-
- fn into_custom(val: Self::Builtin) -> uniffi::Result<Self> {
- Ok(Uuid { val })
- }
-
- fn from_custom(obj: Self) -> Self::Builtin {
- obj.val
- }
-}
-```
-
-There's also a `uniffi::custom_newtype!` macro, designed for custom types which use the
-"new type" idiom. You still need to specify the type name and builtin type, but because UniFFI
-is able to make assumptions about how the type is laid out, `UniffiCustomTypeConverter`
-is implemented automatically.
-
-```rust
-uniffi::custom_newtype!(NewTypeHandle, i64);
-pub struct NewtypeHandle(i64);
-```
-
-and that's it!
+See the [UDL documentation for Custom Types](../udl/custom_types.md). It works exactly the same for
+proc-macros.
 
 ## The `uniffi::Error` derive
 
@@ -463,32 +427,7 @@ pub trait Person {
 // }
 ```
 
-## Types from dependent crates
-
-When using proc-macros, you can use types from dependent crates in your exported library, as long as
-the dependent crate annotates the type with one of the UniFFI derives. However, there are a couple
-exceptions:
-
-### Types from UDL-based dependent crates
-
-If the dependent crate uses a UDL file to define their types, then you must invoke one of the
-`uniffi::use_udl_*!` macros, for example:
-
-```rust
-uniffi::use_udl_record!(dependent_crate, RecordType);
-uniffi::use_udl_enum!(dependent_crate, EnumType);
-uniffi::use_udl_error!(dependent_crate, ErrorType);
-uniffi::use_udl_object!(dependent_crate, ObjectType);
-```
-
-### Non-UniFFI types from dependent crates
-
-If the dependent crate doesn't define the type in a UDL file or use one of the UniFFI derive macros,
-then it's currently not possible to use them in an proc-macro exported interface. However, we hope
-to fix this limitation soon.
-
-## Other limitations
+## Mixing UDL and proc-macros
 
-In addition to the per-item limitations of the macros presented above, there is also currently a
-global restriction: You can only use the proc-macros inside a crate whose name is the same as the
-namespace in your UDL file. This restriction will be lifted in the future.
+If you use both UDL and proc-macro generation, then your crate name must match the namespace in your
+UDL file. This restriction will be lifted in the future.

docs/manual/src/udl/custom_types.md

@@ -1,72 +1,95 @@
 # Custom types
 
-Custom types allow you to extend the UniFFI type system to support types from your Rust crate or 3rd
-party libraries. This works by converting to and from some other UniFFI type to move data across the
-FFI. You must provide a `UniffiCustomTypeConverter` implementation to convert the types.
+Custom types allow you to extend the UniFFI type system by converting to and from some other UniFFI
+type to move data across the FFI.
 
 ## Custom types in the scaffolding code
 
-Consider the following trivial Rust abstraction for a "handle" which wraps an integer:
+### custom_type!
+
+Use the `custom_type!` macro to define a new custom type.
 
 ```rust
-pub struct Handle(i64);
+
+// Some complex struct that can be serialized/deserialized to a string.
+use some_mod::SerializableStruct;
+
+/// `SerializableStruct` objects will be passed across the FFI the same way `String` values are.
+uniffi::custom_type!(SerializableStruct, String);
 ```
 
-In this trivial example, the simplest way to expose this is with a macro.
+By default:
+
+ - Values passed to the foreign code will be converted using `<SerializableStruct as Into<String>>` before being lowered as a `String`.
+ - Values passed to the Rust code will be converted using `<String as TryInto<SerializableStruct>>` after lifted as a `String`.
+ - The `TryInto::Error` type can be anything that implements `Into<anyhow::Error>`.
+ - `<String as Into<SerializableStruct>>` will also work, since there is a blanket impl in the core libary.
 
+### custom_type! with manual conversions
+
+You can also manually specify the conversions by passing an extra param to the macro:
+
+```rust
+uniffi::custom_type!(SerializableStruct, String, {
+ from_custom: |s| s.serialize(),
+ try_into_custom: |s| s.deserialize(),
+});
 ```
+
+### custom_newtype!
+
+The `custom_newtype!` can trivially handle newtypes that wrap a UniFFI type.
+
+```rust
+/// handle which wraps an integer
+pub struct Handle(i64);
+
+/// `Handle` objects will be passed across the FFI the same way `i64` values are.
 uniffi::custom_newtype!(Handle, i64);
 ```
 
-Or you can define this in UDL via a `typedef` with a `Custom` attribute,
-defining the builtin type that it's based on.
+### UDL
+
+Define custom types in UDL via a `typedef` with a `Custom` attribute, specifying the UniFFI type
+followed by the custom type.
 
 ```idl
 [Custom]
 typedef i64 Handle;
 ```
 
-For this to work, your Rust code must also implement a special trait named
-`UniffiCustomTypeConverter`.
-
-An implementation is provided if you used the `uniffi::custom_newtype!()` macro.
-But if you use UDL or otherwise need to implement your own:
+**note**: UDL users still need to call the `custom_type!` or `custom_newtype!` macro in their Rust
+code.
 
-This trait is generated by UniFFI and can be found in the generated
-Rust scaffolding - it is defined as:
+## User-defined types
 
-```Rust
-trait UniffiCustomTypeConverter {
- type Builtin;
+All examples so far in this section convert the custom type to a builtin type.
+It's also possible to convert them to a user-defined type (Record, Enum, interface, etc.).
+For example you might want to convert `log::Record` class into a UniFFI record:
 
- fn into_custom(val: Self::Builtin) -> uniffi::Result<Self>
- where
- Self: Sized;
- fn from_custom(obj: Self) -> Self::Builtin;
-}
-```
+```rust
 
-where `Builtin` is the Rust type corresponding to the UniFFI builtin-type - `i64` in the example above. Thus, the trait
-implementation for `Handle` would look something like:
+pub type LogRecord = log::Record;
 
-```rust
-impl UniffiCustomTypeConverter for Handle {
- type Builtin = i64;
+#[derive(uniffi::Record)]
+pub type LogRecordData {
+ level: LogLevel,
+ message: String,
+}
 
- fn into_custom(val: Self::Builtin) -> uniffi::Result<Self> {
- Ok(Handle(val))
+uniffi::custom_type!(LogRecord, LogRecordData, {
+ from_custom: |r| LogRecordData {
+ level: r.level(),
+ message: r.to_string(),
  }
+ try_into_custom: |r| LogRecord::builder()
+ .level(r.level)
+ .args(format_args!("{}", r.message))
+ .build()
+});
 
- fn from_custom(obj: Self) -> Self::Builtin {
- obj.0
- }
-}
 ```
 
-Because `UniffiCustomTypeConverter` is defined in each crate, this means you can use custom types even
-if they are not defined in your crate - see the 'custom_types' example which demonstrates
-`url::Url` as a custom type.
-
 ## Error handling during conversion
 
 You might have noticed that the `into_custom` function returns a `uniffi::Result<Self>` (which is an
@@ -112,20 +135,14 @@ pub enum ExampleError {
  InvalidHandle,
 }
 
-impl UniffiCustomTypeConverter for ExampleHandle {
- type Builtin = i64;
-
- fn into_custom(val: Self::Builtin) -> uniffi::Result<Self> {
- if val == 0 {
- Err(ExampleErrors::InvalidHandle.into())
- } else if val == -1 {
- Err(SomeOtherError.into()) // SomeOtherError decl. not shown.
- } else {
- Ok(Handle(val))
- }
+uniffi::custom_type!(ExampleHandle, Builtin, {
+ from_custom: |handle| handle.0,
+ try_into_custom: |value| match value {
+ 0 => Err(ExampleErrors::InvalidHandle.into()),
+ -1 => Err(SomeOtherError.into()), // SomeOtherError decl. not shown.
+ n => Ok(Handle(n)),
  }
- // ...
-}
+})
 ```
 
 The behavior of the generated scaffolding will be:
@@ -168,12 +185,32 @@ Here's how the configuration works in `uniffi.toml`.
  * `from_custom`: Expression to convert the custom type to the UDL type. `{}` will be replaced with the value of the custom type.
  * `imports` (Optional) list of modules to import for your `into_custom`/`from_custom` functions.
 
-## Using Custom Types from other crates
+## Using custom types from other crates
 
-To use the `Handle` example above from another crate, these other crates just refer to the type
-as a regular `External` type - for example, another crate might use `udl` such as:
+To use custom types from other crates, use a typedef wrapped with the `[External]` attribute.
+For example, if another crate wanted to use the examples above:
 
 ```idl
 [External="crate_defining_handle_name"]
-typedef extern Handle;
+typedef i64 Handle;
+
+[External="crate_defining_log_record_name"]
+typedef dictionary LogRecord;
 ```
+
+## Remote custom types
+
+Custom types that convert [Remote types](./remote_ext_types.md#remote-types) defined in other crates require special handling.
+
+1) Specify `remote` param in the `custom_type!` macro:
+
+```rust
+
+uniffi::custom_type!(SerializableStructFromOtherCrate, String, {
+ remote,
+ from_custom: |s| s.serialize(),
+ try_into_custom: |s| s.deserialize(),
+});
+```
+
+2) To share the custom type implementation with other crates, use the [remote_type! macro](./remote_ext_types.md#external+remote-types).