Add a doc comparing UniFFI with diplomat

[mhammond]

I spent some time having a look at diplomat, primarily to see what UniFFI can learn and opportunities to borrow, and felt it is worth sharing more broadly. I'm not sure if this is the best place for such a doc, and whether we should consider adding a link to README.md (I guess we probably should - explicitly noting diplomat exists in our README could well help some users)

I'd love feedback from everyone, but explicitly flagging just a few - in particular, @Manishearth (who I can't actually flag!) to represent the diplomat team, @rfk as the author of the much referenced #416, and @badboy to see if the document accurately describes his experience trying to use glean with the macro approach in #416. There's no hurry whatsoever, so please continue to enjoy your holidays 🎇

rendered version here

[Manishearth]

Hmm, this is a good idea, but I think there might be some misunderstandings about how diplomat handles modules; left some comments.

[Manishearth]

diplomat takes in a whole crate, not just a module. there may be multiple #[diplomat::bridge] tagged modules in the crate

[rfk]

there may be multiple #[diplomat::bridge] tagged modules in the crate

@mhammond @badboy I wonder how much of a difference this would make to a use-case like Glean, if you could e.g. have one #[diplomat::bridge] declaration per metric type but the tool knows how to find them all and expose them as a single API surface when generating the bindings.

[mhammond]

This is certainly worth experimenting with, especially if we can avoid the "module" limitation - eg, if all types needed [uniffi::magic] then maybe there's actually no reason to insist on a wrapping module? Regardless, I've tried to capture this idea in the doc.

[badboy]

I think in the 416 approach that would still have not worked, because we hash all input to put that into the generated function names as a way to prevent use of the wrong version of a library.
A single #[diplomat::bridge] invocation cannot know all input and thus can't generate one hash.
Of course this is a limitation we built ourselves, so maybe could be lifted.

[badboy]

Re-reading quickly how diplomat works that same hashing could be applied to only the limited bridged module while still giving the same benefits.
Given that diplomat-tool parses the same code, it would come up with the same hash and know what functions to call in the foreign language code.

[Manishearth]

Yeah the reason we have the module wrapper is mostly for convenience, because we need to look at imports as well sometimes.

[mhammond]

Yeah, thanks for clarifying - and while that's interesting context for this discussion, I don't think a discussion of the hashing needs to go into that document. LMK if you disagree though.

[badboy]

Leaving that out sounds right to me.
It's a thing we will need to consider if we try the macro approach again, but not necessary to discuss in comparison.

[rfk]

Re-reading quickly how diplomat works that same hashing could be applied to only the limited bridged module
while still giving the same benefits.

Good point, and in retrospect this thing about including a hash of the component in the name of the generated function probably deserved its own separate design doc. A summary of what's in my head based on the above, in case some of the details aren't obvious:

The main goal of naming our FFI functions like component_<hash>_do_the_thing instead of just component_do_the_thing is to guard against undefined behaviours if foreign-language bindings for version X of the interface are accidentally used with a .so for version Y of the crate. That is: if the details of how to call one of the FFI functions change, then we want the generated name of that function to change.

We don't necessarily have to achieve that by using a single <hash> of the whole API surface. If we split the FFI into several independent macro invocations, then as long as they're self-contained then they could all safely use their own individual hash.

[Manishearth]

diplomat just treats documentation as "yet another backend", which works reasonably well, since the architecture of a diplomat backend is just "here's the type structure, you know what to expect on the FFI layer, do what you want".

[rfk]

By contrast, the only reason that we don't already have documentation as yet-another-backend in UniFFI, is that the off-the-shelf parser that we use for the IDL throws away comments by default :-(

[Manishearth]

i think a difference not really covered in this document is the style of actual interface. UniFFI basically seems to be serializing types across FFI with this cool FFIConverter scheme, whereas Diplomat uses raw repr(C) so backends do not need much ceremony to read things. Both are valid approaches, but it's an interesting difference likely borne out of the choice of languages to prioritize (Diplomat cares heavily about C++)

[rfk]

I agree, and I think there may be some broader philosophical differences as well that feed into some of the technical differences. For example, I get the impression that UniFFI is happier to eat some performance overhead in the bindings, and that this comes from our initial focus on targeting managed languages. (As you say, both valid approaches, but an interesting difference).

[mhammond]

As Ryan notes later, I think this is a reflection of expedience and initial use-cases, but not necessarily a key difference between the tools at a high level - I think UniFFI could support "native" repr(C) types in some cases.

If you still think this is important to communicate after seeing my updated version, please give me a rough idea of what it should say and where :)

[rfk]

@mhammond thanks for writing this up, and for looping me in! I'll try to chew on this a bit over the coming days as I have a little downtime, but some initial thoughts below.

[rfk]

I agree, and I think there may be some broader philosophical differences as well that feed into some of the technical differences. For example, I get the impression that UniFFI is happier to eat some performance overhead in the bindings, and that this comes from our initial focus on targeting managed languages. (As you say, both valid approaches, but an interesting difference).

[rfk]

It may not be obvious why "the entire type system is known when generating" is a good thing, I wonder if it's worth saying a few more words here about how it e.g. allows additional safety assurances.

[mhammond]

Actually, the point I had in mind was simply that the foreign bindings need to know the names and types of struct elements so it can recreate the same struct on the other side. But I've tried to capture that in the new version.

[rfk]

Or just find it a really interesting technical challenge I guess... 😅

[rfk]

This plays a little bit into the discussions around having bindings generators live in separate crates, ref #299.

[rfk]

Thinking a bit more seriously about this, I don't think that there's realistically much value in having a kind of two-way interoperability between UniFFI bits and Diplomat bits. It would require the projects to converge on a great many details of how the FFI layer works, and I don't think there are sufficiently many quick wins to fuel that kind of work. For example, it would almost certainly be quicker for Diplomat to gain a Kotlin backend by writing one from scratch than by trying to iterate the UniFFI Kotlin backend towards something that works for both.

However, what I could see happening in future is UniFFI becoming a kind of higher-level wrapper around Diplomat. I can imagine a Diplomat backend for UniFFI that converts a .udl file into a bridge module and then uses the Diplomat toolchain to generate bindings from it, keeping some of the additional affordances/conveniences we've built for our specific use-cases (e.g. around megazording).

An interesting thought experiment, anyway 😁

[mhammond]

yeah, I did get a bit carried away - I've tried to tone this down a little in the new version.

[rfk]

By contrast, the only reason that we don't already have documentation as yet-another-backend in UniFFI, is that the off-the-shelf parser that we use for the IDL throws away comments by default :-(

[mhammond]

Thanks to both of you for your considered comments. I've made some significant changes in a version I'm about to push, so I'd love continuing to get feedback on the new version. It would also be great if you could "resolve" any comments which you think are suitably represented in the new version, just so I can keep track of what I should consider in future iterations.

[mhammond]

As Ryan notes later, I think this is a reflection of expedience and initial use-cases, but not necessarily a key difference between the tools at a high level - I think UniFFI could support "native" repr(C) types in some cases.

If you still think this is important to communicate after seeing my updated version, please give me a rough idea of what it should say and where :)

[mhammond]

Actually, the point I had in mind was simply that the foreign bindings need to know the names and types of struct elements so it can recreate the same struct on the other side. But I've tried to capture that in the new version.

[mhammond]

This is certainly worth experimenting with, especially if we can avoid the "module" limitation - eg, if all types needed [uniffi::magic] then maybe there's actually no reason to insist on a wrapping module? Regardless, I've tried to capture this idea in the doc.

[mhammond]

yeah, I did get a bit carried away - I've tried to tone this down a little in the new version.

[bendk]

Can you elaborate on this one? If we can use these macros as the source for scaffolding code, why couldn't we generate the bindings code as well?

[mhammond]

I hope I cleared this up in a86e949 - lmk if I need more about this.

[bendk]

This section is some serious food for thought.

For one, as @rfk points out, we're currently duplicating the struct/function definitions in the UDL. The way I see it: both projects have made a similar design decision. @Manishearth describes it as "we do not want changes far away to change the FFI API". I would maybe reword that to "the FFI should be fully defined in one place". For UniFFI, that place is the UDL. For diplomat, it's the ffi module. (Multiple ffi modules complicates this picture a bit, but doesn't fundamentally change things).

However, there is one difference: in some cases you only need code inside the ffi module. This has the potential to reduce duplication. For example, it would be very natural to move our Store definitions inside the ffi module. Also, since types defined inside the ffi module are visible to the rest of the rust code, so we could move types like Login, EncrytpedLogin, CreditCard, Address, etc. into the ffi module. I think external data types would still need duplicate definitions, but maybe that's it.

One potential issue with this is that it couples the library with the FFI code. But this doesn't seem to be a problem for our components. As @MarkH points out, our Rust APIs exists purely to service the FFIs.

It really makes me wonder about switching from a UDL-based approach to a macro based approach. At the start, maybe each consumer would simply refactor their current UDL file to a macro. This shouldn't be much work, maybe we could even automate it. After that, we have the ability to do to small refactors that eliminate the unneeded duplication.

BTW, this also could solve some documentation issues since a) we can actually see docstrings when using syn and b) if there's only 1 place where a type is defined then it's clear where to put the docstrings.

[mhammond]

I added a brief note in a86e949 which doesn't capture all of this comment, but does briefly say why we should consider this more.

[Manishearth]

Seems good!

[Manishearth]

nit: something's missing here?

[mhammond]

Thanks! But I can't see what's missing in this line - which probably means I should reword it :) Can you please let me know what is unclear?

[Manishearth]

the "but we struck" seems to be an incomplete sentence and probably also should have a newline after it?

[rfk]

IIUC you want this sentence to treat the **what's bad about this** as a kind of inline link to suggest "but we struck the bad things noted here". It might be clearer as an actual link :-)

[rfk]

(That is to say, I think this would read more clearly if you make the **what's bad about this** a kind of sub-heading in the same style as the UniFFI section above, and reference it more explicitly in the "UniFFI's experience" part).

[Manishearth]

ah! i see

[Manishearth]

ah, this explains a lot the design decisions :)

[badboy]

Thanks for writing this up!
This actually makes me want to get back to trying a macro-based approach again.
While I like our UDL (probably more than others here) it's duplication of concerns can be a bit annoying.

Regarding the request for my experience: In my #416 comment I mentioned "In the current implementation a lot of errors are hidden", unfortunately I can't remember further details.
What I think I encountered was bugs in both the macro-annotated module and outside of it, where rustc would then only show the errors from the macro. This made iterating on the code base annoying as I couldn't see all errors at once.
But hard to say after the fact whether that's an implementation bug or some more underlying problems.

Maybe we uniffi-devs should get together soon again and make some high-level plans where to take UniFFI in the next months.

[rfk]

Lots of great discussion here, thanks for driving it!

[rfk]

FWIW I don't really understand what's "better" about using a macro than a separate interface-definition file in this regard. IIUC, in UniFFI the API is stable as long as the .udl doesn't change, while in Diplomat the API is stable as long as the contents of hte macro block don't change.

(Making this claim doesn't bother me, I just don't get it and wonder if I'm missing something).

[mhammond]

I've tried to clarify this, but I didn't intend this to mean "better than UniFFI", I meant it to mean "better than allowing any type in the crate to define the FFI"

IOW, I'm trying to say "diplomat's view is that forcing MyFFIType to be declared inside the ffi module is better than allowing it to exist outside the module, because what might be considered an "incidental" change to that type might accidentally change the FFI"

[rfk]

Aha, I see, thanks for clarifying 👍🏻

[rfk]

(That is to say, I think this would read more clearly if you make the **what's bad about this** a kind of sub-heading in the same style as the UniFFI section above, and reference it more explicitly in the "UniFFI's experience" part).

[rfk]

Ah, I think this helps answer my question about about API stability in UniFFI vs Diplomat. I still feel like the .udl serves a similar purpose in the UniFFI world, but because of the way the .udl kind of maps directly onto the underlying Rust code, I can see an argument that changes in a UniFFI component are more likely to change the FFI surface than in Diplomat.

They won't change to FFI surface by stealth, because you'll have to update the .udl file...but changing the .udl file is likely to be the simplest way to accommodate a change in the underlying Rust code. In Diplomat you might instead write some adapter code in the FFI module in order to avoid changing the FFI surface, because there's an obvious place to do it.

[mhammond]

yeah, I think we agree that UniFFI's use of the .udl file and Diplomat's decision to restrict where types are exposed do serve the same purpose in that regard. The broader point I'm trying to make though is that application-services would prefer to not have those guards in place - ie, it would probably prefer a world where the UDL file didn't exist and nor did any limitation about where these types could be defined.

[rfk]

How does this work in Diplomat, does the FFI allow you to pass it around as a pointer but not construct one for yourself on the foreign-language side?

[mhammond]

No - diplomat doesn't have that problem because the struct definition must appear inside the module, so the foreign bindings, which parse that module, do know the struct elements. This is the same as we discovered in #416 - that forcing all type definitions into the single ffi module would technically work, but the impact that would have on how our code is organized made it less appealing than the status quo.

I added a few words here to try and make that clearer.

[rfk]

Suggested change

	maybe can made to work, so long as we are happy to help UniFFI discover where such annotations
	maybe can be made to work, so long as we are happy to help UniFFI discover where such annotations

[rfk]

FWIW, I think the toy example is the worst-case for highlighting the duplication here because the two duplicate declarations are separated by just 7 lines of text. In a real-world crate I would expect the duplication to not feel quite so bad because the source struct and its redeclaration would be further apart.

(That doesn't help with some of the other contra points raised here though)

[mhammond]

I would expect the duplication to not feel quite so bad

This is a good point, but from my POV, that can almost make the duplication feel worse - eg, you get a rust compiler error, and it can be hard to work out whether the UDL needs to change or the rust duplicate of that UDL needs to change - eg, making something optional means adding a ? to the udl and a corresponding Option<> to the Rust - it can be difficult to work out which one you screwed up :)

[rfk]

You could imagine explicitly telling the macro about the path to the redeclared struct, like:

    #[uniffi::declare_imported_type(super::MyFFIType)]
    pub struct MyFFIType {
        pub a: i32,
        pub b: bool,
    }

Then if it wanted to, the code processing this declaration could go find the corresponding super::myFFIType and pull out e.g. its docstring.

I think this would probably be more trouble than it's worth (we don't want to re-implement vast swathes of Rust's name lookup machinery, for example) but it's interesting to think about.

[mhammond]

Thanks all - I also added a link to this to the README. I'm going to merge this now - we can always change it later if we think of something else to add or if it becomes outdated in any way.