Using embind to generate TypeScript typings

[boyanio]

Hello. To export C++ structs and classes, one can use embind. As the export is very explicit, I was thinking, one could use embind to generate TypeScript typings (.d.ts) too. I would be happy to hear your comments on that :-)

[isc30]

Yes, I like that idea. I would go even further and introduce some kind of "ownership" expressed by marker interfaces. Something like this:

interface owns<T> extends T
{
    delete(): void;
}

This way your API can be explicit on what objects you need to call .delete() (this is very similar to IDisposable interface in C# but in a mixin version).

[boyanio]

Good point! If we look at the class example in the documentation, we could generate the following typings:

declare namespace Module {
    export interface Deletable {
        delete();
    }

    export interface MyClass extends Deletable {
        x: number;
        incrementX(): void;
    }

    export var MyClass: {
        new(x: number, y: string): MyClass;

        getStringFromInstance(a: MyClass): string;
    }
}

As not everyone uses TypeScript, this part should be optional. Could be a separate bind switch, for example

emcc --bind --ts-typings -o quick_example.js quick_example.cpp

[isc30]

I think the interface inheritance should go the other way around. You should be able to mark some type as Owning when the delete() function is available. This can be easily achieved with mixin interfaces.

declare namespace Module
{
    export interface Owner<T> : T
    {
        delete();
    }

    export interface MyClass
    {
        x: number;
        incrementX(): void;
    }

    export var MyClass:
    {
        new(x: number, y: string): Owner<MyClass>; // js needs to destroy it
        getSomeNotOwningPointer(): MyClass; // js doesnt need to destroy it

        getStringFromInstance(a: MyClass): string;
    }
}

With this design, Owner<T> is assignable to T so it can be used in non-owning contexts.

[endel]

I believe this is totally possible, given the verbosity and details we provide to EMSCRIPTEN_BINDINGS. Has anyone started implementing this? I'd love to contribute!

[isc30]

as far as I know no one started working on this

[wffurr]

I'm working with a prototype of this, created by a colleague. If I can get it working on a couple of different embind examples, then I will see about contributing it upstream.

[mathiasi]

Any updates on this?

[wffurr]

The tool itself works fine for the subset of embind it supports. I'm having trouble making it work for my project due to project-specific issues. I'm also not sure the best way to integrate it with the Emscripten toolchain. I might have something by the end of the year.

[stale]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 30 days. Feel free to re-open at any time if this issue is still relevant.

[kripken]

Reopening, as this still sounds useful. @wffurr , any updates?

[netpoetica]

@jjrv Did a really great job on this with nbind. I wonder if they might be able to provide some insight to help bring this directly into emscripten? Or if someone who is more familiar with emscripten may be able to draw some inspiration from their work: https://github.com/charto/nbind/blob/master/bin/ndts.js

I have been working with WebIDL with emscripten and auto-generating types from it, but I have run into a brick wall because WebIDL is incredibly unsophisticated when it comes to generics, templates and typedefs, which is one of the main things you want from C++ world. I think at this point for sure I will need to move over to using embind api and manually writing type definitions. I am not sure I am savvy enough on the C++ or emscripten side to contribute auto-generated types but I sure do wish I was :)

[netpoetica]

@wffurr Some of the code in nbind may provide some useful insight for you if you're still interested in tackling this: https://github.com/charto/nbind/blob/master/src/todts.ts and the Reflect.cc/Reflect.ts files I would imagine are the main thing

[ted537]

https://github.com/ted537/tsembind

I've created a rudimentary typescript bindings generator that works on existing EMBind libraries. There are some hacky things present (mostly grabbing context) but are only necessary for an external implementation.

[ssg3d]

@ted537 Am I correct in understanding that your generator will take the abc.js file generated by emscripten during compilation and output a ts file? I tried installing it from your github link but all it installs are a bunch of js/ts files, no executable. Anything I am missing? By the way, we were thinking of writing a js script that would read c++ source files and generate a module.d.ts file. I think your project already does that?

[ted537]

@ted537 Am I correct in understanding that your generator will take the abc.js file generated by emscripten during compilation and output a ts file? I tried installing it from your github link but all it installs are a bunch of js/ts files, no executable. Anything I am missing? By the way, we were thinking of writing a js script that would read c++ source files and generate a module.d.ts file. I think your project already does that?

Yep that's correct! Emscripten registers things like typenames at runtime, so that's why I targeted the .js output. Here's the usage. Note that this is an experimental library so lock down the version if you want something more consistent. Now that it's packed up on npm the install should be more stable (client doesn't need to run the prepare script). As for the executable question, the cmd.js is an executable.

npm i -g tsembind
tsembind my-compiled-lib.js

A similar strategy should be possible from the C++ side as well. I imagine a drop in bind-with-types.h header (which wraps emscripten/bind.h functionality) would be sufficient.

[ssg3d]

Thanks! I get this error when I try to run this on my js file though -

failed to asynchronously prepare wasm: LinkError: WebAssembly.instantiate(): Import #114 module="env" function="memory" error: memory import must be a WebAssembly.Memory object
LinkError: WebAssembly.instantiate(): Import #114 module="env" function="memory" error: memory import must be a WebAssembly.Memory object
RuntimeError: abort(LinkError: WebAssembly.instantiate(): Import #114 module="env" function="memory" error: memory import must be a WebAssembly.Memory object) at Error

[ted537]

Thanks! I get this error when I try to run this on my js file though -

failed to asynchronously prepare wasm: LinkError: WebAssembly.instantiate(): Import #114 module="env" function="memory" error: memory import must be a WebAssembly.Memory object
LinkError: WebAssembly.instantiate(): Import #114 module="env" function="memory" error: memory import must be a WebAssembly.Memory object
RuntimeError: abort(LinkError: WebAssembly.instantiate(): Import #114 module="env" function="memory" error: memory import must be a WebAssembly.Memory object) at Error

Could you create an issue for this? I've only tested it with some of the formats for the .js file Emscripten generates.

[ssg3d]

Done!

[mmarczell-graphisoft]

@ted537

A similar strategy should be possible from the C++ side as well. I imagine a drop in bind-with-types.h header (which wraps emscripten/bind.h functionality) would be sufficient.

I am creating this right now in our product. The approach I took is to compile all the binding C++ code twice, with the second pass using a replacement header and library that replicates Embind's API (instead of the wrapping you mentioned here). Running this second executable writes the declarations to a file and this can be nicely automated from CMake as a custom build step.

[devshgraphicsprogramming]

@ted537

A similar strategy should be possible from the C++ side as well. I imagine a drop in bind-with-types.h header (which wraps emscripten/bind.h functionality) would be sufficient.

I am creating this right now in our product. The approach I took is to compile all the binding C++ code twice, with the second pass using a replacement header and library that replicates Embind's API (instead of the wrapping you mentioned here). Running this second executable writes the declarations to a file and this can be nicely automated from CMake as a custom build step.

lovely, can you make the repo public so we could contribute?

[mmarczell-graphisoft]

@ted537
lovely, can you make the repo public so we could contribute?

I'd love to! But it's not a "repo" I could just "make public" - we use Perforce, and this code is not an isolated library or tool, it is part of our larger codebase. I'd have to work to isolate it, write a readme, etc. and I'd have to go through some corporate procedure to open source it.

[devshgraphicsprogramming]

@ted537
lovely, can you make the repo public so we could contribute?

I'd love to! But it's not a "repo" I could just "make public" - we use Perforce, and this code is not an isolated library or tool, it is part of our larger codebase. I'd have to work to isolate it, write a readme, etc. and I'd have to go through some corporate procedure to open source it.

Well, if you go through all the pain, the least I can promise you that we'll try to use it in production and contribute a bit.

[gracicot]

@mmarczell-graphisoft Honestly, even a simple gists of that replacement header file would be of extreme help. I would contribut too to make it production ready

[devshgraphicsprogramming]

We actually submitted a talk to WASM I/O about generating JavaScript bindings to C# with C++ Reflection TS, Cross Language Polymorphism, Embind and SWIG

Unfortunately got rejected, barrier to entry was too high XD

[marczellm]

@gracicot @devshgraphicsprogramming
Here you are: https://github.com/marczellm/emscripdtsgen
However I can see that https://github.com/ted537/tsembind is seeing much more active development.

[devshgraphicsprogramming]

@gracicot @devshgraphicsprogramming Here you are: https://github.com/marczellm/emscripdtsgen However I can see that https://github.com/ted537/tsembind is seeing much more active development.

TSembind completely failed us with regards to namespaces, nested classes, etc.

I'm hoping that our client allows us to finally mention them by name and accepts our proposal to let us speak about the tooling we developed at WASM Summit.

[mike-lischke]

@marczellm tsembind is useless for ES6 modules, it only works with CommonJS output.

[devshgraphicsprogramming]

@marczellm tsembind is useless for ES6 modules, it only works with CommonJS output.

moreover TSembind won't reconstruct your namespaces and nested classes.

[brendandahl]

I've landed the new emcc option --embind-emit-tsd <filename>. When using the flag your program will be compiled and run under node to generate the TS definitions. The feature is still new, but is supports most of the commonly used embind features. Please try it out and file new issues for feature requests or bugs.

Closing at fixed.

[mike-lischke]

@brendandahl Is this already published? I upgraded to emscripten 3.1.43 and added that option to my build script (which uses -lembind), but instead of a file I got a number of warnings (_embind_register_class_class_property) and a final error

/emscripten_zdgc_erx.rsp.utf-8' failed (returned 1)

If it is already released I can open a new issue for this.

[brendandahl]

That hasn't been implemented. I've filed issues for the remaining features:

• embind typescript - generate class property definitions #19855
• embind typescript - generate static class functions definitions #19856
• embind typescript - generate memory views definitions #19857

[starball5]

Thanks for the work being done here! I've written up a TL;DR on Stack Overflow here that I'll keep updated as things progress.