feat: polars + bun js runtime

[ryanrussell]

The new zig-based runtime for JS looks extremely promising:

Specifically, FFI appears much more performant: https://github.com/oven-sh/bun#bunffi-foreign-functions-interface

@ritchie46 @universalmind303 -- has this made your radar yet?

I was considering giving it a run on my side; but figured it made sense to ask you guys first.

If this is of interest, do you have any guidance on how you would like to see it in the project structurally?

Best,
Ryan

[universalmind303]

reading through the docs...

seems like a really cool project. Not relying on the V8 runtime seems like a game changer in terms of performance & interop.

It looks like they have actually implemented the node_api for their runtime. So in theory, you should be able to just plug-n-play. I'd imagine true FFI bindings would be faster than using napi. However, FFI bindings would be a pretty big undertaking given the number of methods polars exposes. It would likely be the equivalent effort of developing & maintaining the node or python bindings.

[ryanrussell]

Thanks for the feedback; makes sense.

I was waiting on a bugfix on bun to test further - oven-sh/bun#644

This appears resolved, I'm going to take a look around and see if I notice a marked improvement in bun vs node.

It may take some time, but I will report back(read: behind lots of other things in the todo list). Open to anyone else jumping in as well.

Best,
Ryan

[universalmind303]

I did give this a try, it looks like without napi_create_bigint_words support, the module will not load. Will need to wait for that method. See oven-sh/bun#158 for Node-API implementation status.

[Jarred-Sumner]

napi_create_bigint_words is implemented in Bun v0.3.1

It looks like Bun is still missing util/types though

[Jarred-Sumner]

util/types is now implemented in Bun v0.3.1

You can try nodejs-polars in the canary build of Bun:

bun upgrade --canary

Note that the napi_create_bigint_words function is probably slower in Bun than in V8 due to a missing constructor in JSC (have to call setDigit for each value in the array instead of a memcpy). If that's used much, happy to add the fast path

[stinodego]

Moving this to the nodejs-polars repo. Not sure if it is still relevant.

[ryanrussell]

@stinodego

As much as I'd love to help on this, I do not foresee having the bandwidth in the near future.

Feel free to close for housekeeping unless someone else has the cycles. It can always be recreated if someone steps forward to take it on.

[stinodego]

All right, I'll close this for now. Thank you!