From 6879b0d7d1a895bb14ce802f042f2730c8860629 Mon Sep 17 00:00:00 2001 From: ritchie Date: Mon, 20 Nov 2023 13:33:25 +0100 Subject: [PATCH] update readme --- README.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 1942da3..5225b7c 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,17 @@ ## 1. Shared library plugins for Polars -This is new functionality and not entirely stable, but should be preferred over `2.` as this + + + + +This is new functionality and should be preferred over `2.` as this will circumvent the GIL and will be the way we want to support extending polars. Parallelism and optimizations are managed by the default polars runtime. That runtime will call into the plugin function. The plugin functions are compiled separately. -We can therefore keep polars more lean and maybe add support for a `polars-distance`, `polars-geo`, `polars-ml`, etc. Those can then have specialized expressions and don't have to worry as much for code bloat as they can be optionally installed. +We can therefore keep polars more lean and maybe add support for a `polars-distance`, `polars-geo`, `polars-ml`, etc. +Those can then have specialized expressions and don't have to worry as much for code bloat as they can be optionally installed. The idea is that you define an expression in another Rust crate with a proc_macro `polars_expr`. @@ -89,10 +94,6 @@ See the full example in [example/derive_expression]: https://github.com/pola-rs/ ## 2. Pyo3 extensions for Polars - - - - See the `example` directory for a concrete example. Here we send a polars `DataFrame` to rust and then compute a `jaccard similarity` in parallel using `rayon` and rust hash sets.