Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add native expression for Series.split/2 #875

Merged
merged 1 commit into from
Mar 4, 2024
Merged

Conversation

H12
Copy link
Contributor

@H12 H12 commented Mar 3, 2024

Fixes #874

Let me know if I missed anything!

@H12
Copy link
Contributor Author

H12 commented Mar 4, 2024

Here are the results of testing this in iex:

Mix.install([{:explorer, github: "H12/explorer"}])
# => :ok

require Explorer.DataFrame, as: DF
# => Explorer.DataFrame

df = DF.new(a: ["foo,bar", "bar,baz"])
# => Explorer.DataFrame<
#      Polars[2 x 1]
#      a string ["foo,bar", "bar,baz"]
#    >

DF.mutate(df, b: split(a, ","))
# => Explorer.DataFrame<
#      Polars[2 x 2]
#      a string ["foo,bar", "bar,baz"]
#      b list[string] [["foo", "bar"], ["bar", "baz"]]
#    >

Curiously, this does not work in Livebook -- something to do with mismatched rustler versions between explorer and kino_explorer. Bumping the rustler & explorer versions in kino_explorer then results in a compiler failure w/ this error:

== Compilation error in file lib/explorer/polars_backend/native.ex ==
** (ErlangError) Erlang error: :enoent
    (elixir 1.16.0) lib/system.ex:1101: System.cmd("cargo", ["metadata", "--format-version=1"], [cd: "native/explorer"])
    (rustler 0.31.0) lib/rustler/compiler/config.ex:91: Rustler.Compiler.Config.metadata!/1
    (rustler 0.31.0) lib/rustler/compiler/config.ex:78: Rustler.Compiler.Config.build/1
    (rustler 0.31.0) lib/rustler/compiler.ex:8: Rustler.Compiler.compile_crate/3
    lib/explorer/polars_backend/native.ex:32: (module)
could not compile dependency :explorer, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile explorer --force", update it with "mix deps.update explorer" or clean it with "mix deps.clean explorer"

For the record, that output was on the latest nightly build of the native MacOS distribution & the following dependencies:

Mix.install([
  {:kino, "~> 0.12.3"},
  {:kino_explorer, github: "H12/kino_explorer"}
])

I presume this is an issue w/ outdated/missing dependencies in Livebook/Kino that will get resolved eventually, but I'd also be happy to help figure that out too if it's not already being worked on. Wouldn't quite know where to start tho 😅

@josevalim josevalim merged commit acf7455 into elixir-explorer:main Mar 4, 2024
4 checks passed
@josevalim
Copy link
Member

💚 💙 💜 💛 ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Seeing :nif_not_loaded error for Series.split/2 when mutating a dataframe
2 participants