Pyodide integration?

[Alex-Monahan]

Hey folks!!

Data twitter got pretty excited the other day just realizing that SQLite comes with Pyodide... Any idea what it would take to add a WASM DuckDB to Pyodide?

Pyodide Repo

Thanks!
-Alex

[ankoh]

Sounds like a very interesting idea! We should look into that.

[ankoh]

I had a brief look and I think there are a few road blocks ahead of us here.
This would only be maintainable & useful if we could glue the python client of DuckDB to DuckDB-Wasm.
But they don't share the same API today.

I think building a pyodide specific DuckDB-Wasm API is unrealistic and compiling native DuckDB to pyodide would be severly limited due to the lack of, for example, our web filesystem.
We'd probably also have to glue the Pandas package of pyodide to DuckDB-Wasm.
It's not impossible but quite a bit of work.

[hamilton]

cc @rth

[paultiq]

Edit: More examples here jupylite_duckdb

I know this is an old thread, but it's probably the most appropriate place to mention:

I was experimenting with jupyterlite & duckdb... and ended up running duckdb_wasm in pyodide.

Run the below code on Pyodide shell console: https://pyodide.org/en/stable/console.html

import micropip;
await micropip.install('pandas');
await micropip.install('jupylite-duckdb');
import jupylite_duckdb as jd;
conn = await jd.connect();
r1 = await jd.query("pragma version", conn);
r2 = await jd.query("create or replace table xyz as select * from 'https://raw.githubusercontent.com/Teradata/kylo/master/samples/sample-data/parquet/userdata2.parquet'", conn);
r3 = await jd.query("select gender, count(*) as c from xyz group by gender", conn);

print("DuckDb Version:")
print(r1);
print("# of Rows Loaded:")
print(r2);
print("Summary:")
print(r3);

[carlopi]

This has been a great idea / a great thread, but I think the duckdb-pyodide and duckdb-wasm will go two independent tracks, see here (https://github.com/duckdb/duckdb-pyodide) for the current status of duckdb within pyodide.

See for example the (very basic) demo at https://duckdb.github.io/duckdb-pyodide/console.

Thanks everyone for the contribution

[carlopi]

Another update, pyodide/pyodide#4684: DuckDB is now a proper Pyodide package