Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider migrating to Rust based Python package? #319

Open
haoxins opened this issue Feb 21, 2024 · 8 comments
Open

Consider migrating to Rust based Python package? #319

haoxins opened this issue Feb 21, 2024 · 8 comments
Labels
type/feature req Type: feature request

Comments

@haoxins
Copy link
Contributor

haoxins commented Feb 21, 2024

General Question

More and more Python packages are built from the Rust code base. Such as:

  1. https://github.com/pola-rs/polars
  2. https://github.com/Qiskit/rustworkx
  3. https://github.com/pydantic/pydantic-core

The associated tools are:

https://github.com/PyO3

The benefit is that we can build the logic from Rust once and publish it as Rust, Python, and even JavaScript (Node.js) packages.
This is especially the trending solution for the data/ML ecosystem.

When I start to build the services and SDKs based on the Nebula.
The first candidate language is Rust so that I can develop and deploy the services as Rust code, and build the Python SDKs (for data scientists) based on the Rust codebase.
This is the ideal solution for writing code only once.

But the Rust client for Nebula is not ready at last year.

Since we already have the first draft version of the Nebula Rust client, I think this is a good time to share this idea with you guys.

The other tools for more languages that I'm not used to are:
https://github.com/dtolnay/cxx for c++

cc @Nicole00 @wey-gu

@haoxins haoxins changed the title Consider migrated to Rust based Python package Consider migrated to Rust based Python package? Feb 21, 2024
@haoxins haoxins changed the title Consider migrated to Rust based Python package? Consider migrating to Rust based Python package? Feb 21, 2024
@haoxins
Copy link
Contributor Author

haoxins commented Feb 23, 2024

Another example from Temporal SDK core

@QingZ11 QingZ11 added the type/feature req Type: feature request label Feb 23, 2024
@QingZ11
Copy link

QingZ11 commented Feb 23, 2024

It's really a great idea, thank you Haoxin.

@wey-gu
Copy link
Contributor

wey-gu commented Feb 25, 2024

General Question

More and more Python packages are built from the Rust code base. Such as:

  1. https://github.com/pola-rs/polars

  2. https://github.com/Qiskit/rustworkx

  3. https://github.com/pydantic/pydantic-core

The associated tools are:

https://github.com/PyO3

The benefit is that we can build the logic from Rust once and publish it as Rust, Python, and even JavaScript (Node.js) packages.

This is especially the trending solution for the data/ML ecosystem.

When I start to build the services and SDKs based on the Nebula.

The first candidate language is Rust so that I can develop and deploy the services as Rust code, and build the Python SDKs (for data scientists) based on the Rust codebase.

This is the ideal solution for writing code only once.

But the Rust client for Nebula is not ready at last year.

Since we already have the first draft version of the Nebula Rust client, I think this is a good time to share this idea with you guys.

The other tools for more languages that I'm not used to are:

https://github.com/dtolnay/cxx for c++

cc @Nicole00 @wey-gu

Agreed, was considering pyo3 when facing perf issues in wey-gu/nebula-dgl#10

We could do so in free time!

@haoxins
Copy link
Contributor Author

haoxins commented Mar 1, 2024

Also find this https://github.com/jni-rs/jni-rs for Java :)

@wey-gu
Copy link
Contributor

wey-gu commented Mar 21, 2024

Also, recently I had some chance to implement/build something with cython, later I could prepare a PR to add cython build of nebula-python and come with a benchmark.

@Abeautifulsnow
Copy link

👍Both Rust and Cython are excellent choices, but Rust (using PyO3) may be a more popular and convenient option when integrating with both Python and Node, especially if there already has a mature Rust-based repository here. Hope to see it soon.

@wey-gu
Copy link
Contributor

wey-gu commented Sep 12, 2024

@PsiACE
Copy link

PsiACE commented Sep 13, 2024

https://github.com/nebula-contrib/rust-nebula

cc @BeautyyuYanli @PsiACE

Cool idea. We can start by simply checking that the core functionality is aligned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature req Type: feature request
Projects
None yet
Development

No branches or pull requests

5 participants