WIP: Rust wrapper #3181

nyurik · 2025-01-27T17:59:16Z

TODO/Goals

Include core header files
Generate/write a tiny C++ wrapper that Rust can communicate with
Compile core with cmake, possibly including the tiny C++ generated code - resulting in a statically-linkable (from Rust) lib
Use proper Rust strings instead of &CStr and CString where it is not performance critical.
- PROs: consistency with the Rust ecosystem, proper utf-8 validated strings going in and out of the crate.
- CONs: requires extra allocation and validation. On invalid string will panic, or we must introduce Result<String> for proper error reporting.
Fix unit tests not compiling (linking), while cargo run works ok
Camera operations for tile and static image rendering

Implementation ideas

Current thinking is to get cmake to produce a static library. All steps must be initiated by the platform/rust-cxx/build.rs:

cxx bridge generates a header file out of the #[cxx::bridge] mod ffi { ... } in platform/rust-cxx/src/main.rs
cmake crate executes cmake with special flags to pick up that header file, plus any other CC and H files in the rust-cxx dirs, plus the rest of the core and produces static lib
main.rs links to it and calls get_version or some other basic thing

github-actions · 2025-01-29T03:41:02Z

Bloaty Results 🐋

Compared to main

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [ = ]       0  [ = ]       0    TOTAL

Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/bloaty-results/pr-3181-compared-to-main.txt

Compared to d387090 (legacy)

    FILE SIZE        VM SIZE    
 --------------  -------------- 
   +32% +36.5Mi  +439% +26.3Mi    TOTAL

Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/bloaty-results/pr-3181-compared-to-legacy.txt

github-actions · 2025-01-29T03:43:54Z

Benchmark Results ⚡

Benchmark                                                     Time             CPU      Time Old      Time New       CPU Old       CPU New
------------------------------------------------------------------------------------------------------------------------------------------
OVERALL_GEOMEAN                                            +0.0109         +0.0109             0             0             0             0

Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/benchmark-results/pr-3181-compared-to-main.txt

platform/rust/core-src/ARCHITECTURE.md

platform/rust/Cargo.toml

louwers · 2025-01-30T13:59:09Z

platform/rust/src/wrapper.cc

+  return std::make_unique<TileServerOptions>();
+}
+std::unique_ptr<TileServerOptions> TileServerOptions_mapbox() {
+  return std::make_unique<TileServerOptions>(TileServerOptions::MapLibreConfiguration());


These are pending deprecation and removal.

For now these are good for testing, but I agree that we should remove these

for more information, see https://pre-commit.ci

* create map/headless after setting some parameters * once instantiated, can only render and set styles * need to fix camera operations for tiles and for static

for more information, see https://pre-commit.ci

louwers · 2025-03-05T09:55:51Z

.github/workflows/linux-ci.yml

@@ -111,6 +111,14 @@ jobs:
          cmake -B build -GNinja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=RelWithDebInfo -DMLN_WITH_CLANG_TIDY=ON -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DMLN_WITH_COVERAGE=ON ${{ env.renderer_flag_cmake }}
          cmake --build build --target mbgl-core mbgl-test-runner mbgl-render-test-runner mbgl-expression-test mbgl-render mbgl-benchmark-runner

+      # test rust platform
+      - if: matrix.renderer == 'drawable-rust'


We should move this to rust-ci.

louwers · 2025-03-05T09:57:52Z

platform/rust/src/lib.rs

@@ -0,0 +1,10 @@
+// FIXME: Remove this before merging


CommanderStorm · 2025-07-30T00:45:36Z

since we moved things to its own wrapper, I think this PR is moot, right?

nyurik · 2025-07-30T00:54:42Z

tbh, i would much rather move this here to avoid the unneeded extra complexity of the two repos - esp because now you see many other projects are using it, and it clearly shows value?

birkskyum · 2025-07-30T17:19:25Z

Taking a step back, it's not just two repos. There is the option to keep the wrapper in maplibre-native-rs, and just add a CI check here in Native that clones the wrapper repo to test it with upcoming native versions. It's a bit of an "ecosystem ci" check, which is an approach that will soon be used for Qt SDK (which prev lived in maplibre-native) and I think strikes a good balance.

It's inspired by the vite-ecosystem-ci project, which keep the vite core repo clean by making integration tests, and it's protecting as well against regressions as a monorepo would.

Without an ecosystem check like that, everything that depends on native has to be part of native core repo to be sure... maplibre-compose, maplibre-react-native, maplibre-native-qt, flutter-maplibre-gl etc., the rs bindings aren't really different in that regard, so i think it's best we try to establish a solution that'll work for other dependants too.

CommanderStorm · 2025-07-30T18:21:11Z

I see that the issues that we currently have are not very MLN specific.

Given that otherwise I would have to subscribe to all MLN issues or Bart would need to tag me (both horrible) I think keeping us separate and adding something like ecosystem CI would be nice.

The changes that I see as TODO would unnecessarily spam people, making maintaining something as high traffic as MLN more difficult.

nyurik · 2025-07-30T18:24:07Z

oops, i totally misread - i thought this was the PR that added the justfile to the core MLN, not the rust wrapper. I'm fine with the wrapper being separate too

nyurik force-pushed the rust-wrapper branch from 31d4902 to 277e006 Compare January 27, 2025 18:07

nyurik marked this pull request as draft January 27, 2025 18:07

nyurik changed the title ~~Rust wrapper~~ WIP: Rust wrapper Jan 27, 2025

nyurik force-pushed the rust-wrapper branch 2 times, most recently from 611176c to d2ef4df Compare January 27, 2025 19:44

nyurik mentioned this pull request Jan 29, 2025

linker: Remove the "--whole-archive in test mode" backcompat hack rust-lang/rust#128400

Merged

louwers reviewed Jan 29, 2025

View reviewed changes

platform/rust/core-src/ARCHITECTURE.md Outdated Show resolved Hide resolved

louwers reviewed Jan 29, 2025

View reviewed changes

platform/rust/Cargo.toml Outdated Show resolved Hide resolved

nyurik force-pushed the rust-wrapper branch from b3be21a to 567cf09 Compare January 29, 2025 16:37

louwers reviewed Jan 30, 2025

View reviewed changes

nyurik mentioned this pull request Jan 31, 2025

Suggest content for next MapLibre Newsletter maplibre/maplibre.github.io#291

Closed

CommanderStorm mentioned this pull request Oct 6, 2024

Add server-side raster tile rendering maplibre/martin#978

Open

4 tasks

nyurik and others added 16 commits February 4, 2025 18:07

wip

619b05a

wip

cfd4f26

wip

bf1168e

wip

2ae0d90

Add ChatGPT generated build.rs

4a00747

Add ccache, remove linking with STL

e7f90dd

fmt

dc5efbe

[pre-commit.ci] auto fixes from pre-commit.com hooks

da5ff7b

for more information, see https://pre-commit.ci

rm old rust stuff

7baf25c

wip

90b7df6

fix build on 1.82+

0cdc369

ci

1fa4a96

ci wip

06c3292

symlink

34ccd93

add Rust-specific C++ wrapper, renderer args

b85ac6f

[pre-commit.ci] auto fixes from pre-commit.com hooks

27d1a34

for more information, see https://pre-commit.ci

pre-commit-ci bot and others added 20 commits February 9, 2025 05:20

[pre-commit.ci] auto fixes from pre-commit.com hooks

a254edb

for more information, see https://pre-commit.ci

Massive rework, ability to reuse renderer

fcc032e

* create map/headless after setting some parameters * once instantiated, can only render and set styles * need to fix camera operations for tiles and for static

[pre-commit.ci] auto fixes from pre-commit.com hooks

89ffd52

for more information, see https://pre-commit.ci

handle style url/paths, cleanup

6147835

[pre-commit.ci] auto fixes from pre-commit.com hooks

313f101

for more information, see https://pre-commit.ci

inline render

de78b7d

expose camera and debug mode api

9e91990

[pre-commit.ci] auto fixes from pre-commit.com hooks

412840e

for more information, see https://pre-commit.ci

make map chainable

c948159

first pass at tile impl

5bde308

Fix linking issue with dependency order

12685a5

add sqlite3 dyn link workaround

f490037

Merge branch 'main' into rust-wrapper

6b54a8e

Merge branch 'main' into rust-wrapper

23b2dc3

fix unit test

e000e27

Refactor into multiple modules

abb0034

[pre-commit.ci] auto fixes from pre-commit.com hooks

ddd5ac5

for more information, see https://pre-commit.ci

Merge branch 'main' into rust-wrapper

813ee06

Merge branch 'main' into rust-wrapper

b3292c3

Rm sqlite dependency, cleanup

29de0ff

louwers reviewed Mar 5, 2025

View reviewed changes

platform/rust/src/lib.rs

@@ -0,0 +1,10 @@

// FIXME: Remove this before merging

Copy link

Member

louwers Mar 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

nyurik added 2 commits March 5, 2025 14:07

Merge branch 'main' into rust-wrapper

e6ecabb

Force native build to be release

d8fb47c

nyurik closed this Jul 30, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

WIP: Rust wrapper #3181

WIP: Rust wrapper #3181

Uh oh!

nyurik commented Jan 27, 2025 •

edited by louwers

Loading

Uh oh!

github-actions bot commented Jan 29, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jan 29, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

louwers Jan 30, 2025

Uh oh!

nyurik Jan 30, 2025

Uh oh!

louwers Mar 5, 2025

Uh oh!

louwers Mar 5, 2025

Uh oh!

CommanderStorm commented Jul 30, 2025

Uh oh!

nyurik commented Jul 30, 2025

Uh oh!

birkskyum commented Jul 30, 2025 •

edited

Loading

Uh oh!

CommanderStorm commented Jul 30, 2025

Uh oh!

nyurik commented Jul 30, 2025

Uh oh!

Uh oh!

Uh oh!

WIP: Rust wrapper #3181

WIP: Rust wrapper #3181

Uh oh!

Conversation

nyurik commented Jan 27, 2025 • edited by louwers Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODO/Goals

Implementation ideas

Uh oh!

github-actions bot commented Jan 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bloaty Results 🐋

Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/bloaty-results/pr-3181-compared-to-main.txt

Uh oh!

github-actions bot commented Jan 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

louwers Jan 30, 2025

Choose a reason for hiding this comment

Uh oh!

nyurik Jan 30, 2025

Choose a reason for hiding this comment

Uh oh!

louwers Mar 5, 2025

Choose a reason for hiding this comment

Uh oh!

louwers Mar 5, 2025

Choose a reason for hiding this comment

Uh oh!

CommanderStorm commented Jul 30, 2025

Uh oh!

nyurik commented Jul 30, 2025

Uh oh!

birkskyum commented Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CommanderStorm commented Jul 30, 2025

Uh oh!

nyurik commented Jul 30, 2025

Uh oh!

Uh oh!

nyurik commented Jan 27, 2025 •

edited by louwers

Loading

github-actions bot commented Jan 29, 2025 •

edited

Loading

github-actions bot commented Jan 29, 2025 •

edited

Loading

birkskyum commented Jul 30, 2025 •

edited

Loading