diff --git a/README.md b/README.md index fc1bd802cf3f..baaa98dc94c1 100644 --- a/README.md +++ b/README.md @@ -7,9 +7,10 @@ This repository has the source code for Comprehensive Rust 🦀, a multi-day Rust course developed by the Android team. The course covers all aspects of Rust, from basic syntax to generics and error handling. It also includes deep dives on -[Android], [bare-metal], and [concurrency]. +[Android], [Chromium], [bare-metal], and [concurrency]. [Android]: https://google.github.io/comprehensive-rust/android.html +[Chromium]: https://google.github.io/comprehensive-rust/comprehensive.html [bare-metal]: https://google.github.io/comprehensive-rust/bare-metal.html [concurrency]: https://google.github.io/comprehensive-rust/concurrency.html diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 31a060947e93..36db5b6ce58a 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -235,6 +235,42 @@ - [Exercises](exercises/android/morning.md) + +# Chromium + +---- + +- [Welcome](chromium.md) +- [Setup](chromium/setup.md) +- [Using cargo for experimental tools](chromium/cargo.md) +- [Policy](chromium/policy.md) +- [Build Rules](chromium/build-rules.md) + - [Unsafe code](chromium/build-rules/unsafe.md) + - [Depending on Rust code from Chromium C++](chromium/build-rules/depending.md) + - [Visual Studio code](chromium/build-rules/vscode.md) + - [Exercise](exercises/chromium/build-rules.md) +- [Interoperability with C++](chromium/interoperability-with-cpp.md) + - [Example bindings](chromium/interoperability-with-cpp/example-bindings.md) + - [Limitations of cxx](chromium/interoperability-with-cpp/limitations-of-cxx.md) + - [cxx error handling](chromium/interoperability-with-cpp/error-handling.md) + - [Using cxx in Chromium](chromium/interoperability-with-cpp/using-cxx-in-chromium.md) + - [Exercise](exercises/chromium/interoperability-with-cpp.md) +- [Adding third party crates](chromium/adding-third-party-crates.md) + - [Configuring Cargo.toml](chromium/adding-third-party-crates/configuring-cargo-toml.md) + - [Configuring gnrt_config.toml](chromium/adding-third-party-crates/configuring-gnrt-config-toml.md) + - [Downloading crates](chromium/adding-third-party-crates/downloading-crates.md) + - [Generating gn build rules](chromium/adding-third-party-crates/generating-gn-build-rules.md) + - [Resolving problems](chromium/adding-third-party-crates/resolving-problems.md) + - [Build scripts which generate code](chromium/adding-third-party-crates/resolving-problems/build-scripts-which-generate-code.md) + - [Build scripts which build C++ or take arbitrary actions](chromium/adding-third-party-crates/resolving-problems/build-scripts-which-take-arbitrary-actions.md) + - [Depending on a crate](chromium/adding-third-party-crates/depending-on-a-crate.md) + - [Reviews and audits](chromium/adding-third-party-crates/reviews-and-audits.md) + - [Checking into Chromium source code](chromium/adding-third-party-crates/checking-in.md) + - [Keeping crates up to date](chromium/adding-third-party-crates/keeping-up-to-date.md) + - [Exercise](exercises/chromium/third-party.md) +- [Bringing it together - Exercise](exercises/chromium/bringing-it-together.md) + + # Bare Metal: Morning ---- diff --git a/src/chromium.md b/src/chromium.md new file mode 100644 index 000000000000..83926fbdca78 --- /dev/null +++ b/src/chromium.md @@ -0,0 +1,9 @@ +# Welcome to Rust in Chromium + +Rust is supported for third-party libraries in Chromium, with first-party glue +code to connect between Rust and existing Chromium C++ code. + +> Today, we'll call into Rust to do something silly with strings. If you've +> got a corner of the code where you're displaying a UTF8 string to the user, +> feel free to follow this recipe in your part of the codebase instead of +> the exact part we talk about. diff --git a/src/chromium/adding-third-party-crates.md b/src/chromium/adding-third-party-crates.md new file mode 100644 index 000000000000..0dbd24e23f0d --- /dev/null +++ b/src/chromium/adding-third-party-crates.md @@ -0,0 +1,32 @@ +# Adding third party crates + +Rust libraries are called "crates" and are found at [crates.io][0]. It's *very +easy* for Rust crates to depend upon one another. So they do! + +| Property | C++ library | Rust crate | +| --- | --- | --- | +| Build system | Lots | Consistent - `Cargo.toml` | +| Typical library size | Large-ish | Small | +| Transitive dependencies | Few | Lots | + +For a Chromium engineer, this has pros and cons: + +* All crates use a common build system so we can automate their inclusion into + Chromium... +* ... but, crates typically have transitive dependencies, so you will + likely have to bring in multiple libraries. + +We'll discuss: + +* How to put a crate in the Chromium source code tree +* How to make `gn` build rules for it +* How to audit its source code for sufficient safety. + +[0]: https://crates.io + +
+All of the things in the table on this slide are generalizations, and +counter-examples can be found. But in general it's important for students +to understand that most Rust code depends on other Rust libraries, because +it's easy to do so, and that this has both benefits and costs. +
\ No newline at end of file diff --git a/src/chromium/adding-third-party-crates/checking-in.md b/src/chromium/adding-third-party-crates/checking-in.md new file mode 100644 index 000000000000..e9fd9f21804d --- /dev/null +++ b/src/chromium/adding-third-party-crates/checking-in.md @@ -0,0 +1,23 @@ +# Checking crates into Chromium source code + +`git status` should reveal: +* Crate code in `//third_party/rust/chromium_crates_io` +* Metadata (`BUILD.gn` and `README.chromium`) in `//third_party/rust//` + +Please also add an `OWNERS` file in the latter location. + +You should land all this, along with your `Cargo.toml` and `gnrt_config.toml` changes, into +the Chromium repo. + +**Important**: you need to use `git add -f` because otherwise `.gitignore` files +may result in some files being skipped. + +As you do so, you might find presubmit checks fail because of non-inclusive +language. This is because Rust crate data tends to include names of git branches, +and many projects still use non-inclusive terminology there. So you may need +to run: + +```shell +infra/update_inclusive_language_presubmit_exempt_dirs.sh > infra/inclusive_language_presubmit_exempt_dirs.txt +git add -p infra/inclusive_language_presubmit_exempt_dirs.txt # add whatever changes are yours +``` diff --git a/src/chromium/adding-third-party-crates/configuring-cargo-toml.md b/src/chromium/adding-third-party-crates/configuring-cargo-toml.md new file mode 100644 index 000000000000..86aa17649ee4 --- /dev/null +++ b/src/chromium/adding-third-party-crates/configuring-cargo-toml.md @@ -0,0 +1,23 @@ +# Configuring the `Cargo.toml` file to add crates + +Chromium has a single set of centrally-managed direct crate dependencies. +These are managed through a single [`Cargo.toml`][0]: + +```toml +[dependencies] +bitflags = "1" +cfg-if = "1" +cxx = "1" +# lots more... +``` + +As with any other `Cargo.toml`, you can specify [more details about +the dependencies][1] - most commonly, you'll want to specify the `features` that +you wish to enable in the crate. + +When adding a crate to Chromium, you'll often need to provide some extra +information in an additional file, `gnrt_config.toml`, which we'll meet next. + + +[0]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/rust/chromium_crates_io/Cargo.toml +[1]: https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html diff --git a/src/chromium/adding-third-party-crates/configuring-gnrt-config-toml.md b/src/chromium/adding-third-party-crates/configuring-gnrt-config-toml.md new file mode 100644 index 000000000000..6649786f3dc5 --- /dev/null +++ b/src/chromium/adding-third-party-crates/configuring-gnrt-config-toml.md @@ -0,0 +1,29 @@ +# Configuring `gnrt_config.toml` + +Alongside `Cargo.toml` is [`gnrt_config.toml`][0]. This contains Chromium-specific +extensions to crate handling. + +If you add a new crate, you should specify at least the `group`. +This is one of: + +```toml +# 'safe': The library satisfies the rule-of-2 and can be used in any process. +# 'sandbox': The library does not satisfy the rule-of-2 and must be used in +# a sandboxed process such as the renderer or a utility process. +# 'test': The library is only used in tests. +``` + +For instance, + +```toml +[crate.my-new-crate] +group = 'test' # only used in test code +``` + +Depending on the crate source code layout, you may also need to use this +file to specify where its `LICENSE` file(s) can be found. + +Later, we'll see some other things you will need to configure in this file +to resolve problems. + +[0]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/rust/chromium_crates_io/gnrt_config.toml \ No newline at end of file diff --git a/src/chromium/adding-third-party-crates/depending-on-a-crate.md b/src/chromium/adding-third-party-crates/depending-on-a-crate.md new file mode 100644 index 000000000000..a8e3439bcd0b --- /dev/null +++ b/src/chromium/adding-third-party-crates/depending-on-a-crate.md @@ -0,0 +1,23 @@ +# Depending on a crate + +Once you've added a third-party crate and generated build rules, +depending on a crate is simple. Find your `rust_static_library` target, +and add a `dep` on the `:lib` target within your crate. + +Specifically, + +```bob + +------------+ +----------------------+ +"//third_party/rust" | crate name | "/v" | major semver version | "/:lib" + +------------+ +----------------------+ +``` + +For instance, + +```gn +rust_static_library("my_rust_lib") { + crate_root = "lib.rs" + sources = [ "lib.rs" ] + deps = [ "//third_party/rust/example_rust_crate/v1:lib" ] +} +``` \ No newline at end of file diff --git a/src/chromium/adding-third-party-crates/downloading-crates.md b/src/chromium/adding-third-party-crates/downloading-crates.md new file mode 100644 index 000000000000..fdae6ca4ce72 --- /dev/null +++ b/src/chromium/adding-third-party-crates/downloading-crates.md @@ -0,0 +1,30 @@ +# Downloading crates + +A tool called `gnrt` knows how to download crates and how to generate `BUILD.gn` +rules. + +To start, download the crate you want like this: + +```shell +cd chromium/src +vpython3 tools/crates/run_gnrt.py -- vendor +``` + +> Although the `gnrt` tool is part of the Chromium source code, +> by running this command you will be downloading and running its dependencies +> from `crates.io`. See [the earlier section][0] discussing this security +> decision. + +This `vendor` command may download: +* Your crate +* Direct and transitive dependencies +* New versions of other crates, as required by `cargo` to resolve + the complete set of crates required by Chromium. + +If a crate in `//third_party/rust/chromium_crates_io/patches` was updated as +part of vendoring, then reapply patches to it by running +`cd third_party/rust/chromium_crates_io; ./apply_patches.sh`. + + + +[0]: ../cargo.md \ No newline at end of file diff --git a/src/chromium/adding-third-party-crates/generating-gn-build-rules.md b/src/chromium/adding-third-party-crates/generating-gn-build-rules.md new file mode 100644 index 000000000000..9350851d59dc --- /dev/null +++ b/src/chromium/adding-third-party-crates/generating-gn-build-rules.md @@ -0,0 +1,25 @@ +# Generating `gn` build rules + +Once you've downloaded the crate, generate the `BUILD.gn` files like this: + +```shell +vpython3 tools/crates/run_gnrt.py -- gen +``` + +Now run `git status`. You should find: + +* At least one new crate source code in `third_party/rust/chromium_crates_io/vendor` +* At least one new `BUILD.gn` in `third_party/rust//v` +* An appropriate `README.chromium` + +The "major semver version" is a [Rust "semver" version number][0]. + +Take a close look, especially at the things generated in `third_party/rust`. + +
+Talk a little about semver - and specifically the way that in Chromium +it's to allow multiple incompatible versions of a crate, which is discouraged +but sometimes necessary in the cargo ecosystem. + + +[0]: https://doc.rust-lang.org/cargo/reference/semver.html \ No newline at end of file diff --git a/src/chromium/adding-third-party-crates/keeping-up-to-date.md b/src/chromium/adding-third-party-crates/keeping-up-to-date.md new file mode 100644 index 000000000000..92b2c1861a0b --- /dev/null +++ b/src/chromium/adding-third-party-crates/keeping-up-to-date.md @@ -0,0 +1,8 @@ +# Keeping crates up to date + +As the OWNER of any third party Chromium dependency, you are +[expected to keep it up to date with any security fixes][0]. It is hoped +that we will soon automate this for Rust crates, but for now, it's still +your responsibility just as it is for any other third party dependency. + +[0]: https://chromium.googlesource.com/chromium/src/+/main/docs/adding_to_third_party.md#add-owners \ No newline at end of file diff --git a/src/chromium/adding-third-party-crates/resolving-problems.md b/src/chromium/adding-third-party-crates/resolving-problems.md new file mode 100644 index 000000000000..88e23fe2da4f --- /dev/null +++ b/src/chromium/adding-third-party-crates/resolving-problems.md @@ -0,0 +1,21 @@ +# Resolving problems + +If your build fails, it may be because of a `build.rs`: programs which do arbitrary +things at build time. This is fundamentally at odds with the design of `gn` +and `ninja` which aim for static, deterministic, build rules to maximize +parallelism and repeatability of builds. + +Some `build.rs` actions are automatically supported; others require action: + +| build script effect | Supported by our gn templates | Work required by you | +|-----|-----|-----| +| Checking rustc version to configure features on and off | Yes | None | +| Checking platform or CPU to configure features on and off | Yes | None | +| Generating code | Yes | Yes - specify in `gnrt_config.toml` | +| Building C/C++ | No | Patch around it | +| Arbitrary other actions | No | Patch around it | + +Fortunately, most crates don't contain a build script, and fortunately, most +build scripts only do the top two actions. + +[0]: https://doc.rust-lang.org/cargo/reference/build-scripts.html diff --git a/src/chromium/adding-third-party-crates/resolving-problems/build-scripts-which-generate-code.md b/src/chromium/adding-third-party-crates/resolving-problems/build-scripts-which-generate-code.md new file mode 100644 index 000000000000..b59b46da9ae5 --- /dev/null +++ b/src/chromium/adding-third-party-crates/resolving-problems/build-scripts-which-generate-code.md @@ -0,0 +1,22 @@ +# Build scripts which generate code + +If `ninja` complains about missing files, check the `build.rs` to see if it +writes source code files. + +If so, modify [`gnrt_config.toml`][1] to add `build-script-outputs` to the +crate. If this is a transitive dependency, that is, one on which Chromium +code should not directly depend, also add `allow-first-party-usage=false`. +There are several examples already in that file: + +```toml +[crate.unicode-linebreak] +allow-first-party-usage = false +build-script-outputs = [ "tables.rs" ] +``` + +Now rerun [`gnrt.py -- gen`][2] to regenerate `BUILD.gn` files to inform ninja +that this particular output file is input to subsequent build steps. + + +[1]: ../configuring-gnrt-config-toml.md +[2]: ../generating-gn-build-rules.md \ No newline at end of file diff --git a/src/chromium/adding-third-party-crates/resolving-problems/build-scripts-which-take-arbitrary-actions.md b/src/chromium/adding-third-party-crates/resolving-problems/build-scripts-which-take-arbitrary-actions.md new file mode 100644 index 000000000000..cebfe7fc21d3 --- /dev/null +++ b/src/chromium/adding-third-party-crates/resolving-problems/build-scripts-which-take-arbitrary-actions.md @@ -0,0 +1,24 @@ +# Build scripts which build C++ or take arbitrary actions + +Some crates use the [`cc`][2] crate to build and link C/C++ libraries. +Other crates parse C/C++ using [`bindgen`][3] within their build scripts. +These actions can't be supported in a Chromium context - our gn, ninja +and LLVM build system is very specific in expressing relationships between +build actions. + +So, your options are: + +* Avoid these crates +* Apply a patch to the crate. + +Patches should be kept in `third_party/rust/chromium_crates_io/patches/` - +see for example the [patches against the cxx crate][4]. There is currently +no automation - [simply create and apply patches manually][5] to remove the +problematic actions from the build script. + +If your patches modify the `Cargo.toml` file, rerun `gnrt gen`. + +[2]: https://crates.io/crates/cc +[3]: https://crates.io/crates/bindgen +[4]: https://source.chromium.org/chromium/chromium/src/+/main:third_party/rust/chromium_crates_io/patches/cxx/ +[5]: https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/rust.md#patching-third_party-crates diff --git a/src/chromium/adding-third-party-crates/reviews-and-audits.md b/src/chromium/adding-third-party-crates/reviews-and-audits.md new file mode 100644 index 000000000000..e69dde2310b9 --- /dev/null +++ b/src/chromium/adding-third-party-crates/reviews-and-audits.md @@ -0,0 +1,33 @@ +# Auditing third party crates + +Adding new libraries is subject to Chromium's standard [policies][0], but of +course also subject to code review. As you may be bringing in not just a single +crate but also transitive dependencies, there may be a lot of code to review. +On the other hand, safe Rust code can have limited negative side effects. +How should you review it? + +Over time Chromium aims to move to a process based around [cargo vet][1]. + +Meanwhile, for each new crate addition, we are checking for the following: + +* Understand why each crate is used. What's the relationship between crates? + If the build system for each crate contains a `build.rs` or procedural + macros, work out what they're for. Are they compatible with the way + Chromium is normally built? +* Check each crate seems to be reasonably well maintained +* Use `cd third-party/rust/chromium_crates_io; cargo audit` to check for + known vulnerabilities (first you'll need to `cargo install cargo-audit`, + which ironically involves downloading lots of dependencies from the internet[2]) +* Ensure any unsafe code is good enough for the [Rule of Two][3] +* Check for any use of `fs` or `net` APIs +* Read all the code at a sufficient level to look for anything out of place + that might have been maliciously inserted. (You can't realistically aim + for 100% perfection here: there's often just too much code.) + +These are just guidelines - work with reviewers from `security@chromium.org` +to work out the right way to become confident of the crate. + +[0]: https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/rust.md#Third_party-review +[1]: https://mozilla.github.io/cargo-vet/ +[2]: ../cargo.md +[3]: https://chromium.googlesource.com/chromium/src/+/main/docs/security/rule-of-2.md#unsafe-code-in-safe-languages \ No newline at end of file diff --git a/src/chromium/build-rules.md b/src/chromium/build-rules.md new file mode 100644 index 000000000000..e0c31daa54af --- /dev/null +++ b/src/chromium/build-rules.md @@ -0,0 +1,38 @@ +# Build rules + +Rust code is usually built using `cargo`. Chromium builds with `gn` and `ninja` +for efficiency - its static rules allow maximum parallelism. Rust is no exception. + +## Adding Rust code to Chromium + +In some existing Chromium `BUILD.gn` file, declare a `rust_static_library`: + +```gn +import("//build/rust/rust_static_library.gni") + +rust_static_library("my_rust_lib") { + crate_root = "lib.rs" + sources = [ "lib.rs" ] +} +``` + +You can also add `deps` on other Rust targets. Later we'll use this to +depend upon third party code. + +
+You must specify _both_ the crate root, _and_ a full list of sources. +The `crate_root` is the file given to the Rust compiler representing the root +file of the compilation unit - typically `lib.rs`. `sources` is a complete +list of all source files which `ninja` needs in order to determine when rebuilds +are necessary. + +(There's no such thing as a Rust `source_set`, because in Rust, an entire +crate is a compilation unit. A `static_library` is the smallest unit.) + +Students might be wondering why we need a gn template, rather than using +[gn's built-in support for Rust static libraries][0]. +The answer is that this template provides support for cxx interop, Rust features, +and unit tests, some of which we'll use later. +
+ +[0]: https://gn.googlesource.com/gn/+/main/docs/reference.md#func_static_library \ No newline at end of file diff --git a/src/chromium/build-rules/depending.md b/src/chromium/build-rules/depending.md new file mode 100644 index 000000000000..d530576ec801 --- /dev/null +++ b/src/chromium/build-rules/depending.md @@ -0,0 +1,22 @@ +# Depending on Rust code from Chromium C++ + +Simply add the above target to the `deps` of some Chromium C++ target. + +```gn +import("//build/rust/rust_static_library.gni") + +rust_static_library("my_rust_lib") { + crate_root = "lib.rs" + sources = [ "lib.rs" ] +} + +# or source_set, static_library etc. +component("preexisting_cpp") { + deps = [ ":my_rust_lib" ] +} +``` + +
+We'll see that this relationship only works if the Rust code exposes plain C APIs +which can be called from C++, or if we use a C++/Rust interop tool. +
\ No newline at end of file diff --git a/src/chromium/build-rules/unsafe.md b/src/chromium/build-rules/unsafe.md new file mode 100644 index 000000000000..505d9af845d4 --- /dev/null +++ b/src/chromium/build-rules/unsafe.md @@ -0,0 +1,17 @@ +# Including `unsafe` Rust code + +Unsafe Rust code is forbidden in `rust_static_library` by default - it won't +compile. If you need unsafe Rust code, add `allow_unsafe = true` to the +gn target. (Later in the course we'll see circumstances where this is necessary.) + +```gn +import("//build/rust/rust_static_library.gni") + +rust_static_library("my_rust_lib") { + crate_root = "lib.rs" + sources = [ + "lib.rs", + "hippopotamus.rs" + ] + allow_unsafe = true +} \ No newline at end of file diff --git a/src/chromium/build-rules/vscode.md b/src/chromium/build-rules/vscode.md new file mode 100644 index 000000000000..7cc4c9c63e01 --- /dev/null +++ b/src/chromium/build-rules/vscode.md @@ -0,0 +1,17 @@ +# Visual Studio code + +Types are elided in Rust code, which makes a good IDE even more useful than +for C++. Visual Studio code works well for Rust in Chromium. To use it, + +* Ensure your VSCode has the `rust-analyzer` extension, not earlier forms + of Rust support +* `gn gen out/Debug --export-rust-project` (or equivalent for your output + directory) +* `ln -s out/Debug/rust-project.json rust-project.json` + +Example screenshot from VSCode + +
+A demo of some of the code annotation and exploration features of rust-analyzer might be +beneficial if the audience are naturally skeptical of IDEs. +
\ No newline at end of file diff --git a/src/chromium/build-rules/vscode.png b/src/chromium/build-rules/vscode.png new file mode 100644 index 000000000000..4325eadfe36a Binary files /dev/null and b/src/chromium/build-rules/vscode.png differ diff --git a/src/chromium/cargo.md b/src/chromium/cargo.md new file mode 100644 index 000000000000..e5bf6f7b11af --- /dev/null +++ b/src/chromium/cargo.md @@ -0,0 +1,51 @@ +# Using cargo for experimental tools + +Subjectively, + +```bob +High ^ + | x cargo + | +Development | x "cargo --offline" +speed | + | x "gn/ninja" + | "rust_executable(...)" +Low +----------------------------------------------------> + Low Determinism High +``` + +`cargo` works great for pure-Rust tools, but isn't optimized for large multi- +language projects like Chromium. Chromium uses `gn` and `ninja`. + +When writing a tool in Rust, your choices are: + +* Use `gn` and `ninja` (using the `rust_executable` template we'll meet + later) +* Use `cargo`, but [restrict yourself to Chromium's audited toolchain and crates][0] +* Use `cargo`, trusting a [toolchain][1] and [crates downloaded from the internet][2] + +Your organization's policy, and/or common sense, may prohibit you from doing +these things. + +From here on we'll be focusing on `gn` and `ninja`. + +## Mini exercise + +Discuss in small groups the policies within your own team and organization, +and come to a group agreement about what's an acceptable level of risk. + +
+Explain that it might seem strange to write tools in Rust, but this is +increasingly popular across the industry - Rust tools are quicker and work +more reliably. + +Assuming folks taking the course are physically together, ask them to discuss +in small groups of 3-4 people. Then, ask each table whether they've come +to a consensus on the level of risk. + +Later in the course, we'll be running an actual `cargo`-based tool, `gnrt`. +
+ +[0]: https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/rust.md#Using-cargo +[1]: https://rustup.rs/ +[2]: https://crates.io/ \ No newline at end of file diff --git a/src/chromium/interoperability-with-cpp.md b/src/chromium/interoperability-with-cpp.md new file mode 100644 index 000000000000..7367863ba1db --- /dev/null +++ b/src/chromium/interoperability-with-cpp.md @@ -0,0 +1,25 @@ +# Interoperability with C++ + +The Rust community offers multiple options for C++/Rust interop, with new tools +being developed all the time. At the moment, Chromium uses a tool called "cxx". + +You describe your whole language boundary in an interface definition language +(which looks a lot like Rust) and then cxx tools generate declarations for +functions and types in both Rust and C++. + +Overview diagram of cxx, showing that the same interface definition is used to create both C++ and Rust side code which then communicate via a lowest common denominator C API + +See the [CXX tutorial][1] for a full example of using this. + + +[1]: https://cxx.rs/tutorial.html +[2]: https://cxx.rs/bindings.html + + +
+Talk through the diagram. Explain that behind the scenes, this is doing +just the same as you previously did - but by programmatically ensuring that +the C++ and Rust sides match, cxx can ensure there aren't obvious errors +with object lifetimes, string lengths, etc. It reduces lots of fiddly +boilerplate and the resulting code feels more "natural". +
\ No newline at end of file diff --git a/src/chromium/interoperability-with-cpp/error-handling.md b/src/chromium/interoperability-with-cpp/error-handling.md new file mode 100644 index 000000000000..5b13e9807f94 --- /dev/null +++ b/src/chromium/interoperability-with-cpp/error-handling.md @@ -0,0 +1,41 @@ +# cxx error handling + +cxx's support for `Result` relies on C++ exceptions, so we can't use that +in Chromium. Alternatives: + +* Where success can be represented as a simple Boolean, as done in our [QR code generator][1]: + Return a Boolean representing success, and record results using out-parameters: + ```rust,ignore + #[cxx::bridge(namespace = "qr_code_generator")] + mod ffi { + extern "Rust" { + fn generate_qr_code_using_rust( + data: &[u8], + min_version: i16, + out_pixels: Pin<&mut CxxVector>, + out_qr_size: &mut usize, + ) -> bool; + } + } + ``` +* Where success is more complex, provide a Rust + object which can be queried for details of success or failure: + ```rust,ignore + #[cxx::bridge] + mod ffi { + extern "Rust" { + type PngDecoder; + fn create_png_decoder() -> Box; + fn decode(self: &PngDecoder, png: &[u8]) -> bool; // whether successful + fn get_err_code(self: &PngDecoder) -> u32; // or some more complex error type + fn get_decoded_image(self: &PngDecoder) -> &[u8]; + // or some more complex success type + } + } + ``` + + +The best way to learn cxx is by doing, so, another exercise! + +[0]: https://cxx.rs/binding/result.html +[1]: https://source.chromium.org/chromium/chromium/src/+/main:components/qr_code_generator/qr_code_generator_ffi_glue.rs;l=10 \ No newline at end of file diff --git a/src/chromium/interoperability-with-cpp/example-bindings.md b/src/chromium/interoperability-with-cpp/example-bindings.md new file mode 100644 index 000000000000..da49de0499bb --- /dev/null +++ b/src/chromium/interoperability-with-cpp/example-bindings.md @@ -0,0 +1,23 @@ +# Example bindings + +cxx requires you to declare the whole C++/Rust boundary in one of your `.rs` +files. For instance: + +```rust,ignore +{{#include ../../../third_party/cxx/book/snippets.rs:cxx_overview}} +``` + +
+Point out: + +* Native support for C++'s `std::unique_ptr` in Rust +* Native support for Rust slices in C++ +* Calls from C++ to Rust, and Rust types (in the top part) +* Calls from Rust to C++, and C++ types (in the bottom part) +* If the function definitions in C++ or Rust don't match the cxx::bridge, + a compilation failure results. + +**Common misconception**: It _looks_ like a C++ header is being parser by Rust, +but this is misleading. This header is never interpreted by Rust, but simply +`#include`d in the generated C++ code for the benefit of C++ compilers. +
\ No newline at end of file diff --git a/src/chromium/interoperability-with-cpp/limitations-of-cxx.md b/src/chromium/interoperability-with-cpp/limitations-of-cxx.md new file mode 100644 index 000000000000..022086d9de37 --- /dev/null +++ b/src/chromium/interoperability-with-cpp/limitations-of-cxx.md @@ -0,0 +1,31 @@ +## Limitations of cxx + +By far the most useful page when using cxx is the [type reference][1]. + +cxx fundamentally suits cases where: + +* Your Rust-C++ interface is sufficiently simple that you can declare all of it. +* You're using only the types natively supported by cxx already, for example + `std::unique_ptr`, `std::string`, `&[u8]` etc. + +It has many limitations - for example lack of support for Rust's `Option` type. + +These limitations constrain us to using Rust in Chromium only for well isolated +"leaf nodes" rather than for arbitrary Rust-C++ interop. When considering +a use-case for Rust in Chromium, a good starting point is to draft the cxx +bindings for the language boundary to see if it appears simple enough. + + +[1]: https://cxx.rs/bindings.html + +
+In addition, right now, Rust code in one component cannot depend on Rust +code in another, due to linking details in our component build. That's another +reason to restrict Rust to use in leaf nodes. + +You should also discuss some of the other sticky points with cxx, for example: + +* Its error handling is based around C++ exceptions (given on the next slide) +* Function pointers are awkward to use. + +
\ No newline at end of file diff --git a/src/chromium/interoperability-with-cpp/using-cxx-in-chromium.md b/src/chromium/interoperability-with-cpp/using-cxx-in-chromium.md new file mode 100644 index 000000000000..8df5fc4faa4a --- /dev/null +++ b/src/chromium/interoperability-with-cpp/using-cxx-in-chromium.md @@ -0,0 +1,41 @@ +## Using cxx in Chromium + +In Chromium, we define an independent `#[cxx::bridge] mod` for each leaf-node +where we want to use Rust. You'd typically have one for each +`rust_static_library`. Just add + +```gn +cxx_bindings = [ "my_rust_file.rs" ] + # list of files containing #[cxx::bridge], not all source files +allow_unsafe = true +``` + +to your existing `rust_static_library` target alongside `crate_root` and +`sources`. + +C++ headers will be generated at a sensible location, so you can just + +```cpp +#include "ui/base/my_rust_file.rs.h" +``` + +You will find some utility functions in `//base` to convert to/from Chromium +C++ types to cxx Rust types - for example [`SpanToRustSlice`][0]. + +
+Students may ask - why do we still need `allow_unsafe = true`? + +The broad answer is that no C/C++ code is "safe" by the normal Rust standards. +Calling back and forth to C/C++ from Rust may do arbitrary things to memory, and +compromise the safety of Rust's own data layouts. Presence of _too many_ +`unsafe` keywords in C/C++ interop can harm the signal-to-noise ratio of +such a keyword, and is [controversial][1], but strictly, bringing any foreign +code into a Rust binary can cause unexpected behavior from Rust's perspective. + +The narrow answer lies in the diagram at the top of this page - behind the +scenes, cxx generates Rust `unsafe` and `extern "C"` functions just like +we did manually in the previous section. +
+ +[0]: https://source.chromium.org/chromium/chromium/src/+/main:base/containers/span_rust.h;l=21 +[1]: https://steveklabnik.com/writing/the-cxx-debate \ No newline at end of file diff --git a/src/chromium/policy.md b/src/chromium/policy.md new file mode 100644 index 000000000000..6516446b424d --- /dev/null +++ b/src/chromium/policy.md @@ -0,0 +1,37 @@ +# Chromium Rust policy + +Chromium does not yet allow first-party Rust except in rare cases as approved +by Chromium's [Area Tech Leads](https://source.chromium.org/chromium/chromium/src/+/main:ATL_OWNERS). + +Chromium's policy on third party libraries is outlined [here](https://chromium.googlesource.com/chromium/src/+/main/docs/adding_to_third_party.md#rust) - +Rust is allowed for third party libraries under various circumstances, including +if they're the best option for performance or for security. + +Very few Rust libraries directly expose a C/C++ API, so that means that nearly +all such libraries will require a small amount of first-party glue code. + +```bob +C++ Rust +.- - - - - - - - - -. .- - - - - - - - - - - - - - - - - - - - - - -. +: : : : +: Existing Chromium : : Chromium Rust Existing Rust : +: "C++" : : "wrapper" crate : +: +---------------+ : : +----------------+ +-------------+ : +: | | : : | | | | : +: | o-----+-+-----------+-+-> o-+----------+--> | : +: | | : Language : | | Crate | | : +: +---------------+ : boundary : +----------------+ API +-------------+ : +: : : : +`- - - - - - - - - -' `- - - - - - - - - - - - - - - - - - - - - - -' +``` + +> First-party Rust glue code for a particular third-party crate should +> normally be kept in `third_party/rust///wrapper`. + +Because of this, today's course will be heavily focused on: + +* Bringing in third-party Rust libraries ("crates") +* Writing glue code to be able to use those crates from Chromium C++. + +If this policy changes over time, the course will evolve to keep up. + diff --git a/src/chromium/setup.md b/src/chromium/setup.md new file mode 100644 index 000000000000..b9aad9894c41 --- /dev/null +++ b/src/chromium/setup.md @@ -0,0 +1,27 @@ +# Setup + +Make sure you can build and run Chromium. Any platform and set of build flags is +OK, so long as your code is relatively recent (commit position 1223636 onwards, +corresponding to November 2023): + +```shell +gn gen out/Debug +autoninja -C out/Debug chrome +out/Debug/chrome # or on Mac, out/Debug/Chromium.app/Contents/MacOS/Chromium +``` + +(A component, debug build is recommended for quickest iteration time. This +is the default!) + +See [How to build Chromium](https://www.chromium.org/developers/how-tos/get-the-code/) +if you aren't already at that point. Be warned - setting up to build Chromium +takes time. + +It's also recommended that you have Visual Studio code installed. + +# About the exercises + +This part of the course has a series of exercises which build on each other. +We'll be doing them spread throughout the course instead of just at the end. +If you don't have time to complete a certain part, don't worry: you can +catch up in the next slot. \ No newline at end of file diff --git a/src/exercises/chromium/bringing-it-together.md b/src/exercises/chromium/bringing-it-together.md new file mode 100644 index 000000000000..08549304d9ae --- /dev/null +++ b/src/exercises/chromium/bringing-it-together.md @@ -0,0 +1,61 @@ +# Bringing it together - Exercise + +In this exercise, you're going to add a whole new Chromium feature, bringing +together everything you already learned. + +## The brief from Product Management + +A community of pixies has been discovered living in a remote rainforest. +It's important that we get Chromium for Pixies delivered to them as soon +as possible. + +The requirement is to translate all Chromium's UI strings into Pixie language. + +There's not time to wait for proper translations, but fortunately pixie +language is very close to English, and it turns out there's a Rust crate +which does the translation. + +In fact, you already [imported that crate in the previous exercise][0]. + +(Obviously, real translations of Chrome require incredible care and +diligence. Don't ship this!) + +## Steps + +Modify `ResourceBundle::MaybeMangleLocalizedString` so that it uwuifies +all strings before display. In this special build of Chromium, it should +always do this irrespective of the setting of `mangle_localized_strings_`. + +If you've done everything right across all these exercises, congratulations, +you should have created Chrome for pixies! + +Chromium UI screenshot with uwu language + +
+Students will likely need some hints here. Hints include: + +* UTF16 vs UTF8. Students should be aware that Rust strings are always + UTF8, and will probably decide that it's better to do the conversion + on the C++ side using `base::UTF16ToUTF8` and back again. +* If students decide to do the conversion on the Rust side, they'll need to + consider [`std::string::from_utf16`][1], consider error handling, and + consider which [cxx supported types can transfer a lot of u16s][2]. +* Students may design the C++/Rust boundary in several different ways, + e.g. taking and returning strings by value, or taking a mutable reference + to a string. If a mutable reference is used, cxx will likely + tell the student that they need to use [`Pin`][3]. You may need to explain + what `Pin` does, and then explain why `cxx` needs it for mutable references + to C++ data: the answer is that C++ data can't be moved around like Rust + data, because it may contain self-referential pointers. +* The C++ target containing `ResourceBundle::MaybeMangleLocalizedString` + will need to depend on a `rust_static_library` target. The student + probably already did this. +* The `rust_static_library` target will need to depend on + `//third_party/rust/uwuify/v0_2:lib`. + +
+ +[0]: https://crates.io/crates/uwuify +[1]: https://doc.rust-lang.org/std/string/struct.String.html#method.from_utf16 +[2]: https://cxx.rs/binding/slice.html +[3]: https://doc.rust-lang.org/std/pin/ diff --git a/src/exercises/chromium/build-rules.md b/src/exercises/chromium/build-rules.md new file mode 100644 index 000000000000..2863cba6ecde --- /dev/null +++ b/src/exercises/chromium/build-rules.md @@ -0,0 +1,58 @@ +# Build rules exercise + +In your Chromium build, add a new Rust target to `//ui/base/BUILD.gn` containing: + +```rust +#[no_mangle] +pub extern "C" fn hello_from_rust() { + println!("Hello from Rust!") +} +``` +**Important**: note that `no_mangle` here is considered a type of unsafety +by the Rust compiler, so you'll need to to allow unsafe code in your +`gn` target. + +Add this new Rust target as a dependency of `//ui/base:base`. +Declare this function at the top of `ui/base/resource/resource_bundle.cc` +(later, we'll see how this can be automated by bindings generation tools): +```cpp +extern "C" void hello_from_rust(); +``` + +Call this function from somewhere in `ui/base/resource/resource_bundle.cc` - +we suggest the top of `ResourceBundle::MaybeMangleLocalizedString`. +Build and run Chromium, and ensure that "Hello from Rust!" is printed lots of times. + +If you use VSCode, now set up Rust to work well in VSCode. It will be useful +in subsequent exercises. If you've succeeded, you will be able to use +right-click "Go to definition" on `println!`. + +## Where to find help + +* The options available to the [`rust_static_library` gn template][0] +* Information about [`#[no_mangle]`][1] +* Information about [`extern "C"`][2] +* Information about gn's [`--export-rust-project`][3] switch +* [How to install rust-analyzer in VSCode][4] + +
+It's really important that students get this running, because future exercises +will build on it. + +This example is unusual because it boils down to the lowest-common-denominator +interop language, C. Both C++ and Rust can natively declare and call C ABI +functions. Later in the course, we'll connect C++ directly to Rust. + +`allow_unsafe = true` is required here because `#[no_mangle]` might allow Rust +to generate two functions with the same name, and Rust can no longer guarantee +that the right one is called. + +If you need a pure Rust executable, you can also do that using the +`rust_executable` gn template. +
+ +[0]: https://source.chromium.org/chromium/chromium/src/+/main:build/rust/rust_static_library.gni;l=16 +[1]: https://doc.rust-lang.org/beta/reference/abi.html#the-no_mangle-attribute +[2]: https://doc.rust-lang.org/std/keyword.extern.html +[3]: https://gn.googlesource.com/gn/+/main/docs/reference.md#compilation-database +[4]: https://code.visualstudio.com/docs/languages/rust \ No newline at end of file diff --git a/src/exercises/chromium/chwomium.png b/src/exercises/chromium/chwomium.png new file mode 100644 index 000000000000..00e708c64f51 Binary files /dev/null and b/src/exercises/chromium/chwomium.png differ diff --git a/src/exercises/chromium/interoperability-with-cpp.md b/src/exercises/chromium/interoperability-with-cpp.md new file mode 100644 index 000000000000..5c8394d0a8be --- /dev/null +++ b/src/exercises/chromium/interoperability-with-cpp.md @@ -0,0 +1,71 @@ +# Exercise: Interoperability with C++ + +## Part one + +* In the Rust file you previously created, add a `#[cxx::bridge]` which specifies a single function, + to be called from C++, called `hello_from_rust`, taking no parameters and returning + no value. +* Modify your previous `hello_from_rust` function to remove `extern "C"` and `#[no_mangle]`. + This is now just a standard Rust function. +* Modify your `gn` target to build these bindings. +* In your C++ code, remove the forward-declaration of `hello_from_rust`. Instead, include + the generated header file. +* Build and run! + +## Part two + +It's a good idea to play with cxx a little. It helps you think about how flexible +Rust in Chromium actually is. + +Some things to try: + +* Call back into C++ from Rust. You will need: + * An additional header file which you can `include!` from your `cxx::bridge`. + You'll need to declare your C++ function in that new header file. + * An `unsafe` block to call such a function, or alternatively specify the `unsafe` + keyword in your `#[cxx::bridge]` [as described here][0]. + * You may also need to `#include "third_party/rust/cxx/v1/crate/include/cxx.h"` +* Pass a C++ string from C++ into Rust. +* Pass a reference to a C++ object into Rust. +* Intentionally get the Rust function signatures mismatched from the `#[cxx::bridge]`, + and get used to the errors you see. +* Intentionally get the C++ function signatures mismatched from the `#[cxx::bridge]`, + and get used to the errors you see. +* Pass a `std::unique_ptr` of some type from C++ into Rust, so that Rust + can own some C++ object. +* Create a Rust object and pass it into C++, so that C++ owns it. (Hint: + you need a `Box`). +* Declare some methods on a C++ type. Call them from Rust. +* Declare some methods on a Rust type. Call them from C++. + +## Part three + +Now you understand the strengths and limitations of cxx interop, think of +a couple of use-cases for Rust in Chromium where the interface would be +sufficiently simple. Sketch how you might define that interface. + +## Where to find help + +* The [cxx binding reference][1] +* The [`rust_static_library` gn template][2] + +
+As students explore Part Two, they're bound to have lots of questions about how +to achieve these things, and also how cxx works behind the scenes. + +Some of the questions you may encounter: +* I'm seeing a problem initializing a variable of type X with type Y, where + X and Y are both function types. + This is because your C++ function doesn't quite match the declaration in your + `cxx::bridge`. +* I seem to be able to freely convert C++ references into Rust references. + Doesn't that risk UB? + For cxx's _opaque_ types, no, because they are zero-sized. For cxx trivial types + yes, it's _possible_ to cause UB, although cxx's design makes it quite + difficult to craft such an example. +
+ + +[0]: https://cxx.rs/extern-c++.html#functions-and-member-functions +[1]: https://cxx.rs/bindings.html +[2]: https://source.chromium.org/chromium/chromium/src/+/main:build/rust/rust_static_library.gni;l=16 \ No newline at end of file diff --git a/src/exercises/chromium/third-party.md b/src/exercises/chromium/third-party.md new file mode 100644 index 000000000000..57221ccc1cd1 --- /dev/null +++ b/src/exercises/chromium/third-party.md @@ -0,0 +1,22 @@ +# Exercise + +Add [uwuify][0] to Chromium, turning off the crate's [default features][1]. +Assume that the crate will be used in shipping Chromium, but won't be used +to handle untrustworthy input. + +(In the next exercise we'll use uwuify from Chromium, but feel free to +skip ahead and do that now if you like. Or, you could create a new +[`rust_executable` target][2] which uses `uwuify`). + +
+Students will need to download lots of transitive dependencies. + +The total crates needed are: uwuify, smallvec, scopeguard, parking_lot, +parking_lot_core, lock_api and instant. If students are downloading even +more than that, they probably forgot to turn off the default features. + + + +[0]: https://crates.io/crates/uwuify +[1]: https://doc.rust-lang.org/cargo/reference/features.html#the-default-feature +[2]: https://source.chromium.org/chromium/chromium/src/+/main:build/rust/rust_executable.gni \ No newline at end of file diff --git a/src/glossary.md b/src/glossary.md index 62a184ce46cc..203c33299bdd 100644 --- a/src/glossary.md +++ b/src/glossary.md @@ -127,6 +127,8 @@ Also, please keep the hard line breaks to ensure a nice formatting. Days 1 to 3 of this course. - Rust in Android:\ See [Rust in Android](android.md). +- Rust in Chromium:\ + See [Rust in Chromium](chromium.md). - safe:\ Refers to code that adheres to Rust's ownership and borrowing rules, preventing memory-related errors. - scope:\ diff --git a/src/index.md b/src/index.md index 727d2b89571d..bcff48cb72bd 100644 --- a/src/index.md +++ b/src/index.md @@ -25,6 +25,9 @@ Building on this, you're invited to dive into one or more specialized topics: * [Android](android.md): a half-day course on using Rust for Android platform development (AOSP). This includes interoperability with C, C++, and Java. +* [Chromium](chromium.md): a half-day course on using Rust within Chromium + based browsers. This includes interoperability with C++ and how to include + third-party crates in Chromium. * [Bare-metal](bare-metal.md): a whole-day class on using Rust for bare-metal (embedded) development. Both microcontrollers and application processors are covered. diff --git a/src/running-the-course/course-structure.md b/src/running-the-course/course-structure.md index c73e9963c049..299093f0dcb1 100644 --- a/src/running-the-course/course-structure.md +++ b/src/running-the-course/course-structure.md @@ -32,6 +32,17 @@ commands it runs and make sure they work when you run them by hand. [1]: https://source.android.com/docs/setup/download/downloading [2]: https://github.com/google/comprehensive-rust +### Rust in Chromium + +The [Rust in Chromium](../chromium.md) deep dive is a half-day course on using +Rust as part of the Chromium browser. It includes using Rust in Chromium's +`gn` build system, bringing in third-party libraries ("crates") and C++ +interoperability. + +You will need to be able to build Chromium - a debug, component build is +[recommended](../chromium/setup.md) for speed but any build will work. +Ensure that you can run the Chromium browser that you've built. + ### Bare-Metal Rust The [Bare-Metal Rust](../bare-metal.md) deep dive is a full day class on using Rust for diff --git a/third_party/cxx/book/snippets.rs b/third_party/cxx/book/snippets.rs index 8b7fd0a39541..a54e4c72e6e9 100644 --- a/third_party/cxx/book/snippets.rs +++ b/third_party/cxx/book/snippets.rs @@ -119,3 +119,25 @@ fn main() { } } // ANCHOR_END: cpp_exception + +// ANCHOR: cxx_overview +#[cxx::bridge] +mod ffi { + extern "Rust" { + type MultiBuf; + + fn next_chunk(buf: &mut MultiBuf) -> &[u8]; + } + + unsafe extern "C++" { + include!("example/include/blobstore.h"); + + type BlobstoreClient; + + fn new_blobstore_client() -> UniquePtr; + fn put(self: &BlobstoreClient, buf: &mut MultiBuf) -> Result; + } +} + +// Definitions of Rust types and functions go here +// ANCHOR_END: cxx_overview \ No newline at end of file