Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Cargo command for cross-compiling and packaging buildpacks #199

Merged
merged 41 commits into from
Dec 3, 2021
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
28044b5
Add libcnb-cargo crate
Malax Nov 23, 2021
486a1f7
Add initial libcnb-cargo implementation
Malax Nov 23, 2021
0fd7bfb
Update CHANGELOG
Malax Nov 24, 2021
6546622
Remove custom packaging config from Ruby example
Malax Nov 24, 2021
e86a02b
Add quick start guide to README
Malax Nov 25, 2021
b57db40
Fix documentation issues
Malax Nov 25, 2021
9d4205a
Update Cargo.toml
Malax Nov 25, 2021
7ce14ee
Update README.md
Malax Nov 25, 2021
7c7bfca
Merge remote-tracking branch 'origin/main' into malax/cargo
Malax Nov 26, 2021
6579f2b
Fix Cargo manifest for publishing
Malax Nov 26, 2021
02bb430
Fix docs.rs link in README.md
Malax Nov 26, 2021
a2d5c52
Fix pack command in quick-start guide
Malax Nov 26, 2021
e50fd75
Fix minor doc issues
Malax Nov 29, 2021
f1b1b87
Enable clippy::pedantic, fix Clippy warnings
Malax Nov 29, 2021
961bc29
Remove unused serde and tempfile dependency
Malax Nov 30, 2021
248451d
Use SubCommand instead of App for subcommands
Malax Nov 30, 2021
1f5de4f
Improve logging output
Malax Nov 30, 2021
0ee8d90
Replace -b with --buildpack in pack hint
Malax Nov 30, 2021
e863d50
Improve Clap config
Malax Nov 30, 2021
8226d4f
Add dedicated README for libcnb-cargo
Malax Nov 30, 2021
91edbdb
Introduce extra variable for readability
Malax Nov 30, 2021
0e4e8e9
Cleanup stderrlog setup
Malax Nov 30, 2021
53ef3c6
Improve error output while packaging
Malax Nov 30, 2021
1e8e24a
Improve error-handling for cargo metadata
Malax Nov 30, 2021
66d27b5
Move setup_cli_parsing into cli.rs
Malax Nov 30, 2021
7f52227
Replace tar file assembly with directory construction
Malax Nov 30, 2021
1ada397
Add newlines after markdown headings
Malax Dec 1, 2021
f53b0ed
Simplify cross_compile#cross_compile_env
Malax Dec 1, 2021
6631880
Update README.md
Malax Dec 1, 2021
97cfb52
Update cross compile help texts
Malax Dec 1, 2021
96fc202
Update README.md
Malax Dec 1, 2021
81d9a61
Add link to blog post about macOS cross-compile config
Malax Dec 1, 2021
3301d24
Use dedicated directories for dev and release profiles
Malax Dec 1, 2021
8344f9a
Update README with updated libcnb-cargo output
Malax Dec 1, 2021
03b0830
Fix error message
Malax Dec 1, 2021
7f35903
Remove custom output path feature
Malax Dec 2, 2021
80d2cc2
Remove buildpack directory before assembling
Malax Dec 2, 2021
fb16494
Refactor cross-compile assistance
Malax Dec 2, 2021
e080fc8
Fix typo
Malax Dec 2, 2021
5326552
Remove version from buildpack output path
Malax Dec 2, 2021
a391c72
Merge remote-tracking branch 'origin/main' into malax/cargo
Malax Dec 2, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,5 +49,6 @@
- `BuildpackTomlError` has been split into `BuildpackApiError` and `StackError`.
- `BuildpackApi` no longer implements `FromStr`, use `BuildpackApi::try_from()` instead.
- Fixed file extension for delimiters when writing `LayerEnv` to disk.
- Add an external Cargo command for packaging libcnb buildpacks. See the README for usage.

## [0.3.0] 2021/09/17
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
[workspace]
members = [
"libcnb",
"libcnb-cargo",
"libcnb-data",
"libcnb-proc-macros",
"examples/example-01-basics",
Expand Down
211 changes: 182 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,79 +13,232 @@

It currently targets version `0.6` of the CNB spec.

## Quick Start Guide
Malax marked this conversation as resolved.
Show resolved Hide resolved

## Installation
Add the following to your `Cargo.toml` file:
This quick start guide walks you through writing a simple Cloud Native Buildpack in Rust with libcnb.rs. It will also
demonstrate how to package your buildpack and how to build an application image with it.

### Development Environment Setup
Malax marked this conversation as resolved.
Show resolved Hide resolved

In addition to the libcnb crate, we need some tools to compile, package and run the buildpack. These steps only need to
be carried out once and don't need to be repeated for each buildpack you will write.
Malax marked this conversation as resolved.
Show resolved Hide resolved

#### libcnb Cargo Command
Malax marked this conversation as resolved.
Show resolved Hide resolved
Malax marked this conversation as resolved.
Show resolved Hide resolved

Start by installing the libcnb Cargo command which we will later use to package our buildpack:

```shell
$ cargo install libcnb-cargo
```

#### Cross-compilation prerequisites
Malax marked this conversation as resolved.
Show resolved Hide resolved

It is common to write and build your buildpack on a platform that is different from the platform the buildpack will
eventually run on. This means we have to cross-compile our buildpack. The `libcnb package` Cargo command tries to help you setting
up your environment depending on your host platform, but we always need the appropriate target platform for Rust which
we can install with `rustup`:

```shell
$ rustup target add x86_64-unknown-linux-musl
```

#### Docker
Malax marked this conversation as resolved.
Show resolved Hide resolved

If you don't have it already, we need to install Docker. Refer to the Docker documentation on how to install it for your
operating system: https://docs.docker.com/engine/install/

#### pack

To run our buildpack locally, we will use `pack`, a tool maintained by the Cloud Native Buildpacks project to support
the use of buildpacks. It's the tool that we will eventually use to run our buildpack and create application images.
Find their documentation about installing it here: https://buildpacks.io/docs/tools/pack/

### Project Setup

After we've successfully set up our development environment, we can move on and create a new Rust project for our
buildpack. First, we create a new binary crate with Cargo:

```shell
$ cargo new my-buildpack
Created binary (application) `my-buildpack` package
```

Then, modify your project's `Cargo.toml` to include the `libcnb` dependency:

```toml
[package]
name = "my-buildpack"
version = "0.1.0"
edition = "2021"

[dependencies]
libcnb = "0.5.0"
libcnb = "0.4.0"
```

*Compiler support requires rustc 1.56+ for 2021 edition*
Since we're writing a Cloud Native Buildpack, we also need a `buildpack.toml`. Use the template below and write it to a
file named `buildpack.toml` in the root of your project, right next to `Cargo.toml`.

## Usage
View the [examples](./examples) for some buildpack samples.
```toml
api = "0.6"

All spec data files are implemented in the [`libcnb-data`](https://docs.rs/libcnb-data) crate and
can be used without the framework to implement Cloud Native Buildpacks tooling in Rust.
[buildpack]
id = "libcnb-examples/my-buildpack"
version = "0.1.0"
name = "My Buildpack"

### Hello World Buildpack
[[stacks]]
id = "heroku-20"
```

A basic hello world buildpack looks like this:
That's all we need! We can now move on to finally write some buildpack code!


### Writing the Buildpack
As aforementioned, the buildpack we're writing will be very simple. We will just log a "Hello World" message during the build
and set the default process type to a command that will also emit "Hello World" when the application image is run.
Find more complex example buildpacks in the [examples directory](examples).

Modify the project's `src/main.rs` file to contain the following:

```rust,no_run
use libcnb::build::{BuildContext, BuildResult, BuildResultBuilder};
use libcnb::data::launch::{Launch, Process};
use libcnb::data::process_type;
use libcnb::detect::{DetectContext, DetectResult, DetectResultBuilder};
use libcnb::generic::{GenericError, GenericMetadata, GenericPlatform};
use libcnb::{buildpack_main, Buildpack};

struct HelloWorldBuildpack;

impl Buildpack for HelloWorldBuildpack {
// The CNB platform this buildpack targets, usually GenericPlatform. See the CNB spec for
// more information about platforms:
// The CNB platform this buildpack targets, usually `GenericPlatform`. See the CNB spec for
// more information about platforms:
// https://github.com/buildpacks/spec/blob/main/buildpack.md
type Platform = GenericPlatform;

// The type for the metadata of the buildpack itself. This is the data found in the
// `[metadata]` section of your buildpack's buildpack.toml. The framework will automatically
// try to parse it into the specified type. This example buildpack uses GenericMetadata which
// The type for the metadata of the buildpack itself. This is the data found in the
// `[metadata]` section of your buildpack's `buildpack.toml`. The framework will automatically
// try to parse it into the specified type. This example buildpack uses GenericMetadata which
// provides low-level access to the TOML table.
type Metadata = GenericMetadata;

// The error type for this buildpack. Buildpack authors usually implement an enum with
// specific errors that can happen during buildpack execution. This error type should
// only contain error specific to this buildpack, such as CouldNotExecuteMaven or
// InvalidGemfileLock. This example buildpack uses GenericError which means this buildpack
// The error type for this buildpack. Buildpack authors usually implement an enum with
// specific errors that can happen during buildpack execution. This error type should
// only contain error specific to this buildpack, such as `CouldNotExecuteMaven` or
// `InvalidGemfileLock`. This example buildpack uses `GenericError` which means this buildpack
// does not specify any errors.
//
// More generic errors that happen during buildpack execution such as I/O errors while
// Common errors that happen during buildpack execution such as I/O errors while
// writing CNB TOML files are handled by libcnb.rs itself.
type Error = GenericError;

// This method will be called when the CNB lifecycle calls detect. Use the DetectContext to
// access CNB data such as the stack this buildpack is currently executed on, the app
// directory and similar things. When using libcnb.rs, you never have to read environment
// This method will be called when the CNB lifecycle calls detect. Use the `DetectContext` to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// This method will be called when the CNB lifecycle calls detect. Use the `DetectContext` to
// This method will be called when the CNB lifecycle calls `bin/detect`. Use the `DetectContext` to

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you open a dedicated PR for this if you feel strongly? This comment wasn't added in this PR and I'd like to keep this as clean as possible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// access CNB data such as the stack this buildpack is currently executed on, the app
// directory and similar things. When using libcnb.rs, you never have to read environment
// variables or read/write files to disk to interact with the CNB lifecycle.
//
// One example of this is the return type of this method. DetectResult encapsulates the
// required exit code as well as the data written to the build plan. libcnb.rs will,
// according to the returned value, handle both writing the build plan and exiting with
// One example of this is the return type of this method. `DetectResult` encapsulates the
// required exit code as well as the data written to the build plan. libcnb.rs will,
// according to the returned value, handle both writing the build plan and exiting with
// the correct status code for you.
fn detect(&self, context: DetectContext<Self>) -> libcnb::Result<DetectResult, Self::Error> {
fn detect(&self, _context: DetectContext<Self>) -> libcnb::Result<DetectResult, Self::Error> {
DetectResultBuilder::pass().build()
}

// Similar to detect, this method will be called when the CNB lifecycle executes the
// Similar to detect, this method will be called when the CNB lifecycle executes the
// build phase.
Malax marked this conversation as resolved.
Show resolved Hide resolved
fn build(&self, context: BuildContext<Self>) -> libcnb::Result<BuildResult, Self::Error> {
println!("Hello World!");
println!("Build runs on stack {}!", context.stack_id);
BuildResultBuilder::new().build()

BuildResultBuilder::new()
.launch(Launch::new().process(Process::new(
process_type!("web"),
"echo",
vec!["Hello World!"],
false,
true,
schneems marked this conversation as resolved.
Show resolved Hide resolved
)))
.build()
}
}

// Implements the main function and wires up the framework for the given buildpack.
buildpack_main!(HelloWorldBuildpack);
```

### Packaging the Buildpack
Malax marked this conversation as resolved.
Show resolved Hide resolved

Now that our buildpack is written, it's time to package it, so that it can be run. If you followed the steps to setup
your development environment, you have access to the `libcnb` Cargo command that will handle packaging for you.

In your project directory, run `cargo libcnb package` to start packaging:

```shell
$ cargo libcnb package
INFO - Reading buildpack metadata...
INFO - Found buildpack libcnb-examples/my-buildpack with version 0.1.0.
INFO - Building buildpack binary (x86_64-unknown-linux-musl)...
Compiling my-buildpack v0.1.0 (/Users/manuel.fuchs/projects/my-buildpack)
# Omitting compilation output...
Finished dev [unoptimized + debuginfo] target(s) in 2.67s
INFO - Writing buildpack directory...
INFO - Successfully wrote buildpack directory: target/debug/libcnb-examples_my-buildpack_0.1.0 (53.1M)
INFO - Packaging successfully finished!
INFO - Hint: To test your buildpack locally with pack, run: pack build my-image --buildpack target/debug/libcnb-examples_my-buildpack_0.1.0 --path /path/to/application
```

If you get errors with hints about how to install required tools to cross-compile from your host platform to the
target platform, follow them and try again. These differ from platform to platform, which is why they're not part of this
quick start guide.

### Running the Buildpack
Malax marked this conversation as resolved.
Show resolved Hide resolved

You might have seen in the output that we're now ready to run our buildpack locally
with `pack`. Before we can do this, we need an application to build. Since our buildpack does not interact with the
application code at all, we just create an empty directory and use that as our application:

```shell
$ mkdir bogus-app
$ pack build my-image --buildpack target/debug/libcnb-examples_my-buildpack_0.1.0 --path bogus-app --builder heroku/buildpacks:20
20: Pulling from heroku/buildpacks
Digest: sha256:04e8ea7a1f482f289d432d9518edcfaaf9f3a10432cd1b624e58225f22e7c416
Status: Image is up to date for heroku/buildpacks:20
20: Pulling from heroku/pack
Digest: sha256:21ea4b85bc937b47e017bf43136a5295ec08e093e3c210b20e69e9c1b0f2bd57
Status: Image is up to date for heroku/pack:20
===> DETECTING
libcnb-examples/my-buildpack 0.1.0
===> ANALYZING
Skipping buildpack layer analysis
===> BUILDING
Hello World!
Build runs on stack heroku-20!
===> EXPORTING
Reusing 1/1 app layer(s)
Reusing layer 'launcher'
Reusing layer 'config'
Reusing layer 'process-types'
Adding label 'io.buildpacks.lifecycle.metadata'
Adding label 'io.buildpacks.build.metadata'
Adding label 'io.buildpacks.project.metadata'
Setting default process type 'web'
Saving my-image...
*** Images (d4f67a828236):
my-image
Successfully built image my-image
```

### Running the image
The newly created Docker image can be run in the same way as you would a Docker image created via `docker build`.
If all went well, you should see our "Hello World!" message in your terminal:

```shell
$ docker run my-image
Hello World!
```

### Next Steps
While the buildpack we've written in this quick start guide is not very useful, it can serve as a starting point for a
more useful buildpack. To discover more of the libcnb API, browse the [examples directory](example) and the
[documentation on docs.rs][docs.rs].
105 changes: 0 additions & 105 deletions examples/example-02-ruby-sample/Makefile.toml

This file was deleted.

2 changes: 0 additions & 2 deletions examples/example-02-ruby-sample/package.toml

This file was deleted.

Loading