From d17070390c992e0aa584efde072bd9f0442cf855 Mon Sep 17 00:00:00 2001 From: Marek Kubica Date: Mon, 23 Sep 2024 16:59:38 +0200 Subject: [PATCH] doc: `dune pkg` tutorial (#10920) Signed-off-by: Marek Kubica --- .../dune-package-management/dependencies.md | 140 ++++++++++++++++++ .../dependencies/constraints | 8 + .../dune-package-management/dependencies/dune | 3 + .../dependencies/dune-project | 8 + .../dependencies/test.ml | 5 + .../dune-package-management/index.md | 27 ++++ .../dune-package-management/pinning.md | 50 +++++++ .../pinning/dune-project | 12 ++ .../dune-package-management/repos.md | 52 +++++++ .../repos/dune-workspace | 8 + .../dune-package-management/setup.md | 109 ++++++++++++++ .../dune-package-management/setup/dune | 2 + .../setup/dune-project | 7 + .../dune-package-management/setup/test.ml | 5 + doc/tutorials/index.md | 1 + 15 files changed, 437 insertions(+) create mode 100644 doc/tutorials/dune-package-management/dependencies.md create mode 100644 doc/tutorials/dune-package-management/dependencies/constraints create mode 100644 doc/tutorials/dune-package-management/dependencies/dune create mode 100644 doc/tutorials/dune-package-management/dependencies/dune-project create mode 100644 doc/tutorials/dune-package-management/dependencies/test.ml create mode 100644 doc/tutorials/dune-package-management/index.md create mode 100644 doc/tutorials/dune-package-management/pinning.md create mode 100644 doc/tutorials/dune-package-management/pinning/dune-project create mode 100644 doc/tutorials/dune-package-management/repos.md create mode 100644 doc/tutorials/dune-package-management/repos/dune-workspace create mode 100644 doc/tutorials/dune-package-management/setup.md create mode 100644 doc/tutorials/dune-package-management/setup/dune create mode 100644 doc/tutorials/dune-package-management/setup/dune-project create mode 100644 doc/tutorials/dune-package-management/setup/test.ml diff --git a/doc/tutorials/dune-package-management/dependencies.md b/doc/tutorials/dune-package-management/dependencies.md new file mode 100644 index 00000000000..30cba77c47e --- /dev/null +++ b/doc/tutorials/dune-package-management/dependencies.md @@ -0,0 +1,140 @@ +# Managing Dependencies + +The OCaml ecosystem has a wealth of third-party packages that are available for +use. In this section we will look into how to use them with Dune. + +## Adding Dependencies + +Much like in regular projects, to add a library we need to add a dependency to +it. For simplicity we will use the popular `fmt` library as an example, but any +package from the [package repository](https://ocaml.org/packages) can be used. + +First we update the `dune-project` file to add a dependeny on the opam package. + +::::{dropdown} `dune-project` +:icon: file-code + +:::{literalinclude} dependencies/dune-project +:language: dune +:emphasize-lines: 8 +::: + +:::: + +After this change to our project dependencies, we need to relock dependencies +to update our lock directory with the new packages. + +```sh +$ dune pkg lock +Solution for dune.lock: +- base-unix.base +- fmt.0.9.0 +- ocaml.5.2.0 +- ocaml-base-compiler.5.2.0 +- ocaml-config.3 +- ocamlbuild.0.15.0+dune +- ocamlfind.1.9.6+dune +- topkg.1.0.7 +``` + +You can see a lot of new dependencies, among these `fmt`. + +:::{note} +The list of packages being output includes all dependencies of your project, +including transitive dependencies. +::: + +This will take care of installing the dependencies, but we still need to add it to +our build as a library, as usual: + +::::{dropdown} `dune` +:icon: file-code + +:::{literalinclude} dependencies/dune +:language: dune +:emphasize-lines: 3 +::: + +Adding a library dependency to our `dune` file via the `libraries` stanza. + +:::: + +This will allow us to use the `Fmt` module in our OCaml code. + +::::{dropdown} `test.ml` +:icon: file-code + +:::{literalinclude} dependencies/test.ml +:language: ocaml +:emphasize-lines: 4 +::: + +We update the code to define an `Fmt.t` pretty-printer for the list of strings +and then use it to print the value. + +:::: + +To build it we just call `build` again. + +```sh +$ dune build +``` + +which will download and install the new dependencies and build our project as +before. + +As we see, the code works and uses `fmt` to do the pretty-printing: + +```sh +$ dune exec ./test.exe +Hello, OCaml, Rust! +``` + +### Dependency Constraints + +Packages are often only compatible with some versions of dependencies. To +specify a version range, use the regular Dune dependency syntax +used for opam dependencies in the `dune-project` file. + +::::{dropdown} `dune-project` +:icon: file-code + +:::{literalinclude} dependencies/constraints +:language: dune +:emphasize-lines: 7-8 +::: + +:::: + +This ensures the `fmt` package to install will be compatible with +our request. These constraints will be taken into account the next time the +package is locked: + +```sh +$ dune pkg lock +Solution for dune.lock: +- base-unix.base +- fmt.0.9.0 +- ocaml.5.2.0 +- ocaml-base-compiler.5.2.0 +- ocaml-config.3 +- ocamlbuild.0.15.0+dune +- ocamlfind.1.9.6+dune +- topkg.1.0.7 +``` + +The version of `fmt` picked is indeed between `0.6` and `1.0`. + +## Removing Dependencies + +Given all dependencies are defined in the `dune-project` file, removing a +dependency means to remove the dependency from the `depends` field of your +`dune-project` and relocking the project. + +The new lock directory will not depend on the package anymore, and in future +builds, the package will not be accessible as `library` anymore. + +:::{note} +The removed dependency might still be part of the lock directory if some other +dependency of your project depends on it. +::: diff --git a/doc/tutorials/dune-package-management/dependencies/constraints b/doc/tutorials/dune-package-management/dependencies/constraints new file mode 100644 index 00000000000..8457e2fa084 --- /dev/null +++ b/doc/tutorials/dune-package-management/dependencies/constraints @@ -0,0 +1,8 @@ +(lang dune 3.17) +(name test) + +(package + (name test) + (depends + (ocaml (>= 4.14)) + (fmt (and (>= 0.6) (< 1.0))))) diff --git a/doc/tutorials/dune-package-management/dependencies/dune b/doc/tutorials/dune-package-management/dependencies/dune new file mode 100644 index 00000000000..eb7ff88832b --- /dev/null +++ b/doc/tutorials/dune-package-management/dependencies/dune @@ -0,0 +1,3 @@ +(executable + (public_name test) + (libraries fmt)) diff --git a/doc/tutorials/dune-package-management/dependencies/dune-project b/doc/tutorials/dune-package-management/dependencies/dune-project new file mode 100644 index 00000000000..9dd0e2eddce --- /dev/null +++ b/doc/tutorials/dune-package-management/dependencies/dune-project @@ -0,0 +1,8 @@ +(lang dune 3.17) +(name test) + +(package + (name test) + (depends + (ocaml (>= 4.14)) + fmt)) diff --git a/doc/tutorials/dune-package-management/dependencies/test.ml b/doc/tutorials/dune-package-management/dependencies/test.ml new file mode 100644 index 00000000000..1583eb1d32f --- /dev/null +++ b/doc/tutorials/dune-package-management/dependencies/test.ml @@ -0,0 +1,5 @@ +let langs = ["OCaml"; "Rust"] + +let () = + let pp_langs = Fmt.(list ~sep:(any ", ") string) in + Format.printf "Hello, %a!\n" pp_langs langs diff --git a/doc/tutorials/dune-package-management/index.md b/doc/tutorials/dune-package-management/index.md new file mode 100644 index 00000000000..9dc6069ea56 --- /dev/null +++ b/doc/tutorials/dune-package-management/index.md @@ -0,0 +1,27 @@ +--- +author: Marek Kubica +--- + +OCaml Package Management With Dune +================================== + +:::{warning} +Dune Package Management is not final yet and details are still subject to +change. +::: + +In this tutorial we will be looking at how to use Dune for managing project +dependencies. This enables users to install the compiler as well as third-party +dependencies using a single tool which takes care of building code and +dependencies. + +To get started you only need Dune. Head to {doc}`setup` to begin the setup. + +:::{toctree} +:hidden: +:maxdepth: 1 +setup +dependencies +pinning +repos +::: diff --git a/doc/tutorials/dune-package-management/pinning.md b/doc/tutorials/dune-package-management/pinning.md new file mode 100644 index 00000000000..c487555fef1 --- /dev/null +++ b/doc/tutorials/dune-package-management/pinning.md @@ -0,0 +1,50 @@ +# Pinning Projects + +When Dune is looking up packages to lock, it uses the (pre)configured OCaml +package repositories. However it is also possible to manually specify the +sources for packages; for example, if the package is not released in a package +repository. This is called "pinning." + +## Installing Packages From a Pin + +To pin a package, a new `pin` has to be declared in the `dune-project` file. + +::::{dropdown} `dune-project` +:icon: file-code + +:::{literalinclude} pinning/dune-project +:language: dune +:emphasize-lines: 4-6 +::: + +This will create a pin on the `fmt` package and use the specified Git repository +URL to retrieve the sources. + +:::: + +The next time the package is locked, Dune will use this repository instead of +the information from the selected package repositories. + +```sh +$ dune pkg lock +Solution for dune.lock: +- base-unix.base +- fmt.dev +- ocaml.5.0.0 +- ocaml-base-compiler.5.0.0 +- ocaml-config.3 +- ocamlbuild.0.15.0+dune +- ocamlfind.1.9.6+dune +- topkg.1.0.7 +``` + +Unlike previously, the version of the `fmt` library that is picked is `dev`, to +signify a development version. + +The next build will check out the sources from that repository instead of +downloading the release tarball: + +```sh +$ dune exec ./test.exe +Hello, OCaml, Rust! +``` diff --git a/doc/tutorials/dune-package-management/pinning/dune-project b/doc/tutorials/dune-package-management/pinning/dune-project new file mode 100644 index 00000000000..85196454cee --- /dev/null +++ b/doc/tutorials/dune-package-management/pinning/dune-project @@ -0,0 +1,12 @@ +(lang dune 3.17) +(name test) + +(pin + (url "git+https://github.com/dbuenzli/fmt.git") + (package (name fmt))) + +(package + (name test) + (depends + (ocaml (>= 4.14)) + fmt)) diff --git a/doc/tutorials/dune-package-management/repos.md b/doc/tutorials/dune-package-management/repos.md new file mode 100644 index 00000000000..4992c477f3b --- /dev/null +++ b/doc/tutorials/dune-package-management/repos.md @@ -0,0 +1,52 @@ +# Custom Repositories + +By default when locking package versions, Dune looks up packages from two +sources: + + 1. The upstream, community maintained `opam-repository` at + [ocaml/opam-repository](https://github.com/ocaml/opam-repository) for most + packages + 2. An overlay repository with patched software to allow usage in Dune at + [ocaml-dune/opam-overlays](https://github.com/ocaml-dune/opam-overlays) + +To change the presets, the `dune-workspace` file has to be edited (and created +if it didn't exist): + +::::{dropdown} `dune-workspace` +:icon: file-code + +:::{literalinclude} repos/dune-workspace +:language: dune +::: + +:::: + +In this case, we want to select a specific revision of the community repository +instead of always using the most recent one, as it would do by default. We +define a new repository and configure the lock directory to use this +repository. + +When relocking the dependencies, the list of packages that are found as +dependencies changes accordingly: + +```sh +$ dune pkg lock +Solution for dune.lock: +- base-unix.base +- fmt.0.9.0 +- ocaml.5.0.0 +- ocaml-base-compiler.5.0.0 +- ocaml-config.3 +- ocamlbuild.0.15.0+dune +- ocamlfind.1.9.6+dune +- topkg.1.0. +``` + +Compared to before, the OCaml compiler version is older, which shows +that we did indeed pick an older version of the package repository for locking. + +:::{note} +This feature can also be used to make sure the locked dependencies are +reproducible, as fixing all the package repository versions will lead to +equivalent locking results. +::: diff --git a/doc/tutorials/dune-package-management/repos/dune-workspace b/doc/tutorials/dune-package-management/repos/dune-workspace new file mode 100644 index 00000000000..601b59b1cd4 --- /dev/null +++ b/doc/tutorials/dune-package-management/repos/dune-workspace @@ -0,0 +1,8 @@ +(lang dune 3.17) + +(lock_dir + (repositories overlay specific-upstream)) + +(repository + (name specific-upstream) + (source "git+https://github.com/ocaml/opam-repository.git#00ac3727bc4ac0eabd3c89e69c1660d6b63a3d48")) diff --git a/doc/tutorials/dune-package-management/setup.md b/doc/tutorials/dune-package-management/setup.md new file mode 100644 index 00000000000..2c72f920c1f --- /dev/null +++ b/doc/tutorials/dune-package-management/setup.md @@ -0,0 +1,109 @@ +# Setting Up Package Management With Dune + +The idea of package management with Dune has been as unobtrusive as +possible. Thus most projects can easily be built with just the minimum of +changes. + +In this tutorial we will create a simple project to use the integrated package +management feature for the very first time. + +## Declare Dependencies + +The best way to work with the package management is to declare your +dependencies in the `dune-project` file. + +::::{dropdown} `dune-project` +:icon: file-code + +:::{literalinclude} setup/dune-project +:language: dune +:emphasize-lines: 6-7 +::: + +We define a project called `test` and define that to build it we need an OCaml +compiler that is at least version 4.14. + +This is exactly the same information that is used to generate opam files using +the `generate_opam_files` stanza as described in +{doc}`/howto/opam-file-generation`. + +:::: + +::::{dropdown} `test.ml` +:icon: file-code + +:::{literalinclude} setup/test.ml +:language: ocaml +::: + +To show that the build works, this simple program will be built and executed. + +:::: + + +::::{dropdown} `dune` +:icon: file-code + +:::{literalinclude} setup/dune +:language: dune +::: + +To declare our module an executable we need a little bit of configuration, so we +just define the module as an executable. + +:::: + +After our project skeleton is set up, we can proceed to the next step. + +## Locking Dependencies + +After declaring the dependencies, you will need to tell Dune which package +versions to use for your project. This is done by creating a lock directory. +This is easily done with a new Dune command: + +```sh +$ dune pkg lock +Solution for dune.lock: +- ocaml.5.2.0 +- ocaml-base-compiler.5.2.0 +- ocaml-config.3 +``` + +This will update all the required opam repositories, use the newest version of +each and try to find a set of packages and versions that satisfy the +constraints that your project dependencies declare. + +:::{note} +The versions that get locked might be different from this tutorial, as we only +specified the lower bound of `ocaml`; barring any additional configuration, Dune +will pick the newest possible version for each dependency. +::: + +## Build Project + +To build the project, you can just use the regular Dune commands. + +```sh +dune build +``` + +This will download, build, and install all your locked dependencies and then use +those to build your project. This means that the first time building it will take +longer than usual, as the dependencies need to be built first. Subsequent +builds where all dependencies have been built before will be just as fast as +before. + +We can show that the package has been built successfully and works as expected: + +```sh +$ dune exec ./test.exe +Hello, OCaml, Rust! +``` + +## Conclusion + +In this section we learned how to set up a Dune project that picks a compiler +and installs it without need for any additional tooling. + +In the next section {doc}`dependencies` we will look on how to add third party +dependencies. diff --git a/doc/tutorials/dune-package-management/setup/dune b/doc/tutorials/dune-package-management/setup/dune new file mode 100644 index 00000000000..e422d9fbae2 --- /dev/null +++ b/doc/tutorials/dune-package-management/setup/dune @@ -0,0 +1,2 @@ +(executable + (public_name test)) diff --git a/doc/tutorials/dune-package-management/setup/dune-project b/doc/tutorials/dune-package-management/setup/dune-project new file mode 100644 index 00000000000..ac137b377f8 --- /dev/null +++ b/doc/tutorials/dune-package-management/setup/dune-project @@ -0,0 +1,7 @@ +(lang dune 3.17) +(name test) + +(package + (name test) + (depends + (ocaml (>= 4.14)))) diff --git a/doc/tutorials/dune-package-management/setup/test.ml b/doc/tutorials/dune-package-management/setup/test.ml new file mode 100644 index 00000000000..60c76c598a7 --- /dev/null +++ b/doc/tutorials/dune-package-management/setup/test.ml @@ -0,0 +1,5 @@ +let langs = ["OCaml"; "Rust"] + +let () = + let s = String.concat ", " langs in + Format.printf "Hello, %s!\n" s diff --git a/doc/tutorials/index.md b/doc/tutorials/index.md index 2ce22aa1c62..1c6322e96cb 100644 --- a/doc/tutorials/index.md +++ b/doc/tutorials/index.md @@ -7,4 +7,5 @@ These tutorials are hands-on lessons to learn about Dune. :maxdepth: 1 developing-with-dune/index +dune-package-management/index :::