diff --git a/tmpl/wiki/package-quality.md b/tmpl/wiki/package-quality.md index c448ac7d9..1c6cd00b7 100644 --- a/tmpl/wiki/package-quality.md +++ b/tmpl/wiki/package-quality.md @@ -1,10 +1,12 @@ ## Package Quality +NOTE: this is WIP + The MirageOS ecosystem of libraries is booming: ``` $ opam list -a | grep mirage | wc -l - 90 + 95 ``` We now have various libraries covering all aspects of the OS: from network to @@ -17,40 +19,116 @@ developing your own application instead of spending time re-implementing something that the OS would usually do for you. There are of course still gaps in the ecosystem, but things are getting better! -The next problem is knowing the level of confidence to attribute to each of -these libraries. Would be great to be able to answer questions such as: +A MirageOS unikernel contains an unmodified OCaml runtime, but the `Unix` module +is intentionally! not implemented on MirageOS, neither is the `Str` or `Threads` +module. Any OCaml library depending directly or indirectly on these is not +usable with MirageOS. + +Only a very small set of C functions are implemented in the MirageOS lower +layers (such as +[ocaml-freestanding](https://github.com/mirage/ocaml-freestanding)). If an +OCaml library uses external libraries written in C via OCaml's C foreign +function interface, this library will likely not work on MirageOS since the +C functions are missing. It is possible to use C functions on MirageOS, but +the build system of such a library needs to "cross-compile" the C functions for +the different virtualization technologies. +Examples include +[mirage-entropy](https://github.com/mirage/mirage-entropy), +[checkseum](https://github.com/mirage/checkseum), +[digestif](https://github.com/mirage/digestif), +[bigstringaf](https://github.com/inhabitedtype/bigstringaf) +and [nocrypto](https://github.com/mirleft/ocaml-nocrypto) (which +uses [gmp](https://gmplib.org) via [zarith](https://github.com/ocaml/zarith) and +the gmp-freestanding and gmp-xen package in opam-repository), + +A problem is how to know the level of confidence to attribute to a MirageOS +library. Would be great to be able to answer questions such as: +Does this library work with MirageOS? Does the library has a good test coverage? Was it formally verified? -Was it used in production? In order to do this, I would like to introduce -the "MirageOS quality levels". Ideally this will live in a tool (e.g. +Was it used in production? In order to do this, we are introducing +the "MirageOS quality tags". Ideally this will live in a tool (e.g. `mirage lint`) but to start we list the criteria that package aiming for increased quality should follow. -### ![Level 1:](https://img.shields.io/badge/level-1-blue.svg) Follow Packaging Guidelines - -See the [packaging guidelines](https://mirage.io/wiki/packaging) for more -details. In summary: - -- The package should work well with [odig](http://erratique.ch/software/odig) - e.g. `CHANGES.md`, `LICENSE.md` and `README.md` should exist. +### ![ready:](https://img.shields.io/badge/mirageos-ready-orange.svg) usable with MirageOS + +The OCaml library is packaged with [opam](https://opam.ocaml.org), and released +to the [opam repository](https://github.com/ocaml/opam-repository). + +In addition, the library follows our [packaging guidelines](https://mirage.io/wiki/packaging). + +Checks to be in this level: +- The commands `opam lint` and `dune-release lint` or `topkg lint` return no + error. +- The metadata files `CHANGES.md`, `LICENSE.md` and `README.md` exist +- The use of [dune](http://dune.readthedocs.io/en/latest/) is + strongly encouraged but not yet mandatory. +- Immutable strings, since 4.06.0 default (`-safe-string`). +- The not available modules `Unix`, `Str` and `Threads` are not referenced. +- Either there are no C dependencies or the build system cross-compiles to + MirageOS targets. +- If a side-effecting computation, such as network access, random, etc. is + required, an implementation using the [mirage-types](https://github.com/mirage/mirage-types) + interfaces is provided in a sublibrary (e.g. [tls-mirage](https://github.com/mirleft/ocaml-tls/tree/master/mirage). + +## ![integrated:](https://img.shields.io/badge/mirageos-tested-yellow.svg) follows our best practises + +MirageOS follows software development best practises. The library has to +follow the [OCaml programming guidelines](https://ocaml.org/learn/tutorials/guidelines.html). + +For some functionality in MirageOS we agreed on OCaml libraries: +- A MirageOS unikernel should have a unified logging configuration of log level + and subsystems. We use the [logs](http://erratique.ch/software/logs/doc/Logs.html) + library for logging, which nicely separates the log source from the log reporter. + The [mirage-logs](https://github.com/mirage/mirage-logs) contains command-line + logging source and level support, a + [syslog reporter](https://github.com/hannesm/logs-syslog) is integrated. +- If a library reports metrics, e.g. it uses a cache or wants to measure + durations between request and response. The + [metrics](https://github.com/mirage/metrics) is used. + +Recommended OCaml libraries: +- Use [lwt](http://ocsigen.org/lwt/) for asynchronous tasks. +- Use [ipaddr](https://github.com/mirage/ocaml-ipaddr) for Internet Protocol + addresses. +- Use [astring](http://erratique.ch/software/astring/doc/Astring.html) for + string processing. +- Provide pretty printers by using + [fmt](http://erratique.ch/software/fmt/doc/Fmt.html). +- Have unit tests (using [alcotest](https://github.com/mirage/alcotest)), + ideally with automatedcoverage report output. +- [Ptime](http://erratique.ch/software/ptime/doc/Ptime.html) for POSIX time + computations. +- [Duration](https://github.com/hannesm/duration) converts time units + (milliseconds, seconds, ..) +- [Randomconv](https://github.com/hannesm/randomconv) converting bytes to + number ranges. + +Code best pracises: +- Avoid polymorphic equality and comparison -- when possible define your own + specialized functions. +- Avoid global mutable state. +- Use tail recursion. +- Do not expose any exceptions in the public interface, but use the + [result](http://caml.inria.fr/pub/docs/manual-ocaml/libref/Pervasives.html#TYPEresult) + type and [rresult](http://erratique.ch/software/rresult/doc/Rresult.html) + combinators. -- The package should work well with [topkg](http://erratique.ch/software/topkg), - e.g. `pkg/pkg.ml` should exist. +Style best practises +- Have proper indentation (using ocp-indent + checked-in ocp-indent file). +- Use at most 80 columns (`ocp-indent` unfortunately doesn't check this). -- The use of [jbuilder](http://jbuilder.readthedocs.io/en/latest/) is - strongly encouraged but not yet mandatory as 1.0.0 has not been - officially released. However, we plan to enforce its use as soon as - it is the case -- assuming that it has all the features needed for - building MirageOS applications. +The library is documented with a well-defined scope and its API is documented, +accessible online (e.g. [docs.mirage.io](https://docs.mirage.io)) +and can be generated locally with [odig](http://erratique.ch/software/odig). -- Use proper names: same name for opam and ocamlfind package, and also ideally - for the top-level module. +## ![automated:](https://img.shields.io/badge/mirageos-automated-yellow-green.svg) automated with continuous integration -- All direct dependencies are specified in the opam file (e.g. sometimes some - direct dependencies are included by transitivity -- this is not very stable - and could break the package in the future if one of a transitive dependency - is updated) (NOTE: should probably be added in the packaging guideline) +The library uses continuous integration systems to catch issues early: +- The [ocaml-ci-scripts](https://github.com/ocaml/ocaml-ci-scripts) contain + Travis and AppVeyor integration. -### ![Level 2](https://img.shields.io/badge/level-2-blue.svg) Define Package Scope +## ![approved:](https://img.shields.io/badge/mirageos-approved-green.svg) approved member of the MirageOS ecosystem - Have well-identified maintainers. GitHub recently [introduced code owners](https://github.com/blog/2392-introducing-code-owners), @@ -59,80 +137,31 @@ details. In summary: - Have a design document specifying the scope of the library (and esp. what is out of scope) -- Use the correct set of tags in opam metadata - -- Have documentation - -### ![Level 3:](https://img.shields.io/badge/level-3-blue.svg) Use Good Coding Style - -- Have proper indentation (using ocp-indent + checked-in ocp-indent file). - -- Use at most 80 columns (`ocp-indent` unfortunately doesn't check this). - -- Follow [OCaml programming guidelines](https://ocaml.org/learn/tutorials/guidelines.html). - -### ![Level 4:](https://img.shields.io/badge/level-4-blue.svg) Keep your Style Functional - -- Avoid global mutable state. - -- Use tail recursion. - -- Use `-safe-string` (looks like this will be the default in 4.06). - -- Avoid polymorphic equality and comparison -- when possible define your own - specialized functions. - -### ![Level 5:](https://img.shields.io/badge/level-5-blue.svg) Test - -- Have unit tests (using alcotest) (ideally with coverage report). - -### ![Level 6:](https://img.shields.io/badge/level-6-blue.svg) Keep Sane Dependencies - -- Depends only on libraries released in opam. - -- Provide pretty printers by using [fmt](http://erratique.ch/software/fmt/doc/Fmt.html). - -- Do not expose any exceptions in the public interface, but use the - [result](http://caml.inria.fr/pub/docs/manual-ocaml/libref/Pervasives.html#TYPEresult) - type and [rresult](http://erratique.ch/software/rresult/doc/Rresult.html) - combinators. - -- Use [logs](http://erratique.ch/software/logs/doc/Logs.html) for logging. - -- Use [astring](http://erratique.ch/software/astring/doc/Astring.html) for - string processing. - -- Use [bos](http://erratique.ch/software/bos/doc/Bos.html) for operating system - interaction. - -- Use [fpath](http://erratique.ch/software/fpath/doc/Fpath.html) for file paths. - - Avoid the use of C bindings for no good reasons. A good reason would be to improve performance by an order of magnitude, or re-use an existing C library that has not been rewritten yet. -### ![Level 7:](https://img.shields.io/badge/level-7-blue.svg) Randomized Test - -- Have randomized property-based testing. Using QuickCheck-like libraries or - even better using fuzz testing (and crowbar) when the tooling will be ready. +- Has randomized property-based testing. Using QuickCheck-like libraries or + fuzz testing (e.g. [crowbar](https://github.com/stedolan/crowbar/), automated + fuzz testing with [bun](https://github.com/yomimono/ocaml-bun) and + [ppx_deriving_crowbar](https://github.com/yomimono/ppx_deriving_crowbar/)). -### ![Level 8:](https://img.shields.io/badge/level-8-blue.svg) Count with Care - -- Avoid integer overflows (basically every addition and subtraction, as well +- Avoid integer overflows: basically every addition and subtraction, as well as multiplication needs to be guarded unless you know that an overflow can - never happen (in this case, a comment should be suggested)) + never happen (in this case, a comment should be suggested). + +- Work on 32bit (esp. in regards to the last point), tested by CI. -- Work on 32bit (esp. in regards to the last point) -### ![Level 9:](https://img.shields.io/badge/level-9-blue.svg) Used in Production +- Is reproducible by having a way to reproduce the exact binary result. See + [reproducible builds](https://reproducible-builds.org/) for further information. + OCaml produces reproducible binaries. A reproducible MirageOS unikernel needs + to record the exact version of its opam package dependencies, and each opam + package needs to be reproducible. - Have a clear indication if the library is used in production (and if yes by which project). -- For distributed binaries: have a way to reproduce the exact set of packages - needed to build the released version of the binary, for instance by vendoring - opam metadata. - -### ![Level 10:](https://img.shields.io/badge/level-10-blue.svg) Verify Formally +## ![verified:](https://img.shields.io/badge/mirageos-verified-purple.svg) Formally verified -- Has been formally verified. +- Has been formally verified with a proof assistant. diff --git a/tmpl/wiki/packages.md b/tmpl/wiki/packages.md index 6a72382d7..19c1fb6ac 100644 --- a/tmpl/wiki/packages.md +++ b/tmpl/wiki/packages.md @@ -66,6 +66,10 @@ A Mirage library should have and [mirage-block-lwt.1.1.0](https://github.com/mirage/mirage-block/blob/1.1.0/mirage-block-lwt.opam). These should have a github pages `doc:` link in order that `topkg` can detect the upstream repo. +- The opam file specifies all direct dependencies are specified. Sometimes + direct dependencies are included by transitivity, which is not very stable + and could break the package in the future if one of the transitive dependencies + is updated - `Makefile`: contains `jbuilder` invocations including the `--dev` argument to enable warnings as errors for local builds. For example [mirage-block.3.1.2](https://github.com/mirage/mirage-block/blob/1.1.0/Makefile)