Skip to content

Commit

Permalink
Organise the list into levels
Browse files Browse the repository at this point in the history
  • Loading branch information
samoht committed Aug 3, 2017
1 parent 4f7e8fa commit f2f74d3
Showing 1 changed file with 89 additions and 49 deletions.
138 changes: 89 additions & 49 deletions tmpl/wiki/package-quality.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## Package hardening
## Package Quality

The MirageOS ecosystem of libraries is booming:

Expand All @@ -7,7 +7,7 @@ $ opam list -a | grep mirage | wc -l
90
```

We now have various libraries covering all aspect of the OS: from network to
We now have various libraries covering all aspects of the OS: from network to
block device drivers -- at all aspects of the stack: from virtual network cards
to TLS or virtual block devices to virtual filesystems. These libraries also
span multiple hypervisors including Xen and (via solo5) KVM.
Expand All @@ -21,11 +21,40 @@ The next problem is knowing the level of confidence to attribute to each of
these libraries. Would be great to be able to answer questions such as:
Does the library has a good test coverage? Was it formally verified?
Was it used in production? In order to do this, I would like to introduce
the "MirageOS hardening process". Ideally this will live in a tool (e.g.
`mirage lint`) but to start we list the criteria that such an hardened
package should follow.
the "MirageOS quality levels". Ideally this will live in a tool (e.g.
`mirage lint`) but to start we list the criteria that package aiming
for increased quality should follow.

### Project Paperwork
### Level 1: Follow Packaging Guidelines

![level-1](https://img.shields.io/badge/level-1-blue.svg)

See the [packaging guidelines](https://mirage.io/wiki/packaging) for more
details. In summary:

- The package should work well with [odig](http://erratique.ch/software/odig)
e.g. `CHANGES.md`, `LICENSE.md` and `README.md` should exist.

- The package should work well with [topkg](http://erratique.ch/software/topkg),
e.g. `pkg/pkg.ml` should exist.

- The use of [jbuilder](http://jbuilder.readthedocs.io/en/latest/) is
strongly encouraged but not yet mandatory as 1.0.0 has not been
officially released. However, we plan to enforce its use as soon as
it is the case -- assuming that it has all the features needed for
building MirageOS applications.

- Use proper names: same name for opam and ocamlfind package, and also ideally
for the top-level module.

- All direct dependencies are specified in the opam file (e.g. sometimes some
direct dependencies are included by transitivity -- this is not very stable
and could break the package in the future if one of a transitive dependency
is updated) (NOTE: should probably be added in the packaging guideline)

### Level 2: Define Package Scope

![level-2](https://img.shields.io/badge/level-2-blue.svg)

- Have well-identified maintainers. GitHub recently
[introduced code owners](https://github.com/blog/2392-introducing-code-owners),
Expand All @@ -36,83 +65,94 @@ package should follow.

- Use the correct set of tags in opam metadata

- Use proper names: same name for opam and ocamlfind package, and also ideally
for the top-level module (see next point).
- Have documentation

- Use module alias instead of packs to define proper namespaces
(to avoid name-clashes)
### Level 3: Use Good Coding Style

- Have documentation
![level-3](https://img.shields.io/badge/level-3-blue.svg)

- Have proper indentation (using ocp-indent + checked-in ocp-indent file).

- Use at most 80 columns (`ocp-indent` unfortunately doesn't check this).

- Have a clear LICENSE file
- Follow [OCaml programming guidelines](https://ocaml.org/learn/tutorials/guidelines.html).

- Use [topkg](https://github.com/dbuenzli/topkg) to release, in order to ease
the task of the release manager(s).
### Level 4: Keep your Style Functional

- Have proper indentation (using ocp-indent + checked-in ocp-indent file)
![level-4](https://img.shields.io/badge/level-4-blue.svg)

- Use at most 80 columns (`ocp-indent` unfortunately doesn't check this)
- Avoid global mutable state.

- Follow [OCaml programming guidelines](https://ocaml.org/learn/tutorials/guidelines.html)
- Use tail recursion.

- Ideally, use [Jbuilder](https://github.com/janestreet/jbuilder) to build,
in order to have short feedback loops when fixing bugs spanning multiple
packages.
- Use `-safe-string` (looks like this will be the default in 4.06).

- Avoid polymorphic equality and comparison -- when possible define your own
specialized functions.

### Code Good Practice
### Level 5: Test

- Provide pretty printers by using [fmt](http://erratique.ch/software/fmt/doc/Fmt.html)
![level-5](https://img.shields.io/badge/level-4-blue.svg)

- Have unit tests (using alcotest) (ideally with coverage report).

### Level 6: Sane Dependencies

![level-6](https://img.shields.io/badge/level-6-blue.svg)

- Depends only on libraries released in opam.

- Provide pretty printers by using [fmt](http://erratique.ch/software/fmt/doc/Fmt.html).

- Do not expose any exceptions in the public interface, but use the
[result](http://caml.inria.fr/pub/docs/manual-ocaml/libref/Pervasives.html#TYPEresult)
type and [rresult](http://erratique.ch/software/rresult/doc/Rresult.html)
combinators!
combinators.

- Use [logs](http://erratique.ch/software/logs/doc/Logs.html) for logging
- Use [logs](http://erratique.ch/software/logs/doc/Logs.html) for logging.

- Use [astring](http://erratique.ch/software/astring/doc/Astring.html) for
string processing
string processing.

- Use [bos](http://erratique.ch/software/bos/doc/Bos.html) for operating system
interaction
interaction.

- Use [fpath](http://erratique.ch/software/fpath/doc/Fpath.html) for file paths
- Use [fpath](http://erratique.ch/software/fpath/doc/Fpath.html) for file paths.

- Avoid integer overflows (basically every addition and subtraction, as well
as multiplication needs to be guarded unless you know that an overflow can
never happen (in this case, a comment should be suggested))
- Avoid the use of C bindings for no good reasons. A good reason would be to
improve performance by an order of magnitude, or re-use an existing C library
that has not been rewritten yet.

- Work on 32bit (esp. in regards to the last point)
### Level 7: Randomized Test

- Avoid global mutable state
![level-7](https://img.shields.io/badge/level-7-blue.svg)

- Be aware of non-tail recursive functions
- Have randomized property-based testing. Using QuickCheck-like libraries or
even better using fuzz testing (and crowbar) when the tooling will be ready.

- Be aware of polymorphic equality drawbacks -- when possible define your own
specialized functions.
### Level 8: Count with Care

### Project Confidence
![level-8](https://img.shields.io/badge/level-8-blue.svg)

- All direct dependencies are specified in the opam file (e.g. sometimes some
direct dependencies are included by transitivity -- this is not very stable
and could break the package in the future if one of a transitive dependency
is updated)

- Use `-safe-string` (looks like this will be the default in 4.06)
- Avoid integer overflows (basically every addition and subtraction, as well
as multiplication needs to be guarded unless you know that an overflow can
never happen (in this case, a comment should be suggested))

- Have unit tests (using alcotest) (ideally with coverage report)
- Work on 32bit (esp. in regards to the last point)

- Have fuzz tests (using crowbar)
### Level 9: Used in Production

- Depends only on libraries released in opam
![level-9](https://img.shields.io/badge/level-9-blue.svg)

- Is released in opam
- Have a clear indication if the library is used in production (and if yes by
which project).

- For distributed binaries: have a way to reproduce the exact set of packages
needed to build the released version of the binary, for instance by vendoring
opam metadata.

- have a clear indication if the library is used in production (and if yes by
which project)
### Level 10: Verify Formally

![level-10](https://img.shields.io/badge/level-10-blue.svg)

- bonus: has been formally verified
- Has been formally verified.

0 comments on commit f2f74d3

Please sign in to comment.